Performance and Validation of the Prediction Model
All models were evaluated using the test data. The metrics used to
assess performance include Area Under the Curve (AUC), accuracy,
precision, and recall in accordance with previous recommendations for
results reporting of clinical prediction models.14 The
model with the highest performance in the most performance metrics was
chosen as the final model. The permutation feature importance (PFI)
scores were also obtained to illuminate the most significant variables
used in the model’s prediction. The PFI scores are the difference in
model performance determined by the AUC before and after alteration of a
given dependent variable. Thus, the absolute magnitude of a PFI score
reflects the impact an individual variable has on the overall
performance.