Fully Independent Validation.
In fully independent validation, model validation is performed in data
collected by independent investigators, usually at a different location.
Generally, the validation sample is drawn from a different time. It is
important to establish that the model is equally accurate when applied
by independent investigators, as they are unlikely to study identically
selected patients and data collecting tools3. In
addition, the definition of predictors and outcomes and study
participants selection may be slightly different compared with the
development setting in fully independent validation2.
Full independent validation often shows poor results (more unfavorable)
than temporal or geographical validation. There could be several reasons
for that. Some of those reasons are related to the original model’s
development issues such as inadequate model development strategy, small
sample size, and suboptimal statistical analysis. It also happens
frequently that all the variables used to build the original model may
not be available at validation data, which eventually affects the
model’s performance in validation data. In addition, a true difference
between development and validation samples may cause poor validation
results. Fully independent external validation of a model is often more
difficult than anticipated. However, if a model can demonstrate adequate
performance in a fully independent validation in a different setting,
then the results of this model’s performance are more authentic,
acceptable and generalizable.