Fully Independent Validation.
In fully independent validation, model validation is performed in data collected by independent investigators, usually at a different location. Generally, the validation sample is drawn from a different time. It is important to establish that the model is equally accurate when applied by independent investigators, as they are unlikely to study identically selected patients and data collecting tools3. In addition, the definition of predictors and outcomes and study participants selection may be slightly different compared with the development setting in fully independent validation2.
Full independent validation often shows poor results (more unfavorable) than temporal or geographical validation. There could be several reasons for that. Some of those reasons are related to the original model’s development issues such as inadequate model development strategy, small sample size, and suboptimal statistical analysis. It also happens frequently that all the variables used to build the original model may not be available at validation data, which eventually affects the model’s performance in validation data. In addition, a true difference between development and validation samples may cause poor validation results. Fully independent external validation of a model is often more difficult than anticipated. However, if a model can demonstrate adequate performance in a fully independent validation in a different setting, then the results of this model’s performance are more authentic, acceptable and generalizable.