AI improved the doctors’ performance on CNS malformations identification
When facilitated with the AI diagnosis, the overall diagnostic efficacy of three subgroups of doctors got significantly improved (Table 3, Figure 5a, b, c) in terms of accuracy, sensitivity, and AUC. For the experts, the accuracy, sensitivity and AUC were improved from 78.9% to 84.7% (p = 0.002), from 77.5% to 83.4% (p = 0.003), and from 0.853 to 0.910 (p = 0.019), respectively. For the competent doctors, the improvements for accuracy was from 69.6% to 85.1% (p = 0.005), sensitivity was from 67.5% to 84.0% (p = 0.006), and AUC from 0.793 to 0.905(p = 0.002). For trainee doctors, the progress was shown in accuracy (51.5% vs. 80.2%, p = 0.001), sensitivity (48.6% vs.78.7%, p = 0.001), and AUC (0.654 vs. 0.872, p = 0.006), respectively. Whereas, no significant difference was noted in specificity with and without AI assistance. Among the three groups of doctors, the trainee group received maximum improvement with AI assistance, whose diagnostic performance advanced to the level of expert group in terms of accuracy[ (80.2% (95% CI 75.0-85.3%) vs. 78.9 %(95% CI 75.2-85.2%), P = 0.593] and AUC [0.872 (95% CI 0.861-0.882) vs. 0.853(95% CI 0.809-0.905), p = 0.238]. (Table 4).
The average time for diagnosis required by 13 doctors reduced significantly (7040s vs. 11571s, p < 0.001) with AI assistance, compared to that without AI assistance. Compared the time in subgroup, the time required by trainee doctors (7383s vs. 12663s, p = 0.008) and competent doctors (7729s vs. 12801s, p = 0.018) also decreased. However, for experts, no significant time-saving was observed (5923s vs. 8864s, p = 0.114).