VSSI 17 was developed as a procedure-specific rating tool to assess surgeons while performing vaginal hysterectomies. Interestingly, the 13 items in the Likert scale are not procedure specific and can be applied to laparoscopic surgery in general. A limitation is that this transfer of general competencies to a specific rating tool did not prove to be appropriate. Importantly, the authors focus on case mix, where a specific (patient) characteristic is known to potentially effect (surgical) outcome. A recent review on case-mix variables and predictors for outcomes of laparoscopic hysterectomy showed that body mass index, previous operations, adhesions and age were predominate case-mix characteristics.25This knowledge on case mix is important when choosing a surgical case for assessment.
Chou et al. modified an existing global rating scale by adding procedure specific items to develop HASC 18, which targets gynaecologic trainees and aims to evaluate all surgical competencies in gynaecologic surgery. This procedure-specific rating tool is applicable to all types of laparoscopic surgery. The generalizability and lack of a task-specific checklist makes HASC applicable to other gynaecologic programmes. To our knowledge, this applicability has not been demonstrated in other validated studies. The study was not blinded, only trainees were tested and data were collected for all types of surgical procedures, lowering the strength of the study.
The OSALS rating tools is incorporated in the Danish curriculum for assessment of OBGYN trainees. 1926.It comprises five general and five task-specific items and was developed and validated in a blinded study19. There was a wide performance range in the expert group and a narrow performance range in the novice group, which could be explained by case mix and by the fact that categorising surgeons as intermediate or expert can be difficult. The study is limited by a small sample size.
Arguably, a disadvantage of video evaluation is that it is time-consuming, but Larsen et al. underline it as a strength for the objective assessment, an assertion that Langermann et al. support, arguing that video recording in the operating theatre enhances and supports surgical training and can be performed equally good by doctors with different expeince.27 28
Six of the included studies used video recording and blinded observers when evaluating the surgeon’s performance. All of the studies found significant discriminative validity, demonstrating that the assessor can differentiate between novices, advanced beginners and experts. This indicates that video-recorded assessment is a good choice when validating an assessment tool, but as it is time-consuming, it may not be an obvious choice for implementation in daily clinical practice.29