Methods
The echo core Lab of the Institute of Clinical Physiology-CNR in Pisa (LG) coordinated the quality control procedure of all investigators at different centres participating in the RIGHT-NET study. Each center designated one operator that performed or reported at least 100 stress echocardiography studies per year. All readers were certified by national and/ or international societies. The quality control process was designed to be simple, repeatable, and sustainable. The Echo Core Lab issued a User Manual with a detailed description on how to measure each parameter, according to the most recent American and European Recommendations and Guidelines.8-11 The User Manual was sent to all Participating Centers including the reference for transthoracic echocardiography assessment. All participating centers followed the recommended standard operational procedures in terms of data data storage (data format, transfer procedure), and data processing (software used and measurement procedures). All operators performing and reading echocardiographic exams adhered to the quality control protocol. The Echo Core Lab sent ten complete echocardiographic examinations in DICOM format through a safe file sharing platform (Fig. 1) . All participating investigators were invited by email to join the platform, which was protected by user-specific passwords. The platform includes also detailed instructions on how to start the training and allows downloading and uploading of external files. All images and videos were completely anonymized to protect patients’ privacy, according to the new EU directive of protection of personal data (GDPR).6
Reading sessions
We randomly selected 10 cases including healthy subjects and at least one group of patients with overt and/or at risk of pulmonary hypertension (PH), according to clinical classification of PH(Table 1) .12 Data were collected on patients undergoing EDE on a semi-recumbent cycle ergometer with an incremental workload of 25 Watts every 2 minutes up to the symptom-limited maximal tolerated workload including resting, 50 Watts, peak stress and recovery acquisition, as previously described.6 All operators directly measured the requested parameters by uploading the same ten cases from the web platform to their echocardiography machine. The DICOM format enabled to perform assessment of variables in the respective centres. All operators were then asked to enter their measurements in a dedicated excel file, which was then sent to the Coordinating Center for analysis. Table 2 provides the list of the left and the right heart parameters measured by all operators. The gold standard value for each measurement was established by the values measured by the Echo Core Lab.
Statistical analysis
Statistical analysis was performed using standard software (MedCalc version 14.8.1, MedCalc Software Ltd, Belgium; SPSS version 20.0, SPSS, Inc., Chicago, IL). Continuous variables were described by mean values ± standard deviation (SD). Accuracy (in %) for each observer was estimated by comparison with the reference standard (core lab reading). Intra-class correlation coefficient (ICC) was calculated along with the 95% confidence interval, in order to quantify the reliability of measurement process. An ICC of >0.8 indicated excellent agreement with the core lab. Inter-observer variability among 19 observers were examined for resting and peak exercise TRV, RVOT Act, TAPSE, S’, right ventricular fractional area change (RV FAC), LVOT velocity time integral (VTI), mitral early inflow pulsed wave Doppler velocity (E), early diastolic mitral annular lateral and septal velocity by TDI (e’), left ventricular ejection fraction (LVEF). Data are presented as mean of the absolute and relative differences (in %) between measurements.
Intra-observer agreement was tested in 2 observers who volunteered to repeat the measurement session on 2 separate days and ICC was calculated.