Approach Strengths Weaknesses
Method 1 SMDs. Weights based on adjusted standard errors All available data combined Clustering accounted for at level of randomisation Mixture of different outcomes and formats likely to lead to heterogeneity May be difficult to interpret Inconsistent units of analysis (patient/HCP/site) Estimation assumptions may not hold
Method 2 Separate analyses for binary and continuous data. Weights based on adjusted standard errors Likely to lead to less heterogeneity than method 1 as more similar measures are being combined. Little manipulation or estimation required Does not combine all available information in a single analysis, which leads to loss of power and multiplicity Two analyses may give conflicting results
Method 3 SMDs. Weighting by number of HCP
Consistent units – weighted by health care professional
Number of health care professionals not always reported, requiring an estimate to be imputed Weighting may be related to quality of reporting; e.g. poorly reported studies get less weight. Unit of analysis error when not randomised at level of analysis Weights related to the size of the study but not the variability/precision Issues with SMDs as above
Method 4 Albatross plot May include additional studies that report p-value only No assumptions Difficult to check that p-values are correct if not accompanied by other summary data P-values prone to selective reporting Need to adjust sample size in some way for cluster trials