Method 4: Albatross plots
The albatross plot was first described by Harrison et al7 and is also discussed in Higgins 22. This method requires minimal data extraction or manipulation and allows data to be synthesised even in circumstances when outcomes are reported in multiple different formats or where no summary statistics are reported. Reported results are split into two groups according to the direction of effect; and then p-values are plotted against sample size. Where necessary, 1-sided p-values need to be converted to 2-sided p-values (or vice versa) to ensure consistency. An albatross plot allows us to combine outcome data that was reported in a variety of different ways, including from studies where only a p-value was provided. Under an assumption of normality, you would expect results corresponding to the same effect size to lie along a contour, with p-values generally getting smaller as sample size increases. Contours can be added to the plot for a range of different effect sizes based on standardised mean differences, mean differences, odds ratios or other summary of choice. Effect sizes can be estimated according to where the majority of points lie. We have added contours to represent standardised mean differences of 0.3, 0.6 and 0.9. Heterogeneity can also be explored visually by looking at how closely trials tend to group together along a particular contour.
Note that where p-values are obtained from studies that are clustered in some way, adjustment of sample size is necessary. One method of doing this is to calculate the effective sample size (E) using the sample size (S), the reported intra-class correlation coefficient (ICC) and the average cluster size (M) using the formula23
\begin{equation} E=\frac{S}{1+ICC\ \times(M-1)}\nonumber \\ \end{equation}
An alternative is to replace the sample size with the number of health care professionals (or sites) as in method 3.
For illustration we produced a contour plot using the number of health care professionals (or sites) as the sample size (Figure 1)