Method 1: Standardised Mean Differences, weights based on
adjusted standard error
This method is commonly used, including in the SOCIAL systematic review8 and other reviews14-17. This method is
simple to use, it utilises information that is generally reported, and
it can be performed using standard statistical software. All reported
measures of intervention effect are converted into an approximation of
the standardised mean difference (SMD) using the formulae in Table 21819.
The formula for a standardised mean difference for a continuous outcome
(Table 2) refers to Cohen’s d for ease of calculation. As an alternative
Hedges g may be used 19which allows a correction for small sample size.
We applied these methods to the SOCIAL meta-analysis using the inverse
of the squared adjusted standard error as weights (inverse variance
method 13).
Where a trial reported both a continuous and binary outcome measure with
appropriately adjusted standard errors, we utilised the continuous
measure but also calculated the SMDs and standard errors using the
binary measure to check for anomalies. Note that rules such as this
should be pre-specified to avoid post-hoc decisions that could introduce
bias.
Method 2: Separate analyses for binary and continuous
outcomes, weights based on adjusted standard errors
In this method two separate analyses are produced for the same outcome;
one for those that were reported as binary measurements and one for
those that were reported as continuous measurements. This method has
been used in a number of systematic reviews of health behaviour change4,5,16.
This method requires very little data manipulation and adopts a
conservative approach to heterogeneity by keeping the two types of
outcome separate.
For illustration we performed meta-analysis on the SOCIAL data using
odds ratios for binary data and standardised mean differences for
continuous data, using the inverse variance method with weights based on
adjusted standard errors.
Outcomes were reported as continuous measures on a variety of different
scales; therefore they were converted to standardised mean differences
as above 20 . If all
continuous measures had been measured on the same scale, e.g. mean
percentage on a scale of 0 to 100, they could be meta-analysed using
means and standard deviations. For meta-analysis of binary data, all
summaries need to be converted to the same format (odds ratio, risk
ratio or risk difference); it is recommended that this be chosen in
advance at the protocol stage to avoid selective reporting. We chose
odds ratios 18 here as
they have certain desirable mathematical properties; their symmetrical
nature would mean that an analysis where the outcome measure is
‘compliance’ or an analysis where the outcome measure is
‘non-compliance’ would lead to identical conclusions.
Sometimes trials report the same outcome measure in both binary and
continuous formats – for example in a trial where the desired behaviour
is ‘test ordering’; summary data could be reported both in terms of the
overall proportion of patients who had a test ordered, and the mean
proportion of tests ordered by health care professional. Where a trial
has reported an outcome in both binary and continuous formats, we
included both measures in the two separate meta-analyses. Note that when
using this method, continuous and binary results may not be later
combined together as this would lead to double counting of the same
participants.