loading page

Consequences of multiple imputation of missing standard deviations and sample sizes in meta-analysis
  • +3
  • Stephan Kambach,
  • Helge Bruelheide,
  • Katharina Gerstner,
  • Jessica Gurevitch,
  • Michael Beckmann,
  • Ralf Seppelt
Stephan Kambach
German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig
Author Profile
Helge Bruelheide
Martin Luther University Halle Wittenberg, Martin Luther University Halle-Wittenberg
Author Profile
Katharina Gerstner
German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig
Author Profile
Jessica Gurevitch
Stony Brook University
Author Profile
Michael Beckmann
Helmholtz-Centre for Environmental Research - UFZ
Author Profile
Ralf Seppelt
Helmholtz-Centre for Environmental Research - UFZ
Author Profile

Abstract

Meta-analyses often encounter studies with incompletely reported variance measures (e.g. standard deviation values) or sample sizes, both needed to conduct weighted meta-analyses. Here, we first present a systematic literature survey on the frequency and treatment of missing data in published ecological meta-analyses showing that the majority of meta-analyses encountered incompletely reported studies. We then simulated meta-analysis data sets to investigate the performance of 14 options to treat or impute missing SDs and/or SSs. Performance was thereby assessed using results from fully informed weighted analyses on (hypothetically) complete data sets. We show that the omission of incompletely reported studies is not a viable solution. Unweighted and sample size-based variance approximation can yield unbiased grand means if effect sizes are independent of their corresponding SDs and SSs. The performance of different imputation methods depends on the structure of the meta-analysis data set, especially in the case of correlated effect sizes and standard deviations or sample sizes. In a best-case scenario, which assumes that SDs and/or SSs are both missing at random and are unrelated to effect sizes, our simulations show that the imputation of up to 90% of missing data still yields grand means and confidence intervals that are similar to those obtained with fully informed weighted analyses. We conclude that multiple imputation of missing variance measures and sample sizes could help overcome the problem of incompletely reported primary studies, not only in the field of ecological meta-analyses. Still, caution must be exercised in consideration of potential correlations and pattern of missingness.

Peer review status:ACCEPTED

20 Aug 2020Submitted to Ecology and Evolution
21 Aug 2020Assigned to Editor
21 Aug 2020Submission Checks Completed
22 Aug 2020Review(s) Completed, Editorial Evaluation Pending
24 Aug 2020Editorial Decision: Accept