A meta-evaluation of the quality of reporting and execution in ecological meta-analyses
- Smithsonian Environmental Research Center, Tiburon, California (United States); OSTI
- Lanzhou University, Gansu (China)
- Northern Arizona University, Flagstaff, AZ (United States)
- University of Georgia, Athens, GA (United States)
Quantitatively summarizing results from a collection of primary studies with meta-analysis can help answer ecological questions and identify knowledge gaps. The accuracy of the answers depends on the quality of the meta-analysis. We reviewed the literature assessing the quality of ecological meta-analyses to evaluate current practices and highlight areas that need improvement. From each of the 18 review papers that evaluated the quality of meta-analyses, we calculated the percentage of meta-analyses that met criteria related to specific steps taken in the meta-analysis process (i.e., execution) and the clarity with which those steps were articulated (i.e., reporting). We also re-evaluated all the meta-analyses available from Pappalardo et al. to extract new information on ten additional criteria and to assess how the meta-analyses recognized and addressed non-independence. In general, we observed better performance for criteria related to reporting than for criteria related to execution; however, there was a wide variation among criteria and meta-analyses. Meta-analyses had low compliance with regard to correcting for phylogenetic non-independence, exploring temporal trends in effect sizes, and conducting a multifactorial analysis of moderators (i.e., explanatory variables). In addition, although most meta-analyses included multiple effect sizes per study, only 66% acknowledged some type of non-independence. The types of non-independence reported were most often related to the design of the original experiment (e.g., the use of a shared control) than to other sources (e.g., phylogeny). We suggest that providing specific training and encouraging authors to follow the PRISMA EcoEvo checklist recently developed by O’Dea et al. can improve the quality of ecological meta-analyses.
- Research Organization:
- Northern Arizona University, Flagstaff, AZ (United States)
- Sponsoring Organization:
- National Science Foundation (NSF); USDOE Office of Science (SC), Biological and Environmental Research (BER)
- Grant/Contract Number:
- SC0010632
- OSTI ID:
- 2471996
- Journal Information:
- PLoS ONE, Journal Name: PLoS ONE Journal Issue: 10 Vol. 18; ISSN 1932-6203
- Publisher:
- Public Library of ScienceCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Similar Records
Data from: Comparing traditional and Bayesian approaches to ecological meta-analysis
Comparing traditional and Bayesian approaches to ecological meta‐analysis
Dataset
·
Tue Jul 14 00:00:00 EDT 2020
·
OSTI ID:1873870
Comparing traditional and Bayesian approaches to ecological meta‐analysis
Journal Article
·
Sat Jul 25 20:00:00 EDT 2020
· Methods in Ecology and Evolution (Online)
·
OSTI ID:1643175