skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Bias in meta-analyses using Hedges’ d

Abstract

The type of metric and weighting method used in meta-analysis can create bias and alter coverage of confidence intervals when the estimated effect size and its weight are correlated. Here, we investigate bias associated with the common metric, Hedges’ d, under conditions common in ecological meta-analyses. We simulated data from experiments, computed effect sizes and their variances, and performed meta-analyses applying three weighting schemes (inverse variance, sample size, and unweighted) for varying levels of effect size, within-study replication, number of studies in the meta-analysis, and among-study variance. Unweighted analyses, and those using weights based on sample size, were close to unbiased and yielded coverages close to the nominal level of 0.95. In contrast, the inverse-variance weighting scheme led to bias and low coverage, especially for meta-analyses based on studies with low replication. This bias arose because of a correlation between the estimated effect and its weight when using the inverse-variance method. In many cases, the sample size weighting scheme was most efficient, and, when not, the differences in efficiency among the three methods were relatively minor. Thus, if using Hedges’ d, we recommend using weights based upon sample size that do not involve individual study estimates of the effect size.

Authors:
ORCiD logo [1]; ORCiD logo [1];  [2];  [3]; ORCiD logo [1]
  1. Univ. of Georgia, Athens, GA (United States). Odum School of Ecology
  2. Michigan State Univ., East Lansing, MI (United States). Quantitative Fisheries Center. Dept. of Fisheries and Wildlife
  3. Michigan State Univ., East Lansing, MI (United States). Dept. of Fisheries and Wildlife
Publication Date:
Research Org.:
Univ. of Georgia, Athens, GA (United States); Michigan State Univ., East Lansing, MI (United States)
Sponsoring Org.:
USDOE Office of Science (SC), Biological and Environmental Research (BER) (SC-23); National Science Foundation (NSF)
OSTI Identifier:
1473748
Alternate Identifier(s):
OSTI ID: 1476656; OSTI ID: 1483359
Grant/Contract Number:  
SC0010632; DEB-1655426; DEB-1655394
Resource Type:
Published Article
Journal Name:
Ecosphere
Additional Journal Information:
Journal Volume: 9; Journal Issue: 9; Journal ID: ISSN 2150-8925
Publisher:
Ecological Society of America
Country of Publication:
United States
Language:
English
Subject:
54 ENVIRONMENTAL SCIENCES; bias; coverage; effect size; Hedges' d; meta-analysis; sample size; weights

Citation Formats

Hamman, Elizabeth A., Pappalardo, Paula, Bence, James R., Peacor, Scott D., and Osenberg, Craig W. Bias in meta-analyses using Hedges’ d. United States: N. p., 2018. Web. doi:10.1002/ecs2.2419.
Hamman, Elizabeth A., Pappalardo, Paula, Bence, James R., Peacor, Scott D., & Osenberg, Craig W. Bias in meta-analyses using Hedges’ d. United States. doi:10.1002/ecs2.2419.
Hamman, Elizabeth A., Pappalardo, Paula, Bence, James R., Peacor, Scott D., and Osenberg, Craig W. Tue . "Bias in meta-analyses using Hedges’ d". United States. doi:10.1002/ecs2.2419.
@article{osti_1473748,
title = {Bias in meta-analyses using Hedges’ d},
author = {Hamman, Elizabeth A. and Pappalardo, Paula and Bence, James R. and Peacor, Scott D. and Osenberg, Craig W.},
abstractNote = {The type of metric and weighting method used in meta-analysis can create bias and alter coverage of confidence intervals when the estimated effect size and its weight are correlated. Here, we investigate bias associated with the common metric, Hedges’ d, under conditions common in ecological meta-analyses. We simulated data from experiments, computed effect sizes and their variances, and performed meta-analyses applying three weighting schemes (inverse variance, sample size, and unweighted) for varying levels of effect size, within-study replication, number of studies in the meta-analysis, and among-study variance. Unweighted analyses, and those using weights based on sample size, were close to unbiased and yielded coverages close to the nominal level of 0.95. In contrast, the inverse-variance weighting scheme led to bias and low coverage, especially for meta-analyses based on studies with low replication. This bias arose because of a correlation between the estimated effect and its weight when using the inverse-variance method. In many cases, the sample size weighting scheme was most efficient, and, when not, the differences in efficiency among the three methods were relatively minor. Thus, if using Hedges’ d, we recommend using weights based upon sample size that do not involve individual study estimates of the effect size.},
doi = {10.1002/ecs2.2419},
journal = {Ecosphere},
number = 9,
volume = 9,
place = {United States},
year = {2018},
month = {9}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
DOI: 10.1002/ecs2.2419

Citation Metrics:
Cited by: 1 work
Citation information provided by
Web of Science

Save / Share: