skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Integrating the evidence from evidence factors in observational studies

Abstract

Summary A sensitivity analysis for an observational study assesses how much bias, due to nonrandom assignment of treatment, would be necessary to change the conclusions of an analysis that assumes treatment assignment was effectively random. The evidence for a treatment effect can be strengthened if two different analyses, which could be affected by different types of biases, are both somewhat insensitive to bias. The finding from the observational study is then said to be replicated. Evidence factors allow for two independent analyses to be constructed from the same dataset. When combining the evidence factors, the Type I error rate must be controlled to obtain valid inference. A powerful method is developed for controlling the familywise error rate for sensitivity analyses with evidence factors. It is shown that the Bahadur efficiency of sensitivity analysis for the combined evidence is greater than for either evidence factor alone. The proposed methods are illustrated through a study of the effect of radiation exposure on the risk of cancer. An R package, evidenceFactors, is available from CRAN to implement the methods of the paper.

Authors:
 [1];  [2];  [1]
  1. Department of Statistics, The Wharton School, University of Pennsylvania, 3730 Walnut Street, Philadelphia, Pennsylvania 19104-6340, U.S.A
  2. Department of Statistics, Radiation Effects Research Foundation, 5-2 Hijiyama Park, Minami-ku, Hiroshima 732-0815, Japan
Publication Date:
Sponsoring Org.:
USDOE
OSTI Identifier:
1511779
Resource Type:
Published Article
Journal Name:
Biometrika
Additional Journal Information:
Journal Name: Biometrika Journal Volume: 106 Journal Issue: 2; Journal ID: ISSN 0006-3444
Publisher:
Oxford University Press
Country of Publication:
United Kingdom
Language:
English

Citation Formats

Karmakar, B., French, B., and Small, D. S. Integrating the evidence from evidence factors in observational studies. United Kingdom: N. p., 2019. Web. doi:10.1093/biomet/asz003.
Karmakar, B., French, B., & Small, D. S. Integrating the evidence from evidence factors in observational studies. United Kingdom. doi:10.1093/biomet/asz003.
Karmakar, B., French, B., and Small, D. S. Wed . "Integrating the evidence from evidence factors in observational studies". United Kingdom. doi:10.1093/biomet/asz003.
@article{osti_1511779,
title = {Integrating the evidence from evidence factors in observational studies},
author = {Karmakar, B. and French, B. and Small, D. S.},
abstractNote = {Summary A sensitivity analysis for an observational study assesses how much bias, due to nonrandom assignment of treatment, would be necessary to change the conclusions of an analysis that assumes treatment assignment was effectively random. The evidence for a treatment effect can be strengthened if two different analyses, which could be affected by different types of biases, are both somewhat insensitive to bias. The finding from the observational study is then said to be replicated. Evidence factors allow for two independent analyses to be constructed from the same dataset. When combining the evidence factors, the Type I error rate must be controlled to obtain valid inference. A powerful method is developed for controlling the familywise error rate for sensitivity analyses with evidence factors. It is shown that the Bahadur efficiency of sensitivity analysis for the combined evidence is greater than for either evidence factor alone. The proposed methods are illustrated through a study of the effect of radiation exposure on the risk of cancer. An R package, evidenceFactors, is available from CRAN to implement the methods of the paper.},
doi = {10.1093/biomet/asz003},
journal = {Biometrika},
number = 2,
volume = 106,
place = {United Kingdom},
year = {2019},
month = {4}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record
DOI: 10.1093/biomet/asz003

Save / Share:

Works referenced in this record:

On the optimally weighted <mml:math altimg="si137.gif" display="inline" overflow="scroll" xmlns:xocs="http://www.elsevier.com/xml/xocs/dtd" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.elsevier.com/xml/ja/dtd" xmlns:ja="http://www.elsevier.com/xml/ja/dtd" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:tb="http://www.elsevier.com/xml/common/table/dtd" xmlns:sb="http://www.elsevier.com/xml/common/struct-bib/dtd" xmlns:ce="http://www.elsevier.com/xml/common/dtd" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:cals="http://www.elsevier.com/xml/common/cals/dtd" xmlns:sa="http://www.elsevier.com/xml/common/struct-aff/dtd"><mml:mi>z</mml:mi></mml:math>-test for combining probabilities from independent studies
journal, February 2014


Selection Bias in Cancer Risk Estimation from A-Bomb Survivors
journal, June 2007

  • Pierce, Donald A.; Vaeth, Michael; Shimizu, Yukiko
  • Radiation Research, Vol. 167, Issue 6
  • DOI: 10.1667/RR0349.1

The sensitivity of linear regression coefficients’ confidence limits to the omission of a confounder
journal, June 2010

  • Hosman, Carrie A.; Hansen, Ben B.; Holland, Paul W.
  • The Annals of Applied Statistics, Vol. 4, Issue 2
  • DOI: 10.1214/09-AOAS315

Sensitivity analysis for certain permutation inferences in matched observational studies
journal, January 1987


Solid Cancer Incidence in Atomic Bomb Survivors: 1958–1998
journal, July 2007

  • Preston, D. L.; Ron, E.; Tokuoka, S.
  • Radiation Research, Vol. 168, Issue 1
  • DOI: 10.1667/RR0763.1

Effect Modification and Design Sensitivity in Observational Studies
journal, March 2013

  • Hsu, Jesse Y.; Small, Dylan S.; Rosenbaum, Paul R.
  • Journal of the American Statistical Association, Vol. 108, Issue 501
  • DOI: 10.1080/01621459.2012.742018

Radiation-Related Cancer Risks at Low Doses among Atomic Bomb Survivors
journal, August 2000


Truncated product method for combiningP-values
journal, January 2002

  • Zaykin, D. V.; Zhivotovsky, Lev A.; Westfall, P. H.
  • Genetic Epidemiology, Vol. 22, Issue 2
  • DOI: 10.1002/gepi.0042

A comparison of Bayesian and Monte Carlo sensitivity analysis for unmeasured confounding: L. C. MCCANDLESS AND P. GUSTAFSON
journal, April 2017

  • McCandless, Lawrence C.; Gustafson, Paul
  • Statistics in Medicine, Vol. 36, Issue 18
  • DOI: 10.1002/sim.7298

Some Approximate Evidence Factors in Observational Studies
journal, March 2011


Sensitivity Analysis Without Assumptions
journal, January 2016


Evidence factors in observational studies
journal, April 2010


On closed testing procedures with special reference to ordered analysis of variance
journal, January 1976


Estimating Causal Effects in Observational Studies Using Electronic Health Data: Challenges and (some) Solutions
journal, December 2013

  • Stuart, Elizabeth A.; DuGof, Eva; Abrams, Michael
  • eGEMs (Generating Evidence & Methods to improve patient outcomes), Vol. 1, Issue 3
  • DOI: 10.13063/2327-9214.1038

On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9
journal, November 1990

  • Splawa-Neyman, Jerzy; Dabrowska, D. M.; Speed, T. P.
  • Statistical Science, Vol. 5, Issue 4
  • DOI: 10.1214/ss/1177012031

Amplification of Sensitivity Analysis in Matched Observational Studies
journal, December 2009

  • Rosenbaum, Paul R.; Silber, Jeffrey H.
  • Journal of the American Statistical Association, Vol. 104, Issue 488
  • DOI: 10.1198/jasa.2009.tm08470

Choosing an optimal method to combine P -values
journal, May 2009

  • Won, Sungho; Morris, Nathan; Lu, Qing
  • Statistics in Medicine, Vol. 28, Issue 11
  • DOI: 10.1002/sim.3569

Selection of reference groups in the Life Span Study of atomic bomb survivors
journal, December 2017

  • French, Benjamin; Cologne, John; Sakata, Ritsu
  • European Journal of Epidemiology, Vol. 32, Issue 12
  • DOI: 10.1007/s10654-017-0337-9

Design sensitivity in observational studies
journal, March 2004


Bahadur Efficiency of Sensitivity Analyses in Observational Studies
journal, January 2015


Constructed Second Control Groups and Attenuation of Unmeasured Biases
journal, July 2016

  • Pimentel, Samuel D.; Small, Dylan S.; Rosenbaum, Paul R.
  • Journal of the American Statistical Association, Vol. 111, Issue 515
  • DOI: 10.1080/01621459.2015.1076342

Contrasting Evidence Within and Between Institutions That Provide Treatment in an Observational Study of Alternate Forms of Anesthesia
journal, May 2012

  • Zubizarreta, José R.; Neuman, Mark; Silber, Jeffrey H.
  • Journal of the American Statistical Association, Vol. 107, Issue 499
  • DOI: 10.1080/01621459.2012.682533

Asymptotic Optimality of Fisher's Method of Combining Independent Tests
journal, December 1971


How Much Is Minnesota Like Wisconsin? Assumptions and Counterfactuals in Causal Inference with Observational Data
journal, January 2013

  • Keele, Luke; Minozzi, William
  • Political Analysis, Vol. 21, Issue 2
  • DOI: 10.1093/pan/mps041

Rates of Convergence of Estimates and Test Statistics
journal, April 1967