Comparing the reliability of related populations with the probability of agreement

Stevens, Nathaniel T.; Anderson-Cook, Christine M.

doi:10.1080/00401706.2016.1214180

Title: Comparing the reliability of related populations with the probability of agreement

Abstract

Combining information from different populations to improve precision, simplify future predictions, or improve underlying understanding of relationships can be advantageous when considering the reliability of several related sets of systems. Using the probability of agreement to help quantify the similarities of populations can help to give a realistic assessment of whether the systems have reliability that are sufficiently similar for practical purposes to be treated as a homogeneous population. In addition, the new method is described and illustrated with an example involving two generations of a complex system where the reliability is modeled using either a logistic or probit regression model. Note that supplementary materials including code, datasets, and added discussion are available online.

Authors:

Stevens, Nathaniel T. ^[1]; Anderson-Cook, Christine M. ^[2]

Univ. of San Francisco, San Francisco, CA (United States)
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Publication Date:: Tue Jul 26 00:00:00 EDT 2016

Research Org.:: Los Alamos National Laboratory (LANL), Los Alamos, NM (United States)

Sponsoring Org.:: USDOD; USDOE

OSTI Identifier:: 1304827

Report Number(s):: LA-UR-15-27900
Journal ID: ISSN 0040-1706

Grant/Contract Number:: AC52-06NA25396

Resource Type:: Accepted Manuscript

Journal Name:: Technometrics

Additional Journal Information:: Journal Name: Technometrics; Journal ID: ISSN 0040-1706

Publisher:: Taylor & Francis

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING; generalized linear models; reliability; equivalence testing; homogeneity of population characteristics; probability of agreement

Citation Formats


                    Stevens, Nathaniel T., and Anderson-Cook, Christine M. Comparing the reliability of related populations with the probability of agreement.  United States: N. p., 2016. 
Web.  doi:10.1080/00401706.2016.1214180.

Copy to clipboard


                    Stevens, Nathaniel T., & Anderson-Cook, Christine M. Comparing the reliability of related populations with the probability of agreement.  United States.  https://doi.org/10.1080/00401706.2016.1214180

Copy to clipboard


                    Stevens, Nathaniel T., and Anderson-Cook, Christine M. Tue .  
"Comparing the reliability of related populations with the probability of agreement".  United States.  https://doi.org/10.1080/00401706.2016.1214180.  https://www.osti.gov/servlets/purl/1304827.

Copy to clipboard


                    
@article{osti_1304827,

  title        = {Comparing the reliability of related populations with the probability of agreement},

  author       = {Stevens, Nathaniel T. and Anderson-Cook, Christine M.},

  abstractNote = {Combining information from different populations to improve precision, simplify future predictions, or improve underlying understanding of relationships can be advantageous when considering the reliability of several related sets of systems. Using the probability of agreement to help quantify the similarities of populations can help to give a realistic assessment of whether the systems have reliability that are sufficiently similar for practical purposes to be treated as a homogeneous population. In addition, the new method is described and illustrated with an example involving two generations of a complex system where the reliability is modeled using either a logistic or probit regression model. Note that supplementary materials including code, datasets, and added discussion are available online.},

  doi          = {10.1080/00401706.2016.1214180},

  journal      = {Technometrics},

  number       = ,

  volume       = ,

  place        = {United States},

  year         = {Tue Jul 26 00:00:00 EDT 2016},

  month        = {Tue Jul 26 00:00:00 EDT 2016}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1080/00401706.2016.1214180

Other availability

Search WorldCat to find libraries that may hold this journal

Citation Metrics:

Cited by: 10 works

Citation information provided by
Web of Science

Figures / Tables:

Table 1:: Summary of bias and ratio distributions by age. Entries correspond to mean (95% CI).

All figures and tables (7 total)

Save / Share:

Export Metadata

Save to My Library

Works referenced in this record:

Improving Reliability Understanding Through Estimation and Prediction with Usage Information
journal, June 2015

Lu, Lu; Anderson-Cook, Christine M.
Quality Engineering, Vol. 27, Issue 3
DOI: 10.1080/08982112.2014.990033

Statistical Model Selection for Better Prediction and Discovering Science Mechanisms That Affect Reliability
journal, August 2015

Anderson-Cook, Christine; Morzinski, Jerome; Blecker, Kenneth
Systems, Vol. 3, Issue 3
DOI: 10.3390/systems3030109

Generic Drugs: Therapeutic Equivalence
journal, January 1996

Meredith, Peter A.
Drug Safety, Vol. 15, Issue 4
DOI: 10.2165/00002018-199615040-00001

Assessing agreement between two measurement systems: An alternative to the limits of agreement approach
journal, September 2015

Stevens, Nathaniel T.; Steiner, Stefan H.; MacKay, R. Jock
Statistical Methods in Medical Research, Vol. 26, Issue 6
DOI: 10.1177/0962280215601133

Equivalence and Noninferiority Tests for Quality, Manufacturing and Test Engineers
journal, October 2014

Szarka, John L.
Journal of Quality Technology, Vol. 46, Issue 4
DOI: 10.1080/00224065.2014.11917979

Logistic Regression
book, January 2000

Pampel, Fred C.
DOI: 10.4135/9781412984805

Prediction of Reliability of an Arbitrary System from a Finite Population
journal, December 2010

Lu, Lu; Anderson-Cook, Christine M.
Quality Engineering, Vol. 23, Issue 1
DOI: 10.1080/08982112.2010.495259

Logistic Regression
book, October 2013

Weisburd, David; Britt, Chester
Statistics in Criminal Justice
DOI: 10.1007/978-1-4614-9170-5_18

Generalized Linear Models
journal, December 2000

McCulloch, Charles E.
Journal of the American Statistical Association, Vol. 95, Issue 452
DOI: 10.1080/01621459.2000.10474340

Generic Drugs
reference-book, January 2009

Rovner, Julie
Health Care Policy and Politics A to Z
DOI: 10.4135/9781452240121.n137

Logistic Regression
book, October 2018

George, Darren; Mallery, Paul
IBM SPSS Statistics 25 Step by Step
DOI: 10.4324/9781351033909-32

Generic Drugs
book, January 2004

Stahl, Michael J.
Encyclopedia of Health Care Management
DOI: 10.4135/9781412950602.n328

Logistic regression
null, January 2011

Claire Hoede, Erick Denamur
Figshare
DOI: 10.6084/m9.figshare.10769

Logistic Regression
book, January 2009

Acton, Ciaran; Miller, Robert; Maltby, John
SPSS for Social Scientists
DOI: 10.1007/978-1-137-01390-3_12

Works referencing / citing this record:

Design and analysis of confirmation experiments
journal, April 2019

Stevens, Nathaniel T.; Anderson-Cook, Christine M.
Journal of Quality Technology, Vol. 51, Issue 2
DOI: 10.1080/00224065.2019.1571344

How to Host An Effective Data Competition: Statistical Advice for Competition Design and Analysis
journal, February 2019

Anderson‐Cook, Christine M.; Myers, Kary L.; Lu, Lu
Statistical Analysis and Data Mining: The ASA Data Science Journal, Vol. 12, Issue 4
DOI: 10.1002/sam.11404

Bayesian probability of agreement for comparing the similarity of response surfaces
journal, April 2019

Stevens, Nathaniel T.; Rigdon, Steven E.; Anderson-Cook, Christine M.
Journal of Quality Technology
DOI: 10.1080/00224065.2019.1569961

Bayesian probability of predictive agreement for comparing the outcome of two separate regressions: Bayesian probability of predictive agreement for comparing the outcome of two separate regressions
journal, April 2018

Stevens, Nathaniel T.; Rigdon, Steven E.; Anderson-Cook, Christine M.
Quality and Reliability Engineering International, Vol. 34, Issue 6
DOI: 10.1002/qre.2284

Figures / Tables found in this record:

Figures/Tables have been extracted from DOE-funded journal article accepted manuscripts.

Similar Records in DOE PAGES and OSTI.GOV collections:

Quantifying similarity in reliability surfaces using the probability of agreement

Journal Article Stevens, Nathaniel T. ; Anderson-Cook, Christine Michaela - Quality Engineering

When separate populations exhibit similar reliability as a function of multiple explanatory variables, combining them into a single population is tempting. This can simplify future predictions and reduce uncertainty associated with estimation. However, combining these populations may introduce bias if the underlying relationships are in fact different. The probability of agreement formally and intuitively quantifies the similarity of estimated reliability surfaces across a two-factor input space. An example from the reliability literature demonstrates the utility of the approach when deciding whether to combine two populations or to keep them as distinct. As a result, new graphical summaries provide strategies formore » visualizing the results.« less
Cited by 8
https://doi.org/10.1080/08982112.2017.1312004

Full Text Available
Bayesian probability of agreement for comparing the similarity of response surfaces

Journal Article Stevens, Nathaniel T. ; Rigdon, Steven E. ; Anderson-Cook, Christine M. - Journal of Quality Technology

For a variety of linear and generalized linear models, there may be interest in evaluating whether the expected response values in two different populations are sufficiently similar to be considered practically equivalent. The probability of agreement has been introduced as a strategy for quantifying the similarity between two groups. We propose a Bayesian version of the probability of agreement to quantify the similarity between linear and generalized linear response surfaces. The proposed methodology is based on Markov chain Monte Carlo estimation and it allows for an intuitive interpretation based directly on population parameters. As demonstrated, the methodology can be flexiblymore »« less
Cited by 7
https://doi.org/10.1080/00224065.2019.1569961

Full Text Available
Policy, Permit, Perform: Using City Benchmarking Data and Building Construction Permit History to Identify Energy Performance Improvements

Technical Report Stevens, Casey ; Tyler, Matt ; Lee, Allen ; ...

The City of Portland (City) and Multnomah County 2015 Climate Action Plan targets a 40 percent reduction in carbon emissions below 1990 levels by 2030. To reach this goal, the plan includes a key objective to reduce the total energy use of existing buildings by 25 percent. Buildings are responsible for one-half of carbon emissions in Portland, Oregon, U.S., and improving their performance is critical to achieving the City’s climate goals. As part of its climate action planning, the City passed an ordinance on April 22, 2015, requiring commercial buildings 20,000 ft2 and larger to benchmark and disclose annual energymore »« less
https://doi.org/10.2172/1477793

Full Text Available
The weighted priors approach for combining expert opinions in logistic regression experiments

Journal Article Quinlan, Kevin R. ; Anderson-Cook, Christine M. ; Myers, Kary L. - Quality Engineering

When modeling the reliability of a system or component, it is not uncommon for more than one expert to provide very different prior estimates of the expected reliability as a function of an explanatory variable such as age or temperature. Our goal in this paper is to incorporate all information from the experts when choosing a design about which units to test. Bayesian design of experiments has been shown to be very successful for generalized linear models, including logistic regression models. We use this approach to develop methodology for the case where there are several potentially non-overlapping priors under consideration.more »« less
Cited by 3
https://doi.org/10.1080/08982112.2017.1319956

Full Text Available
Predicting departure from nucleate boiling with advanced data- and physics-driven approaches

Technical Report Zhao, Xingang

The subcooled and low-quality saturated flow boiling critical heat flux (CHF) corresponding to the departure from nucleate boiling (DNB) is one of the major limiting factors in the design and operation of pressurized water reactors (PWRs). Due to lack of agreement on its physical mechanisms, development of an accurate DNB-type CHF model has been elusive. The various steady-state (SS) predictive tools that have been proposed can be grouped into two categories: (1) data-driven methods—namely best-fit empirical correlations and look-up tables that result in relatively good agreement with specific experimental datasets but often fail to extend beyond their ranges of validity,more »« less
https://doi.org/10.2172/1651417

Full Text Available

Other research related to this record: