Design and analysis of confirmation experiments

Stevens, Nathaniel T.; Anderson-Cook, Christine M.

doi:10.1080/00224065.2019.1571344

Title: Design and analysis of confirmation experiments

Journal Article · Wed Apr 03 00:00:00 EDT 2019 · Journal of Quality Technology

DOI:https://doi.org/10.1080/00224065.2019.1571344· OSTI ID:1524387

Stevens, Nathaniel T. ^[1];

^[2]

Univ. of California, San Francisco, CA (United States)
Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

The statistical literature and practitioners have long advocated the use of confirmation experiments as the final stage of a sequence of designed experiments to verify that the optimal operating conditions identified as part of a response surface methodology strategy are attainable and able to achieve the value of the response desired. Yet, until recently there has been a gap between this recommendation and details about how to perform an analysis to quantitatively assess whether the confirmation runs are adequate. Likewise, there has been little in the way of specific recommendations for the number and nature of the confirmation runs that should be performed. In this article, we propose analysis methods to assess agreement between the mean response from previous experiments and the confirmation experiment, and suggest a strategy for the design of confirmation experiments that more fully explores the region around the optimum.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Sponsoring Organization:: USDOE

Grant/Contract Number:: 89233218CNA000001

OSTI ID:: 1524387

Report Number(s):: LA-UR-17-31289

Journal Information:: Journal of Quality Technology, Vol. 51, Issue 2; ISSN 0022-4065

Publisher:: American Society for Quality (ASQ)Copyright Statement

Country of Publication:: United States

Language:: English

Citation Metrics:

Cited by: 10 works

Citation information provided by
Web of Science

References (49)

Optimization of the Adsorption Conditions for the Decolorization and COD Reduction of Methylene Blue Aqueous Solution using Low-Cost Adsorbent Low, Ling Wei; Teng, Tjoon Tow; Alkarkhi, Abbas F. M. Water, Air, & Soil Pollution, Vol. 214, Issue 1-4 https://doi.org/10.1007/s11270-010-0414-0	journal	April 2010
Ultrasonic-Assisted Enzymolysis to Improve the Antioxidant Activities of Peanut (Arachin conarachin L.) Antioxidant Hydrolysate Yu, Lina; Sun, Jie; Liu, Shaofang International Journal of Molecular Sciences, Vol. 13, Issue 7 https://doi.org/10.3390/ijms13079051	journal	July 2012
Design augmentation for response optimization and model estimation Nachtsheim, Christopher J.; Jones, Bradley Quality Engineering, Vol. 30, Issue 1 https://doi.org/10.1080/08982112.2017.1382298	journal	December 2017
Enhanced Production of a Novel Cyclic Hexapeptide Antibiotic (NW-G01) by Streptomyces alboflavus 313 Using Response Surface Methodology Guo, Zhengyan; Shen, Ling; Ji, Zhiqin International Journal of Molecular Sciences, Vol. 13, Issue 4 https://doi.org/10.3390/ijms13045230	journal	April 2012
Production of specific-structured lipids by enzymatic interesterification: Elucidation of Acyl migration by response surface design Xu, X.; Skands, A. R. H.; Høy, C. -E. Journal of the American Oil Chemists' Society, Vol. 75, Issue 12 https://doi.org/10.1007/s11746-998-0309-z	journal	December 1998
Design and Analysis of Mixture Experiments Snee, Ronald D. Journal of Quality Technology, Vol. 3, Issue 4 https://doi.org/10.1080/00224065.1971.11980489	journal	October 1971
Incorporating response variability and estimation uncertainty into Pareto front optimization Chapman, Jessica L.; Lu, Lu; Anderson-Cook, Christine M. Computers & Industrial Engineering, Vol. 76 https://doi.org/10.1016/j.cie.2014.07.028	journal	October 2014
The Design of Optimum Multifactorial Experiments Plackett, R. L.; Burman, J. P. Biometrika, Vol. 33, Issue 4 https://doi.org/10.1093/biomet/33.4.305	journal	January 1946
Modelling and analysis of hybrid electrochemical turning-magnetic abrasive finishing of 6061 Al/Al2O3 composite El-Taweel, T. A. The International Journal of Advanced Manufacturing Technology, Vol. 37, Issue 7-8 https://doi.org/10.1007/s00170-007-1019-7	journal	April 2007
Extraction optimization of pepsin-soluble collagen from eggshell membrane by response surface methodology (RSM) Mohammadi, Reza; Mohammadifar, Mohammad Amin; Mortazavian, Amir Mohammad Food Chemistry, Vol. 190 https://doi.org/10.1016/j.foodchem.2015.05.073	journal	January 2016
Enhanced exoinulinase production from Kluyveromyces marxianus YS-1 using response surface methodology Singh, Ram Sarup; Lotey, Shveta Brazilian Archives of Biology and Technology, Vol. 53, Issue 5 https://doi.org/10.1590/S1516-89132010000500002	journal	October 2010
Response Surface Optimization of the Critical Medium Components for Pullulan Production by Aureobasidium pullulans FB-1 Singh, Ram Sarup; Singh, Harpreet; Saini, Gaganpreet Kaur Applied Biochemistry and Biotechnology, Vol. 152, Issue 1 https://doi.org/10.1007/s12010-008-8180-9	journal	April 2008
Improved Production of Biosurfactant with Newly Isolated Pseudomonas aeruginosa S2 Chen, Shan-Yu; Lu, Wei-Bin; Wei, Yu-Hong Biotechnology Progress, Vol. 23, Issue 3 https://doi.org/10.1021/bp0700152	journal	January 2007
From spc to doe: a case Study at Meco, inc. Hale-Bennett, Cynthia; Lin, Dennis K. J. Quality Engineering, Vol. 9, Issue 3 https://doi.org/10.1080/08982119708919068	journal	January 1997
The Coordinate-Exchange Algorithm for Constructing Exact Optimal Experimental Designs Meyer, Ruth K.; Nachtsheim, Christopher J. Technometrics, Vol. 37, Issue 1 https://doi.org/10.1080/00401706.1995.10485889	journal	February 1995
Optimization of Alkali, Switchgrass, and Extruder Parameters for Maximum Sugar Recovery Karunanithy, C.; Muthukumarappan, K. Chemical Engineering & Technology, Vol. 34, Issue 9 https://doi.org/10.1002/ceat.201000378	journal	August 2011
Comparing heteroscedastic measurement systems with the probability of agreement Stevens, Nathaniel T.; Steiner, Stefan H.; MacKay, R. Jock Statistical Methods in Medical Research, Vol. 27, Issue 11 https://doi.org/10.1177/0962280217702540	journal	May 2017
Process Optimization Using Response Surface Design and Pilot Plant Production of Dietary Diacylglycerols by Lipase-Catalyzed Glycerolysis Kristensen, Janni Brogaard; Xu, Xuebing; Mu, Huiling Journal of Agricultural and Food Chemistry, Vol. 53, Issue 18 https://doi.org/10.1021/jf0507745	journal	September 2005
Confirmation Runs in Design of Experiments Jensen, Willis A. Journal of Quality Technology, Vol. 48, Issue 2 https://doi.org/10.1080/00224065.2016.11918157	journal	April 2016
Quantifying similarity in reliability surfaces using the probability of agreement Stevens, Nathaniel T.; Anderson-Cook, Christine M. Quality Engineering, Vol. 29, Issue 3 https://doi.org/10.1080/08982112.2017.1312004	journal	March 2017
Bayesian probability of agreement for comparing the similarity of response surfaces Stevens, Nathaniel T.; Rigdon, Steven E.; Anderson-Cook, Christine M. Journal of Quality Technology https://doi.org/10.1080/00224065.2019.1569961	journal	April 2019
The 2 ^k — ^p Fractional Factorial Designs Box, G. E. P.; Hunter, J. S. Technometrics, Vol. 3, Issue 3 https://doi.org/10.1080/00401706.1961.10489951	journal	August 1961
GLMs for the Analysis of Robust Designs with Dynamic Characteristics Lesperance, M. L.; Park, Sung-Min Journal of Quality Technology, Vol. 35, Issue 3 https://doi.org/10.1080/00224065.2003.11980219	journal	July 2003
Lipase-catalyzed production of medium-chain triacylglycerols from palm kernel oil distillate: Optimization using response surface methodology Low, Chee Tein; Mohamad, Rosfarizan; Tan, Chin Ping European Journal of Lipid Science and Technology, Vol. 109, Issue 2 https://doi.org/10.1002/ejlt.200600201	journal	February 2007
Distributing many points on a sphere Saff, E. B.; Kuijlaars, A. B. J. The Mathematical Intelligencer, Vol. 19, Issue 1 https://doi.org/10.1007/BF03024331	journal	December 1997
Response surface experiments: A meta-analysis Ockuly, Rebecca A.; Weese, Maria L.; Smucker, Byran J. Chemometrics and Intelligent Laboratory Systems, Vol. 164 https://doi.org/10.1016/j.chemolab.2017.03.009	journal	May 2017
Improvement in Preform Weights from a 48-Cavity PET Injection Molder Dodd, Kevin; Ney, Kendall; Neaves, Salvador Quality Engineering, Vol. 14, Issue 3 https://doi.org/10.1081/QEN-120001886	journal	March 2002
Optimal Experimental Design for Efficient Optical Manipulation of Magnetic Beads Seeded in a Microwell Array Decrop, D.; Lammertyn, J.; Goos, P. Journal of Quality Technology, Vol. 49, Issue 4 https://doi.org/10.1080/00224065.2017.11918005	journal	October 2017
Optimum Designs in Regression Problems Kiefer, J.; Wolfowitz, J. The Annals of Mathematical Statistics, Vol. 30, Issue 2 https://doi.org/10.1214/aoms/1177706252	journal	June 1959
On the Experimental Attainment of Optimum Conditions Box, G. E. P.; Wilson, K. B. Journal of the Royal Statistical Society: Series B (Methodological), Vol. 13, Issue 1 https://doi.org/10.1111/j.2517-6161.1951.tb00067.x	journal	January 1951
Using Genetic Algorithms to Design Experiments: A Review: Using Genetic Algorithms to Design Experiments: A Review Lin, C. Devon; Anderson-Cook, Christine M.; Hamada, Michael S. Quality and Reliability Engineering International, Vol. 31, Issue 2 https://doi.org/10.1002/qre.1591	journal	January 2014
Comparing the Reliability of Related Populations With the Probability of Agreement Stevens, Nathaniel T.; Anderson-Cook, Christine M. Technometrics, Vol. 59, Issue 3 https://doi.org/10.1080/00401706.2016.1214180	journal	July 2016
Assessing agreement between two measurement systems: An alternative to the limits of agreement approach Stevens, Nathaniel T.; Steiner, Stefan H.; MacKay, R. Jock Statistical Methods in Medical Research, Vol. 26, Issue 6 https://doi.org/10.1177/0962280215601133	journal	September 2015
Some New Three Level Designs for the Study of Quantitative Variables Box, G. E. P.; Behnken, D. W. Technometrics, Vol. 2, Issue 4 https://doi.org/10.1080/00401706.1960.10489912	journal	November 1960
Achieving Continuous Improvement with the dual Approach: a Demonstration of the Roman Catapult Luner, Jeffery J. Quality Engineering, Vol. 6, Issue 4 https://doi.org/10.1080/08982119408918759	journal	January 1994
Optimization technique for sewage sludge conditioning with polymer and skeleton builders Benítez, Jaime; Rodríguez, Abraham; Suárez, Alejandro Water Research, Vol. 28, Issue 10 https://doi.org/10.1016/0043-1354(94)90016-7	journal	October 1994
The 2 ^k — ^p Fractional Factorial Designs Part II. Box, G. E. P.; Hunter, J. S. Technometrics, Vol. 3, Issue 4 https://doi.org/10.1080/00401706.1961.10489967	journal	November 1961
A Class of Three-Level Designs for Definitive Screening in the Presence of Second-Order Effects Jones, Bradley; Nachtsheim, Christopher J. Journal of Quality Technology, Vol. 43, Issue 1 https://doi.org/10.1080/00224065.2011.11917841	journal	January 2011
Combining Taguchi and Response Surface Philosophies: A Dual Response Approach Vining, G. Geoffrey; Myers, Raymond H. Journal of Quality Technology, Vol. 22, Issue 1 https://doi.org/10.1080/00224065.1990.11979204	journal	January 1990
An Efficient Split-Plot Approach for Modeling Nonlinear Aerodynamic Effects English, Teresa G.; Simpson, James R.; Landman, Drew Quality Engineering, Vol. 24, Issue 4 https://doi.org/10.1080/08982112.2012.710164	journal	October 2012
Screening and optimization of the factors of a detergent admixture preparation Baati, Rochdi; Kamoun, Amel; Chaabouni, Moncef Chemometrics and Intelligent Laboratory Systems, Vol. 80, Issue 2 https://doi.org/10.1016/j.chemolab.2005.06.008	journal	February 2006
Uniform distribution of points on a hyper-sphere with applications to vector bit-plane encoding Lovisolo, L.; da Silva, E. A. B. IEE Proceedings - Vision, Image, and Signal Processing, Vol. 148, Issue 3 https://doi.org/10.1049/ip-vis:20010361	journal	January 2001
Winning back the Customer'S Confidence: a case Study on the Application of Design of Experiments to an Injection-Molding Process Acharya, U. H.; Mahesh, C. Quality Engineering, Vol. 11, Issue 3 https://doi.org/10.1080/08982119908919251	journal	March 1999
Some New Three Level Designs for the Study of Quantitative Variables Box, G. E. P.; Behnken, D. W. Technometrics, Vol. 2, Issue 4 https://doi.org/10.2307/1266454	journal	November 1960
The Coordinate-Exchange Algorithm for Constructing Exact Optimal Experimental Designs Meyer, Ruth K.; Nachtsheim, Christopher J. Technometrics, Vol. 37, Issue 1 https://doi.org/10.2307/1269153	journal	February 1995
Optimal Design of Experiments Tobias, Randall D. Journal of Quality Technology, Vol. 27, Issue 1 https://doi.org/10.1080/00224065.1995.11979568	journal	January 1995
Comparing the Reliability of Related Populations With the Probability of Agreement Stevens, Nathaniel T.; Anderson-Cook, Christine M. Figshare, 1.08 MB https://doi.org/10.6084/m9.figshare.3502052.v2	dataset	April 2017
Production of specific-structured lipids by enzymatic interesterification: Elucidation of acyl migration by response surface design Xu, X.; Skands, A. R. H.; Høy, C. -E. Journal of the American Oil Chemists' Society, Vol. 75, Issue 9 https://doi.org/10.1007/s11746-998-0132-6	journal	September 1998
Comparing the Reliability of Related Populations With the Probability of Agreement [Supplemental Data] Stevens, Nathaniel; Anderson-Cook, Christine figshare-Supplementary information for journal article at DOI: 10.1080/00401706.2016.1214180, 1 PDF file (1.08 MB) https://doi.org/10.6084/m9.figshare.3502052	dataset	April 2017

Cited By (1)

Bayesian probability of agreement for comparing the similarity of response surfaces Stevens, Nathaniel T.; Rigdon, Steven E.; Anderson-Cook, Christine M. Journal of Quality Technology https://doi.org/10.1080/00224065.2019.1569961	journal	April 2019

Similar Records

Methods for Detector Placement and Analysis of Criticality Accident Alarm Systems

Conference · Sun Jun 24 00:00:00 EDT 2012 · Transactions of the American Nuclear Society · OSTI ID:1524387

Peplow, Douglas E.; Wetzel, Larry

Development of a technical approach for assessing environmental release and migration characteristics of Hanford Grout

Technical Report · Sun Sep 01 00:00:00 EDT 1985 · OSTI ID:1524387

Serne, R. J.; Treat, R. L.; Lokken, R. O.

2015 Advanced Site Investigation and Monitoring Report Riverton, Wyoming, Processing Site September 2016

Technical Report · Thu Sep 01 00:00:00 EDT 2016 · OSTI ID:1524387

Frazier, William; Campbell, Sam

Related Subjects

97 MATHEMATICS AND COMPUTING
commensurate probability
confirmation runs
equiradial design
probability of agreement
response surface methodology
sequential experimentation
validation

Design and analysis of confirmation experiments Stevens, Nathaniel T.; Anderson-Cook, Christine M. https://doi.org/10.6084/m9.figshare.7946624 The current record is supplemented by this collection	collection	January 2019
Design and analysis of confirmation experiments Stevens, Nathaniel T.; Anderson-Cook, Christine M. https://doi.org/10.6084/m9.figshare.7946624.v1 The current record is supplemented by this collection	collection	January 2019

Title: Design and analysis of confirmation experiments

Citation Formats

References (49)

Cited By (1)

Linked Research (2)

Similar Records

Related Subjects