Best Bang for the Buck: Part 1 – The Size of Experiments Relative to Design Performance

Anderson-Cook, Christine Michaela; Lu, Lu

Title: Best Bang for the Buck: Part 1 – The Size of Experiments Relative to Design Performance

Full Record
Other Related Research

Abstract

There are many choices to make, when designing an experiment for a study, such as: what design factors to consider, which levels of the factors to use and which model to focus on. One aspect of design, however, is often left unquestioned: the size of the experiment. When learning about design of experiments, problems are often posed as "select a design for a particular objective with N runs." It’s tempting to consider the design size as a given constraint in the design-selection process. If you think of learning through designed experiments as a sequential process, however, strategically planning for the use of resources at different stages of data collection can be beneficial: Saving experimental runs for later is advantageous if you can efficiently learn with less in the early stages. Alternatively, if you’re too frugal in the early stages, you might not learn enough to proceed confidently with the next stages. Therefore, choosing the right-sized experiment is important—not too large or too small, but with a thoughtful balance to maximize the knowledge gained given the available resources. It can be a great advantage to think about the design size as flexible and include it as an aspect for comparisons. Sometimesmore »« less

Authors:

Anderson-Cook, Christine Michaela ^[1]; Lu, Lu ^[1]

Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Publication Date:: Sat Oct 01 00:00:00 EDT 2016

Research Org.:: Los Alamos National Lab. (LANL), Los Alamos, NM (United States)

Sponsoring Org.:: USDOE

OSTI Identifier:: 1337114

Report Number(s):: LA-UR-16-26820
Journal ID: ISSN 0033-524X

Grant/Contract Number:: AC52-06NA25396

Resource Type:: Accepted Manuscript

Journal Name:: Quality Progress

Additional Journal Information:: Journal Volume: 49; Journal Issue: 10; Journal ID: ISSN 0033-524X

Publisher:: American Society for Quality Control

Country of Publication:: United States

Language:: English

Subject:: 99 GENERAL AND MISCELLANEOUS; 97 MATHEMATICS AND COMPUTING; Mathematics

Citation Formats


                    Anderson-Cook, Christine Michaela, and Lu, Lu. Best Bang for the Buck: Part 1 – The Size of Experiments Relative to Design Performance.  United States: N. p., 2016. 
Web.

Copy to clipboard


                    Anderson-Cook, Christine Michaela, & Lu, Lu. Best Bang for the Buck: Part 1 – The Size of Experiments Relative to Design Performance.  United States.

Copy to clipboard


                    Anderson-Cook, Christine Michaela, and Lu, Lu. Sat .  
"Best Bang for the Buck: Part 1 – The Size of Experiments Relative to Design Performance".  United States.  https://www.osti.gov/servlets/purl/1337114.

Copy to clipboard


                    
@article{osti_1337114,

  title        = {Best Bang for the Buck: Part 1 – The Size of Experiments Relative to Design Performance},

  author       = {Anderson-Cook, Christine Michaela and Lu, Lu},

  abstractNote = {There are many choices to make, when designing an experiment for a study, such as: what design factors to consider, which levels of the factors to use and which model to focus on. One aspect of design, however, is often left unquestioned: the size of the experiment. When learning about design of experiments, problems are often posed as "select a design for a particular objective with N runs." It’s tempting to consider the design size as a given constraint in the design-selection process. If you think of learning through designed experiments as a sequential process, however, strategically planning for the use of resources at different stages of data collection can be beneficial: Saving experimental runs for later is advantageous if you can efficiently learn with less in the early stages. Alternatively, if you’re too frugal in the early stages, you might not learn enough to proceed confidently with the next stages. Therefore, choosing the right-sized experiment is important—not too large or too small, but with a thoughtful balance to maximize the knowledge gained given the available resources. It can be a great advantage to think about the design size as flexible and include it as an aspect for comparisons. Sometimes you’re asked to provide a small design that is too ambitious for the goals of the study. Finally, if you can show quantitatively how the suggested design size might be inadequate or lead to problems during analysis—and also offer a formal comparison to some alternatives of different (likely larger) sizes—you may have a better chance to ask for additional resources to deliver statistically sound and satisfying results},

  doi          = {},

  journal      = {Quality Progress},

  number       = 10,

  volume       = 49,

  place        = {United States},

  year         = {Sat Oct 01 00:00:00 EDT 2016},

  month        = {Sat Oct 01 00:00:00 EDT 2016}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

The DOI is not currently available

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Best Bang for the Buck: Part 2 – Choosing Between Different Sized Designs

Journal Article Anderson-Cook, Christine Michaela ; Lu, Lu - Quality Progress

Last month, we considered four potential designs of varying sizes and a variety of their attributes for an engineering problem involving a screening design with seven factors. The engineer’s original request was to find a design with 14 runs that could estimate all of the main effects, as well as have some potential for finding important two-factor interactions and possibly curvature in the underlying surface. This assessment was that the proposed size of design was potentially a bit too ambitious for the experiment’s goals, so we explored three other competing designs that had slightly larger sizes but were able tomore » improve several aspects of the design.« less
Full Text Available
"Test" is a Four Letter Word

Journal Article Pope, G M - Better Software Magazine, N/A, N/A, October 1, 2005, N/A

For a number of years I had the pleasure of teaching Testing Seminars all over the world and meeting and learning from others in our field. Over a twelve year period, I always asked the following questions to Software Developers, Test Engineers, and Managers who took my two or three day seminar on Software Testing: 'When was the first time you heard the word test'? 'Where were you when you first heard the word test'? 'Who said the word test'? 'How did the word test make you feel'? Most of the thousands of responses were similar to 'It was mymore »« less
Full Text Available
Using Ancillary Information to Reduce Sample Size in Discovery Sampling and the Effects of Measurement Error

Technical Report Axelrod, M

Discovery sampling is a tool used in a discovery auditing. The purpose of such an audit is to provide evidence that some (usually large) inventory of items complies with a defined set of criteria by inspecting (or measuring) a representative sample drawn from the inventory. If any of the items in the sample fail compliance (defective items), then the audit has discovered an impropriety, which often triggers some action. However finding defective items in a sample is an unusual event--auditors expect the inventory to be in compliance because they come to the audit with an ''innocent until proven guilty attitude''.more »« less
https://doi.org/10.2172/877925

Full Text Available
Hampshire College Center for Science Education. Final Report on Activities Supported by the Department of Energy Grant No. DE-FG02-06ER64256

Technical Report Stillings, Neil ; Wenk, Laura

Hampshire College's Center for Science Education (Center) focuses on teacher professional development, curriculum development, and student enrichment programs. The Center also maintains research programs on teacher change, student learning and instructional effectiveness. The Center's work promotes learning that persists over time and transfers to new situations in and out of school. The projects develop the implications of the increasing agreement among teachers and researchers that effective learning involves active concept mastery and consistent practice with inquiry and critical thinking. The Center's objective is to help strengthen the pipeline of U.S. students pursuing postsecondary study in STEM fields. The Center achievesmore »« less
https://doi.org/10.2172/969756

Full Text Available
Essay: Bob Siemann-SLC Days at SLAC

Journal Article Raubenheimer, Tor O ; /SLAC - Physical Review Special Topics - Accelerators and Beams

Bob Siemann was a great experimentalist and an excellent teacher.We will greatly miss him. Bob came to SLAC in early 1991 to work on the Stanford Linear Collider (SLC). The SLC was a challenging accelerator which began operating in the late 1980's but still had numerous obstacles to be overcome years into operation. One of the compounding difficulties was making reproducible measurements, since the stability of the collider was poor and the diagnostics were insufficient. Bob dove into this challenge and helped design experiments and diagnostics that provided further clarity. I first got to know Bob while I was stillmore »« less
Full Text Available

Similar Records