skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Modeling and simulation technology readiness levels.

Abstract

This report summarizes the results of an effort to establish a framework for assigning and communicating technology readiness levels (TRLs) for the modeling and simulation (ModSim) capabilities at Sandia National Laboratories. This effort was undertaken as a special assignment for the Weapon Simulation and Computing (WSC) program office led by Art Hale, and lasted from January to September 2006. This report summarizes the results, conclusions, and recommendations, and is intended to help guide the program office in their decisions about the future direction of this work. The work was broken out into several distinct phases, starting with establishing the scope and definition of the assignment. These are characterized in a set of key assertions provided in the body of this report. Fundamentally, the assignment involved establishing an intellectual framework for TRL assignments to Sandia's modeling and simulation capabilities, including the development and testing of a process to conduct the assignments. To that end, we proposed a methodology for both assigning and understanding the TRLs, and outlined some of the restrictions that need to be placed on this process and the expected use of the result. One of the first assumptions we overturned was the notion of a ''static'' TRL--rather wemore » concluded that problem context was essential in any TRL assignment, and that leads to dynamic results (i.e., a ModSim tool's readiness level depends on how it is used, and by whom). While we leveraged the classic TRL results from NASA, DoD, and Sandia's NW program, we came up with a substantially revised version of the TRL definitions, maintaining consistency with the classic level definitions and the Predictive Capability Maturity Model (PCMM) approach. In fact, we substantially leveraged the foundation the PCMM team provided, and augmented that as needed. Given the modeling and simulation TRL definitions and our proposed assignment methodology, we conducted four ''field trials'' to examine how this would work in practice. The results varied substantially, but did indicate that establishing the capability dependencies and making the TRL assignments was manageable and not particularly time consuming. The key differences arose in perceptions of how this information might be used, and what value it would have (opinions ranged from negative to positive value). The use cases and field trial results are included in this report. Taken together, the results suggest that we can make reasonably reliable TRL assignments, but that using those without the context of the information that led to those results (i.e., examining the measures suggested by the PCMM table, and extended for ModSim TRL purposes) produces an oversimplified result--that is, you cannot really boil things down to just a scalar value without losing critical information.« less

Authors:
; ; ;
Publication Date:
Research Org.:
Sandia National Laboratories
Sponsoring Org.:
USDOE
OSTI Identifier:
901709
Report Number(s):
SAND2007-0570
TRN: US200716%%320
DOE Contract Number:
AC04-94AL85000
Resource Type:
Technical Report
Country of Publication:
United States
Language:
English
Subject:
99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; SANDIA NATIONAL LABORATORIES; COMPUTERIZED SIMULATION; TESTING; WEAPONS; Military readiness.; Simulation methods.; Information modeling.; Technology-Computer simulation.; Data modeling.

Citation Formats

Clay, Robert L., Shneider, Max S., Marburger, S. J., and Trucano, Timothy Guy. Modeling and simulation technology readiness levels.. United States: N. p., 2006. Web. doi:10.2172/901709.
Clay, Robert L., Shneider, Max S., Marburger, S. J., & Trucano, Timothy Guy. Modeling and simulation technology readiness levels.. United States. doi:10.2172/901709.
Clay, Robert L., Shneider, Max S., Marburger, S. J., and Trucano, Timothy Guy. Sun . "Modeling and simulation technology readiness levels.". United States. doi:10.2172/901709. https://www.osti.gov/servlets/purl/901709.
@article{osti_901709,
title = {Modeling and simulation technology readiness levels.},
author = {Clay, Robert L. and Shneider, Max S. and Marburger, S. J. and Trucano, Timothy Guy},
abstractNote = {This report summarizes the results of an effort to establish a framework for assigning and communicating technology readiness levels (TRLs) for the modeling and simulation (ModSim) capabilities at Sandia National Laboratories. This effort was undertaken as a special assignment for the Weapon Simulation and Computing (WSC) program office led by Art Hale, and lasted from January to September 2006. This report summarizes the results, conclusions, and recommendations, and is intended to help guide the program office in their decisions about the future direction of this work. The work was broken out into several distinct phases, starting with establishing the scope and definition of the assignment. These are characterized in a set of key assertions provided in the body of this report. Fundamentally, the assignment involved establishing an intellectual framework for TRL assignments to Sandia's modeling and simulation capabilities, including the development and testing of a process to conduct the assignments. To that end, we proposed a methodology for both assigning and understanding the TRLs, and outlined some of the restrictions that need to be placed on this process and the expected use of the result. One of the first assumptions we overturned was the notion of a ''static'' TRL--rather we concluded that problem context was essential in any TRL assignment, and that leads to dynamic results (i.e., a ModSim tool's readiness level depends on how it is used, and by whom). While we leveraged the classic TRL results from NASA, DoD, and Sandia's NW program, we came up with a substantially revised version of the TRL definitions, maintaining consistency with the classic level definitions and the Predictive Capability Maturity Model (PCMM) approach. In fact, we substantially leveraged the foundation the PCMM team provided, and augmented that as needed. Given the modeling and simulation TRL definitions and our proposed assignment methodology, we conducted four ''field trials'' to examine how this would work in practice. The results varied substantially, but did indicate that establishing the capability dependencies and making the TRL assignments was manageable and not particularly time consuming. The key differences arose in perceptions of how this information might be used, and what value it would have (opinions ranged from negative to positive value). The use cases and field trial results are included in this report. Taken together, the results suggest that we can make reasonably reliable TRL assignments, but that using those without the context of the information that led to those results (i.e., examining the measures suggested by the PCMM table, and extended for ModSim TRL purposes) produces an oversimplified result--that is, you cannot really boil things down to just a scalar value without losing critical information.},
doi = {10.2172/901709},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Sun Jan 01 00:00:00 EST 2006},
month = {Sun Jan 01 00:00:00 EST 2006}
}

Technical Report:

Save / Share:
  • Integrating technology readiness levels (TRL) into the management of engineering projects is critical to the mitigation of risk and improved customer/supplier communications. TRLs provide a common framework and language with which consistent comparisons of different technologies and approaches can be made. At Sandia National Laboratories, where technologies are developed, integrated and deployed into high consequence systems, the use of TRLs may be transformational. They are technology independent and span the full range of technology development including scientific and applied research, identification of customer requirements, modeling and simulation, identification of environments, testing and integration. With this report, we provide a referencemore » set of definitions for TRLs and a brief history of TRLs at Sandia National Laboratories. We then propose and describe two approaches that may be used to integrate TRLs into the NW SMU business practices. In the first approach, we analyze how TRLs can be integrated within concurrent qualification as documented in TBP-100 [1]. In the second approach we take a look at the product realization process (PRP) as documented in TBP-PRP [2]. Both concurrent qualification and product realization are fundamental to the way weapons engineering work is conducted at this laboratory and the NWC (nuclear weapons complex) as a whole. Given the current structure and definitions laid out in the TBP-100 and TBP-PRP, we believe that integrating TRLs into concurrent qualification (TBP-100) rather than TBP-PRP is optimal. Finally, we note that our charter was to explore and develop ways of integrating TRLs into the NW SMU and therefore we do not significantly cover the development and history of TRLs. This work was executed under the auspices and direction of Sandia's Weapon Engineering Program. Please contact Gerry Sleefe, Deputy Program Director, for further information.« less
  • To quantitatively assess the maturity of a given technology, the Technology Readiness Level (TRL) process is used. The TRL process has been developed and successfully used by the Department of Defense (DOD) for development and deployment of new technology and systems for defense applications. In addition, NASA has also successfully used the TRL process to develop and deploy new systems for space applications. Transmutation fuel development is a critical technology needed for closing the nuclear fuel cycle. Because the deployment of a new nuclear fuel forms requires a lengthy and expensive research, development, and demonstration program, applying the TRL conceptmore » to the transmutation fuel development program is very useful as a management and tracking tool. This report provides definition of the technology readiness level assessment process as defined for use in assessing nuclear fuel technology development for the Transuranic Fuel Development Campaign.« less
  • This is the manual for the Carbon Capture Simulation Initiative (CCSI) Technology Readiness Level Likelihood model based on PNNL velo.
  • The Technology Readiness Level (TRL) process is used to quantitatively assess the maturity of a given technology. The TRL process has been developed and successfully used by the Department of Defense (DOD) for development and deployment of new technology and systems for defense applications. In addition, NASA has also successfully used the TRL process to develop and deploy new systems for space applications. Advanced nuclear fuels and materials development is a critical technology needed for closing the nuclear fuel cycle. Because the deployment of a new nuclear fuel forms requires a lengthy and expensive research, development, and demonstration program, applyingmore » the TRL concept to the advanced fuel development program is very useful as a management and tracking tool. This report provides definition of the technology readiness level assessment process as defined for use in assessing nuclear fuel technology development for the Advanced Fuel Campaign (AFC).« less
  • The Landing Craft Air Cushion is a high-speed, over-the-beach, fully amphibious landing craft capable of carrying a 60-75 ton payload. The LCAC fleet can serve to transport weapons systems, equipment, cargo and personnel from ship to shore and across the beach. This transport system is an integral part of our military arsenal and, as such, its readiness is an important consideration for our national security. Further, the best way to expend financial resources that have been allocated to maintain this fleet is a critical Issue. There is a clear coupling between the measure of Fleet Readiness as defined by themore » customer for this project and the information that is provided by Sandia's ProOpta methodology. Further, there is a richness in the data that provides even more value to the analyst. This report provides an analytic framework for understanding the connection between Fleet Readiness and the output provided by Sandia's ProOpta software. Further, this report highlights valuable information that can also be made available using the ProOpta output and concepts from basic probability theory. Finally, enabling assumptions along with areas that warrant consideration for further study are identified.« less