Staghorn: An Automated Large-Scale Distributed System Analysis Platform

Gabert, Kasimir; Burns, Ian; Elliott, Steven; Kallaher, Jenna; Vail, Adam

doi:10.2172/1411885

Title: Staghorn: An Automated Large-Scale Distributed System Analysis Platform

Full Record
Other Related Research

Abstract

Conducting experiments on large-scale distributed computing systems is becoming significantly easier with the assistance of emulation. Researchers can now create a model of a distributed computing environment and then generate a virtual, laboratory copy of the entire system composed of potentially thousands of virtual machines, switches, and software. The use of real software, running at clock rate in full virtual machines, allows experiments to produce meaningful results without necessitating a full understanding of all model components. However, the ability to inspect and modify elements within these models is bound by the limitation that such modifications must compete with the model, either running in or alongside it. This inhibits entire classes of analyses from being conducted upon these models. We developed a mechanism to snapshot an entire emulation-based model as it is running. This allows us to \freeze time" and subsequently fork execution, replay execution, modify arbitrary parts of the model, or deeply explore the model. This snapshot includes capturing packets in transit and other input/output state along with the running virtual machines. We were able to build this system in Linux using Open vSwitch and Kernel Virtual Machines on top of Sandia's emulation platform Firewheel. This primitive opens the doormore »« less

Authors:

Gabert, Kasimir ^[1]; Burns, Ian ^[1]; Elliott, Steven ^[1]; Kallaher, Jenna ^[1]; Vail, Adam ^[1]

Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Publication Date:: 2016-09-01

Research Org.:: Sandia National Lab. (SNL-NM), Albuquerque, NM (United States)

Sponsoring Org.:: USDOE National Nuclear Security Administration (NNSA)

OSTI Identifier:: 1411885

Report Number(s):: SAND2016-9616
657048

DOE Contract Number:: AC04-94AL85000

Resource Type:: Technical Report

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Gabert, Kasimir, Burns, Ian, Elliott, Steven, Kallaher, Jenna, and Vail, Adam. Staghorn: An Automated Large-Scale Distributed System Analysis Platform.  United States: N. p., 2016. 
        Web.  doi:10.2172/1411885.

Copy to clipboard


                    Gabert, Kasimir, Burns, Ian, Elliott, Steven, Kallaher, Jenna, & Vail, Adam. Staghorn: An Automated Large-Scale Distributed System Analysis Platform.  United States.  https://doi.org/10.2172/1411885

Copy to clipboard


                    Gabert, Kasimir, Burns, Ian, Elliott, Steven, Kallaher, Jenna, and Vail, Adam. 2016.  
        "Staghorn: An Automated Large-Scale Distributed System Analysis Platform".  United States.  https://doi.org/10.2172/1411885.  https://www.osti.gov/servlets/purl/1411885.

Copy to clipboard


                    
@article{osti_1411885,

  title        = {Staghorn: An Automated Large-Scale Distributed System Analysis Platform},

  author       = {Gabert, Kasimir and Burns, Ian and Elliott, Steven and Kallaher, Jenna and Vail, Adam},

  abstractNote = {Conducting experiments on large-scale distributed computing systems is becoming significantly easier with the assistance of emulation. Researchers can now create a model of a distributed computing environment and then generate a virtual, laboratory copy of the entire system composed of potentially thousands of virtual machines, switches, and software. The use of real software, running at clock rate in full virtual machines, allows experiments to produce meaningful results without necessitating a full understanding of all model components. However, the ability to inspect and modify elements within these models is bound by the limitation that such modifications must compete with the model, either running in or alongside it. This inhibits entire classes of analyses from being conducted upon these models. We developed a mechanism to snapshot an entire emulation-based model as it is running. This allows us to \freeze time" and subsequently fork execution, replay execution, modify arbitrary parts of the model, or deeply explore the model. This snapshot includes capturing packets in transit and other input/output state along with the running virtual machines. We were able to build this system in Linux using Open vSwitch and Kernel Virtual Machines on top of Sandia's emulation platform Firewheel. This primitive opens the door to numerous subsequent analyses on models, including state space exploration, debugging distributed systems, performance optimizations, improved training environments, and improved experiment repeatability.},

  doi          = {10.2172/1411885},

  url          = {https://www.osti.gov/biblio/1411885},
  journal      = {},
number       = ,

  volume       = ,

  place        = {United States},

  year         = {2016},

  month        = {9}

}

Copy to clipboard

Technical Report:

View Technical Report (1.62 MB)

https://doi.org/10.2172/1411885

Save / Share:

Export Metadata

Save to My Library

Similar records in OSTI.GOV collections:

Build and Execute Environment

Software Guan, Qiang

At exascale, the challenge becomes to develop applications that run at scale and use exascale platforms reliably, efficiently, and flexibly. Workflows become much more complex because they must seamlessly integrate simulation and data analytics. They must include down-sampling, post-processing, feature extraction, and visualization. Power and data transfer limitations require these analysis tasks to be run in-situ or in-transit. We expect successful workflows will comprise multiple linked simulations along with tens of analysis routines. Users will have limited development time at scale and, therefore, must have rich tools to develop, debug, test, and deploy applications. At this scale, successful workflows willmore »« less
View Software
Logic in Memory Emulator

Software Jain, Abhishek; Macaraeg, Christopher; Gokhale, Maya; ...

Logic in Memory Emulator (LiME) is a hardware/software tool specially designed for memory system evaluation and experiment. Emerging memories display a wide range of bandwidths, latencies, and capacities, making it challenging for the computer architect to navigate the design space of potential memory configurations, and for the application developer to assess performance implications of using such memories. With the LiME framework, architectural ideas can be prototyped in great detail yet with sufficient performance to support realistic evaluation on long running applications. LiME consists of two fundamental components: 1) the hardware and OS infrastructure for the emulator, and 2) a suitemore »« less
https://doi.org/10.11578/dc.20200728.1

View Software
Review of Enabling Technologies to Facilitate Secure Compute Customization

Technical Report Aderholdt, Ferrol; Caldwell, Blake; Hicks, Susan; ...

High performance computing environments are often used for a wide variety of workloads ranging from simulation, data transformation and analysis, and complex workflows to name just a few. These systems may process data for a variety of users, often requiring strong separation between job allocations. There are many challenges to establishing these secure enclaves within the shared infrastructure of high-performance computing (HPC) environments. The isolation mechanisms in the system software are the basic building blocks for enabling secure compute enclaves. There are a variety of approaches and the focus of this report is to review the different virtualization technologies thatmore »« less
https://doi.org/10.2172/1195817

Full Text Available
Langston University - High Energy Physics (LU-HEP)

Technical Report Snow, Dr., Joel

This final report is presented by Langston University (LU) for the project entitled "Langston University High Energy Physics" (LUHEP) under the direction of principal investigator (PI) and project director Professor Joel Snow. The project encompassed high energy physics research performed at hadron colliders. The PI is a collaborator on the DZero experiment at Fermi National Accelerator Laboratory in Batavia, IL, USA and the ATLAS experiment at CERN in Geneva, Switzerland and was during the entire project period from April 1, 1999 until May 14, 2012. Both experiments seek to understand the fundamental constituents of the physical universe and the forcesmore »« less
https://doi.org/10.2172/1048139

Full Text Available
Using Discrete Event Simulation for Programming Model Exploration at Extreme-Scale: Macroscale Components for the Structural Simulation Toolkit (SST).

Technical Report Wilke, Jeremiah; Kenny, Joseph

Discrete event simulation provides a powerful mechanism for designing and testing new extreme- scale programming models for high-performance computing. Rather than debug, run, and wait for results on an actual system, design can first iterate through a simulator. This is particularly useful when test beds cannot be used, i.e. to explore hardware or scales that do not yet exist or are inaccessible. Here we detail the macroscale components of the structural simulation toolkit (SST). Instead of depending on trace replay or state machines, the simulator is architected to execute real code on real software stacks. Our particular user-space threading frameworkmore »« less
https://doi.org/10.2172/1170619

Full Text Available

Similar Records