SDN for End-to-end Networked Science at the Exascale (SENSE) - Final Technical Report
The Software-defined network for End-to-end Networked Science at Exascale (SENSE) research developed intelligent network services to accelerate scientific discovery in the era of big data driven by Exascale, cloud computing, machine learning and artificial intelligence. The SENSE system allows National Labs and Universities to request and provision end-to-end intelligent network services for their application workflows leveraging Software Defined Network (SDN) capabilities. The SENSE system includes a model-based architecture, implementation, and deployment which enables end-to-end network service instantiation across administrative domains. An intent based interface allows applications to express their high-level service requirements. Intelligent orchestrator and software defined systems allow for custom tailoring of scalability and realtime responsiveness based on individual application and infrastructure operator requirements. This allows the science applications to manage the network as a first-class schedulable resource in a manner similar to instruments, compute, and storage. The SENSE includes a model-based orchestration system which operates between the SDN layer controlling the individual networks/end-sites, and science workflow agents/middleware. In addition, SENSE has developed model-based "Network Resource Manager" and "End-Site Resource Manager" components which enable advanced features in the areas of multi-resource integration, real time responsiveness, and user interactions. The SENSE project was motivated by a belief that Domain Science application workflows need real-time, interactive, end-to-end orchestrated SDN services across large, distributed, multi-domain networks. Domain science applications and workflow processes are currently forced to view the network as an opaque infrastructure into which they inject data and hope that it emerges at the destination with an acceptable Quality of Experience. There is little ability for applications to interact with the network to exchange information, negotiate performance parameters, discover expected performance metrics, or receive status/troubleshooting information in real time. The SENSE project was motivated by a vision for a new smart network and smart application ecosystem that will solve these issues and provide a more deterministic and interactive environment for domain science workflows. The SENSE project developed an architecture and implementation to address this vision. Key contributions of this work include the architecture definition, reference implementation, and deployment as the basis for further innovation of smart network services. The SENSE system with its model-based, realtime, multi-resource cyberinfrastructure awareness provides a data source and framework to leverage artificial intelligence and machine learning technologies to enhance network monitoring, provisioning, optimization, troubleshooting, and workflow services operations. The SENSE project team is led by Energy Sciences Network (ESnet) and included participation by Lawrence Berkeley National Laboratory (LBL), Fermi National Accelerator Laboratory (Fermilab), Argonne National Laboratory (ANL), California Institute of Technology (Caltech), and University of Maryland/Mid-Atlantic Crossroads (UMD/MAX).
- Research Organization:
- Univ. of Maryland, College Park, MD (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC)
- DOE Contract Number:
- SC0016585
- OSTI ID:
- 1577127
- Report Number(s):
- DOE-UMD-6585-1
- Country of Publication:
- United States
- Language:
- English
Similar Records
Software-Defined Network for End-to-end Networked Science at the Exascale
SDN-NGenIA, a Software Defined Next Generation Integrated Architecture for HEP and Data Intensive Science