Software-Defined Network for End-to-end Networked Science at the Exascale

Journal Article · Mon Apr 13 00:00:00 EDT 2020 · Future Generations Computer Systems

DOI:https://doi.org/10.1016/j.future.2020.04.018· OSTI ID:1780747

Monga, Inder ^[1]; Guok, Chin ^[1]; MacAuley, John ^[1]; Sim, Alex ^[1]; Newman, Harvey ^[2]; Balcas, Justas ^[2]; DeMar, Phil ^[3]; Winkler, Linda ^[4]; Lehman, Tom ^[5]; Yang, Xi ^[6]

---

1. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
2. California Institute of Technology (CalTech), Pasadena, CA (United States)
3. Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
4. Argonne National Lab. (ANL), Argonne, IL (United States)
5. Virnao, Arlington, VA (United States)
6. Univ. of Maryland, College Park, MD (United States)

Domain science applications and workflow processes are currently forced to view the network as an opaque infrastructure into which they inject data and hope that it emerges at the destination with an acceptable Quality of Experience. There is little ability for applications to interact with the network to exchange information, negotiate performance parameters, discover expected performance metrics, or receive status/troubleshooting information in real time. The work we presen here is motivated by a vision for a new smart network and smart application ecosystem that will provide a more deterministic and interactive environment for domain science workflows. The Software-Defined Network for End-to-end Networked Science at Exascale (SENSE) system includes a model-based architecture, implementation, and deployment which enables automated end-to-end network service instantiation across administrative domains. An intent based interface allows applications to express their high-level service requirements, an intelligent orchestrator and resource control systems allow for custom tailoring of scalability and real-time responsiveness based on individual application and infrastructure operator requirements. This allows the science applications to manage the network as a first-class schedulable resource as is the current practice for instruments, compute, and storage systems. Deployment and experiments on production networks and testbeds have validated SENSE functions and performance. Emulation based testing verified the scalability needed to support research and education infrastructures. Key contributions of this work include an architecture definition, reference implementation, and deployment. Finally, this provides the basis for further innovation of smart network services to accelerate scientific discovery in the era of big data, cloud computing, machine learning and artificial intelligence.

• Research Organization: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States); Argonne National Laboratory (ANL), Argonne, IL (United States); Fermi National Accelerator Laboratory (FNAL), Batavia, IL (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); National Science Foundation (NSF)
• Grant/Contract Number: AC02-05CH11231; AC02-07CH11359; AC02-06CH11357; SC0007346; SC0015527; SC0015528; SC0016585; SC0010716; 1246133; 1341024; 1120138
• OSTI ID: 1780747
• Alternate ID(s): OSTI ID: 1615391; OSTI ID: 1833521
• Report Number(s): FERMILAB-PUB-20-684-CCD; ark:/13030/qt08w3f6zf
• Journal Information: Future Generations Computer Systems, Vol. 110; ISSN 0167-739X
• Publisher: Elsevier
• Country of Publication: United States
• Language: English

Related Subjects

97 MATHEMATICS AND COMPUTING
intent based networking
end-to-end orchestration
intelligent network services
distributed infrastructure
resource modeling
software defined networking
real-time
interactive