Machine-learning-aided cognitive reconfiguration for flexible-bandwidth HPC and data center networks [Invited]

Chen, Xiaoliang; Proietti, Roberto; Fariborz, Marjan; Liu, Che-Yu; Yoo, S. J. Ben

doi:10.1364/JOCN.412360

Machine-learning-aided cognitive reconfiguration for flexible-bandwidth HPC and data center networks [Invited]

Journal Article · Wed Jan 20 00:00:00 EST 2021 · Journal of Optical Communications and Networking

DOI:https://doi.org/10.1364/JOCN.412360· OSTI ID:1842377

; ; Fariborz, Marjan; Liu, Che-Yu; Yoo, S. J. Ben

This paper proposes a machine-learning (ML)-aided cognitive approach for effective bandwidth reconfiguration in optically interconnected datacenter/high-performance computing (HPC) systems. The proposed approach relies on a Hyper-X-like architecture augmented with flexible-bandwidth photonic interconnections at large scales using a hierarchical intra/inter-POD photonic switching layout. We first formulate the problem of the connectivity graph and routing scheme optimization as a mixed-integer linear programming model. A two-phase heuristic algorithm and a joint optimization approach are devised to solve the problem with low time complexity. Then, we propose an ML-based end-to-end performance estimator design to assist the network control plane with intelligent decision making for bandwidth reconfiguration. Numerical simulations using traffic distribution profiles extracted from HPC applications traces as well as random traffic matrices verify the accuracy performance of the ML design estimator ( $< <#comment/> 9 % <#comment/>$ error) and demonstrate up to $5 \times <#comment/>$ throughput gain from the proposed approach compared with the baseline Hyper-X network using fixed all-to-all intra/inter-portable data center interconnects.

View Accepted Manuscript (Publisher)

Sponsoring Organization:: USDOE

Grant/Contract Number:: SC0019526; SC0019582

OSTI ID:: 1842377

Alternate ID(s):: OSTI ID: 1853355

Journal Information:: Journal of Optical Communications and Networking, Journal Name: Journal of Optical Communications and Networking Journal Issue: 6 Vol. 13; ISSN JOCNBB; ISSN 1943-0620

Publisher:: Optical Society of AmericaCopyright Statement

Country of Publication:: United States

Language:: English

References (22)

Dynamic Service Provisioning in Elastic Optical Networks With Hybrid Single-/Multi-Path Routing Zhu, Zuqing; Lu, Wei; Zhang, Liang Journal of Lightwave Technology, Vol. 31, Issue 1 https://doi.org/10.1109/JLT.2012.2227683	journal	January 2013
Experimental Demonstration of Flexible Bandwidth Optical Data Center Core Network With All-to-All Interconnectivity Cao, Zheng; Proietti, Roberto; Clements, Matthew Journal of Lightwave Technology, Vol. 33, Issue 8 https://doi.org/10.1109/JLT.2014.2387205	journal	April 2015
A Scalable, Low-Latency, High-Throughput, Optical Interconnect Architecture Based on Arrayed Waveguide Grating Routers Proietti, Roberto; Cao, Zheng; Nitta, Christopher J. Journal of Lightwave Technology, Vol. 33, Issue 4 https://doi.org/10.1109/JLT.2015.2395352	journal	February 2015
Predictive Analytics Based Knowledge-Defined Orchestration in a Hybrid Optical/Electrical Datacenter Network Testbed Fang, Hongqiang; Lu, Wei; Li, Qinhezi Journal of Lightwave Technology, Vol. 37, Issue 19 https://doi.org/10.1109/JLT.2019.2925827	journal	October 2019
Silicon Photonic Flex-LIONS for Bandwidth-Reconfigurable Optical Interconnects Xiao, Xian; Proietti, Roberto; Liu, Gengchen IEEE Journal of Selected Topics in Quantum Electronics, Vol. 26, Issue 2 https://doi.org/10.1109/JSTQE.2019.2950770	journal	March 2020
Highly efficient data migration and backup for big data applications in elastic optical inter-data-center networks Lu, Ping; Zhang, Liang; Liu, Xiahe IEEE Network, Vol. 29, Issue 5 https://doi.org/10.1109/MNET.2015.7293303	journal	September 2015
Flexfly: Enabling a Reconfigurable Dragonfly through Silicon Photonics Wen, Ke; Samadi, Payman; Rumley, Sebastien SC16: International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1109/SC.2016.14	conference	November 2016
A Survey on Optical Interconnects for Data Centers Kachris, Christoforos; Tomkos, Ioannis IEEE Communications Surveys & Tutorials, Vol. 14, Issue 4 https://doi.org/10.1109/SURV.2011.122111.00069	journal	January 2012
OSA: An Optical Switching Architecture for Data Center Networks With Unprecedented Flexibility Chen, Kai; Singla, Ankit; Singh, Atul IEEE/ACM Transactions on Networking, Vol. 22, Issue 2 https://doi.org/10.1109/TNET.2013.2253120	journal	April 2014
Latest standardization status and its future directions for high speed optical transceivers Isono, Hideki Metro and Data Center Optical Networks and Short-Reach Links II https://doi.org/10.1117/12.2507321	conference	March 2019
A scalable, commodity data center network architecture Al-Fares, Mohammad; Loukissas, Alexander; Vahdat, Amin ACM SIGCOMM Computer Communication Review, Vol. 38, Issue 4 https://doi.org/10.1145/1402946.1402967	journal	October 2008
Understanding data center traffic characteristics Benson, Theophilus; Anand, Ashok; Akella, Aditya ACM SIGCOMM Computer Communication Review, Vol. 40, Issue 1 https://doi.org/10.1145/1672308.1672325	journal	January 2010
c-Through: part-time optics in data centers Wang, Guohui; Andersen, David G.; Kaminsky, Michael ACM SIGCOMM Computer Communication Review, Vol. 40, Issue 4 https://doi.org/10.1145/1851275.1851222	journal	August 2010
Helios: a hybrid electrical/optical switch architecture for modular data centers Farrington, Nathan; Porter, George; Radhakrishnan, Sivasankar ACM SIGCOMM Computer Communication Review, Vol. 40, Issue 4 https://doi.org/10.1145/1851275.1851223	journal	August 2010
Xpander: Towards Optimal-Performance Datacenters Valadarsky, Asaf; Shahaf, Gal; Dinitz, Michael Proceedings of the 12th International on Conference on emerging Networking EXperiments and Technologies - CoNEXT '16 https://doi.org/10.1145/2999572.2999580	conference	January 2016
Neural Network Meets DCN: Traffic-driven Topology Adaptation with Deep Learning Wang, Mowei; Cui, Yong; Xiao, Shihan Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 2, Issue 2 https://doi.org/10.1145/3224421	journal	June 2018
HyperX topology: first at-scale implementation and comparison to the fat-tree Domke, Jens; Matsuoka, Satoshi; Ivanov, Ivan R. SC '19: The International Conference for High Performance Computing, Networking, Storage, and Analysis, Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis https://doi.org/10.1145/3295500.3356140	conference	November 2019
MEMS-Actuated 8×8 Silicon Photonic Wavelength-Selective Switches with 8 Wavelength Channels Seok, Tae Joon; Luo, Jianheng; Huang, Zhilei CLEO: Science and Innovations, Conference on Lasers and Electro-Optics https://doi.org/10.1364/CLEO_SI.2018.STu4B.1	conference	January 2018
HiFOST: A Scalable and Low-Latency Hybrid Data Center Network Architecture Based on Flow-Controlled Fast Optical Switches Yan, Fulong; Xue, Xuwei; Calabretta, Nicola Journal of Optical Communications and Networking, Vol. 10, Issue 7 https://doi.org/10.1364/JOCN.10.0000B1	journal	January 2018
Flexspander: augmenting expander networks in high-performance systems with optical bandwidth steering Teh, Min Yee; Wu, Zhenguo; Bergman, Keren Journal of Optical Communications and Networking, Vol. 12, Issue 4 https://doi.org/10.1364/JOCN.379487	journal	January 2020
Photonic switching in high performance datacenters [Invited] Cheng, Qixiang; Rumley, Sébastien; Bahadori, Meisam Optics Express, Vol. 26, Issue 12 https://doi.org/10.1364/OE.26.016022	journal	January 2018
Autonomous Dynamic Bandwidth Steering with Silicon Photonic-Based Wavelength and Spatial Switching for Datacom Networks Shen, Yiwen; Gazman, Alexander; Zhu, Ziyi Optical Fiber Communication Conference https://doi.org/10.1364/OFC.2018.Tu3F.2	conference	January 2018

Similar Records

Efficient and compact thermo-optic phase shifter in silicon-rich silicon nitride

Journal Article · Mon Sep 13 20:00:00 EDT 2021 · Optics Letters · OSTI ID:1819771

Ultra-high extinction ratio polarization beam splitter with extreme skin-depth waveguide

Journal Article · Tue Apr 27 20:00:00 EDT 2021 · Optics Letters · OSTI ID:1779911

Machine-learning-aided cognitive reconfiguration for flexible-bandwidth HPC and data center networks [Invited]

Citation Formats

References (22)

Similar Records

Related Subjects