Optically connected memory for disaggregated data centers

Gonzalez, Jorge; Palma, Mauricio G.; Hattink, Maarten; Rubio-Noriega, Ruth; Orosa, Lois; Mutlu, Onur; Bergman, Keren; Azevedo, Rodolfo

doi:10.1016/j.jpdc.2022.01.013

Title: Optically connected memory for disaggregated data centers

Journal Article · 28 January 2022 · Journal of Parallel and Distributed Computing

DOI: https://doi.org/10.1016/j.jpdc.2022.01.013 · OSTI ID:1977385

^[1];

^[2]; Hattink, Maarten ^[3]; Rubio-Noriega, Ruth ^[4]; Orosa, Lois ^[5]; Mutlu, Onur ^[5]; Bergman, Keren ^[3]; Azevedo, Rodolfo ^[2]

University of Engineering and Technology, Lima (Peru); Univ. of Campinas (UNICAMP), Sao Paulo (Brazil); OSTI
Univ. of Campinas (UNICAMP), Sao Paulo (Brazil)
Columbia Univ., New York, NY (United States)
Instituto Nacional de Investigación y Capacitación de Telecomunicaciones de la Universidad Nacional de Ingeniería (INICTEL-UNI) (Peru)
Eidgenoessische Technische Hochschule (ETH), Zurich (Switzerland)

Recent advances in integrated photonics enable the implementation of reconfigurable, high-bandwidth, and low energy-per-bit interconnects in next-generation data centers. We propose and evaluate an Optically Connected Memory (OCM) architecture that disaggregates the main memory from the computation nodes in data centers. OCM is based on micro-ring resonators (MRRs), and it does not require any modification to the DRAM memory modules. We calculate energy consumption from real photonic devices and integrate them into a system simulator to evaluate performance. Here, our results show that (1) OCM is capable of interconnecting four DDR4 memory channels to a computing node using two fibers with 1.02 pJ energy-per-bit consumption and (2) OCM performs up to 5.5× faster than a disaggregated memory with 40G PCIe NIC connectors to computing nodes.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Analog Photonics LLC, Boston, MA (United States)

Sponsoring Organization:: USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR)

Grant/Contract Number:: SC0017182

OSTI ID:: 1977385

Journal Information:: Journal of Parallel and Distributed Computing, Journal Name: Journal of Parallel and Distributed Computing Journal Issue: C Vol. 163; ISSN 0743-7315

Publisher:: ElsevierCopyright Statement

Country of Publication:: United States

Language:: English

References (31)

Progress, Challenges, and Opportunities for 2D Material Based Photodetectors Long, Mingsheng; Wang, Peng; Fang, Hehai Advanced Functional Materials, Vol. 29, Issue 19 https://doi.org/10.1002/adfm.201803807	journal	September 2018
Novel Light Source Integration Approaches for Silicon Photonics: Novel Light Source Integration Approaches for Silicon Photonics Wang, Zhechao; Abbasi, Amin; Dave, Utsav Laser & Photonics Reviews, Vol. 11, Issue 4 https://doi.org/10.1002/lpor.201700063	journal	July 2017
Versatile, scalable, and accurate simulation of distributed applications and platforms Casanova, Henri; Giersch, Arnaud; Legrand, Arnaud Journal of Parallel and Distributed Computing, Vol. 74, Issue 10 https://doi.org/10.1016/j.jpdc.2014.06.008	journal	October 2014
Low-loss composite photonic platform based on 2D semiconductor monolayers Datta, Ipshita; Chae, Sang Hoon; Bhatt, Gaurang R. Nature Photonics, Vol. 14, Issue 4 https://doi.org/10.1038/s41566-020-0590-4	journal	February 2020
Dense wavelength division multiplexing networks: principles and applications Brackett, C. A. IEEE Journal on Selected Areas in Communications, Vol. 8, Issue 6 https://doi.org/10.1109/49.57798	journal	January 1990
Comprehensive Design Space Exploration of Silicon Photonic Interconnects Bahadori, Meisam; Rumley, Sebastien; Nikolova, Dessislava Journal of Lightwave Technology, Vol. 34, Issue 12 https://doi.org/10.1109/JLT.2015.2503120	journal	June 2016
All-Optical Programmable Disaggregated Data Centre Network Realized by FPGA-Based Switch and Interface Card Yan, Yan; Saridis, George M.; Shu, Yi Journal of Lightwave Technology, Vol. 34, Issue 8 https://doi.org/10.1109/JLT.2016.2518492	journal	April 2016
Thermal Rectification of Integrated Microheaters for Microring Resonators in Silicon Photonics Platform Bahadori, Meisam; Gazman, Alexander; Janosik, Natalie Journal of Lightwave Technology, Vol. 36, Issue 3 https://doi.org/10.1109/JLT.2017.2781131	journal	February 2018
Realization and Application of Large-Scale Fast Optical Circuit Switch for Data Center Networking Sato, Ken-ichi Journal of Lightwave Technology, Vol. 36, Issue 7 https://doi.org/10.1109/JLT.2018.2801308	journal	April 2018
Design Space Exploration of Microring Resonators in Silicon Photonic Interconnects: Impact of the Ring Curvature Bahadori, Meisam; Bergman, Keren; Nikdast, Mahdi Journal of Lightwave Technology, Vol. 36, Issue 13 https://doi.org/10.1109/JLT.2018.2821359	journal	July 2018
Silicon Photonics for Extreme Scale Systems Shen, Yiwen; Meng, Xiang; Cheng, Qixiang Journal of Lightwave Technology, Vol. 37, Issue 2 https://doi.org/10.1109/JLT.2019.2897365	journal	January 2019
Silicon Photonic 2.5D Multi-Chip Module Transceiver for High-Performance Data Centers Abrams, Nathan C.; Cheng, Qixiang; Glick, Madeleine Journal of Lightwave Technology, Vol. 38, Issue 13 https://doi.org/10.1109/JLT.2020.2967235	journal	July 2020
An Energy-Efficient Optically Connected Memory Module for Hybrid Packet- and Circuit-Switched Optical Networks Brunina, D.; Bergman, K. IEEE Journal of Selected Topics in Quantum Electronics, Vol. 19, Issue 2 https://doi.org/10.1109/JSTQE.2012.2224096	journal	March 2013
Enabling Efficient and Scalable Hybrid Memories Using Fine-Granularity DRAM Cache Management Meza, Justin; Chang, Jichuan; Yoon, HanBin IEEE Computer Architecture Letters, Vol. 11, Issue 2 https://doi.org/10.1109/L-CA.2012.2	journal	July 2012
Size Influence on the Propagation Loss Induced by Sidewall Roughness in Ultrasmall SOI Waveguides Grillot, F.; Vivien, L.; Laval, S. IEEE Photonics Technology Letters, Vol. 16, Issue 7 https://doi.org/10.1109/LPT.2004.828497	journal	July 2004
WDM Source Based on High-Power, Efficient 1280-nm DFB Lasers for Terabit Interconnect Technologies Buckley, Bob B.; Fryslie, Stewart T. M.; Guinn, Keith IEEE Photonics Technology Letters, Vol. 30, Issue 22 https://doi.org/10.1109/LPT.2018.2872597	journal	November 2018
1.3- $\mu$ m Reflection Insensitive InAs/GaAs Quantum Dot Lasers Directly Grown on Silicon Duan, Jianan; Huang, Heming; Dong, Bozhang IEEE Photonics Technology Letters, Vol. 31, Issue 5 https://doi.org/10.1109/LPT.2019.2895049	journal	March 2019
The End of Moore's Law: A New Beginning for Information Technology Theis, Thomas N.; Wong, H. -S. Philip Computing in Science & Engineering, Vol. 19, Issue 2 https://doi.org/10.1109/MCSE.2017.29	journal	March 2017
Building Many-Core Processor-to-DRAM Networks with Monolithic CMOS Silicon Photonics Batten, Christopher; Joshi, Ajay; Orcutt, Jason IEEE Micro, Vol. 29, Issue 4 https://doi.org/10.1109/MM.2009.60	journal	July 2009
Simulating MPI Applications: The SMPI Approach Degomme, Augustin; Legrand, Arnaud; Markomanolis, George S. IEEE Transactions on Parallel and Distributed Systems, Vol. 28, Issue 8 https://doi.org/10.1109/TPDS.2017.2669305	journal	August 2017
Efficiency Optimization of Silicon Photonic Links in 65-nm CMOS and 28-nm FDSOI Technology Nodes Polster, Robert; Thonnart, Yvain; Waltener, Guillaume IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 24, Issue 12 https://doi.org/10.1109/TVLSI.2016.2553673	journal	December 2016
Architectural Impacts of RFiop: RF to Address I/O Pad and Memory Controller Scalability Marino, Mario Donato IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 26, Issue 8 https://doi.org/10.1109/TVLSI.2018.2821004	journal	August 2018
SPEC CPU2006 benchmark descriptions Henning, John L. ACM SIGARCH Computer Architecture News, Vol. 34, Issue 4 https://doi.org/10.1145/1186736.1186737	journal	September 2006
Valgrind: a framework for heavyweight dynamic binary instrumentation Nethercote, Nicholas; Seward, Julian ACM SIGPLAN Notices, Vol. 42, Issue 6 https://doi.org/10.1145/1273442.1250746	journal	June 2007
Roofline: an insightful visual performance model for multicore architectures Williams, Samuel; Waterman, Andrew; Patterson, David Communications of the ACM, Vol. 52, Issue 4 https://doi.org/10.1145/1498765.1498785	journal	April 2009
Optically Disaggregated Data Centers With Minimal Remote Memory Latency: Technologies, Architectures, and Resource Allocation [Invited] Zervas, Georgios; Yuan, Hui; Saljoghei, Arsalan Journal of Optical Communications and Networking, Vol. 10, Issue 2 https://doi.org/10.1364/JOCN.10.00A270	journal	January 2018
Building Data Centers With Optically Connected Memory Brunina, Daniel; Lai, Caroline P.; Garg, Ajay S. Journal of Optical Communications and Networking, Vol. 3, Issue 8 https://doi.org/10.1364/JOCN.3.000A40	journal	July 2011
Responsivity optimization of a high-speed germanium-on-silicon photodetector Fard, Monireh Moayedi Pour; Cowan, Glenn; Liboiron-Ladouceur, Odile Optics Express, Vol. 24, Issue 24 https://doi.org/10.1364/OE.24.027738	journal	January 2016
Ultra-thin DVS-BCB adhesive bonding of III-V wafers, dies and multiple dies to a patterned silicon-on-insulator substrate Keyvaninia, S.; Muneeb, M.; Stanković, S. Optical Materials Express, Vol. 3, Issue 1 https://doi.org/10.1364/OME.3.000035	journal	December 2012
A Roadmap for Integrated Photonics Glick, Madeleine; Kimmerling, Lionel C.; Pfahl, Robert C. Optics and Photonics News, Vol. 29, Issue 3 https://doi.org/10.1364/OPN.29.3.000036	journal	January 2018
High-channel-count 20 GHz passively mode-locked quantum dot laser directly grown on Si with 41 Tbit/s transmission capacity Liu, Songtao; Wu, Xinru; Jung, Daehwan Optica, Vol. 6, Issue 2 https://doi.org/10.1364/OPTICA.6.000128	journal	January 2019

Similar Records

Data Movement Dominates: Advanced Memory Technology to Address the Real Exascale Power Problem

Technical Report · 2014 · OSTI ID:1151423

Bergman, Keren

New trends in photonic switching and optical networking architectures for data centers and computing systems [Invited]

Journal Article · 2022 · Journal of Optical Communications and Networking · OSTI ID:2421350

Yoo, S. J. Ben

New trends in photonic switching and optical networking architectures for data centers and computing systems [Invited]

Journal Article · 2023 · Journal of Optical Communications and Networking · OSTI ID:1992555

Yoo, S. J. Ben

Related Subjects

97 MATHEMATICS AND COMPUTING
DRAM
Data-centers
Disaggregated memory
Memory systems
Photonics

Title: Optically connected memory for disaggregated data centers

Citation Formats

References (31)

Similar Records

Related Subjects