skip to main content

DOE PAGESDOE PAGES

This content will become publicly available on June 7, 2019

Title: NSTX-U advances in real-time deterministic PCIe-based internode communication

Distributing control mechanisms across modern commercial off the shelf (COTS) GNU/Linux systems often introduces difficult to mitigate non-deterministic behavior. Existing methods to address this problem involve non-real-time technologies such as Remote Direct Memory Access (RDMA) over Infiniband or custom Ethernet solutions, such as RDMA over Converged Ethernet (RoCE) or userspace Ethernet drivers, that trade determinism for ease of use or lower cost. The National Spherical Torus Experiment Upgrade (NSTX-U) is pursuing a new design that allows direct communication between heterogeneous systems with scalable, microsecond latency with 1 μs of jitter on that latency, outside of the constant transmission delay at the physical layer. The future design of the NSTX-U Real-time Communication System will utilize direct PCIe-to-PCIe communication with kernel support tuned for low overhead, allowing two (or more, through a switch) real-time (RT) systems to communicate and share resources as one larger entity. As a result, this greatly increases the processing capability of the primary Plasma Control System (PCS), turning previously insurmountable computational challenges into a more manageable divide and conquer parallel task.
Authors:
ORCiD logo [1] ;  [1] ;  [2]
  1. Princeton Univ. Plasma Physics Lab, Princeton, NJ (United States)
  2. Concurrent Real-Time, Pompano Beach, FL (United States)
Publication Date:
Grant/Contract Number:
AC02-09CH11466
Type:
Accepted Manuscript
Journal Name:
Fusion Engineering and Design
Additional Journal Information:
Journal Volume: 133; Journal Issue: C; Journal ID: ISSN 0920-3796
Publisher:
Elsevier
Research Org:
Princeton Plasma Physics Lab. (PPPL), Princeton, NJ (United States)
Sponsoring Org:
USDOE
Country of Publication:
United States
Language:
English
Subject:
70 PLASMA PHYSICS AND FUSION TECHNOLOGY; Real-time Linux; Reflective memory; RDMA; PCIe
OSTI Identifier:
1460738

Erickson, Keith G., Boyer, M. Dan, and Higgins, David. NSTX-U advances in real-time deterministic PCIe-based internode communication. United States: N. p., Web. doi:10.1016/j.fusengdes.2018.02.055.
Erickson, Keith G., Boyer, M. Dan, & Higgins, David. NSTX-U advances in real-time deterministic PCIe-based internode communication. United States. doi:10.1016/j.fusengdes.2018.02.055.
Erickson, Keith G., Boyer, M. Dan, and Higgins, David. 2018. "NSTX-U advances in real-time deterministic PCIe-based internode communication". United States. doi:10.1016/j.fusengdes.2018.02.055.
@article{osti_1460738,
title = {NSTX-U advances in real-time deterministic PCIe-based internode communication},
author = {Erickson, Keith G. and Boyer, M. Dan and Higgins, David},
abstractNote = {Distributing control mechanisms across modern commercial off the shelf (COTS) GNU/Linux systems often introduces difficult to mitigate non-deterministic behavior. Existing methods to address this problem involve non-real-time technologies such as Remote Direct Memory Access (RDMA) over Infiniband or custom Ethernet solutions, such as RDMA over Converged Ethernet (RoCE) or userspace Ethernet drivers, that trade determinism for ease of use or lower cost. The National Spherical Torus Experiment Upgrade (NSTX-U) is pursuing a new design that allows direct communication between heterogeneous systems with scalable, microsecond latency with 1 μs of jitter on that latency, outside of the constant transmission delay at the physical layer. The future design of the NSTX-U Real-time Communication System will utilize direct PCIe-to-PCIe communication with kernel support tuned for low overhead, allowing two (or more, through a switch) real-time (RT) systems to communicate and share resources as one larger entity. As a result, this greatly increases the processing capability of the primary Plasma Control System (PCS), turning previously insurmountable computational challenges into a more manageable divide and conquer parallel task.},
doi = {10.1016/j.fusengdes.2018.02.055},
journal = {Fusion Engineering and Design},
number = C,
volume = 133,
place = {United States},
year = {2018},
month = {6}
}