Fan-In Communications On A Cray Gemini Interconnect

Conference · Wed Jan 01 04:00:00 EST 2014

OSTI ID:1131533

Jones, Terry R ^[1]; Settlemyer, Bradley W ^[1]

---

1. ORNL

Using the Cray Gemini interconnect as our platform, we present a study of an important class of communication operations the fan-in communication pattern. By its nature, fan-in communications form hot spots that present significant challenges for any interconnect fabric and communication software stack. Yet despite the inherent challenges, these communication patterns are common in both applications (which often perform reductions and other collective operations that include fan-in communication such as barriers) and system software (where they assume an important role within parallel file systems and other components requiring high-bandwidth or low-latency I/O). Our study determines the effectiveness of differing clientserver fan-in strategies. We describe fan-in performance in terms of aggregate bandwidth in the presence of varying degrees of congestion, as well as several other key attributes. Comparison numbers are presented for the Cray Aries interconnect. Finally, we provide recommended communication strategies based on our findings.

• Research Organization: Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States). Oak Ridge Leadership Computing Facility (OLCF)
• Sponsoring Organization: ORNL LDRD Director's R&D
• DOE Contract Number: AC05-00OR22725
• OSTI ID: 1131533
• Country of Publication: United States
• Language: English