HPC top 10 InfiniBand Machine : a 3D Torus IB interconnect on Red Sky.

Conference · Mon Mar 01 04:00:00 EST 2010

OSTI ID:988974

Monk, Stephen Todd; Rajan, Mahesh; Epperson, Marcus R; Schutt, James Alan; Bohnsack, Matthew Paul; Doerfler, Douglas W; Naegle, John Hunt

This presentation discusses the following topics: (1) Red Sky Background; (2) 3D Torus Interconnect Concepts; (3) Difficulties of Torus in IB; (4) New Routing Code for IB a 3D Torus; (5) Red Sky 3D Torus Implementation; and (6) Managing a Large IB Machine. Computing at Sandia: (1) Capability Computing - Designed for scaling of single large runs, Usually proprietary for maximum performance, and Red Storm is Sandia's current capability machine; (2) Capacity Computing - Computing for the masses, 100s of jobs and 100s of users, Extreme reliability required, Flexibility for changing workload, Thunderbird will be decommissioned this quarter, Red Sky is our future capacity computing platform, and Red Mesa machine for National Renewable Energy Lab. Red Sky main themes are: (1) Cheaper - 5X capacity of Tbird at 2/3 the cost, Substantially cheaper per flop than our last large capacity machine purchase; (2) Leaner - Lower operational costs, Three security environments via modular fabric, Expandable, upgradeable, extensible, and Designed for 6yr. life cycle; and (3) Greener - 15% less power-1/6th power per flop, 40% less water-5M gallons saved annually, 10X better cooling efficiency, and 4x denser footprint.

• Research Organization: Sandia National Laboratories
• Sponsoring Organization: USDOE
• DOE Contract Number: AC04-94AL85000
• OSTI ID: 988974
• Report Number(s): SAND2010-1502C
• Country of Publication: United States
• Language: English

Related Subjects

99 GENERAL AND MISCELLANEOUS
CAPACITY
EFFICIENCY
FLEXIBILITY
IMPLEMENTATION
LIFE CYCLE
PERFORMANCE
RELIABILITY
ROUTING
SECURITY
SKY
STORMS