skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Preparing NERSC users for Cori, a Cray XC40 system with Intel many integrated cores

Abstract

The newest NERSC supercomputer Cori is a Cray XC40 system consisting of 2,388 Intel Xeon Haswell nodes and 9,688 Intel Xeon-Phi “Knights Landing” (KNL) nodes. Compared to the Xeon-based clusters NERSC users are familiar with, optimal performance on Cori requires consideration of KNL mode settings; process, thread, and memory affinity; fine-grain parallelization; vectorization; and use of the high-bandwidth MCDRAM memory. This paper describes our efforts preparing NERSC users for KNL through the NERSC Exascale Science Application Program, Web documentation, and user training. We discuss how we configured the Cori system for usability and productivity, addressing programming concerns, batch system configurations, and default KNL cluster and memory modes. Here, system usage data, job completion analysis, programming and running jobs issues, and a few successful user stories on KNL are presented.

Authors:
ORCiD logo [1];  [1];  [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1]; ORCiD logo [1];  [1];  [1]; ORCiD logo [2];  [3]
  1. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
  2. Univ. of Oklahoma, Norman, OK (United States)
  3. Hamburger Sternwarte, Hamburg (Germany)
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1459400
Grant/Contract Number:  
AC02-05CH11231
Resource Type:
Accepted Manuscript
Journal Name:
Concurrency and Computation. Practice and Experience
Additional Journal Information:
Journal Volume: 30; Journal Issue: 1; Related Information: Copyright © 2017 John Wiley & Sons, Ltd.; Journal ID: ISSN 1532-0626
Publisher:
Wiley
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; cross compilation; Intel Xeon Phi; KNL; performance optimization; process and thread affinity; user support; training; web documentation; heterogeneous; cluster and memory modes; NESAP

Citation Formats

He, Yun, Cook, Brandon, Deslippe, Jack, Friesen, Brian, Gerber, Richard, Hartman-Baker, Rebecca, Koniges, Alice, Kurth, Thorsten, Leak, Stephen, Yang, Woo -Sun, Zhao, Zhengji, Baron, Eddie, and Hauschildt, Peter. Preparing NERSC users for Cori, a Cray XC40 system with Intel many integrated cores. United States: N. p., 2017. Web. doi:10.1002/cpe.4291.
He, Yun, Cook, Brandon, Deslippe, Jack, Friesen, Brian, Gerber, Richard, Hartman-Baker, Rebecca, Koniges, Alice, Kurth, Thorsten, Leak, Stephen, Yang, Woo -Sun, Zhao, Zhengji, Baron, Eddie, & Hauschildt, Peter. Preparing NERSC users for Cori, a Cray XC40 system with Intel many integrated cores. United States. doi:10.1002/cpe.4291.
He, Yun, Cook, Brandon, Deslippe, Jack, Friesen, Brian, Gerber, Richard, Hartman-Baker, Rebecca, Koniges, Alice, Kurth, Thorsten, Leak, Stephen, Yang, Woo -Sun, Zhao, Zhengji, Baron, Eddie, and Hauschildt, Peter. Fri . "Preparing NERSC users for Cori, a Cray XC40 system with Intel many integrated cores". United States. doi:10.1002/cpe.4291. https://www.osti.gov/servlets/purl/1459400.
@article{osti_1459400,
title = {Preparing NERSC users for Cori, a Cray XC40 system with Intel many integrated cores},
author = {He, Yun and Cook, Brandon and Deslippe, Jack and Friesen, Brian and Gerber, Richard and Hartman-Baker, Rebecca and Koniges, Alice and Kurth, Thorsten and Leak, Stephen and Yang, Woo -Sun and Zhao, Zhengji and Baron, Eddie and Hauschildt, Peter},
abstractNote = {The newest NERSC supercomputer Cori is a Cray XC40 system consisting of 2,388 Intel Xeon Haswell nodes and 9,688 Intel Xeon-Phi “Knights Landing” (KNL) nodes. Compared to the Xeon-based clusters NERSC users are familiar with, optimal performance on Cori requires consideration of KNL mode settings; process, thread, and memory affinity; fine-grain parallelization; vectorization; and use of the high-bandwidth MCDRAM memory. This paper describes our efforts preparing NERSC users for KNL through the NERSC Exascale Science Application Program, Web documentation, and user training. We discuss how we configured the Cori system for usability and productivity, addressing programming concerns, batch system configurations, and default KNL cluster and memory modes. Here, system usage data, job completion analysis, programming and running jobs issues, and a few successful user stories on KNL are presented.},
doi = {10.1002/cpe.4291},
journal = {Concurrency and Computation. Practice and Experience},
number = 1,
volume = 30,
place = {United States},
year = {2017},
month = {8}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Citation Metrics:
Cited by: 1 work
Citation information provided by
Web of Science

Save / Share:

Works referenced in this record:

Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set
journal, July 1996