Interactive Supercomputing With Jupyter

Journal Article · Mon Mar 01 04:00:00 EST 2021 · Computing in Science and Engineering

DOI:https://doi.org/10.1109/mcse.2021.3059037· OSTI ID:1826909

Thomas, Rollin; Cholia, Shreyas; Mohror, Kathryn; Shalf, John M.

Rich user interfaces like Jupyter have the potential to make interacting with a supercomputer easier and more productive, consequently attracting new kinds of users and helping to expand the application of supercomputing to new science domains. For the scientist-user, the ideal rich user interface delivers a familiar, responsive, introspective, modular, and customizable platform upon which to build, run, capture, document, re-run, and share analysis workflows. From the provider or system administrator perspective, such a platform would also be easy to configure, deploy securely, update, customize, and support. Jupyter checks most if not all of these boxes. But from the perspective of leadership computing organizations that provide supercomputing power to users, such a platform should also make the unique features of a supercomputer center more accessible to users and more composable with high performance computing (HPC) workflows. Project Jupyter’s core design philosophy of extensibility, abstraction, and agnostic deployment, has allowed HPC centers like NERSC to bring in advanced supercomputing capabilities that can extend the interactive notebook environment. This has enabled a rich scientific discovery platform, particularly for experimental facility data analysis and machine learning problems.

• Research Organization: Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Basic Energy Sciences (BES)
• Grant/Contract Number: AC02-05CH11231
• OSTI ID: 1826909
• Alternate ID(s): OSTI ID: 1826911
• Journal Information: Computing in Science and Engineering, Journal Name: Computing in Science and Engineering Journal Issue: 2 Vol. 23; ISSN 1521-9615
• Publisher: IEEE Computer Society
• Country of Publication: United States
• Language: English