skip to main content
DOE PAGES title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: The Modern Research Data Portal: a design pattern for networked, data-intensive science

Abstract

We describe best practices for providing convenient, high-speed, secure access to large data via research data portals. Here, we capture these best practices in a new design pattern, the Modern Research Data Portal, that disaggregates the traditional monolithic web-based data portal to achieve orders-of-magnitude increases in data transfer performance, support new deployment architectures that decouple control logic from data storage, and reduce development and operations costs. We introduce the design pattern; explain how it leverages high-performance data enclaves and cloud-based data management services; review representative examples at research laboratories and universities, including both experimental facilities and supercomputer sites; describe how to leverage Python APIs for authentication, authorization, data transfer, and data sharing; and use coding examples to demonstrate how these APIs can be used to implement a range of research data portal capabilities. Sample code at a companion web site,https://docs.globus.org/mrdp, provides application skeletons that readers can adapt to realize their own research data portals.

Authors:
 [1];  [2];  [1];  [1];  [1];  [1]
  1. Univ. of Chicago, IL (United States); Argonne National Lab. (ANL), Argonne, IL (United States)
  2. Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). Energy Sciences Network
Publication Date:
Research Org.:
Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States)
Sponsoring Org.:
USDOE Office of Science (SC)
OSTI Identifier:
1425432
Grant/Contract Number:  
AC02-05CH11231
Resource Type:
Accepted Manuscript
Journal Name:
PeerJ. Computer Science
Additional Journal Information:
Journal Volume: 4; Journal Issue: 1; Journal ID: ISSN 2376-5992
Publisher:
PeerJ Inc.
Country of Publication:
United States
Language:
English
Subject:
97 MATHEMATICS AND COMPUTING; Portal; High-speed network; Globus; Science DMZ; Data transfer node

Citation Formats

Chard, Kyle, Dart, Eli, Foster, Ian, Shifflett, David, Tuecke, Steven, and Williams, Jason. The Modern Research Data Portal: a design pattern for networked, data-intensive science. United States: N. p., 2018. Web. doi:10.7717/peerj-cs.144.
Chard, Kyle, Dart, Eli, Foster, Ian, Shifflett, David, Tuecke, Steven, & Williams, Jason. The Modern Research Data Portal: a design pattern for networked, data-intensive science. United States. doi:10.7717/peerj-cs.144.
Chard, Kyle, Dart, Eli, Foster, Ian, Shifflett, David, Tuecke, Steven, and Williams, Jason. Mon . "The Modern Research Data Portal: a design pattern for networked, data-intensive science". United States. doi:10.7717/peerj-cs.144. https://www.osti.gov/servlets/purl/1425432.
@article{osti_1425432,
title = {The Modern Research Data Portal: a design pattern for networked, data-intensive science},
author = {Chard, Kyle and Dart, Eli and Foster, Ian and Shifflett, David and Tuecke, Steven and Williams, Jason},
abstractNote = {We describe best practices for providing convenient, high-speed, secure access to large data via research data portals. Here, we capture these best practices in a new design pattern, the Modern Research Data Portal, that disaggregates the traditional monolithic web-based data portal to achieve orders-of-magnitude increases in data transfer performance, support new deployment architectures that decouple control logic from data storage, and reduce development and operations costs. We introduce the design pattern; explain how it leverages high-performance data enclaves and cloud-based data management services; review representative examples at research laboratories and universities, including both experimental facilities and supercomputer sites; describe how to leverage Python APIs for authentication, authorization, data transfer, and data sharing; and use coding examples to demonstrate how these APIs can be used to implement a range of research data portal capabilities. Sample code at a companion web site,https://docs.globus.org/mrdp, provides application skeletons that readers can adapt to realize their own research data portals.},
doi = {10.7717/peerj-cs.144},
journal = {PeerJ. Computer Science},
number = 1,
volume = 4,
place = {United States},
year = {2018},
month = {1}
}

Journal Article:
Free Publicly Available Full Text
Publisher's Version of Record

Save / Share: