Experimental Data Curation at Large Instrument Facilities with Open Source Software

Journal Article · 11 September 2019 · International Journal of Digital Curation

DOI: https://doi.org/10.2218/ijdc.v14i1.637 · OSTI ID:1561332

Pouchard, Line  ^[1]; Kleese van Dam, Kerstin  ^[2]; Campbell, Stuart I.  ^[2]

---

1. Brookhaven National Lab. (BNL), Upton, NY (United States); Purdue Univ., West Lafayette, IN (United States)
2. Brookhaven National Lab. (BNL), Upton, NY (United States)

The National Synchrotron Light Source II operating at Brookhaven National Laboratory since 2014 for the US Department of Energy is one of the newest and brightest storage-ring synchrotron facility in the world.  NSLS-II, like other facilities, provides pre-processing of the raw data and some analysis capabilities to its users. We describe the research collaborations and open source infrastructure  developed at large instrument facilities such as NSLS-II for the purpose of curating high value scientific data along the early stages of the data lifecycle.  Data acquisition and curation tasks include storing experiment configuration, detector metadata, raw data acquisition with infrastructure that converts proprietary instrument formats to industry standards.  In addition, we describe a specific effort for discovering sample information at NSLS-II and tracing the provenance of analysis performed on acquired images.  We show that curation tasks must be embedded into software along the data life cycle for effectiveness and ease of use, and that loosely defined collaborations evolve around shared open source tools.  Finally we discuss best practices for experimental metadata capture in such facilities, data access and the new challenges of scale and complexity posed by AI-based discovery for the synthesis of new materials.

• Research Organization: Energy Frontier Research Centers (EFRC) (United States). A Next Generation Synthesis Center (GENESIS); State Univ. of New York (SUNY), Syracuse, NY (United States); Brookhaven National Laboratory (BNL), Upton, NY (United States)
• Sponsoring Organization: USDOE Office of Science (SC), Basic Energy Sciences (BES)
• Grant/Contract Number: SC0019212; SC0012704
• OSTI ID: 1561332
• Alternate ID(s): OSTI ID: 1597286
• Journal Information: International Journal of Digital Curation, Vol. 14, Issue 1; ISSN 1746-8256
• Publisher: Edinburgh UniversityCopyright Statement
• Country of Publication: United States
• Language: English