Exploring Two Approaches for an End-to-End Scientific Analysis Workflow
- Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States); Univ. of Chicago, IL (United States)
- Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
The advance of the scientific discovery process is accomplished by the integration of independently-developed programs run on disparate computing facilities into coherent workflows usable by scientists who are not experts in computing. For such advancement, we need a system which scientists can use to formulate analysis workflows, to integrate new components to these workflows, and to execute different components on resources that are best suited to run those components. In addition, we need to monitor the status of the workflow as components get scheduled and executed, and to access the intermediate and final output for visual exploration and analysis. Finally, it is important for scientists to be able to share their workflows with collaborators. Moreover we have explored two approaches for such an analysis framework for the Large Synoptic Survey Telescope (LSST) Dark Energy Science Collaboration (DESC), the first one is based on the use and extension of Galaxy, a web-based portal for biomedical research, and the second one is based on a programming language, Python. In our paper, we present a brief description of the two approaches, describe the kinds of extensions to the Galaxy system we have found necessary in order to support the wide variety of scientific analysis in the cosmology community, and discuss how similar efforts might be of benefit to the HEP community.
- Research Organization:
- Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC), High Energy Physics (HEP)
- Grant/Contract Number:
- AC02-07CH11359
- OSTI ID:
- 1260268
- Report Number(s):
- FERMILAB-PUB-15-608-CD; 1413975
- Journal Information:
- Journal of Physics. Conference Series, Vol. 664, Issue 6; ISSN 1742-6588
- Publisher:
- IOP PublishingCopyright Statement
- Country of Publication:
- United States
- Language:
- English
Similar Records
Numerical Codes for the DESC-LSST Analysis Pipeline: Core Cosmology Library Standard Modules and Beyond wCDM Modules (Final Technical Report)
Probing the Physics of Dark Energy with LSST: Large Scale Structure and Supernovae