Distributed processing and analysis of physics data in the D0 SAM system at Fermilab
- Fermilab
SAM (Sequential Access through Meta-data) is the data access system for the D0 high energy physics (HEP) experiment at Fermilab. The system is being developed and used to handle the Petabyte-scale experiment data. The D0 applications, like virtually all HEP applications, are data-intensive, which poses special problems for the data management and job control facilities in the distributed environment. The fundamental problem is to bring the user applications and the data together, and SAM attacks the problems from both sides. First, we describe how the system moves the data through the distributed disk cache. Second, we describe how SAM interacts with the batch system to synchronize parallel user jobs with the data availability. All the design solutions herein have been implemented in a real system that handles the mission-critical data of the D0 experiment; thus, we present our work from the standpoint of real experience.
- Research Organization:
- Fermi National Accelerator Lab., Batavia, IL (US)
- Sponsoring Organization:
- USDOE Office of Energy Research (ER) (US)
- DOE Contract Number:
- AC02-76CH03000
- OSTI ID:
- 785150
- Report Number(s):
- FERMILAB-TM-2156
- Country of Publication:
- United States
- Language:
- English
Similar Records
Distributed data access and resource management in the D0 SAM system
The D0 Experiment Data Grid: SAM
Distributed data access in the sequential access model at the D0 experiment at Fermilab
Conference
·
Tue Jun 26 00:00:00 EDT 2001
·
OSTI ID:781989
The D0 Experiment Data Grid: SAM
Journal Article
·
Sat Sep 01 00:00:00 EDT 2001
· Lect.Notes Comput.Sci.
·
OSTI ID:1900204
Distributed data access in the sequential access model at the D0 experiment at Fermilab
Conference
·
Wed Jul 05 00:00:00 EDT 2000
·
OSTI ID:757586