Designing a Multi-Petabyte Database for LSST

Conference · Wed Jan 10 04:00:00 EST 2007

OSTI ID:897453

Becla, Jacek; Hanushevsky, Andrew; Nikolaev, Sergei; Abdulla, Ghaleb; Szalay, Alex; Nieto-Santisteban, Maria; Thakar, Ani; Gray, Jim

The 3.2 giga-pixel LSST camera will produce approximately half a petabyte of archive images every month. These data need to be reduced in under a minute to produce real-time transient alerts, and then added to the cumulative catalog for further analysis. The catalog is expected to grow about three hundred terabytes per year. The data volume, the real-time transient alerting requirements of the LSST, and its spatio-temporal aspects require innovative techniques to build an efficient data access system at reasonable cost. As currently envisioned, the system will rely on a database for catalogs and metadata. Several database systems are being evaluated to understand how they perform at these data rates, data volumes, and access patterns. This paper describes the LSST requirements, the challenges they impose, the data access philosophy, results to date from evaluating available database technologies against LSST requirements, and the proposed database architecture to meet the data challenges.

• Research Organization: Stanford Linear Accelerator Center (SLAC)
• Sponsoring Organization: USDOE
• DOE Contract Number: AC02-76SF00515
• OSTI ID: 897453
• Report Number(s): SLAC-PUB-12292
• Country of Publication: United States
• Language: English

Related Subjects

71 CLASSICAL AND QUANTUM MECHANICS
GENERAL PHYSICS
99 GENERAL AND MISCELLANEOUS
ASTROPHYSICS
Accelerators
ASTRO
CAMERAS
DATA BASE MANAGEMENT
DESIGN
INFORMATION SYSTEMS
TELESCOPES