Designing a multi-petabyte database for LSST

Conference · Wed Dec 21 04:00:00 EST 2005

DOI:https://doi.org/10.1117/12.671721· OSTI ID:883571

Becla, J; Hanushevsky, A

The 3.2 giga-pixel LSST camera will produce over half a petabyte of raw images every month. This data needs to be reduced in under a minute to produce real-time transient alerts, and then cataloged and indexed to allow efficient access and simplify further analysis. The indexed catalogs alone are expected to grow at a speed of about 600 terabytes per year. The sheer volume of data, the real-time transient alerting requirements of the LSST, and its spatio-temporal aspects require cutting-edge techniques to build an efficient data access system at reasonable cost. As currently envisioned, the system will rely on a database for catalogs and metadata. Several database systems are being evaluated to understand how they will scale and perform at these data volumes in anticipated LSST access patterns. This paper describes the LSST requirements, the challenges they impose, the data access philosophy, and the database architecture that is expected to be adopted in order to meet the data challenges.

• Research Organization: Lawrence Livermore National Laboratory (LLNL), Livermore, CA
• Sponsoring Organization: USDOE
• DOE Contract Number: W-7405-ENG-48
• OSTI ID: 883571
• Report Number(s): UCRL-CONF-217858
• Country of Publication: United States
• Language: English

Related Subjects

99 GENERAL AND MISCELLANEOUS
ARCHITECTURE
CAMERAS
TRANSIENTS
VELOCITY