Designing a multi-petabyte database for LSST
The 3.2 giga-pixel LSST camera will produce over half a petabyte of raw images every month. This data needs to be reduced in under a minute to produce real-time transient alerts, and then cataloged and indexed to allow efficient access and simplify further analysis. The indexed catalogs alone are expected to grow at a speed of about 600 terabytes per year. The sheer volume of data, the real-time transient alerting requirements of the LSST, and its spatio-temporal aspects require cutting-edge techniques to build an efficient data access system at reasonable cost. As currently envisioned, the system will rely on a database for catalogs and metadata. Several database systems are being evaluated to understand how they will scale and perform at these data volumes in anticipated LSST access patterns. This paper describes the LSST requirements, the challenges they impose, the data access philosophy, and the database architecture that is expected to be adopted in order to meet the data challenges.
- Research Organization:
- Lawrence Livermore National Lab. (LLNL), Livermore, CA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- W-7405-ENG-48
- OSTI ID:
- 883571
- Report Number(s):
- UCRL-CONF-217858; TRN: US200615%%141
- Resource Relation:
- Journal Volume: 6270; Conference: Presented at: Designing a multi-petabyte database for LSST, Orlando, FL, United States, May 24 - May 31, 2005
- Country of Publication:
- United States
- Language:
- English
Similar Records
Organizing the Extremely Large LSST Database forReal-Time Astronomical Processing
A Wide New Window on the Universe