skip to main content

Title: Supporting large-scale computational science

A study has been carried out to determine the feasibility of using commercial database management systems (DBMSs) to support large-scale computational science. Conventional wisdom in the past has been that DBMSs are too slow for such data. Several events over the past few years have muddied the clarity of this mindset: 1. 2. 3. 4. Several commercial DBMS systems have demonstrated storage and ad-hoc quer access to Terabyte data sets. Several large-scale science teams, such as EOSDIS [NAS91], high energy physics [MM97] and human genome [Kin93] have adopted (or make frequent use of) commercial DBMS systems as the central part of their data management scheme. Several major DBMS vendors have introduced their first object-relational products (ORDBMSs), which have the potential to support large, array-oriented data. In some cases, performance is a moot issue. This is true in particular if the performance of legacy applications is not reduced while new, albeit slow, capabilities are added to the system. The basic assessment is still that DBMSs do not scale to large computational data. However, many of the reasons have changed, and there is an expiration date attached to that prognosis. This document expands on this conclusion, identifies the advantages and disadvantages ofmore » various commercial approaches, and describes the studies carried out in exploring this area. The document is meant to be brief, technical and informative, rather than a motivational pitch. The conclusions within are very likely to become outdated within the next 5-7 years, as market forces will have a significant impact on the state of the art in scientific data management over the next decade.« less
Authors:
Publication Date:
OSTI Identifier:
8429
Report Number(s):
UCRL-ID-129903; DP0101035
DP0101035; TRN: AH200117%%148
DOE Contract Number:
W-7405-ENG-48
Resource Type:
Technical Report
Resource Relation:
Other Information: PBD: 1 Oct 1998
Research Org:
Lawrence Livermore National Lab., CA (US)
Sponsoring Org:
USDOE Office of Defense Programs (DP) (US)
Country of Publication:
United States
Language:
English
Subject:
71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSICS; HIGH ENERGY PHYSICS; MANAGEMENT; MARKET; PERFORMANCE; STORAGE