skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Framework for Interactive Parallel Dataset Analysis on the Grid

Abstract

We present a framework for use at a typical Grid site to facilitate custom interactive parallel dataset analysis targeting terabyte-scale datasets of the type typically produced by large multi-institutional science experiments. We summarize the needs for interactive analysis and show a prototype solution that satisfies those needs. The solution consists of desktop client tool and a set of Web Services that allow scientists to sign onto a Grid site, compose analysis script code to carry out physics analysis on datasets, distribute the code and datasets to worker nodes, collect the results back to the client, and to construct professional-quality visualizations of the results.

Authors:
; ; ; ; ;
Publication Date:
Research Org.:
Stanford Linear Accelerator Center (SLAC)
Sponsoring Org.:
USDOE
OSTI Identifier:
897459
Report Number(s):
SLAC-PUB-12289
TRN: US200705%%330
DOE Contract Number:
AC02-76SF00515
Resource Type:
Conference
Resource Relation:
Conference: To appear in the proceedings of 2006 International Conference on Parallel Processing Workshops (ICPPW 06), Columbus, Ohio, 14-18 Aug 2006
Country of Publication:
United States
Language:
English
Subject:
99 GENERAL AND MISCELLANEOUS//MATHEMATICS, COMPUTING, AND INFORMATION SCIENCE; PARALLEL PROCESSING; PHYSICS; GRIDS; Computing,HEPEX

Citation Formats

Alexander, David A., Ananthan, Balamurali, /Tech-X Corp., Johnson, Tony, Serbo, Victor, and /SLAC. Framework for Interactive Parallel Dataset Analysis on the Grid. United States: N. p., 2007. Web.
Alexander, David A., Ananthan, Balamurali, /Tech-X Corp., Johnson, Tony, Serbo, Victor, & /SLAC. Framework for Interactive Parallel Dataset Analysis on the Grid. United States.
Alexander, David A., Ananthan, Balamurali, /Tech-X Corp., Johnson, Tony, Serbo, Victor, and /SLAC. Wed . "Framework for Interactive Parallel Dataset Analysis on the Grid". United States. doi:. https://www.osti.gov/servlets/purl/897459.
@article{osti_897459,
title = {Framework for Interactive Parallel Dataset Analysis on the Grid},
author = {Alexander, David A. and Ananthan, Balamurali and /Tech-X Corp. and Johnson, Tony and Serbo, Victor and /SLAC},
abstractNote = {We present a framework for use at a typical Grid site to facilitate custom interactive parallel dataset analysis targeting terabyte-scale datasets of the type typically produced by large multi-institutional science experiments. We summarize the needs for interactive analysis and show a prototype solution that satisfies those needs. The solution consists of desktop client tool and a set of Web Services that allow scientists to sign onto a Grid site, compose analysis script code to carry out physics analysis on datasets, distribute the code and datasets to worker nodes, collect the results back to the client, and to construct professional-quality visualizations of the results.},
doi = {},
journal = {},
number = ,
volume = ,
place = {United States},
year = {Wed Jan 10 00:00:00 EST 2007},
month = {Wed Jan 10 00:00:00 EST 2007}
}

Conference:
Other availability
Please see Document Availability for additional information on obtaining the full-text document. Library patrons may search WorldCat to identify libraries that hold this conference proceeding.

Save / Share:
  • A framework for parallel visualization at Pacific Northwest National Laboratory (PNNL) is being developed that utilizes the IBM Scaleable Graphics Engine (SGE) and IBM SP parallel computers. The SGE allows disjoint regions of pixel data to be transferred simultaneously from multiple compute nodes into a unified frame buffer. The joined graphics data is displayed on monitors attached to the SGE. Three parallel applications have been developed that write pixel data directly to local buffers and transfer the buffers to the SGE. A library is being developed to allow OpenGL applications to run in parallel and utilize the SGE. The librarymore » and SGE hardware will be an interactive framework for parallel visualization applications.« less
  • For today's physicists, who work in large geographically distributed collaborations, the data grid promises significantly greater capabilities for analysis of experimental data and production of physics results than is possible with today's ''remote access'' technologies. The goal of letting scientists at their home institutions interact with and analyze data as if they were physically present at the major laboratory that houses their detector and computer center has yet to be accomplished. The Particle Physics DataGrid project (www.ppdg.net) has recently embarked on an effort to ''Interface and Integrate Interactive Data Analysis Tools with the grid and identify Common Components and Services.''more » The initial activities are to collect known and identify new requirements for grid services and analysis tools from a range of current and future experiments (ALICE, ATLAS, BaBar, D0, CMS, JLab, STAR, others welcome), to determine if existing plans for tools and services meet these requirements. Follow-on activities will foster the interaction between grid service developers, analysis tool developers, experiment analysis frame work developers and end user physicists, and will identify and carry out specific development/integration work so that interactive analysis tools utilizing grid services actually provide the capabilities that users need. This talk will summarize what we know of requirements for analysis tools and grid services, as well as describe the identified areas where more development work is needed.« less
  • We demonstrate an interactive visualization and analysis system for integrating climate data with other geospatial data sets, such as environmental and demographic data. The \eviz system is a desktop-based visualization and analysis environment which allows seamless integration of multiple geospatial data sets from varied sources and provides an interface to interactively analyze the different data sets and apply sophisticated data analysis and mining algorithms in a near real time fashion. The framework is highly desirable in domains such as earth and climate sciences where great emphasis is placed on simultaneous analysis of different data sets such as remote sensing images,more » climate model simulation outputs, and other environmental and demographic databases, to understand weather and climate systems and the impact of climate change on nature and people.« less
  • A parallel array of eight minicomputers has been assembled in an attempt to deal with kiloparameter data events. By exporting computer system functions to a separate processor, the authors have been able to achieve computer amplification linearly proportional to the number of executing processors.