Application of the SAMGrid test-harness for performance evaluation and tuning of a distributed cluster implementation of data handling services
The SAMGrid team has recently refactored its test harness suite for greater flexibility and easier configuration. This makes possible more interesting applications of the test harness, for component tests, integration tests, and stress tests. This new implementation of the test harness is a Python framework which uses XML for configuration and small plug-in python modules for specific test purposes We report on the architecture of the test harness and its recent application to stress tests of a new analysis cluster at Fermilab, to explore the extremes of analysis use cases and the relevant parameters for tuning in the SAMGrid station services. One current testing application is running on a 128-CPU analysis cluster with access to 6 TB distributed cache and also to a 2 TB centralized cache, permitting studies of different cache strategies. We have also studied the service parameters which affect the performance of retrieving data from tape storage. The use cases studied vary from those which will require rapid file delivery with short processing time per file, to the opposite extreme of long processing time per file. These results are interesting for their implications with regard to Grid operations, and illustrate the type of monitoring and test facilities required to accomplish such performance tuning.
- Research Organization:
- Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC02-76CH03000
- OSTI ID:
- 15016975
- Report Number(s):
- FERMILAB-CONF-04-454-CD; TRN: US200621%%451
- Resource Relation:
- Conference: Prepared for Computing in High-Energy Physics (CHEP '04), Interlaken, Switzerland, 27 Sep - 1 Oct 2004
- Country of Publication:
- United States
- Language:
- English
Similar Records
Center for Technology for Advanced Scientific Componet Software (TASCS)
The SAMGrid database server component: its upgraded infrastructure and future development path