GRACC: New generation of the OSG accounting

Retzke, K.; Weitzel, D.; Bhat, S.; Levshina, T.; Bockelman, B.; Jayatilaka, B.; Sehgal, C.; Quick, R.; Wuerthwein, F.

doi:10.1088/1742-6596/898/9/092044

Title: GRACC: New generation of the OSG accounting

Full Record
Other Related Research

Abstract

Throughout the last decade the Open Science Grid (OSG) has been fielding requests from user communities, resource owners, and funding agencies to provide information about utilization of OSG resources. Requested data include traditional accounting - core-hours utilized - as well as users certicate Distinguished Name, their affiliations, and eld of science. The OSG accounting service, Gratia, developed in 2006, is able to provide this information and much more. However, with the rapid expansion and transformation of the OSG resources and access to them, we are faced with several challenges in adapting and maintaining the current accounting service. The newest changes include, but are not limited to, acceptance of users from numerous university campuses, whose jobs are flocking to OSG resources, expansion into new types of resources (public and private clouds, allocation-based HPC resources, and GPU farms), migration to pilot-based systems, and migration to multicore environments. In order to have a scalable, sustainable and expandable accounting service for the next few years, we are embarking on the development of the next-generation OSG accounting service, GRACC, that will be based on open-source technology and will be compatible with the existing system. It will consist of swappable, independent components, such as Logstash, Elasticsearch,more »« less

Authors:

Retzke, K. ^[1]; Weitzel, D. ^[2]; Bhat, S. ^[1]; Levshina, T. ^[1]; Bockelman, B. ^[2]; Jayatilaka, B. ^[1]; Sehgal, C. ^[1]; Quick, R. ^[3]; Wuerthwein, F. ^[4]

Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
Univ. of Nebraska, Lincoln, NE (United States)
Indiana Univ., Bloomington, IN (United States)
Univ. of California, San Diego, CA (United States)

Publication Date:: 2017-10-01

Research Org.:: Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)

Sponsoring Org.:: USDOE Office of Science (SC), High Energy Physics (HEP)

OSTI Identifier:: 1346351

Report Number(s):: FERMILAB-CONF-17-054-CD
Journal ID: ISSN 1742-6588; 1516366

Grant/Contract Number:: AC02-07CH11359

Resource Type:: Accepted Manuscript

Journal Name:: Journal of Physics. Conference Series

Additional Journal Information:: Journal Volume: 898; Conference: CHEP 2016, San Francisco, CA (United States), 8-14 Oct 2016; Journal ID: ISSN 1742-6588

Publisher:: IOP Publishing

Country of Publication:: United States

Language:: English

Subject:: 97 MATHEMATICS AND COMPUTING

Citation Formats


                    Retzke, K., Weitzel, D., Bhat, S., Levshina, T., Bockelman, B., Jayatilaka, B., Sehgal, C., Quick, R., and Wuerthwein, F.. GRACC: New generation of the OSG accounting.  United States: N. p., 2017. 
Web.  doi:10.1088/1742-6596/898/9/092044.

Copy to clipboard


                    Retzke, K., Weitzel, D., Bhat, S., Levshina, T., Bockelman, B., Jayatilaka, B., Sehgal, C., Quick, R., & Wuerthwein, F.. GRACC: New generation of the OSG accounting.  United States.  https://doi.org/10.1088/1742-6596/898/9/092044

Copy to clipboard


                    Retzke, K., Weitzel, D., Bhat, S., Levshina, T., Bockelman, B., Jayatilaka, B., Sehgal, C., Quick, R., and Wuerthwein, F.. Sun .  
"GRACC: New generation of the OSG accounting".  United States.  https://doi.org/10.1088/1742-6596/898/9/092044.  https://www.osti.gov/servlets/purl/1346351.

Copy to clipboard


                    
@article{osti_1346351,

  title        = {GRACC: New generation of the OSG accounting},

  author       = {Retzke, K. and Weitzel, D. and Bhat, S. and Levshina, T. and Bockelman, B. and Jayatilaka, B. and Sehgal, C. and Quick, R. and Wuerthwein, F.},

  abstractNote = {Throughout the last decade the Open Science Grid (OSG) has been fielding requests from user communities, resource owners, and funding agencies to provide information about utilization of OSG resources. Requested data include traditional accounting - core-hours utilized - as well as users certicate Distinguished Name, their affiliations, and eld of science. The OSG accounting service, Gratia, developed in 2006, is able to provide this information and much more. However, with the rapid expansion and transformation of the OSG resources and access to them, we are faced with several challenges in adapting and maintaining the current accounting service. The newest changes include, but are not limited to, acceptance of users from numerous university campuses, whose jobs are flocking to OSG resources, expansion into new types of resources (public and private clouds, allocation-based HPC resources, and GPU farms), migration to pilot-based systems, and migration to multicore environments. In order to have a scalable, sustainable and expandable accounting service for the next few years, we are embarking on the development of the next-generation OSG accounting service, GRACC, that will be based on open-source technology and will be compatible with the existing system. It will consist of swappable, independent components, such as Logstash, Elasticsearch, Grafana, and RabbitMQ, that communicate through a data exchange. GRACC will continue to interface EGI and XSEDE accounting services and provide information in accordance with existing agreements. Lastly, we will present the current architecture and working prototype.},

  doi          = {10.1088/1742-6596/898/9/092044},

  journal      = {Journal of Physics. Conference Series},

  number       = ,

  volume       = 898,

  place        = {United States},

  year         = {2017},

  month        = {10}

}

Copy to clipboard

Journal Article:

Free Publicly Available Full Text

Accepted Manuscript (DOE)

Publisher's Version of Record

https://doi.org/10.1088/1742-6596/898/9/092044

Other availability

Search WorldCat to find libraries that may hold this journal

Save / Share:

Export Metadata

Save to My Library

Similar Records in DOE PAGES and OSTI.GOV collections:

Advances in Grid Computing for the FabrIc for Frontier Experiments Project at Fermialb

Conference Herner, K. ; Alba Hernandex, A. F. ; Bhat, S. ; ...

The FabrIc for Frontier Experiments (FIFE) project is a major initiative within the Fermilab Scientic Computing Division charged with leading the computing model for Fermilab experiments. Work within the FIFE project creates close collaboration between experimenters and computing professionals to serve high-energy physics experiments of diering size, scope, and physics area. The FIFE project has worked to develop common tools for job submission, certicate management, software and reference data distribution through CVMFS repositories, robust data transfer, job monitoring, and databases for project tracking. Since the projects inception the experiments under the FIFE umbrella have signicantly matured, and present an increasinglymore »« less
Full Text Available
Homogenizing OSG and XSEDE: Providing Access to XSEDE Allocations through OSG Infrastructure

Conference Thapa, Thapa,Suchandra ; Gardner, Robert Jr. ; Lesny, David ; ...

We present a system that allows individual researchers and virtual organizations (VOs) to access allocations on Stampede2 and Bridges through the Open Science Grid (OSG), a national grid infrastructure for running high throughput computing (HTC) tasks. Using this system, VOs and researchers are able to run larger workflows than can be done with OSG resources alone. This system allows a VO or user to run on XSEDE resources (with their allocation) using the same framework used with OSG resources. The system consists of two parts: the compute element (CE) that routes workloads to the appropriate user accounts and allocation onmore »« less
https://doi.org/10.1145/3219104.3219157

Full Text Available
Grid accounting service: state and future development

Conference Levshina, T. ; Sehgal, C. ; Bockelman, B. ; ... - J.Phys.Conf.Ser.

During the last decade, large-scale federated distributed infrastructures have been continually developed and expanded. One of the crucial components of a cyber-infrastructure is an accounting service that collects data related to resource utilization and identity of users using resources. The accounting service is important for verifying pledged resource allocation per particular groups and users, providing reports for funding agencies and resource providers, and understanding hardware provisioning requirements. It can also be used for end-to-end troubleshooting as well as billing purposes. In this work we describe Gratia, a federated accounting service jointly developed at Fermilab and Holland Computing Center at Universitymore »« less
https://doi.org/10.1088/1742-6596/513/3/032056

Full Text Available
Supporting shared resource usage for a diverse user community: The OSG experience and lessons learned

Conference Garzoglio, Gabriele ; Levshina, Tanya ; Rynge, Mats ; ... - J.Phys.Conf.Ser.

The Open Science Grid (OSG) supports a diverse community of new and existing users in adopting and making effective use of the Distributed High Throughput Computing (DHTC) model. The LHC user community has deep local support within the experiments. For other smaller communities and individual users the OSG provides consulting and technical services through the User Support area. We describe these sometimes successful and sometimes not so successful experiences and analyze lessons learned that are helping us improve our services. The services offered include forums to enable shared learning and mutual support, tutorials and documentation for new technology, and troubleshootingmore »« less
https://doi.org/10.1088/1742-6596/396/3/032046

Full Text Available
The OSG Open Facility: an on-ramp for opportunistic scientific computing

Journal Article Jayatilaka, B. ; Levshina, T. ; Sehgal, C. ; ... - Journal of Physics. Conference Series

The Open Science Grid (OSG) is a large, robust computing grid that started primarily as a collection of sites associated with large HEP experiments such as ATLAS, CDF, CMS, and DZero, but has evolved in recent years to a much larger user and resource platform. In addition to meeting the US LHC community’s computational needs, the OSG continues to be one of the largest providers of distributed high-throughput computing (DHTC) to researchers from a wide variety of disciplines via the OSG Open Facility. The Open Facility consists of OSG resources that are available opportunistically to users other than resource ownersmore »« less
https://doi.org/10.1088/1742-6596/898/8/082048

Full Text Available

Similar Records