Greatly improved cache update times for conditions data with Frontier/Squid
The CMS detector project loads copies of conditions data to over 100,000 computer cores worldwide by using a software subsystem called Frontier. This subsystem translates database queries into HTTP, looks up the results in a central database at CERN, and caches the results in an industry-standard HTTP proxy/caching server called Squid. One of the most challenging aspects of any cache system is coherency, that is, ensuring that changes made to the underlying data get propagated out to all clients in a timely manner. Recently, the Frontier system was enhanced to drastically reduce the time for changes to be propagated everywhere without heavily loading servers. The propagation time is now as low as 15 minutes for some kinds of data and no more than 60 minutes for the rest of the data. This was accomplished by taking advantage of an HTTP and Squid feature called If-Modified-Since. In order to use this feature, the Frontier server sends a Last-Modified timestamp, but since modification times are not normally tracked by Oracle databases, a PL/SQL program was developed to track the modification times of database tables. We discuss the details of this caching scheme and the obstacles overcome including database and Squid bugs.
- Research Organization:
- Fermi National Accelerator Lab. (FNAL), Batavia, IL (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC02-07CH11359
- OSTI ID:
- 960258
- Report Number(s):
- FERMILAB-CONF-09-231-CD-CMS; TRN: US0904593
- Resource Relation:
- Conference: Presented at 17th International Conference on Computing in High Energy and Nuclear Physics (CHEP 09), Prague, Czech Republic, 21-27 Mar 2009
- Country of Publication:
- United States
- Language:
- English
Similar Records
Understanding the evolution of conditions data access through Frontier for the ATLAS Experiment
CMS conditions data access using FroNTier