Bridging the gaps : joining information sources with Splunk.
Supercomputers are composed of many diverse components, operated at a variety of scales, and function as a coherent whole. The resulting logs are thus diverse in format, interrelated at multiple scales, and provide evidence of faults across subsystems. When combined with system configuration information, insights on both the downstream effects and upstream causes of events can be determined. However, difficulties in joining the data and expressing complex queries slow the speed at which actionable insights can be obtained. Effectively connecting data experts and data miners faces similar hurdles. This paper describes our experience with applying the Splunk log analysis tool as a vehicle to combine both data, and people. Splunk's search language, lookups, macros, and subsearches reduce hours of tedium to seconds of simplicity, and its tags, saved searches, and dashboards offer both operational insights and collaborative vehicles.
- Research Organization:
- Sandia National Laboratories (SNL), Albuquerque, NM, and Livermore, CA (United States)
- Sponsoring Organization:
- USDOE
- DOE Contract Number:
- AC04-94AL85000
- OSTI ID:
- 1028434
- Report Number(s):
- SAND2010-7414C; TRN: US201122%%302
- Resource Relation:
- Conference: Proposed for presentation at the Workshop on Managing Systems via Log Analysis and Machine Learning Techniques (SLAML '10) held October 3, 2010 in Vancouver, CA.
- Country of Publication:
- United States
- Language:
- English
Similar Records
Using Splunk® Enterprise Search Commands for Advanced Analysis of Ivanti Connect Secure© Logs
Domain-specific chatbots for science using embeddings