Abstract
The NetLogger Toolkit is designed to monitor, under actual operating conditions, the behavior of all the elements of the application-to-application communication path in order to determine exactly where time is spent within a complex system. Using NetLogger, distributed application components are modified to produce timestamped logs of "interesting" events at all the critical points of the distributed system. Events from each component are correlated, which allows one to characterize the performance of all aspects of the system and network in detail. The NetLogger Toolkit itself consists of four components an API and library of functions to simplify the generation of application-level event logs, a set of tools for collecting and sorting log files, an event archive system, and a tool for visualization and analysis of the log files. In order to instrument an application to produce event logs, the application developer inserts calls to the NetLogger API at all the critical points in the code, then links the application with the NetLogger library. All the tools in the NetLogger Toolkit share a common log format, and assume the existence of accurate and synchronized system clocks. NetLogger messages can be logged using an easy-to-read text based format based on the lETF-proposed
More>>
- Developers:
- Release Date:
- 2003-03-28
- Project Type:
- Open Source, No Publicly Available Repository
- Software Type:
- Scientific
- Licenses:
-
Other (Commercial or Open-Source): http://netlogger.lbl.gov/software/netlogger-license
- Sponsoring Org.:
-
USDOEPrimary Award/Contract Number:AC03-76SF00098B&R Codes:KJ0102000
- Code ID:
- 56969
- Site Accession Number:
- 4077; CR-1503
- Research Org.:
- Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)
- Country of Origin:
- United States
Citation Formats
Gunter, Dan, Lee, Jason, Stoufer, Martin, and Tierney, Brian.
The NetLogger Toolkit V2.0.
Computer Software.
USDOE.
28 Mar. 2003.
Web.
doi:10.11578/dc.20210521.21.
Gunter, Dan, Lee, Jason, Stoufer, Martin, & Tierney, Brian.
(2003, March 28).
The NetLogger Toolkit V2.0.
[Computer software].
https://doi.org/10.11578/dc.20210521.21.
Gunter, Dan, Lee, Jason, Stoufer, Martin, and Tierney, Brian.
"The NetLogger Toolkit V2.0." Computer software.
March 28, 2003.
https://doi.org/10.11578/dc.20210521.21.
@misc{
doecode_56969,
title = {The NetLogger Toolkit V2.0},
author = {Gunter, Dan and Lee, Jason and Stoufer, Martin and Tierney, Brian},
abstractNote = {The NetLogger Toolkit is designed to monitor, under actual operating conditions, the behavior of all the elements of the application-to-application communication path in order to determine exactly where time is spent within a complex system. Using NetLogger, distributed application components are modified to produce timestamped logs of "interesting" events at all the critical points of the distributed system. Events from each component are correlated, which allows one to characterize the performance of all aspects of the system and network in detail. The NetLogger Toolkit itself consists of four components an API and library of functions to simplify the generation of application-level event logs, a set of tools for collecting and sorting log files, an event archive system, and a tool for visualization and analysis of the log files. In order to instrument an application to produce event logs, the application developer inserts calls to the NetLogger API at all the critical points in the code, then links the application with the NetLogger library. All the tools in the NetLogger Toolkit share a common log format, and assume the existence of accurate and synchronized system clocks. NetLogger messages can be logged using an easy-to-read text based format based on the lETF-proposed ULM format, or a binary format that can still be used through the same API but that is several times faster and smaller, with performance comparable or better than binary message formats such as MPI, XDR, SDDF-Binary, and PBIO. The NetLogger binary format is both highly efficient and self-describing, thus optimized for the dynamic message construction and parsing of application instrumentation. NetLogger includes an "activation" API that allows NetLogger logging to be turned on, off, or modified by changing an external file This IS useful for activating logging in daemons/services (e g GndFTP server). The NetLogger reliability API provides the ability to specify backup logging locations and periodically try to reconnect broken TCP pipe. A typical use for this is to store data on local disk while net is down. An event archiver can log one or more incoming NetLogger streams to a local disk file (netlogd) or to a mySQL database (netarchd). We have found exploratory, visual analysis of the log event data to be the most useful means of determining the causes of performance anomalies The NetLogger Visualization tool, niv, has been developed to provide a flexible and interactive graphical representation of system-level and application-level events.},
doi = {10.11578/dc.20210521.21},
url = {https://doi.org/10.11578/dc.20210521.21},
howpublished = {[Computer Software] \url{https://doi.org/10.11578/dc.20210521.21}},
year = {2003},
month = {mar}
}