Sciweavers

Share
HIPC
2009
Springer

Extracting the textual and temporal structure of supercomputing logs

8 years 8 months ago
Extracting the textual and temporal structure of supercomputing logs
Supercomputers are prone to frequent faults that adversely affect their performance, reliability and functionality. System logs collected on these systems are a valuable resource of information about their operational status and health. However, their massive size, complexity, and lack of standard format makes it difficult to automatically extract information that can be used to improve system management. In this work we propose a novel method to succinctly represent the contents of supercomputing logs, by using textual clustering to automatically find the syntactic structures of log messages. This information is used to automatically classify messages into semantic groups via an online clustering algorithm. Further, we describe a methodology for using the temporal proximity between groups of log messages to identify correlated events in the system. We apply our proposed methods to two large, publicly available supercomputing logs and show that our technique features nearly perfect acc...
Sourabh Jain, Inderpreet Singh, Abhishek Chandra,
Added 18 Feb 2011
Updated 18 Feb 2011
Type Journal
Year 2009
Where HIPC
Authors Sourabh Jain, Inderpreet Singh, Abhishek Chandra, Zhi-Li Zhang, Greg Bronevetsky
Comments (0)
books