Sciweavers

DSN
2009
IEEE

System log pre-processing to improve failure prediction

13 years 10 months ago
System log pre-processing to improve failure prediction
Log preprocessing, a process applied on the raw log before applying a predictive method, is of paramount importance to failure prediction and diagnosis. While existing filtering methods have demonstrated good compression rate, they fail to preserve important failure patterns that are crucial for failure analysis. To address the problem, in this paper we present a log preprocessing method. It consists of three integrated steps: (1) event categorization to uniformly classify system events and identify fatal events; (2) event filtering to remove temporal and spatial redundant records, while also preserving necessary failure patterns for failure analysis; (3) causality-related filtering to combine correlated events for filtering through apriori association rule mining. We demonstrate the effectiveness of our preprocessing method by using real failure logs collected from the Cray XT4 at ORNL and the Blue Gene/L system at SDSC. Experiments show that our method can preserve more failure ...
Ziming Zheng, Zhiling Lan, Byung-Hoon Park, Al Gei
Added 20 May 2010
Updated 20 May 2010
Type Conference
Year 2009
Where DSN
Authors Ziming Zheng, Zhiling Lan, Byung-Hoon Park, Al Geist
Comments (0)