Sciweavers

OSDI
2004
ACM

Understanding and Dealing with Operator Mistakes in Internet Services

14 years 3 months ago
Understanding and Dealing with Operator Mistakes in Internet Services
Operator mistakes are a significant source of unavailability in modern Internet services. In this paper, we first characterize these mistakes by performing an extensive set of experiments using human operators and a realistic three-tier auction service. The mistakes we observed range from software misconfiguration, to fault misdiagnosis, to incorrect software restarts. We next propose to validate operator actions before they are made visible to the rest of the system. We demonstrate how to accomplish this task via the creation of a validation environment that is an extension of the online system, where components can be validated using real workloads before they are migrated into the running service. We show that our prototype validation system can detect 66% of the operator mistakes that we have observed.
Fábio Oliveira, Kiran Nagaraja, Ricardo Bia
Added 03 Dec 2009
Updated 03 Dec 2009
Type Conference
Year 2004
Where OSDI
Authors Fábio Oliveira, Kiran Nagaraja, Ricardo Bianchini, Richard P. Martin, Thu D. Nguyen
Comments (0)