LazyBase: freshness vs. performance in information management

Information management applications exhibit a wide range of query performance and result freshness goals. Some applications, such as web search, require interactive performance, but may safely operate on stale data. Others, such as policy violation detection, require up-to-date results, but can tolerate relaxed performance goals. Furthermore, information processing applications must be able to ingest updates at the scale of an entire organization. In this paper, we present LazyBase, a system that allows users to trade off query performance and result freshness in order to satisfy the full range of users’ goals. LazyBase breaks up data ingestion into a pipeline of operations to minimize ingest time and uses models of processing and query performance to execute user queries. Initial results with LazyBase illustrate the feasibility of the pipelined model, highlight a rich space of trade-offs between result freshness and query performance, and often outperform existing solutions in the ...

• Type: Journal
• Year: 2010
• Where: SIGOPS
• Authors: Kimberly Keeton, Charles B. Morrey III, Craig A. N. Soules, Alistair C. Veitch
• Keywords: Performance, Policy Violation Detection, Query Performance, SIGOPS 2010