A Lightweight Evaluation Framework for Table Layouts in MapReduce Based Query Systems

8 years 4 days ago

Download files.cnblogs.com

Abstract. Table layout determines the way how the relational rowcolumn data values are organized and stored. In recent years, considerable candidates have been developed in MapReduce based query systems; they diﬀer on storage space utilization, data loading time, query performance and so on. In most time, users are confronted with the problem of choosing the comprehensive optimum table layout given the workloads and the schema of tables. The straightforward way to run queries on generated data and compare the results is time consuming, and incurs the inaccuracy due to the MapReduce’s nondeterministic execution runtime. In this paper, we propose a lightweight framework to evaluate table layouts without running the query. The framework adopts the black box method to test critical metrics, and the query aware strategy that extracts table-layout-related operations from query. Based on the metrics and operations, the framework makes suggestions to users. We conduct extensive experiments...

Feng Zhu, Jie Liu, Lijie Xu, Dan Ye, Jun Wei, Tao

Real-time Traffic