Sciweavers

BTW
2015
Springer

Scaling Out the Discovery of Inclusion Dependencies

7 years 11 months ago
Scaling Out the Discovery of Inclusion Dependencies
Abstract: Inclusion dependencies are among the most important database dependencies. In addition to their most prominent application – foreign key discovery – inclusion dependencies are an important input to data integration, query optimization, and schema redesign. With their discovery being a recurring data profiling task, previous research has proposed different algorithms to discover all inclusion dependencies within a given dataset. However, none of the proposed algorithms is designed to scale out, i.e., none can be distributed across multiple nodes in a computer cluster to increase the performance. So on large datasets with many inclusion dependencies, these algorithms can take days to complete, even on high-performance computers. We introduce SINDY, an algorithm that efficiently discovers all unary inclusion dependencies of a given relational dataset in a distributed fashion and that is not tied to main memory requirements. We give a practical implementation of SINDY that ...
Sebastian Kruse, Thorsten Papenbrock, Felix Nauman
Added 17 Apr 2016
Updated 17 Apr 2016
Type Journal
Year 2015
Where BTW
Authors Sebastian Kruse, Thorsten Papenbrock, Felix Naumann
Comments (0)