Sciweavers

CORR
2016
Springer

Dot-Product Join: An Array-Relation Join Operator for Big Model Analytics

7 years 12 months ago
Dot-Product Join: An Array-Relation Join Operator for Big Model Analytics
Big Data analytics has been approached exclusively from a data-parallel perspective, where data are partitioned to multiple workers – threads or separate servers – and model training is executed concurrently over different partitions, under various synchronization schemes that guarantee speedup and/or convergence. The dual – Big Model – problem that, surprisingly, has received no attention in database analytics, is how to manage models with millions if not billions of parameters that do not fit in memory. This distinction in model representation changes fundamentally how in-database analytics tasks are carried out. In this paper, we introduce the first secondary storage array-relation dot-product join operator between a set of sparse arrays and a dense relation targeted. The paramount challenge in designing such an operator is how to optimally schedule access to the dense relation based on the sparse non-contiguous entries in the sparse arrays. We prove that this problem is ...
Chengjie Qin, Florin Rusu
Added 01 Apr 2016
Updated 01 Apr 2016
Type Journal
Year 2016
Where CORR
Authors Chengjie Qin, Florin Rusu
Comments (0)