Sciweavers

CIDR
2009

RIOT: I/O-Efficient Numerical Computing without SQL

13 years 4 months ago
RIOT: I/O-Efficient Numerical Computing without SQL
R is a numerical computing environment that is widely popular for statistical data analysis. Like many such environments, R performs poorly for large datasets whose sizes exceed that of physical memory. We present our vision of RIOT (R with I/O Transparency), a system that makes R programs I/O-efficient in a way transparent to the users. We describe our experience with RIOT-DB, an initial prototype that uses a relational database system as a backend. Despite the overhead and inadequacy of generic database systems in handling array data and numerical computation, RIOT-DB significantly outperforms R in many large-data scenarios, thanks to a suite of high-level, inter-operation optimizations that integrate seamlessly into R. While many techniques in RIOT are inspired by databases (and, for RIOT-DB, realized by a database system), RIOT users are insulated from anything database related. Compared with previous approaches that require users to learn new languages and rewrite their programs ...
Yi Zhang 0011, Herodotos Herodotou, Jun Yang 0001
Added 08 Nov 2010
Updated 08 Nov 2010
Type Conference
Year 2009
Where CIDR
Authors Yi Zhang 0011, Herodotos Herodotou, Jun Yang 0001
Comments (0)