Partitioning real-time ETL workflows

—Many organizations are aiming to move away from traditional batch processing ETL to real-time ETL (RT-ETL). This move is motivated by a need to analyze and take decisions on as fresh a data as possible. The RT-ETL engines operate on raction of data flow executed on parallel architectures. For high throughput and low response times, there is a need for partitioning the data over the large number of nodes in the engine. In this paper, we consider the problem of partitioning realtime ETL flows and we propose a high level architecture for that.

• Type: Conference
• Year: 2010
• Where: ICDE
• Authors: Alkis Simitsis, Chetan Gupta, Song Wang, Umeshwar Dayal
• Keywords: Database, ICDE 2010, Low Response Times, Parallel Architectures, Traditional Batch Processing