A Hybrid Approach to High Availability in Stream Processing Systems

Abstract—Stream processing is widely used by today’s applications such as financial data analysis and disaster response. In distributed stream processing systems, machine fail-stop events are handled by either active standby or passive standby. However, existing high availability (HA) schemes have not sufficiently addressed the situation when a machine becomes temporarily unavailable due to data rate spikes, intensive analysis or job sharing, which happens frequently but lasts for short time. It is not clear how well active and passive standby fare against such transient unavailability. In this paper, we first critically examine the suitability of active and passive standby against transient unavailability in a real testbed environment. We find that both approaches have advantages and drawbacks, but neither is ideal to provide fast recovery at low overhead as required to handle transient unavailability. Based on the insights gained, we propose a novel hybrid HA method that swit...

• Type: Conference
• Year: 2010
• Where: ICDCS
• Authors: Zhe Zhang, Yu Gu, Fan Ye, Hao Yang, Minkyong Kim, Hui Lei, Zhen Liu
• Keywords: Active Standby, Distributed And Parallel Computing, ICDCS 2010, Passive Standby, Transient Unavailability