SINGA: A Distributed Deep Learning Platform

Deep learning has shown outstanding performance in various machine learning tasks. However, the deep complex model structure and massive training data make it expensive to train. In this paper, we present a distributed deep learning system, called SINGA, for training big models over large datasets. An intuitive programming model based on r abstraction is provided, which supports a variety of popular deep learning models. SINGA architecture supports both synchronous and asynchronous training frameworks. Hybrid training frameworks can also be customized to achieve good scalability. SINGA provides different neural net partitioning schemes for training large models. SINGA is an Apache Incubator project released under Apache License 2. Categories and Subject Descriptors I.5.1 [Pattern Recognition]: Models—Neural Nets; H.3.4 [Information Storage and Retrieval]: Systems and Software—Distributed System General Terms Design, Experimentation, Performance Keywords Deep learning; Distributed...

• Type: Journal
• Year: 2015
• Where: MM
• Authors: Beng Chin Ooi, Kian-Lee Tan, Sheng Wang, Wei Wang, Qingchao Cai, Gang Chen, Jinyang Gao, Zhaojing Luo, Anthony K. H. Tung, Yuan Wang, Zhongle Xie, Meihui Zhang, Kaiping Zheng
• Keywords: MM 2015, Multimedia