A Parallel Decomposition Solver for SVM: Distributed dual ascend using Fenchel Duality

We introduce a distributed algorithm for solving large scale Support Vector Machines (SVM) problems. The algorithm divides the training set into a number of processing nodes each running independently an SVM sub-problem associated with its subset of training data. The algorithm is a parallel (Jacobi) block-update scheme derived from the convex conjugate (Fenchel Duality) form of the original SVM problem. Each update step consists of a modified SVM solver running in parallel over the sub-problems followed by a simple global update. We derive bounds on the number of updates showing that the number of iterations (independent SVM applications on sub-problems) required to obtain a solution of accuracy is O(log(1/ )). We demonstrate the efficiency and applicability of our algorithms by running on large scale experiments on standardized datasets while comparing the results to the state-of-the-art SVM solvers.

• Type: Conference
• Year: 2008
• Where: CVPR
• Authors: Tamir Hazan, Amit Man, Amnon Shashua
• Keywords: Computer Vision, CVPR 2008, Independent Svm Applications, Original Svm Problem, State-of-the-art Svm Solvers, Svm Solver, Svm Sub-problem