Sciweavers

Share
UAI
2003

On the Convergence of Bound Optimization Algorithms

10 years 7 months ago
On the Convergence of Bound Optimization Algorithms
Many practitioners who use EM and related algorithms complain that they are sometimes slow. When does this happen, and what can be done about it? In this paper, we study the general class of bound optimization algorithms – including EM, Iterative Scaling, Non-negative Matrix Factorization, CCCP – and their relationship to direct optimization algorithms such as gradientbased methods for parameter learning. We derive a general relationship between the updates performed by bound optimization methods and those of gradient and second-order methods and identify analytic conditions under which bound optimization algorithms exhibit quasi-Newton behavior, and conditions under which they possess poor, first-order convergence. Based on this analysis, we consider several specific algorithms, interpret and analyze their convergence properties and provide some recipes for preprocessing input to these algorithms to yield faster convergence behavior. We report empirical results supporting our a...
Ruslan Salakhutdinov, Sam T. Roweis, Zoubin Ghahra
Added 01 Nov 2010
Updated 01 Nov 2010
Type Conference
Year 2003
Where UAI
Authors Ruslan Salakhutdinov, Sam T. Roweis, Zoubin Ghahramani
Comments (0)
books