Lagrangian Relaxation for MAP Estimation in Graphical Models

Abstract— We develop a general framework for MAP estimation in discrete and Gaussian graphical models using Lagrangian relaxation techniques. The key idea is to reformulate an intractable estimation problem as one defined on a more tractable graph, but subject to additional constraints. Relaxing these constraints gives a tractable dual problem, one defined by a thin graph, which is then optimized by an iterative procedure. When this iterative optimization leads to a consistent estimate, one which also satisfies the constraints, then it corresponds to an optimal MAP estimate of the original model. Otherwise there is a “duality gap”, and we obtain a bound on the optimal solution. Thus, our approach combines convex optimization with dynamic programming techniques applicable for thin graphs. The popular tree-reweighted maxproduct (TRMP) method may be seen as solving a particular class of such relaxations, where the intractable graph is relaxed to a set of spanning trees. We also c...

• Type: Journal
• Year: 2007
• Where: CORR
• Authors: Jason K. Johnson, Dmitry M. Malioutov, Alan S. Willsky
• Keywords: CORR 2007, Dual Problem, Education, Intractable Estimation Problem, Iterative Optimization