Incremental Sigmoid Belief Networks for Grammar Learning

9 years 11 months ago
Incremental Sigmoid Belief Networks for Grammar Learning
We propose a class of Bayesian networks appropriate for structured prediction problems where the Bayesian network's model structure is a function of the predicted output structure. These incremental sigmoid belief networks (ISBNs) make decoding possible because inference with partial output structures does not require summing over the unboundedly many compatible model structures, due to their directed edges and incrementally specified model structure. ISBNs are specifically targeted at challenging structured prediction problems such as natural language parsing, where learning the domain's complex statistical dependencies benefits from large numbers of latent variables. While exact inference in ISBNs with large numbers of latent variables is not tractable, we propose two efficient approximations. First, we demonstrate that a previous neural network parsing model can be viewed as a coarse mean-field approximation to inference with ISBNs. We then derive a more accurate but stil...
James Henderson, Ivan Titov
Added 19 May 2011
Updated 19 May 2011
Type Journal
Year 2010
Where JMLR
Authors James Henderson, Ivan Titov
Comments (0)