Variational Inference for Grammar Induction with Prior Knowledge

Variational EM has become a popular technique in probabilistic NLP with hidden variables. Commonly, for computational tractability, we make strong independence assumptions, such as the meanfield assumption, in approximating posterior distributions over hidden variables. We show how a looser restriction on the approximate posterior, requiring it to be a mixture, can help inject prior knowledge to exploit soft constraints during the variational E-step.

• Type: Journal
• Year: 2009
• Where: ACL
• Authors: Shay B. Cohen, Noah A. Smith
• Keywords: ACL 2009, Computational Linguistics, Hidden Variables, Strong Independence Assumptions, Variational EM