Text Segmentation with LDA-Based Fisher Kernel

In this paper we propose a domainindependent text segmentation method, which consists of three components. Latent Dirichlet allocation (LDA) is employed to compute words semantic distribution, and we measure semantic similarity by the Fisher kernel. Finally global best segmentation is achieved by dynamic programming. Experiments on Chinese data sets with the technique show it can be effective. Introducing latent semantic information, our algorithm is robust on irregular-sized segments.

• Type: Conference
• Year: 2008
• Where: ACL
• Authors: Qi Sun, Runxin Li, Dingsheng Luo, Xihong Wu
• Keywords: ACL 2008, Computational Linguistics, Latent Dirichlet Allocation, Semantic, Words Semantic Distribution