Multilingual Summarization Evaluation without Human Models

We study correlation of rankings of text summarization systems using evaluation methods with and without human models. We apply our comparison framework to various well-established contentbased evaluation measures in text summarization such as coverage, Responsiveness, Pyramids and ROUGE studying their associations in various text summarization tasks including generic and focus-based multi-document summarization in English and generic single-document summarization in French and Spanish. The research is carried out using a new content-based evaluation framework called FRESA to compute a variety of divergences among probability distributions.

• Type: Journal
• Year: 2010
• Where: COLING
• Authors: Horacio Saggion, Juan Manuel Torres Moreno, Iria da Cunha, Eric SanJuan, Patricia Velázquez-Morales
• Keywords: COLING 2010, Computational Linguistics, Text Summarization, Text Summarization Systems, Text Summarization Tasks