Evaluation of a Sentence Ranker for Text Summarization Based on Roget's Thesaurus

Abstract. Evaluation is one of the hardest tasks in automatic text summarization. It is perhaps even harder to determine how much a particular component of a summarization system contributes to the success of the whole system. We examine how to evaluate the sentence ranking component using a corpus which has been partially labelled with Summary Content Units. To demonstrate this technique, we apply it to the evaluation of a new sentence-ranking system which uses Roget’s Thesaurus. This corpus provides a quick and nearly automatic method of evaluating the quality of sentence ranking. 1 Motivation and Related Work One of the hardest tasks in Natural Language Processing is text summarization: given a document or a collection of related documents, generate a (much) shorter text which presents only the main points. A summary can be generic – no restrictions other than the required compression – or query-driven, when the summary must answer a few questions or focus on the topic of the ...

• Type: Journal
• Year: 2010
• Where: TSD
• Authors: Alistair Kennedy, Stan Szpakowicz
• Keywords: Ranking, Signal Processing, Summary, Text Summarization, TSD 2010