An Empirical Study of Information Synthesis Task

This paper describes an empirical study of the "Information Synthesis" task, defined as the process of (given a complex information need) extracting, organizing and inter-relating the pieces of information contained in a set of relevant documents, in order to obtain a comprehensive, non redundant report that satisfies the information need. Two main results are presented: a) the creation of an Information Synthesis testbed with 72 reports manually generated by nine subjects for eight complex topics with 100 relevant documents each; and b) an empirical comparison of similarity metrics between reports, under the hypothesis that the best metric is the one that best distinguishes between manual and automatically generated reports. A metric based on key concepts overlap gives better results than metrics based on n-gram overlap (such as ROUGE) or sentence overlap.

• Type: Conference
• Year: 2004
• Where: ACL
• Authors: Enrique Amigó, Julio Gonzalo, Víctor Peinado, Anselmo Peñas, Felisa Verdejo
• Keywords: ACL 2004, ACL 2007, Information Need, Information Synthesis, Relevant Documents