Sciweavers

Share
22 search results - page 1 / 5
» Assessing the Effect of Inconsistent Assessors on Summarizat...
Sort
View
ACL
2012
9 years 3 months ago
Assessing the Effect of Inconsistent Assessors on Summarization Evaluation
We investigate the consistency of human assessors involved in summarization evaluation to understand its effect on system ranking and automatic evaluation techniques. Using Text A...
Karolina Owczarzak, Peter A. Rankel, Hoa Trang Dan...
SIGIR
2012
ACM
9 years 3 months ago
Effect of written instructions on assessor agreement
Assessors frequently disagree on the topical relevance of documents. How much of this disagreement is due to ambiguity in assessment instructions? We have two assessors assess TRE...
William Webber, Bryan Toth, Marjorie Desamito
CIVR
2008
Springer
152views Image Analysis» more  CIVR 2008»
11 years 3 months ago
Evaluating audio skimming and frame rate acceleration for summarizing BBC rushes
For the first time in 2007, TRECVID considered structured evaluation of automated video summarization, utilizing BBC rushes video. In 2007, we conducted user evaluations with the ...
Michael G. Christel, Wei-Hao Lin, Bryan Maher
SIGIR
2005
ACM
11 years 7 months ago
Information retrieval system evaluation: effort, sensitivity, and reliability
The effectiveness of information retrieval systems is measured by comparing performance on a common set of queries and documents. Significance tests are often used to evaluate the...
Mark Sanderson, Justin Zobel
TRECVID
2007
11 years 2 months ago
Summarizing BBC Rushes the Informedia Way
For the first time in 2007, TRECVID considered structured evaluation of automated video summarization, utilizing BBC rushes video. This paper discusses in detail our approaches fo...
Alexander G. Hauptmann, Michael G. Christel, Wei-H...
books