Probabilistic combination of text classifiers using reliability indicators: models and results

The intuition that different text classifiers behave in qualitatively different ways has long motivated attempts to build a better metaclassifier via some combination of classifiers. We introduce a probabilistic method for combining classifiers that considers the contextsensitive reliabilities of contributing classifiers. The method harnesses reliability indicators--variables that provide a valuable signal about the performance of classifiers in different situations. We provide background, present procedures for building metaclassifiers that take into consideration both reliability indicators and classifier outputs, and review a set of comparative studies undertaken to evaluate the methodology. Categories and Subject Descriptors H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval; I.2.6 [Artificial Intelligence]: Learning; I.5.1 [Pattern Recognition]: Models General Terms Algorithms, Experimentation. Keywords Text classification, classifier combination, metacla...

• Type: Journal
• Year: 2002
• Where: SIGIR
• Authors: Paul N. Bennett, Susan T. Dumais, Eric Horvitz
• Keywords: Better Metaclassifier, Information Technology, Probabilistic Method, Reliability Indicators, SIGIR 2002