Sciweavers

CSL
1998
Springer

Evaluating spoken dialogue agents with PARADISE: Two case studies

13 years 11 months ago
Evaluating spoken dialogue agents with PARADISE: Two case studies
This paper presents PARADISE PARAdigm for DIalogue System Evaluation, a general framework for evaluating and comparing the performance of spoken dialogue agents. The framework decouples task requirements from an agent's dialogue behaviors, supports comparisons among dialogue strategies, enables the calculation of performance over subdialogues and whole dialogues, speci es the relative contribution of various factors to performance, and makes it possible to compare agents performing di erent tasks by normalizing for task complexity. After presenting PARADISE, we illustrate its application to two di erent spoken dialogue agents. We show how to derive a performance function for each agent and how to generalize results across agents. We then show that once such a performance function has been derived, that it can be used both for making predictions about future versions of an agent, and as feedback to the agent so that the agent can learn to optimize its behavior based on its exper...
Marilyn A. Walker, Diane J. Litman, Candace A. Kam
Added 22 Dec 2010
Updated 22 Dec 2010
Type Journal
Year 1998
Where CSL
Authors Marilyn A. Walker, Diane J. Litman, Candace A. Kamm, Alicia Abella
Comments (0)