Estimation Method of User Satisfaction Using N-gram-based Dialog History Model for Spoken Dialog System

In this paper, we propose an estimation method of user satisfaction for a spoken dialog system using an N-gram-based dialog history model. We have collected a large amount of spoken dialog data accompanied by usability evaluation scores by users in real environments. The database is made by a field-test in which naive users used a client-server music retrieval system with a spoken dialog interface on their own PCs. An N-gram model is trained from the sequences that consist of users' dialog acts and/or the system's dialog acts for each one of six user satisfaction levels: from 1 to 5 and (task not completed). Then, the satisfaction level is estimated based on the N-gram likelihood. Experiments were conducted on the large real data and the results show that our proposed method achieved good classification performance; the classification accuracy was 94.7% in the experiment on a classification into dialogs with task completion and those without task completion. Even if the cla...

• Type: Conference
• Year: 2010
• Where: LREC
• Authors: Sunao Hara, Norihide Kitaoka, Kazuya Takeda
• Keywords: Dialog, Education, LREC 2010, Spoken Dialog, Spoken Dialog System