Access the full text.
Sign up today, get DeepDyve free for 14 days.
Loading next page...
/lp/springer-journals/testing-the-performance-of-spoken-dialogue-systems-by-means-of-an-2fIQocvXaW
References (45)
-
James Allen, D. Byron, M. Dzikovska, G. Ferguson, Lucian Galescu, Amanda Stent (2001)
Toward Conversational Human-Computer InteractionAI Mag., 22
-
R. López-Cózar, Á. Torre, J. Segura, A. Rubio, J. López-Soler (2002)
A new method for testing dialogue systems based on simulations of real-world conditions -
(1995)
Dialogue control in automatic inquiry systems -
M McTear (2004)
Spoken dialogue technology: toward the conversational user interface -
E. Levin, R. Pieraccini, W. Eckert (2000)
A stochastic model of human-machine interaction for learning dialog strategiesIEEE Trans. Speech Audio Process., 8
-
J. Schatzmann, Kallirroi Georgila, S. Young (2005)
Quantitative Evaluation of User Simulation Techniques for Spoken Dialogue Systems -
Thomas Hain, P. Woodland, T. Niesler, E. Whittaker (1999)
The 1998 HTK system for transcription of conversational telephone speech1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258), 1
-
R. López-Cózar, P. García-Teodoro, J. Verdejo, A. Rubio (1997)
A voice activated dialogue system for fast-food restaurant applications -
M. Walker, D. Litman, C. Kamm, A. Abella (1997)
PARADISE: A Framework for Evaluating Spoken Dialogue AgentsArXiv, cmp-lg/9704004
-
R. Pieraccini, J. Huerta (2005)
Where do we go from here? Research and Commercial Spoken Dialog Systems -
James Allen, D. Byron, M. Dzikovska, G. Ferguson, Lucian Galescu (2000)
Towards Conversational Human-Computer Interaction -
Simulated user : No [Recognition hypothesis: two (0.4642)] (15) Saplen: I have to confirm your telephone number again. Did you say 2? Please say yes or no -
K. Scheffler, S. Young (2000)
Probabilistic simulation of human-machine dialogues2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100), 2
-
H. Cuayáhuitl, S. Renals, Oliver Lemon, H. Shimodaira (2005)
Human-computer dialogue simulation using hidden Markov modelsIEEE Workshop on Automatic Speech Recognition and Understanding, 2005.
-
1) Saplen: Please say your telephone number -
Michael Kearns, D. Litman, Satinder Singh, M. Walker (2011)
Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun SystemJ. Artif. Intell. Res., 16
-
W. Eckert, E. Levin, R. Pieraccini (1997)
User modeling for spoken dialogue system evaluation1997 IEEE Workshop on Automatic Speech Recognition and Understanding Proceedings
-
Steven Dow, B. MacIntyre, Jaemin Lee, C. Oezbek, J. Bolter, Maribeth Coleman (2005)
Wizard of Oz support throughout an iterative design processIEEE Pervasive Computing, 4
-
R. López-Cózar, Diego Milone (2001)
A new technique based on augmented language models to improve the performance of spoken dialogue systems -
R. López-Cózar, A. Rubio, J. Verdejo, Á. Torre (2000)
Evaluation of a Dialogue System Based on a Generic Model that Combines Robust Speech Understanding and Mixed-initiative Control -
J. Schatzmann, Matthew Stuttle, K. Weilhammer, S. Young (2005)
Effects of the user model on simulation-based learning of dialogue strategiesIEEE Workshop on Automatic Speech Recognition and Understanding, 2005.
-
M. McTear (2004)
Spoken Dialogue Technology -
Xuedong Huang, A. Acero, H. Hon, R. Reddy (2001)
Spoken Language Processing: A Guide to Theory, Algorithm and System Development -
R. Delgado, Masahiro Araki (2005)
Spoken, Multilingual and Multimodal Dialogue Systems: Development and Assessment -
R. López-Cózar, A. Rubio, María García, J. Segura (1998)
A spoken dialogue system based on dialogue corpues analysis -
O. Pietquin, T. Dutoit (2006)
A probabilistic framework for dialog simulation and optimal strategy learningIEEE Transactions on Audio, Speech, and Language Processing, 14
-
S. Möller, R. Englert, Klaus-Peter Engelbrecht, V. Hafner, A. Jameson, Antti Oulasvirta, A. Raake, Norbert Reithinger (2006)
Memo: towards automatic usability evaluation of spoken dialogue services by user error simulations -
Bor-Shen Lin, Lin-Shan Lee (2001)
Computer-aided analysis and design for spoken dialogue systems based on quantitative simulationsIEEE Trans. Speech Audio Process., 9
-
A. Sciullo (2009)
Natural Language Understanding -
S. Carbini, L. Delphin-Poulat, L. Perron, J. Viallet (2006)
From a Wizard of Oz experiment to a real time speech and gesture multimodal interfaceSignal Process., 86
-
Masayuki Okamoto, Kenta Cho, Yuzo Okamoto, Tomohiro Yamasaki, Masanori Hattori (2005)
User-model-based adaptability evaluation for context-aware systemsICPS '05. Proceedings. International Conference on Pervasive Services, 2005.
-
L. Rabiner, B. Juang (1993)
Fundamentals of speech recognition -
D. Litman, Shimei Pan (2002)
Designing and Evaluating an Adaptive Spoken Dialogue SystemUser Modeling and User-Adapted Interaction, 12
-
Z. Tan, P. Dalsgaard (2002)
Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP'2002), Denver-Colorado, USA, September 2002 -
R. López-Cózar, Zoraida Carrión (2006)
Combining language models in the input interface of a spoken dialogue systemComput. Speech Lang., 20
-
O. Pietquin, T. Dutoit (2006)
Dynamic Bayesian Networks for NLU Simulation with Applications to Dialog Optimal Strategy Learning2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, 1
-
(2001)
Corpus - based dialogue simulation for automatic strategy learning and evaluation -
reached before all the scenario goals were achieved. These problems are illustrated in the following sample dialogue -
Kallirroi Georgila, James Henderson, Oliver Lemon (2005)
Learning user simulations for information state update dialogue systems -
R. López-Cózar, Á. Torre, J. Segura, A. Rubio (2003)
Assessment of dialogue systems by means of a new simulation techniqueSpeech Commun., 40
-
Sebastian Mller (2004)
Quality of Telephone-Based Spoken Dialogue Systems -
S. Young (2002)
Talking to machines (statistically speaking) -
M. Walker (2000)
An Application of Reinforcement Learning to Dialogue Strategy Selection in a Spoken Dialogue System for EmailJ. Artif. Intell. Res., 12
-
James Allen (1995)
Natural language understanding (2nd ed.) -
Grace Chung (2004)
Developing a Flexible Spoken Dialog System Using Simulation
- Publisher
- Springer Journals
- Copyright
- Copyright © 2007 by Springer Science+Business Media B.V.
- Subject
- Computer Science; Complexity; Computer Science, general ; Artificial Intelligence (incl. Robotics)
- ISSN
- 0269-2821
- eISSN
- 1573-7462
- DOI
- 10.1007/s10462-007-9059-9
- Publisher site
- See Article on Publisher Site
Abstract
This paper proposes a new technique to test the performance of spoken dialogue systems by artificially simulating the behaviour of three types of user (very cooperative, cooperative and not very cooperative) interacting with a system by means of spoken dialogues. Experiments using the technique were carried out to test the performance of a previously developed dialogue system designed for the fast-food domain and working with two kinds of language model for automatic speech recognition: one based on 17 prompt-dependent language models, and the other based on one prompt-independent language model. The use of the simulated user enables the identification of problems relating to the speech recognition, spoken language understanding, and dialogue management components of the system. In particular, in these experiments problems were encountered with the recognition and understanding of postal codes and addresses and with the lengthy sequences of repetitive confirmation turns required to correct these errors. By employing a simulated user in a range of different experimental conditions sufficient data can be generated to support a systematic analysis of potential problems and to enable fine-grained tuning of the system.
Journal
Artificial Intelligence Review – Springer Journals
Published: Nov 23, 2007