Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Multimodal behavior and interaction as indicators of cognitive load

Multimodal behavior and interaction as indicators of cognitive load Multimodal Behavior and Interaction as Indicators of Cognitive Load FANG CHEN, NATALIE RUIZ, ERIC CHOI, JULIEN EPPS, M. ASIF KHAWAJA, RONNIE TAIB, BO YIN, and YANG WANG, NICTA, Australia High cognitive load arises from complex time and safety-critical tasks, for example, mapping out flight paths, monitoring traffic, or even managing nuclear reactors, causing stress, errors, and lowered performance. Over the last five years, our research has focused on using the multimodal interaction paradigm to detect fluctuations in cognitive load in user behavior during system interaction. Cognitive load variations have been found to impact interactive behavior: by monitoring variations in specific modal input features executed in tasks of varying complexity, we gain an understanding of the communicative changes that occur when cognitive load is high. So far, we have identified specific changes in: speech, namely acoustic, prosodic, and linguistic changes; interactive gesture; and digital pen input, both interactive and freeform. As ground-truth measurements, galvanic skin response, subjective, and performance ratings have been used to verify task complexity. The data suggest that it is feasible to use features extracted from behavioral changes in multiple modal inputs as indices of cognitive load. The speech-based indicators of load, based on data collected http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Interactive Intelligent Systems (TiiS) Association for Computing Machinery

Multimodal behavior and interaction as indicators of cognitive load

Download PDF
36 pages

Loading next page...
 
/lp/association-for-computing-machinery/multimodal-behavior-and-interaction-as-indicators-of-cognitive-load-SF0TWO0ip0

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2012 by ACM Inc.
ISSN
2160-6455
DOI
10.1145/2395123.2395127
Publisher site
See Article on Publisher Site

Abstract

Multimodal Behavior and Interaction as Indicators of Cognitive Load FANG CHEN, NATALIE RUIZ, ERIC CHOI, JULIEN EPPS, M. ASIF KHAWAJA, RONNIE TAIB, BO YIN, and YANG WANG, NICTA, Australia High cognitive load arises from complex time and safety-critical tasks, for example, mapping out flight paths, monitoring traffic, or even managing nuclear reactors, causing stress, errors, and lowered performance. Over the last five years, our research has focused on using the multimodal interaction paradigm to detect fluctuations in cognitive load in user behavior during system interaction. Cognitive load variations have been found to impact interactive behavior: by monitoring variations in specific modal input features executed in tasks of varying complexity, we gain an understanding of the communicative changes that occur when cognitive load is high. So far, we have identified specific changes in: speech, namely acoustic, prosodic, and linguistic changes; interactive gesture; and digital pen input, both interactive and freeform. As ground-truth measurements, galvanic skin response, subjective, and performance ratings have been used to verify task complexity. The data suggest that it is feasible to use features extracted from behavioral changes in multiple modal inputs as indices of cognitive load. The speech-based indicators of load, based on data collected

Journal

ACM Transactions on Interactive Intelligent Systems (TiiS) – Association for Computing Machinery

Published: Dec 1, 2012

There are no references for this article.