Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Intermittent Control as a Model of Mouse Movements

Intermittent Control as a Model of Mouse Movements We present Intermittent Control (IC) models as a candidate framework for modelling human input movements in Human–Computer Interaction (HCI). IC differs from continuous control in that users are not assumed to use feedback to adjust their movements continuously, but only when the difference between the observed pointer position and predicted pointer positions becomes large. We use a parameter optimisation approach to identify the parameters of an intermittent controller from experimental data, where users performed one-dimensional mouse movements in a reciprocal pointing task. Compared to previous published work with continuous control models, based on the Kullback–Leibler divergence from the experimental observations, IC is better able to generatively reproduce the distinctive dynamical features and variability of the pointing task across participants and over repeated tasks. IC is compatible with current physiological and psychological theory and provides insight into the source of variability in HCI tasks. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Computer-Human Interaction (TOCHI) Association for Computing Machinery

Intermittent Control as a Model of Mouse Movements

Download PDF
46 pages

Loading next page...
 
/lp/association-for-computing-machinery/intermittent-control-as-a-model-of-mouse-movements-1QKCrzB2YV

References

References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.

Publisher
Association for Computing Machinery
Copyright
Copyright © 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ISSN
1073-0516
eISSN
1557-7325
DOI
10.1145/3461836
Publisher site
See Article on Publisher Site

Abstract

We present Intermittent Control (IC) models as a candidate framework for modelling human input movements in Human–Computer Interaction (HCI). IC differs from continuous control in that users are not assumed to use feedback to adjust their movements continuously, but only when the difference between the observed pointer position and predicted pointer positions becomes large. We use a parameter optimisation approach to identify the parameters of an intermittent controller from experimental data, where users performed one-dimensional mouse movements in a reciprocal pointing task. Compared to previous published work with continuous control models, based on the Kullback–Leibler divergence from the experimental observations, IC is better able to generatively reproduce the distinctive dynamical features and variability of the pointing task across participants and over repeated tasks. IC is compatible with current physiological and psychological theory and provides insight into the source of variability in HCI tasks.

Journal

ACM Transactions on Computer-Human Interaction (TOCHI) – Association for Computing Machinery

Published: Aug 21, 2021

Keywords: Control theory

References

  • Beyond Fitts’ law: Models for trajectory-based HCI tasks
    Accot, Johnny; Zhai, Shumin
  • More than dotting the i’s—foundations for crossing-based interfaces
    Accot, Johnny; Zhai, Shumin
  • Modeling pointing tasks in mouse-based human-computer interactions
    Aranovskiy, S.; Ushirobira, R.; Efimov, D.; Casiez, G.
  • A switched dynamic model for pointing tasks with a computer mouse
    Aranovskiy, Stanislav; Ushirobira, Rosane; Efimov, Denis; Casiez, Géry
  • Biomechanical Models for Human-Computer Interaction
    Bachynskyi, Myroslav
  • Informing the design of novel input methods with muscle coactivation clustering
    Bachynskyi, Myroslav; Palmas, Gregorio; Oulasvirta, Antti; Weinkauf, Tino
  • ‘Beating’ Fitts’ law: Virtual enhancements for pointing facilitation
    Balakrishnan, Ravin
  • Negative inertia: A dynamic pointing function
    Barrett, R. C.; Selker, E. J.; Rutledge, J. D.; Olyha, R. S.
  • Computational principles of sensorimotor control that minimize uncertainty and variability
    Bays, Paul M.; Wolpert, Daniel M.
  • Computational nature of human adaptive control during learning of reaching movements in force fields
    Bhushan, Nikhil; Shadmehr, Reza
  • The dynamics of human isometric pointing movements under varying accuracy requirements
    Billon, Magali; Bootsma, Reinoud J.; Mottet, Denis
  • Semantic pointing: Improving target acquisition with control-display ratio adaptation
    Blanch, Renaud; Guiard, Yves; Beaudouin-Lafon, Michel
  • Behind Fitts’ law: Kinematic patterns in goal-directed movements
    Bootsma, R. J.; Fernandez, L.; Mottet, D.
  • The BUMP model of response planning: Variable horizon predictive control accounts for the speed–accuracy tradeoffs and velocity profiles of aimed movement
    Bye, Robin T.; Neilson, Peter D.
  • The model human processor: An engineering model for human performance
    Card, S. K.; Moran, T. P.; Newell, A.
  • No more bricolage! Methods and tools to characterize, replicate and compare pointing transfer functions
    Casiez, Géry; Roussel, Nicolas
  • The impact of control-display gain on user performance in pointing tasks
    Casiez, Géry; Vogel, Daniel; Balakrishnan, Ravin; Cockburn, Andy
  • Fitts’ Law in the Wild: A Field Study of Aimed Movements
    Chapuis, Olivier; Blanch, Renaud; Beaudouin-Lafon, Michel
  • Multi-context photo browsing on mobile devices based on tilt dynamics
    Cho, Sung-Jung; Murray-Smith, Roderick; Kim, Yeun-Bae
  • Theory of the human operator in control systems
    Craik, Kenneth J. W.
  • Theory of the human operator in control systems; man as an element in a control system
    Craik, Kenneth J. W.
  • Feedback control of hand-movement and Fitts’ law
    Crossman, E. R. F. W.; Goodeve, P. J.
  • The scaling of motor noise with muscle strength and motor unit number in humans
    de C. Hamilton, Antonia F.; Jones, Kelvin E.; Wolpert, Daniel M.
  • The role of variability in motor learning
    Dhawale, Ashesh K.; Smith, Maurice A.; Ölveczky, Bence P.
  • Tilt-based automatic zooming and scaling in mobile devices – A state-space implementation
    Eslambolchilar, Parisa; Murray-Smith, Roderick
  • Model-based, multimodal interaction in document browsing
    Eslambolchilar, Parisa; Murray-Smith, Roderick
  • Control centric approach in designing scrolling and zooming user interfaces
    Eslambolchilar, Parisa; Murray-Smith, Roderick
  • Stochastic simulations on the reliability of action potential propagation in thin axons
    Faisal, A. Aldo; Laughlin, Simon B.
  • Noise in the nervous system
    Faisal, A. Aldo; Selen, Luc P. J.; Wolpert, Daniel M.
  • Motion-pointing: Target selection using elliptical motions
    Fekete, Jean-Daniel; Elmqvist, Niklas; Guiard, Yves
  • Reinforcement learning control of a biomechanical model of the upper extremity
    Fischer, Florian; Bachinski, Miroslav; Klar, Markus; Fleig, Arthur; Müller, Jörg
  • The information capacity of the human motor system in controlling the amplitude of movement
    Fitts, Paul M.
  • Closed-loop identification revisited
    Forssell, U.; Ljung, L.
  • Intermittent control in man and machine
    Gawthrop, P.; Gollee, H.; Loram, I.
  • Intermittent control models of human standing: Similarities and differences
    Gawthrop, P.; Loram, I.; Gollee, H.; Lakie, M.
  • Intermittent control: A computational theory of human control
    Gawthrop, Peter; Loram, Ian; Lakie, Martin; Gollee, Henrik
  • Intermittent model predictive control
    Gawthrop, P.; Wang, L.
  • The system-matched hold and the intermittent control separation principle
    Gawthrop, P.; Wang, L.
  • Towards model-based continuous-time identification of the human balance controller
    Gawthrop, Peter J.; Wang, Liuping
  • Event-driven intermittent control
    Gawthrop, Peter J.; Wang, Liuping
  • Visuo-manual tracking: Does intermittent control with aperiodic sampling explain linear power and non-linear remnant without sensorimotor noise?The Journal of Physiology 595, 21 (2017), 6751–6770
    Gollee, Henrik; Gawthrop, Peter J.; Lakie, Martin; Loram, Ian D.
  • Control System Design
    Goodwin, G. C.; Graebe, S. F.; Salgado, M. E.
  • Modeling the Speed-Accuracy Tradeoff Using the Tools of Information Theory
    Gori, Julien
  • Speed-accuracy tradeoff: A Formal Information-Theoretic Transmission Scheme (FITTS)
    Gori, Julien; Rioul, Olivier; Guiard, Yves
  • On Fitts’s and Hooke’s laws: Simple harmonic movement in upper-limb cyclical aiming
    Guiard, Yves
  • Fitts’ law 50 years later: Applications and contributions from human-computer interaction
    Guiard, Yves; Beaudouin-Lafon, Michel
  • Target acquisition in multiscale electronic worlds
    Guiard, Y.; Beaudouin-Lafon, M.
  • Modelling the controller of a process
    Hollnagel, Erik
  • Joint Cognitive Systems: Foundations of Cognitive Systems Engineering
    Hollnagel, E.; Woods, D. D
  • A qualitative look at feedback control theory as a style of describing behavior
    Jagacinski, R. J.
  • Control Theory for Humans: Quantitative Approaches to Modeling Performance
    Jagacinski, Richard J.; Flach, John M.
  • Sources of signal-dependent noise during isometric force production
    Jones, Kelvin E.; de C. Hamilton, Antonia F.; Wolpert, Daniel M.
  • Optimal control of linear systems with time-delay and observation noise
    Kleinman, David L.
  • An optimal control model of human response part I: Theory and validation
    Kleinman, D. L.; Baron, S.; Levison, W. H.
  • Semi-automatic zooming for mobile map navigation
    Kratz, Sven; Brodien, Ivo; Rohs, Michael
  • A model for human controller remnant
    Levison, W. H.; Baron, S.; Kleinman, D. L.
  • Intermittent control of unstable multivariate systems
    Loram, Ian; Gawthrop, Peter; Gollee, Henrik
  • The frequency of human, manual adjustments in balancing an inverted pendulum is constrained by intrinsic physiological factors
    Loram, Ian D.; Gawthrop, Peter J.; Lakie, Martin
  • Human control of an inverted pendulum: iIs continuous control necessary? Is intermittent control effective? Is intermittent control physiological?The Journal of Physiology 589, 2 (2011), 307–324
    Loram, Ian D.; Gollee, Henrik; Lakie, Martin; Gawthrop, Peter J.
  • Human balancing of an inverted pendulum: Position control by small, ballistic-like, throw and catch movements
    Loram, I. D.; Lakie, M.
  • Identification of intermittent control in man and machine
    Loram, I. D.; van de Kamp, Cornelis; Gollee, H.; Gawthrop, P. J.
  • Does the motor system need intermittent control?Exercise and Sport Sciences Reviews 42, 3 (2014), 117–125
    Loram, Ian David; van De Kamp, Cornelis; Lakie, Martin; Gollee, Henrik; Gawthrop, Peter J.
  • Fitts’ law as a research and design tool in human–computer interaction
    MacKenzie, I. Scott
  • UMAP: Uniform Manifold Approximation and Projection for dimension reduction
    McInnes, Leland; Healy, John; Melville, James
  • Optimality in human motor performance: Ideal control of rapid aimed movements
    Meyer, D. E.; Abrams, R.; Kornblum, S.; Wright, C. E.; Smith, J. E.
  • Speed-accuracy trade-offs in aimed movements: Toward a theory of rapid voluntary action
    Meyer, D. E.; Keith-Smith, J. E.; Kornblum, S.; Abrams, R. A.; Wright, C. E.
  • Simple or complex systems?Behavioral and Brain Sciences 9, 4 (1986), 734–734
    Miall, R. Christopher
  • Is the cerebellum a smith predictor?Journal of Motor Behavior 25, 3 (1993), 203–216
    Miall, R. C.; Weir, D. Jo; Wolpert, Daniel M.; Stein, J. F.
  • The contractile properties of human motor units during voluntary isometric contractions
    Milner-Brown, H. S.; Stein, R. B.; Yemm, R.
  • Control theoretic models of pointing
    Müller, Jörg; Oulasvirta, Antti; Murray-Smith, Roderick
  • Control theory, dynamics and continuous interaction
    Murray-Smith, Roderick
  • Sampling or intermittency in hand control system dynamics
    Navas, T.; Stark, L.
  • Influence of lntermittency and Synergy on Grasping
    Nielson, P. D.
  • Degrees of freedom and motor planning in purposive movement
    Oytam, Y.; Neilson, P. D.; O’Dwyer, N.
  • An intermittent click planning model
    Park, Eunji; Lee, Byungjoo
  • On estimation of a probability density function and model
    Parzen, Emanuel
  • Tracking Skill and Manual Control
    Poulton, E. C.
  • Touch scrolling transfer functions
    Quinn, Philip; Malacria, Sylvain; Cockburn, Andy
  • Modeling gesture-typing movements
    Quinn, Philip; Zhai, Shumin
  • Open-loop intermittent feedback control: Practical continuous-time GPC
    Ronco, E.; Arsan, T.; Gawthrop, P. J.
  • Remarks on some nonparametric estimates of a density function
    Rosenblatt, Murray
  • Sources of inaccuracy in rapid movement
    Schmidt, Richard A.; Zelaznik, Howard N.; Frank, James S.
  • User representations in human-computer interaction
    Seinfeld, Sofia; Feuchtner, Tiare; Maselli, Antonella; Müller, Jörg
  • Biological Learning and Control: How the Brain Builds Representations, Predicts Events, and Makes Decisions
    Shadmehr, Reza; Mussa-Ivaldi, Sandro
  • Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts’ law research in HCI
    Soukoreff, R. William; MacKenzie, I. Scott
  • The refractory phase of voluntary and associative responses
    Telford, C. W.
  • On the convergence of pattern search algorithms
    Torczon, Virginia
  • Information-theoretic characterization of uncertainty in manual control
    Trendafilov, Dari; Murray-Smith, Roderick
  • Refractoriness in sustained visuo-manual control: Is the refractory duration intrinsic or does it depend on external system properties?PLoS Comput Biol 9, 1 (Jan
    van de Kamp, Cornelis; Gawthrop, P. J.; Gollee, H.; Loram, I. D.
  • Postural responses evoked by platform pertubations are dominated by continuous feedback
    Van Der Kooij, Herman; De Vlugt, Erwin
  • Non-linear stimulus-response behavior of the human stance control system is predicted by optimization of a system with sensory and motor noise
    Van Der Kooij, Herman; Peterka, Robert J.
  • Motion correlation: Selecting objects by matching their movement
    Velloso, Eduardo; Carter, Marcus; Newn, Joshua; Esteves, Augusto; Clarke, Christopher; Gellersen, Hans
  • Divergence estimation for multidimensional densities via $k$-Nearest-Neighbor distances
    Wang, Qing; Kulkarni, Sanjeev R.; Verdu, Sergio
  • Pointing without a pointer
    Williamson, John; Murray-Smith, Roderick
  • Shoogle: Excitatory multimodal interaction on mobile devices
    Williamson, John; Murray-Smith, Roderick; Hughes, Stephen
  • An error model for pointing based on Fitts’ law
    Wobbrock, Jacob O.; Cutrell, Edward; Harada, Susumu; MacKenzie, I. Scott
  • Computational principles of movement neuroscience
    Wolpert, Daniel M.; Ghahramani, Zoubin
  • Internal models in the cerebellum
    Wolpert, Daniel M.; Miall, R. Chris; Kawato, Mitsuo
  • Temporal structure of motor variability is dynamically regulated and predicts motor learning ability
    Wu, Howard G.; Miyamoto, Yohsuke R.; Castro, Luis Nicolas Gonzalez; Ölveczky, Bence P.; Smith, Maurice A.