Access the full text.
Sign up today, get DeepDyve free for 14 days.
James Allen, Lenhart Schubert (1991)
The TRAINS Project
B. Grosz, C. Sidner (1986)
Attention, Intentions, and the Structure of DiscourseComput. Linguistics, 12
T. Regier, Laura Carlson (2001)
Grounding spatial language in perception: an empirical and computational investigation.Journal of experimental psychology. General, 130 2
Susanne Salmon-Alt, Laurent Romary (2009)
Reference Resolution within the Framework of Cognitive GrammarArXiv, abs/0909.2626
M. Spivey-Knowlton, M. Tanenhaus, K. Eberhard, Julie Sedivy (1998)
Integration of visuospatial and linguistic information: language comprehension in real time and real space
Ivana Kruijff-Korbayová, Eva (1997)
A comparison of the salience-based approach and the Centering theory
J. Hobbs (1985)
On the coherence and structure of discourse
M Tanenhaus, M Spivey-Knowlton, K Eberhard, J Spivey (1995)
Integration of visual and linguistic information in spoken language comprehensionScience, 268
John Kelleher, F. Costello, Josef Genabith (2005)
Dynamically structuring, updating and interrelating representations of visual and linguistic discourse contextArtif. Intell., 167
J Kelleher, J Genabith (2004)
Visual salience and reference resolution in simulated 3D environmentsAI Rev, 21
John Kelleher, Josef Genabith (2004)
Visual Salience and Reference Resolution in Simulated 3-D EnvironmentsArtificial Intelligence Review, 21
Frédéric Landragin, L. Romary (2003)
Referring to Objects Through Sub-Contexts in Multimodal Human-Computer Interaction
B. Grosz (1977)
The representation and use of focus in dialogue understanding.
Nicholas Asher, A. Lascarides (2005)
Logics of Conversation
Massimo Poesio (1993)
A Situation-Theoretic Formalization of Definite Description Interpretation in Plan Elaboration Dialo
E. Hajicová (1993)
Issues of Sentence Structure and Discourse Patterns.
I Kruijff-Korbayová, E Hajicová (1997)
Topics and centers: a comparison of the salience-based approach and the centering theoryPrague Bull Math Linguist, 67
P. Oliver, Klaus-Peter Gapp (1998)
Representation and Processing of Spatial Expressions
Peter Gorniak, D. Roy (2011)
Grounded Semantic Composition for Visual ScenesJ. Artif. Intell. Res., 21
Barbara Grosz, Aravind Joshi, S. Weinstein (1995)
Centering: A Framework for Modeling the Local Coherence of DiscourseComput. Linguistics, 21
J Gundel, N Hedberg, R Zacharski (1993)
Cognitive status and the form of referring expression in discourseLanguage, 69
Shalom Lappin, Herbert Leass (1994)
An Algorithm for Pronominal Anaphora ResolutionComput. Linguistics, 20
J. Hopfinger, M. Buonocore, G. Mangun (2000)
The neural mechanisms of top-down attentional controlNature Neuroscience, 3
(1985)
Connectionist Models in Psychology
E. Krahmer, M. Theune (2002)
Efficient context-sensitive generation of referring expressions, 143
(1995)
Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning
J. Enns, Ronald Rensink (1990)
Influence of scene-based properties on visual search.Science, 247 4943
H. Alshawi (1987)
Memory and context for language interpretation
Allen Allport (1989)
Visual attention
L. Kievit, P. Piwek, R. Beun, H. Bunt (1998)
Multimodal Cooperative Resolution of Referential Expressions in the DenK System
W Mann, S Thompson (1987)
Natural language generation: new results in artificial intelligence, psychology and linguistics
W. Mann, S. Thompson (1987)
Rhetorical Structure Theory: Description and Construction of Text Structures
L. Itti, C. Koch (2001)
Computational modelling of visual attentionNature Reviews Neuroscience, 2
L. Itti, C. Koch (2000)
A saliency-based search mechanism for overt and covert shifts of visual attentionVision Research, 40
Kees Deemter, R. Kibble (2002)
Information sharing : reference and presupposition in language generation and interpretation
P. Kevitt (1995)
Integration of Natural Language and Vision Processing
(1997)
Towards a cognitive model of linguistic reference
J. Barwise, J. Gawron, G. Plotkin, S. Tutiya (1995)
Situation Theory and its ApplicationsLanguage, 71
L Kievit, P Piwek, R Beun, H Bunt (2001)
Cooperative multimodal communication: Lecture Notes in Artificial Intelligence 2155
A. Yarbus (1967)
Eye Movements and Vision
Mira Ariel (1990)
Accessing Noun-Phrase Antecedents
D. Heinke, G. Humphreys (2005)
Computational models of visual selective attention: A review
In recent years a number of psycholinguistic experiments have pointed to the interaction between language and vision. In particular, the interaction between visual attention and linguistic reference. In parallel with this, several theories of discourse have attempted to provide an account of the relationship between types of referential expressions on the one hand and the degree of mental activation on the other. Building on both of these traditions, this paper describes an attention based approach to visually situated reference resolution. The framework uses the relationship between referential form and preferred mode of interpretation as a basis for a weighted integration of linguistic and visual attention scores for each entity in the multimodal context. The resulting integrated attention scores are then used to rank the candidate referents during the resolution process, with the candidate scoring the highest selected as the referent. One advantage of this approach is that the resolution process occurs within the full multimodal context, in so far as the referent is selected from a full list of the objects in the multimodal context. As a result situations where the intended target of the reference is erroneously excluded, due to an individual assumption within the resolution process, are avoided. Moreover, the system can recognise situations where attention cues from different modalities make a reference potentially ambiguous.
Artificial Intelligence Review – Springer Journals
Published: Aug 25, 2007
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.