Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Multilingual visual sentiment concept clustering and analysis

Multilingual visual sentiment concept clustering and analysis Visual content is a rich medium that can be used to communicate not only facts and events, but also emotions and opinions. In some cases, visual content may carry a universal affective bias (e.g., natural disasters or beautiful scenes). Often however, to achieve a parity in the affections a visual media invokes in its recipient compared to the one an author intended requires a deep understanding and even sharing of cultural backgrounds. In this study, we propose a computational framework for the clustering and analysis of multilingual visual affective concepts used in different languages which enable us to pinpoint alignable differences (via similar concepts) and nonalignable differences (via unique concepts) across cultures. To do so, we crowdsource sentiment labels for the MVSO dataset, which contains 16 K multilingual visual sentiment concepts and 7.3M images tagged with these concepts. We then represent these concepts in a distribution-based word vector space via (1) pivotal translation or (2) cross-lingual semantic alignment. We then evaluate these representations on three tasks: affective concept retrieval, concept clustering, and sentiment prediction—all across languages. The proposed clustering framework enables the analysis of the large multilingual dataset both quantitatively and qualitatively. We also show a novel use case consisting of a facial image data subset and explore cultural insights about visual sentiment concepts in such portrait-focused images. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png International Journal of Multimedia Information Retrieval Springer Journals

Multilingual visual sentiment concept clustering and analysis

Download PDF
20 pages

Loading next page...
 
/lp/springer-journals/multilingual-visual-sentiment-concept-clustering-and-analysis-V8exPkBWmW
Publisher
Springer Journals
Copyright
Copyright © 2017 by Springer-Verlag London
Subject
Computer Science; Multimedia Information Systems; Information Storage and Retrieval; Information Systems Applications (incl.Internet); Data Mining and Knowledge Discovery; Image Processing and Computer Vision; Computer Science, general
ISSN
2192-6611
eISSN
2192-662X
DOI
10.1007/s13735-017-0120-4
Publisher site
See Article on Publisher Site

Abstract

Visual content is a rich medium that can be used to communicate not only facts and events, but also emotions and opinions. In some cases, visual content may carry a universal affective bias (e.g., natural disasters or beautiful scenes). Often however, to achieve a parity in the affections a visual media invokes in its recipient compared to the one an author intended requires a deep understanding and even sharing of cultural backgrounds. In this study, we propose a computational framework for the clustering and analysis of multilingual visual affective concepts used in different languages which enable us to pinpoint alignable differences (via similar concepts) and nonalignable differences (via unique concepts) across cultures. To do so, we crowdsource sentiment labels for the MVSO dataset, which contains 16 K multilingual visual sentiment concepts and 7.3M images tagged with these concepts. We then represent these concepts in a distribution-based word vector space via (1) pivotal translation or (2) cross-lingual semantic alignment. We then evaluate these representations on three tasks: affective concept retrieval, concept clustering, and sentiment prediction—all across languages. The proposed clustering framework enables the analysis of the large multilingual dataset both quantitatively and qualitatively. We also show a novel use case consisting of a facial image data subset and explore cultural insights about visual sentiment concepts in such portrait-focused images.

Journal

International Journal of Multimedia Information Retrieval – Springer Journals

Published: Feb 20, 2017

References

  • Natural language processing (almost) from scratch
    Collobert, R; Weston, J; Bottou, L; Karlen, M; Kavukcuoglu, K; Kuksa, P
  • Multimodal distributional semantics
    Bruni, E; Tran, NK; Baroni, M
  • Grounded compositional semantics for finding and describing images with sentences
    Socher, R; Karpathy, A; Le, QV; Manning, CD; Ng, AY
  • Improving distributional similarity with lessons learned from word embeddings
    Levy, O; Goldberg, Y; Dagan, I
  • Distributed representations of words and phrases and their compositionality
    Mikolov, T; Sutskever, I; Chen, K; Corrado, GS; Dean, J
  • Neurons that keep a straight face
    Freiwald, WA; Tsao, DY
  • A fast and accurate unconstrained face detector
    Liao, S; Jain, AK; Li, SZ
  • Opinion mining and sentiment analysis
    Bo, Pang; Lee, Lillian
  • Multimodal sentiment analysis of Spanish online videos
    Rosas, Veronica; Mihalcea, Rada; Morency, Louis-Philippe
  • New avenues in opinion mining and sentiment analysis
    Cambria, Erik; Schuller, Bjorn; Xia, Yunqing; Havasi, Catherine
  • Fusing audio, visual and textual clues for sentiment analysis from multimodal content
    Poria, Soujanya; Cambria, Erik; Howard, Newton; Huang, Guang-Bin; Hussain, Amir