Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Mining exoticism from visual content with fusion-based deep neural networks

Mining exoticism from visual content with fusion-based deep neural networks Exoticism is the charm of the unfamiliar or something remote. It has received significant interest in different kinds of arts, but although visual concept classification in images and videos for semantic multimedia retrieval has been researched for years, the visual concept of exoticism has not been investigated yet from a computational perspective. In this paper, we present the first approach to automatically classify images as exotic or non-exotic. We have gathered two large datasets that cover exoticism in a general as well as a concept-specific way. The datasets have been annotated in a crowdsourcing approach. To circumvent cultural differences in the annotation, only North American crowdworkers are employed for this task. Two deep learning architectures to learn the concept of exoticism are evaluated. Besides deep learning features, we also investigate the usefulness of hand-crafted features, which are combined with deep features in our proposed fusion-based approach. Different machine learning models are compared with the fusion-based approach, which is the best performing one, reaching an accuracy over 83% and 91% on two different datasets. Comprehensive experimental results provide insights into which features contribute at most to recognizing exoticism. The estimation of image exoticism could be applied in fields like advertising and travel suggestions, as well as to increase serendipity and diversity of recommendations and search results. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png International Journal of Multimedia Information Retrieval Springer Journals

Mining exoticism from visual content with fusion-based deep neural networks

Download PDF
15 pages

Loading next page...
 
/lp/springer-journals/mining-exoticism-from-visual-content-with-fusion-based-deep-neural-aqMLdhe3sX
Publisher
Springer Journals
Copyright
Copyright © 2019 by Springer-Verlag London Ltd., part of Springer Nature
Subject
Computer Science; Multimedia Information Systems; Information Storage and Retrieval; Information Systems Applications (incl.Internet); Data Mining and Knowledge Discovery; Image Processing and Computer Vision; Database Management
ISSN
2192-6611
eISSN
2192-662X
DOI
10.1007/s13735-018-00165-4
Publisher site
See Article on Publisher Site

Abstract

Exoticism is the charm of the unfamiliar or something remote. It has received significant interest in different kinds of arts, but although visual concept classification in images and videos for semantic multimedia retrieval has been researched for years, the visual concept of exoticism has not been investigated yet from a computational perspective. In this paper, we present the first approach to automatically classify images as exotic or non-exotic. We have gathered two large datasets that cover exoticism in a general as well as a concept-specific way. The datasets have been annotated in a crowdsourcing approach. To circumvent cultural differences in the annotation, only North American crowdworkers are employed for this task. Two deep learning architectures to learn the concept of exoticism are evaluated. Besides deep learning features, we also investigate the usefulness of hand-crafted features, which are combined with deep features in our proposed fusion-based approach. Different machine learning models are compared with the fusion-based approach, which is the best performing one, reaching an accuracy over 83% and 91% on two different datasets. Comprehensive experimental results provide insights into which features contribute at most to recognizing exoticism. The estimation of image exoticism could be applied in fields like advertising and travel suggestions, as well as to increase serendipity and diversity of recommendations and search results.

Journal

International Journal of Multimedia Information Retrieval – Springer Journals

Published: Jan 23, 2019

References

  • On unexpectedness in recommender systems: or how to better expect the unexpected
    Adamopoulos, P; Tuzhilin, A
  • A machine learning approach to sentiment analysis in multilingual web texts
    Boiy, E; Moens, M-F
  • The openCV library
    Bradski, G
  • Increasing cheat robustness of crowdsourcing tasks
    Eickhoff, C; Vries, AP
  • The pascal visual object classes challenge: a retrospective
    Everingham, M; Eslami, SMA; Gool, L; Williams, CKI; Winn, J; Zisserman, A
  • Advances in information retrieval
    Ewerth, R; Springstein, M; Phan-Vogtmann, LA; Schütze, J
  • Primitivism in modern art
    Goldwater, RJ
  • Challenges for the multilingual web of data
    Gracia, J; Montiel-Ponsoda, E; Cimiano, P; Gómez-Pérez, A; Buitelaar, P; McCrae, J
  • The WEKA data mining software: an update
    Hall, M; Frank, E; Holmes, G; Pfahringer, B; Reutemann, P; Witten, IH
  • Statistical and structural approaches to texture
    Haralick, RM
  • Understanding and measuring tourist destination images
    Jenkins, OH
  • Diversity, serendipity, novelty, and coverage: a survey and empirical analysis of beyond-accuracy objectives in recommender systems
    Kaminskas, M; Bridge, D
  • The measurement of observer agreement for categorical data
    Landis, JR; Koch, GG
  • Musical exoticism. Images and reflections
    Locke, RP
  • Imagenet large scale visual recognition challenge
    Russakovsky, O; Deng, J; Su, H; Krause, J; Satheesh, S; Ma, S; Huang, Z; Karpathy, A; Khosla, A; Bernstein, M
  • Essay on exoticism: an aesthetics of diversity
    Segalen, V
  • The CIEDE2000 color-difference formula: implementation notes, supplementary test data, and mathematical observations
    Sharma, G; Wu, W; Dalal, EN
  • Collaborative filtering beyond the user-item matrix: a survey of the state of the art and future challenges
    Shi, Y; Larson, M; Hanjalic, A
  • Textural features corresponding to visual perception
    Tamura, H; Mori, S; Yamawaki, T
  • An examination of the role of beneficial image in tourist destination selection
    Tapachai, N; Waryszak, R
  • Yfcc100m: the new data in multimedia research
    Thomee, B; Shamma, DA; Friedland, G; Elizalde, B; Ni, K; Poland, D; Borth, D; Li, LJ