Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Being the Center of Attention

Being the Center of Attention This article proposes a novel study on personality recognition using video data from different scenarios. Our goal is to jointly model nonverbal behavioral cues with contextual information for a robust, multi-scenario, personality recognition system. Therefore, we build a novel multi-stream Convolutional Neural Network (CNN) framework, which considers multiple sources of information. From a given scenario, we extract spatio-temporal motion descriptors from every individual in the scene, spatio-temporal motion descriptors encoding social group dynamics, and proxemics descriptors to encode the interaction with the surrounding context. All the proposed descriptors are mapped to the same feature space facilitating the overall learning effort. Experiments on two public datasets demonstrate the effectiveness of jointly modeling the mutual Person-Context information, outperforming the state-of-the art-results for personality recognition in two different scenarios. Last, we present CNN class activation maps for each personality trait, shedding light on behavioral patterns linked with personality attributes. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Interactive Intelligent Systems (TiiS) Association for Computing Machinery

Being the Center of Attention

Download PDF
20 pages

Loading next page...
 
/lp/association-for-computing-machinery/being-the-center-of-attention-iNlr8yx4KM
Publisher
Association for Computing Machinery
Copyright
Copyright © 2020 ACM
ISSN
2160-6455
eISSN
2160-6463
DOI
10.1145/3338245
Publisher site
See Article on Publisher Site

Abstract

This article proposes a novel study on personality recognition using video data from different scenarios. Our goal is to jointly model nonverbal behavioral cues with contextual information for a robust, multi-scenario, personality recognition system. Therefore, we build a novel multi-stream Convolutional Neural Network (CNN) framework, which considers multiple sources of information. From a given scenario, we extract spatio-temporal motion descriptors from every individual in the scene, spatio-temporal motion descriptors encoding social group dynamics, and proxemics descriptors to encode the interaction with the surrounding context. All the proposed descriptors are mapped to the same feature space facilitating the overall learning effort. Experiments on two public datasets demonstrate the effectiveness of jointly modeling the mutual Person-Context information, outperforming the state-of-the art-results for personality recognition in two different scenarios. Last, we present CNN class activation maps for each personality trait, shedding light on behavioral patterns linked with personality attributes.

Journal

ACM Transactions on Interactive Intelligent Systems (TiiS) – Association for Computing Machinery

Published: Nov 9, 2020

Keywords: CNN networks

References

  • Buildings with an attitude
    Achten, Henri
  • Salsa: A novel dataset for multimodal group behavior analysis
    Alameda-Pineda, Xavier; Staiano, Jacopo; Subramanian, Ramanathan; Batrinca, Ligia; Ricci, Elisa; Lepri, Bruno; Lanz, Oswald; Sebe, Nicu
  • Analyzing free-standing conversational groups: A multimodal approach
    Alameda-Pineda, Xavier; Yan, Yan; Ricci, Elisa; Lanz, Oswald; Sebe, Nicu
  • Multimodal depression detection: Fusion analysis of paralinguistic, head pose and eye gaze behaviors
    Alghowinem, Sharifa; Goecke, Roland; Wagner, Michael; Epps, Julien; Hyett, Matthew; Parker, Gordon; Breakspear, Michael
  • Social scene understanding: End-to-end multi-person action localization and collective activity recognition
    Bagautdinov, Timur M.; Alahi, Alexandre; Fleuret, François; Fua, Pascal; Savarese, Silvio
  • Model-based gaussian and non-gaussian clustering
    Banfield, Jeffrey D.; Raftery, Adrian E.
  • Investigation of small group social interactions using deep visual activity-based nonverbal features
    Beyan, Cigdem; Shahid, Muhammad; Murino, Vittorio
  • Structural recurrent neural network (SRNN) for group activity analysis
    Biswas, Sovan; Gall, Juergen
  • The role of ego-control and ego-resiliency in the organization of behavior
    Block, Jack; Block, Jeanne H.
  • Face recognition technology: Security versus privacy
    Bowyer, Kevin W.
  • The opencv library
    Bradski, G.
  • Realtime multi-person 2d pose estimation using part affinity fields
    Cao, Zhe; Simon, Tomas; Wei, Shih-En; Sheikh, Yaser
  • Multimodal human-human-robot interactions (mhhri) dataset for studying personality and engagement
    Celiktutan, Oya; Skordos, Efstratios; Gunes, Hatice
  • Multiple sequence alignment with hierarchical clustering
    Corpet, Florence
  • Social interaction discovery by statistical analysis of f-formations
    Cristani, Marco; Bazzani, Loris; Paggetti, Giulia; Fossati, Andrea; Tosato, Diego; Del Bue, Alessio; Menegaz, Gloria; Murino, Vittorio
  • Socially intelligent surveillance and monitoring: Analysing social dimensions of physical space
    Cristani, Marco; Murino, Vittorio; Vinciarelli, Alessandro
  • Towards computational proxemics: Inferring social relations from interpersonal distances
    Cristani, Marco; Paggetti, Giulia; Vinciarelli, Alessandro; Bazzani, Loris; Menegaz, Gloria; Murino, Vittorio
  • Behavior and personality analysis in a nonsocial context dataset
    Dotti, Dario; Popa, Mirela; Asteriadis, Stylianos
  • On seeing human: A three-factor theory of anthropomorphism
    Epley, Nicholas; Waytz, Adam; Cacioppo, John T.
  • Multimodal first impression analysis with deep residual networks
    Güçlütürk, Yağmur; Güçlü, Umut; Baro, Xavier; Escalante, Hugo Jair; Guyon, Isabelle; Escalera, Sergio; Van Gerven, Marcel A. J.; Van Lier, Rob
  • User engineering principles for interactive systems
    Hansen, Wilfred J.
  • Labanotation
    Hutchinson, Ann
  • The big five trait taxonomy: History, measurement, and theoretical perspectives
    John, Oliver P.; Srivastava, Sanjay
  • A new representation of skeleton sequences for 3D action recognition
    Ke, Qiuhong; Bennamoun, Mohammed; An, Senjian; Sohel, Ferdous; Boussaid, Farid
  • Motion cues that make an impression: Predicting perceived personality by minimal motion information
    Koppensteiner, Markus
  • Interplay between personal goals and classroom goal structures in predicting student outcomes: A multilevel analysis of person-context interactions
    Lau, Shun; Nie, Youyan
  • Using interlocutor-modulated attention BLSTM to predict personality traits in small group interaction
    Lin, Yun-Shao; Lee, Chi-Chun
  • Skepxels: Spatio-temporal image representation of human skeleton joints for action recognition
    Liu, Jian; Akhtar, Naveed; Mian, Ajmal
  • Size matters: Variation in personal network size, personality and effect on information transmission
    Lu, Yu-En; Roberts, Sam; Lio, Pietro; Dunbar, Robin; Crowcroft, Jon
  • Towards personality-based user adaptation: Psychologically informed stylistic language generation
    Mairesse, François; Walker, Marilyn A.
  • Actions in context
    Marszalek, Marcin; Laptev, Ivan; Schmid, Cordelia
  • Personality and personal networks
    McCarty, Christopher; Green, H. D.
  • An introduction to the five-factor model and its applications
    McCrae, Robert R.; John, Oliver P.
  • AMIGOS: A dataset for affect, personality and mood research on individuals and groups
    Miranda-Correa, Juan Abdon; Abadi, Mojtaba Khomami; Sebe, Nicu; Patras, Ioannis
  • Analyzing tracklets for the detection of abnormal crowd behavior
    Mousavi, Hossein; Mohammadi, Sadegh; Perina, Alessandro; Chellali, Ryad; Mur, Vittorio
  • Rectified linear units improve restricted boltzmann machines
    Nair, Vinod; Hinton, Geoffrey E.
  • Chalearn lap 2016: First round challenge on first impressions-dataset and results
    Ponce-López, Víctor; Chen, Baiyu; Oliu, Marc; Corneanu, Ciprian; Clapés, Albert; Guyon, Isabelle; Baró, Xavier; Escalante, Hugo Jair; Escalera, Sergio
  • Measuring personality in one minute or less: A 10-item short version of the big five inventory in english and german
    Rammstedt, Beatrice; John, Oliver P.
  • Probabilistic human interaction understanding: Exploring relationship between human body motion and the environmental context
    Roudposhti, Kamrad Khoshhal; Dias, Jorge
  • Probabilistic social behavior analysis by exploring body motion-based patterns
    Roudposhti, Kamrad Khoshhal; Nunes, Urbano; Dias, Jorge
  • Imagenet large scale visual recognition challenge
    Russakovsky, Olga; Deng, Jia; Su, Hao; Krause, Jonathan; Satheesh, Sanjeev; Ma, Sean; Huang, Zhiheng; Karpathy, Andrej; Khosla, Aditya; Bernstein, Michael; et al,
  • A nonverbal behavior approach to identify emergent leaders in small groups
    Sanchez-Cortes, Dairazalia; Aran, Oya; Mast, Marianne Schmid; Gatica-Perez, Daniel
  • The geographic distribution of big five personality traits: Patterns and profiles of human self-description across 56 nations
    Schmitt, David P.; Allik, Jüri; McCrae, Robert R.; Benet-Martínez, Verónica
  • Two-stream convolutional networks for action recognition in videos
    Simonyan, Karen; Zisserman, Andrew
  • Very deep convolutional networks for large-scale image recognition
    Simonyan, Karen; Zisserman, Andrew
  • Early versus late fusion in semantic video analysis
    Snoek, Cees G. M.; Worring, Marcel; Smeulders, Arnold W. M.
  • Personality and sexual relations
    Snyder, Mark; Simpson, Jeffry A.; Gangestad, Steve
  • Hands-off therapist robot behavior adaptation to user personality for post-stroke rehabilitation therapy
    Tapus, Adriana; Tapus, Cristian; Mataric, Maja J
  • A survey of personality computing
    Vinciarelli, Alessandro; Mohammadi, Gelareh
  • Towards intelligent caring agents for aging-in-place: Issues and challenges
    Wang, D.; Subagdja, B.; Kang, Y.; Tan, A. H.; Zhang, D.
  • Recurrent modeling of interaction context for collective activity recognition
    Wang, Minsi; Ni, Bingbing; Yang, Xiaokang
  • RGB-D-based human motion recognition with deep learning: A survey
    Wang, P.; Li, W.; Ogunbona, P.; Wan, J.; Escalera, S.
  • Sensing behavioral change over time: Using within-person variability features from mobile sensing to predict personality traits
    Wang, Weichen; Harari, Gabriella M.; Wang, Rui; Müller, Sandrine R.; Mirjafari, Shayan; Masaba, Kizito; Campbell, Andrew T.
  • Deep bimodal regression of apparent personality traits from short video sequences
    Wei, Xiu-Shen; Zhang, Chen-Lin; Zhang, Hao; Wu, Jianxin
  • Free viewpoint action recognition using motion history volumes
    Weinland, Daniel; Ronfard, Remi; Boyer, Edmond
  • Gender differences in personality across the ten aspects of the big five
    Weisberg, Yanna J.; DeYoung, Colin G.; Hirsh, Jacob B.
  • Principal component analysis
    Wold, Svante; Esbensen, Kim; Geladi, Paul
  • Recognizing human-object interactions in still images by modeling the mutual context of objects and human poses
    Yao, Bangpeng; Fei-Fei, Li
  • Pedestrian behavior modeling from stationary crowds with applications to intelligent surveillance
    Yi, Shuai; Li, Hongsheng; Wang, Xiaogang
  • Space speaks: Towards socially and personality aware visual surveillance
    Zen, Gloria; Lepri, Bruno; Ricci, Elisa; Lanz, Oswald
  • PoseFlow: A deep motion representation for understanding human behaviors in videos
    Zhang, Dingwen; Guo, Guangyu; Huang, Dong; Han, Junwei
  • PersEmoN: A deep network for joint analysis of apparent personality, emotion and their relationship
    Zhang, Le; Peng, Songyou; Winkler, Stefan
  • Heterogeneous features integration via semi-supervised multi-modal deep networks
    Zhao, Lei; Hu, Qinghua; Zhou, Yucan
  • Learning deep features for discriminative localization
    Zhou, Bolei; Khosla, Aditya; Lapedriza, Agata; Oliva, Aude; Torralba, Antonio