Combining deep facial and ambient features for first impression estimation

Furkan Gürpınar, Heysem Kaya*, Albert Ali Salah

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

First impressions influence the behavior of people towards a newly encountered person or a human-like agent. Apart from the physical characteristics of the encountered face, the emotional expressions displayed on it, as well as ambient information affect these impressions. In this work, we propose an approach to predict the first impressions people will have for a given video depicting a face within a context. We employ pre-trained Deep Convolutional Neural Networks to extract facial expressions, as well as ambient information. After video modeling, visual features that represent facial expression and scene are combined and fed to a Kernel Extreme Learning Machine regressor. The proposed system is evaluated on the ChaLearn Challenge Dataset on First Impression Recognition, where the classification target is the “Big Five” personality trait labels for each video. Our system achieved an accuracy of 90.94% on the sequestered test set, 0.36% points below the top system in the competition.

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2016 Workshops, Proceedings
EditorsGang Hua, Herve Jegou
PublisherSpringer
Pages372-385
Number of pages14
ISBN (Print)9783319494081
DOIs
Publication statusPublished - 1 Jan 2016
Event14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands
Duration: 8 Oct 201616 Oct 2016

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9915 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference14th European Conference on Computer Vision, ECCV 2016
Country/TerritoryNetherlands
CityAmsterdam
Period8/10/1616/10/16

Keywords

  • Deep learning
  • ELM
  • First impression
  • Personality traits

Fingerprint

Dive into the research topics of 'Combining deep facial and ambient features for first impression estimation'. Together they form a unique fingerprint.

Cite this