Modeling, Recognizing, and Explaining Apparent Personality from Videos

Hugo Jair Escalante; Heysem Kaya; Albert Ali Salah; Sergio Escalera; Yagmur Gucluturk; Umut Guclu; Xavier Baro; Isabelle Guyon; Julio C.S. Jacques; Meysam Madadi; Stephane Ayache; Evelyne Viegas; Furkan Gurpinar; Achmadnoer Sukma Wicaksana; Cynthia Liem; Marcel A.J. Van Gerven; Rob Van Lier

doi:10.1109/TAFFC.2020.2973984

Modeling, Recognizing, and Explaining Apparent Personality from Videos

Hugo Jair Escalante^*, Heysem Kaya, Albert Ali Salah, Sergio Escalera, Yagmur Gucluturk, Umut Guclu, Xavier Baro, Isabelle Guyon, Julio C.S. Jacques, Meysam Madadi, Stephane Ayache, Evelyne Viegas, Furkan Gurpinar, Achmadnoer Sukma Wicaksana, Cynthia Liem, Marcel A.J. Van Gerven, Rob Van Lier

^*Corresponding author for this work

Sub Social and Affective Computing

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

Explainability and interpretability are two critical aspects of decision support systems. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of apparent personality recognition. To the best of our knowledge, this is the first effort in this direction. We describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, evaluation protocol, proposed solutions and summarize the results of the challenge. We investigate the issue of bias in detail. Finally, derived from our study, we outline research opportunities that we foresee will be relevant in this area in the near future.

Original language	English
Pages (from-to)	894-911
Number of pages	18
Journal	IEEE Transactions on Affective Computing
Volume	13
Issue number	2
Early online date	14 Feb 2020
DOIs	https://doi.org/10.1109/TAFFC.2020.2973984
Publication status	Published - Apr 2022

Keywords

Algorithmic accountability
Computational modeling
Computer vision
Explainable computer vision
Face
First impressions
Interviews
Multimodal information
Personality analysis
Predictive models
Videos
Visualization

Access to Document

10.1109/TAFFC.2020.2973984

Modeling_Recognizing_and_Explaining_Apparent_Personality_From_VideosFinal published version, 3.03 MBLicence: Taverne

Cite this

Escalante, H. J., Kaya, H., Salah, A. A., Escalera, S., Gucluturk, Y., Guclu, U., Baro, X., Guyon, I., Jacques, J. C. S., Madadi, M., Ayache, S., Viegas, E., Gurpinar, F., Wicaksana, A. S., Liem, C., Van Gerven, M. A. J., & Van Lier, R. (2022). Modeling, Recognizing, and Explaining Apparent Personality from Videos. IEEE Transactions on Affective Computing, 13(2), 894-911. https://doi.org/10.1109/TAFFC.2020.2973984

@article{718c31754538492cba7b25ba0c4ccac4,

title = "Modeling, Recognizing, and Explaining Apparent Personality from Videos",

abstract = "Explainability and interpretability are two critical aspects of decision support systems. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of apparent personality recognition. To the best of our knowledge, this is the first effort in this direction. We describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, evaluation protocol, proposed solutions and summarize the results of the challenge. We investigate the issue of bias in detail. Finally, derived from our study, we outline research opportunities that we foresee will be relevant in this area in the near future.",

keywords = "Algorithmic accountability, Computational modeling, Computer vision, Explainable computer vision, Face, First impressions, Interviews, Multimodal information, Personality analysis, Predictive models, Videos, Visualization",

author = "Escalante, {Hugo Jair} and Heysem Kaya and Salah, {Albert Ali} and Sergio Escalera and Yagmur Gucluturk and Umut Guclu and Xavier Baro and Isabelle Guyon and Jacques, {Julio C.S.} and Meysam Madadi and Stephane Ayache and Evelyne Viegas and Furkan Gurpinar and Wicaksana, {Achmadnoer Sukma} and Cynthia Liem and {Van Gerven}, {Marcel A.J.} and {Van Lier}, Rob",

year = "2022",

month = apr,

doi = "10.1109/TAFFC.2020.2973984",

language = "English",

volume = "13",

pages = "894--911",

journal = "IEEE Transactions on Affective Computing",

issn = "1949-3045",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "2",

}

Escalante, HJ, Kaya, H , Salah, AA, Escalera, S, Gucluturk, Y, Guclu, U, Baro, X, Guyon, I, Jacques, JCS, Madadi, M, Ayache, S, Viegas, E, Gurpinar, F, Wicaksana, AS, Liem, C, Van Gerven, MAJ & Van Lier, R 2022, 'Modeling, Recognizing, and Explaining Apparent Personality from Videos', IEEE Transactions on Affective Computing, vol. 13, no. 2, pp. 894-911. https://doi.org/10.1109/TAFFC.2020.2973984

TY - JOUR

T1 - Modeling, Recognizing, and Explaining Apparent Personality from Videos

AU - Escalante, Hugo Jair

AU - Kaya, Heysem

AU - Salah, Albert Ali

AU - Escalera, Sergio

AU - Gucluturk, Yagmur

AU - Guclu, Umut

AU - Baro, Xavier

AU - Guyon, Isabelle

AU - Jacques, Julio C.S.

AU - Madadi, Meysam

AU - Ayache, Stephane

AU - Viegas, Evelyne

AU - Gurpinar, Furkan

AU - Wicaksana, Achmadnoer Sukma

AU - Liem, Cynthia

AU - Van Gerven, Marcel A.J.

AU - Van Lier, Rob

PY - 2022/4

Y1 - 2022/4

N2 - Explainability and interpretability are two critical aspects of decision support systems. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of apparent personality recognition. To the best of our knowledge, this is the first effort in this direction. We describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, evaluation protocol, proposed solutions and summarize the results of the challenge. We investigate the issue of bias in detail. Finally, derived from our study, we outline research opportunities that we foresee will be relevant in this area in the near future.

AB - Explainability and interpretability are two critical aspects of decision support systems. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of apparent personality recognition. To the best of our knowledge, this is the first effort in this direction. We describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, evaluation protocol, proposed solutions and summarize the results of the challenge. We investigate the issue of bias in detail. Finally, derived from our study, we outline research opportunities that we foresee will be relevant in this area in the near future.

KW - Algorithmic accountability

KW - Computational modeling

KW - Computer vision

KW - Explainable computer vision

KW - Face

KW - First impressions

KW - Interviews

KW - Multimodal information

KW - Personality analysis

KW - Predictive models

KW - Videos

KW - Visualization

UR - http://www.scopus.com/inward/record.url?scp=85085992577&partnerID=8YFLogxK

U2 - 10.1109/TAFFC.2020.2973984

DO - 10.1109/TAFFC.2020.2973984

M3 - Article

AN - SCOPUS:85085992577

SN - 1949-3045

VL - 13

SP - 894

EP - 911

JO - IEEE Transactions on Affective Computing

JF - IEEE Transactions on Affective Computing

IS - 2

ER -

Modeling, Recognizing, and Explaining Apparent Personality from Videos

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this