Going Deeper than Tracking: A Survey of Computer-Vision Based Recognition of Animal Pain and Emotions

Sofia Broomé; Marcelo Feighelstein; Anna Zamansky; Gabriel Carreira Lencioni; Pia Haubro Andersen; Francisca Pessanha; Marwa Mahmoud; Hedvig Kjellström; Albert Ali Salah

doi:10.1007/s11263-022-01716-3

Going Deeper than Tracking: A Survey of Computer-Vision Based Recognition of Animal Pain and Emotions

Sofia Broomé^*, Marcelo Feighelstein, Anna Zamansky, Gabriel Carreira Lencioni, Pia Haubro Andersen, Francisca Pessanha, Marwa Mahmoud, Hedvig Kjellström, Albert Ali Salah

^*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

Advances in animal motion tracking and pose recognition have been a game changer in the study of animal behavior. Recently, an increasing number of works go ‘deeper’ than tracking, and address automated recognition of animals’ internal states such as emotions and pain with the aim of improving animal welfare, making this a timely moment for a systematization of the field. This paper provides a comprehensive survey of computer vision-based research on recognition of pain and emotional states in animals, addressing both facial and bodily behavior analysis. We summarize the efforts that have been presented so far within this topic—classifying them across different dimensions, highlight challenges and research gaps, and provide best practice recommendations for advancing the field, and some future directions for research.

Original language	English
Pages (from-to)	572-590
Number of pages	19
Journal	International Journal of Computer Vision
Volume	131
Issue number	2
DOIs	https://doi.org/10.1007/s11263-022-01716-3
Publication status	Published - Feb 2023

Bibliographical note

Funding Information:
The research was partially supported by the grant from the Ministry of Science and Technology of Israel according to the research Project No. 19-57-06007. The second author was additionally supported by the Data Science Research Center (DSRC), University of Haifa. The authors would like to thank Ilan Shimshoni and Shir Amir for their scientific consultations.

Funding Information:
The research was partially supported by the grant from the Ministry of Science and Technology of Israel according to the research Project No. 19-57-06007. The second author was additionally supported by the Data Science Research Center (DSRC), University of Haifa. The authors would like to thank Ilan Shimshoni and Shir Amir for their scientific consultations.

Publisher Copyright:
© 2022, The Author(s).

Funding

The research was partially supported by the grant from the Ministry of Science and Technology of Israel according to the research Project No. 19-57-06007. The second author was additionally supported by the Data Science Research Center (DSRC), University of Haifa. The authors would like to thank Ilan Shimshoni and Shir Amir for their scientific consultations. The research was partially supported by the grant from the Ministry of Science and Technology of Israel according to the research Project No. 19-57-06007. The second author was additionally supported by the Data Science Research Center (DSRC), University of Haifa. The authors would like to thank Ilan Shimshoni and Shir Amir for their scientific consultations.

Funders	Funder number
Royal Institute of Technology

Keywords

Affective computing
Computer vision for animals
Emotion recognition
Non-human behavior analysis
Pain estimation
Pain recognition

Access to Document

10.1007/s11263-022-01716-3Licence: CC BY

s11263-022-01716-3Final published version, 621 KBLicence: CC BY

Cite this

@article{28d46222d6124f388fb410adfe446682,

title = "Going Deeper than Tracking: A Survey of Computer-Vision Based Recognition of Animal Pain and Emotions",

abstract = "Advances in animal motion tracking and pose recognition have been a game changer in the study of animal behavior. Recently, an increasing number of works go {\textquoteleft}deeper{\textquoteright} than tracking, and address automated recognition of animals{\textquoteright} internal states such as emotions and pain with the aim of improving animal welfare, making this a timely moment for a systematization of the field. This paper provides a comprehensive survey of computer vision-based research on recognition of pain and emotional states in animals, addressing both facial and bodily behavior analysis. We summarize the efforts that have been presented so far within this topic—classifying them across different dimensions, highlight challenges and research gaps, and provide best practice recommendations for advancing the field, and some future directions for research.",

keywords = "Affective computing, Computer vision for animals, Emotion recognition, Non-human behavior analysis, Pain estimation, Pain recognition",

author = "Sofia Broom{\'e} and Marcelo Feighelstein and Anna Zamansky and {Carreira Lencioni}, Gabriel and Andersen, {Pia Haubro} and Francisca Pessanha and Marwa Mahmoud and Hedvig Kjellstr{\"o}m and Salah, {Albert Ali}",

note = "Funding Information: The research was partially supported by the grant from the Ministry of Science and Technology of Israel according to the research Project No. 19-57-06007. The second author was additionally supported by the Data Science Research Center (DSRC), University of Haifa. The authors would like to thank Ilan Shimshoni and Shir Amir for their scientific consultations. Funding Information: The research was partially supported by the grant from the Ministry of Science and Technology of Israel according to the research Project No. 19-57-06007. The second author was additionally supported by the Data Science Research Center (DSRC), University of Haifa. The authors would like to thank Ilan Shimshoni and Shir Amir for their scientific consultations. Publisher Copyright: {\textcopyright} 2022, The Author(s).",

year = "2023",

month = feb,

doi = "10.1007/s11263-022-01716-3",

language = "English",

volume = "131",

pages = "572--590",

journal = "International Journal of Computer Vision",

issn = "0920-5691",

publisher = "Springer Netherlands",

number = "2",

}

TY - JOUR

T1 - Going Deeper than Tracking

T2 - A Survey of Computer-Vision Based Recognition of Animal Pain and Emotions

AU - Broomé, Sofia

AU - Feighelstein, Marcelo

AU - Zamansky, Anna

AU - Carreira Lencioni, Gabriel

AU - Andersen, Pia Haubro

AU - Pessanha, Francisca

AU - Mahmoud, Marwa

AU - Kjellström, Hedvig

AU - Salah, Albert Ali

N1 - Funding Information: The research was partially supported by the grant from the Ministry of Science and Technology of Israel according to the research Project No. 19-57-06007. The second author was additionally supported by the Data Science Research Center (DSRC), University of Haifa. The authors would like to thank Ilan Shimshoni and Shir Amir for their scientific consultations. Funding Information: The research was partially supported by the grant from the Ministry of Science and Technology of Israel according to the research Project No. 19-57-06007. The second author was additionally supported by the Data Science Research Center (DSRC), University of Haifa. The authors would like to thank Ilan Shimshoni and Shir Amir for their scientific consultations. Publisher Copyright: © 2022, The Author(s).

PY - 2023/2

Y1 - 2023/2

N2 - Advances in animal motion tracking and pose recognition have been a game changer in the study of animal behavior. Recently, an increasing number of works go ‘deeper’ than tracking, and address automated recognition of animals’ internal states such as emotions and pain with the aim of improving animal welfare, making this a timely moment for a systematization of the field. This paper provides a comprehensive survey of computer vision-based research on recognition of pain and emotional states in animals, addressing both facial and bodily behavior analysis. We summarize the efforts that have been presented so far within this topic—classifying them across different dimensions, highlight challenges and research gaps, and provide best practice recommendations for advancing the field, and some future directions for research.

AB - Advances in animal motion tracking and pose recognition have been a game changer in the study of animal behavior. Recently, an increasing number of works go ‘deeper’ than tracking, and address automated recognition of animals’ internal states such as emotions and pain with the aim of improving animal welfare, making this a timely moment for a systematization of the field. This paper provides a comprehensive survey of computer vision-based research on recognition of pain and emotional states in animals, addressing both facial and bodily behavior analysis. We summarize the efforts that have been presented so far within this topic—classifying them across different dimensions, highlight challenges and research gaps, and provide best practice recommendations for advancing the field, and some future directions for research.

KW - Affective computing

KW - Computer vision for animals

KW - Emotion recognition

KW - Non-human behavior analysis

KW - Pain estimation

KW - Pain recognition

UR - http://www.scopus.com/inward/record.url?scp=85142475831&partnerID=8YFLogxK

U2 - 10.1007/s11263-022-01716-3

DO - 10.1007/s11263-022-01716-3

M3 - Article

AN - SCOPUS:85142475831

SN - 0920-5691

VL - 131

SP - 572

EP - 590

JO - International Journal of Computer Vision

JF - International Journal of Computer Vision

IS - 2

ER -

Going Deeper than Tracking: A Survey of Computer-Vision Based Recognition of Animal Pain and Emotions

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this