SHREC 2024: Recognition of dynamic hand motions molding clay

Ben Veldhuijzen; Remco C. Veltkamp; Omar Ikne; Benjamin Allaert; Hazem Wannous; Marco Emporio; Andrea Giachetti; Joseph J. LaViola; Ruiwen He; Halim Benhabiles; Adnane Cabani; Anthony Fleury; Karim Hammoudi; Konstantinos Gavalas; Christoforos Vlachos; Athanasios Papanikolaou; Ioannis Romanelis; Vlassis Fotis; Gerasimos Arvanitis; Konstantinos Moustakas; Martin Hanik; Esfandiar Nava-Yazdani; Christoph von Tycowicz

doi:10.1016/j.cag.2024.104012

SHREC 2024: Recognition of dynamic hand motions molding clay

Ben Veldhuijzen^*
, Remco C. Veltkamp
, Omar Ikne
, Benjamin Allaert
, Hazem Wannous
, Marco Emporio
, Andrea Giachetti
, Joseph J. LaViola
, Ruiwen He
, Halim Benhabiles
, Adnane Cabani
, Anthony Fleury
, Karim Hammoudi
, Konstantinos Gavalas
, Christoforos Vlachos
, Athanasios Papanikolaou
, Ioannis Romanelis
, Vlassis Fotis
, Gerasimos Arvanitis
, Konstantinos Moustakas

Martin Hanik, Esfandiar Nava-Yazdani, Christoph von Tycowicz

^*Corresponding author for this work

Utrecht University
Universite de Lille 2
University of Verona
University of Central Florida
École supérieure d'ingénieurs Léonard-de-Vinci (ESILV)
IRSEEM - Institut de Recherche en Systèmes Electroniques Embarqués
Université de Haute-Alsace
Université de Strasbourg
University of Patras
Free University of Berlin
Technical University of Berlin
Zuse Institute Berlin

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

Gesture recognition is a tool to enable novel interactions with different techniques and applications, like Mixed Reality and Virtual Reality environments. With all the recent advancements in gesture recognition from skeletal data, it is still unclear how well state-of-the-art techniques perform in a scenario using precise motions with two hands. This paper presents the results of the SHREC 2024 contest organized to evaluate methods for their recognition of highly similar hand motions using the skeletal spatial coordinate data of both hands. The task is the recognition of 7 motion classes given their spatial coordinates in a frame-by-frame motion. The skeletal data has been captured using a Vicon system and pre-processed into a coordinate system using Blender and Vicon Shogun Post. We created a small, novel dataset with a high variety of durations in frames. This paper shows the results of the contest, showing the techniques created by the 5 research groups on this challenging task and comparing them to our baseline method.

Original language	English
Article number	104012
Number of pages	11
Journal	Computers and Graphics (Pergamon)
Volume	123
DOIs	https://doi.org/10.1016/j.cag.2024.104012
Publication status	Published - Oct 2024

Bibliographical note

Publisher Copyright:
© 2024 The Authors

Funding

Capturing the hand movements of modeling clay was initiated by artist Isabel Ferrand, who together with Remco Veltkamp received the PUG prize from Provinciaals Utrechts Genootschap Van Kunsten En Wetenschappen. Thanks to potter Kees Agterberg for having his hand movements recorded. Thanks to Tariq Bakhtali for helping in the motion capture sessions.

Funders
Provinciaals Utrechts Genootschap Van Kunsten En Wetenschappen

Keywords

3D shape retrieval challenge
Gesture recognition
Hand skeleton gestures
Motion capture
Neural networks
SHREC

Access to Document

10.1016/j.cag.2024.104012Licence: CC BY

1-s2.0-S009784932400147X-mainFinal published version, 2.27 MBLicence: CC BY

Cite this

Veldhuijzen, B., Veltkamp, R. C., Ikne, O., Allaert, B., Wannous, H., Emporio, M., Giachetti, A., LaViola, J. J., He, R., Benhabiles, H., Cabani, A., Fleury, A., Hammoudi, K., Gavalas, K., Vlachos, C., Papanikolaou, A., Romanelis, I., Fotis, V., Arvanitis, G., ... von Tycowicz, C. (2024). SHREC 2024: Recognition of dynamic hand motions molding clay. Computers and Graphics (Pergamon), 123, Article 104012. https://doi.org/10.1016/j.cag.2024.104012

@article{a21f0cece5fd4cdb94f22a6ed4354161,

title = "SHREC 2024: Recognition of dynamic hand motions molding clay",

abstract = "Gesture recognition is a tool to enable novel interactions with different techniques and applications, like Mixed Reality and Virtual Reality environments. With all the recent advancements in gesture recognition from skeletal data, it is still unclear how well state-of-the-art techniques perform in a scenario using precise motions with two hands. This paper presents the results of the SHREC 2024 contest organized to evaluate methods for their recognition of highly similar hand motions using the skeletal spatial coordinate data of both hands. The task is the recognition of 7 motion classes given their spatial coordinates in a frame-by-frame motion. The skeletal data has been captured using a Vicon system and pre-processed into a coordinate system using Blender and Vicon Shogun Post. We created a small, novel dataset with a high variety of durations in frames. This paper shows the results of the contest, showing the techniques created by the 5 research groups on this challenging task and comparing them to our baseline method.",

keywords = "3D shape retrieval challenge, Gesture recognition, Hand skeleton gestures, Motion capture, Neural networks, SHREC",

author = "Ben Veldhuijzen and Veltkamp, \{Remco C.\} and Omar Ikne and Benjamin Allaert and Hazem Wannous and Marco Emporio and Andrea Giachetti and LaViola, \{Joseph J.\} and Ruiwen He and Halim Benhabiles and Adnane Cabani and Anthony Fleury and Karim Hammoudi and Konstantinos Gavalas and Christoforos Vlachos and Athanasios Papanikolaou and Ioannis Romanelis and Vlassis Fotis and Gerasimos Arvanitis and Konstantinos Moustakas and Martin Hanik and Esfandiar Nava-Yazdani and \{von Tycowicz\}, Christoph",

note = "Publisher Copyright: {\textcopyright} 2024 The Authors",

year = "2024",

month = oct,

doi = "10.1016/j.cag.2024.104012",

language = "English",

volume = "123",

journal = "Computers and Graphics (Pergamon)",

issn = "0097-8493",

publisher = "Elsevier Ltd",

}

Veldhuijzen, B, Veltkamp, RC, Ikne, O, Allaert, B, Wannous, H, Emporio, M, Giachetti, A, LaViola, JJ, He, R, Benhabiles, H, Cabani, A, Fleury, A, Hammoudi, K, Gavalas, K, Vlachos, C, Papanikolaou, A, Romanelis, I, Fotis, V, Arvanitis, G, Moustakas, K, Hanik, M, Nava-Yazdani, E & von Tycowicz, C 2024, 'SHREC 2024: Recognition of dynamic hand motions molding clay', Computers and Graphics (Pergamon), vol. 123, 104012. https://doi.org/10.1016/j.cag.2024.104012

TY - JOUR

T1 - SHREC 2024

T2 - Recognition of dynamic hand motions molding clay

AU - Veldhuijzen, Ben

AU - Veltkamp, Remco C.

AU - Ikne, Omar

AU - Allaert, Benjamin

AU - Wannous, Hazem

AU - Emporio, Marco

AU - Giachetti, Andrea

AU - LaViola, Joseph J.

AU - He, Ruiwen

AU - Benhabiles, Halim

AU - Cabani, Adnane

AU - Fleury, Anthony

AU - Hammoudi, Karim

AU - Gavalas, Konstantinos

AU - Vlachos, Christoforos

AU - Papanikolaou, Athanasios

AU - Romanelis, Ioannis

AU - Fotis, Vlassis

AU - Arvanitis, Gerasimos

AU - Moustakas, Konstantinos

AU - Hanik, Martin

AU - Nava-Yazdani, Esfandiar

AU - von Tycowicz, Christoph

PY - 2024/10

Y1 - 2024/10

N2 - Gesture recognition is a tool to enable novel interactions with different techniques and applications, like Mixed Reality and Virtual Reality environments. With all the recent advancements in gesture recognition from skeletal data, it is still unclear how well state-of-the-art techniques perform in a scenario using precise motions with two hands. This paper presents the results of the SHREC 2024 contest organized to evaluate methods for their recognition of highly similar hand motions using the skeletal spatial coordinate data of both hands. The task is the recognition of 7 motion classes given their spatial coordinates in a frame-by-frame motion. The skeletal data has been captured using a Vicon system and pre-processed into a coordinate system using Blender and Vicon Shogun Post. We created a small, novel dataset with a high variety of durations in frames. This paper shows the results of the contest, showing the techniques created by the 5 research groups on this challenging task and comparing them to our baseline method.

AB - Gesture recognition is a tool to enable novel interactions with different techniques and applications, like Mixed Reality and Virtual Reality environments. With all the recent advancements in gesture recognition from skeletal data, it is still unclear how well state-of-the-art techniques perform in a scenario using precise motions with two hands. This paper presents the results of the SHREC 2024 contest organized to evaluate methods for their recognition of highly similar hand motions using the skeletal spatial coordinate data of both hands. The task is the recognition of 7 motion classes given their spatial coordinates in a frame-by-frame motion. The skeletal data has been captured using a Vicon system and pre-processed into a coordinate system using Blender and Vicon Shogun Post. We created a small, novel dataset with a high variety of durations in frames. This paper shows the results of the contest, showing the techniques created by the 5 research groups on this challenging task and comparing them to our baseline method.

KW - 3D shape retrieval challenge

KW - Gesture recognition

KW - Hand skeleton gestures

KW - Motion capture

KW - Neural networks

KW - SHREC

UR - https://www.scopus.com/pages/publications/85199347537

U2 - 10.1016/j.cag.2024.104012

DO - 10.1016/j.cag.2024.104012

M3 - Article

AN - SCOPUS:85199347537

SN - 0097-8493

VL - 123

JO - Computers and Graphics (Pergamon)

JF - Computers and Graphics (Pergamon)

M1 - 104012

ER -

SHREC 2024: Recognition of dynamic hand motions molding clay

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this