Automated Detection of Joint Attention and Mutual Gaze in Free Play Parent-Child Interactions

Peitong Li; Hui Lu; Ronald W. Poppe; Albert Ali Salah

doi:10.1145/3610661.3616234

Automated Detection of Joint Attention and Mutual Gaze in Free Play Parent-Child Interactions

Peitong Li, Hui Lu, Ronald W. Poppe, Albert Ali Salah

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Observing a child’s interaction with their parents can provide us with important information about the child’s cognitive development. Nonverbal cues such as joint attention and mutual gaze can indicate a child’s engagement, and have diagnostic value. Since manual coding of gaze events during child-parent interactions is time-consuming and error-prone, there is a need for automatic assessment tools, capable of working with camera recordings without specialized eye-tracking equipment. There are few studies in this setting, and accessing naturalistic parent-child videos is difficult. In this paper, we investigate the feasibility of detecting joint attention and mutual gaze in videos. We test approach on challenging data of a child and a parent engaged in free play. By combining multiple off-the-shelf approaches, we manage to create a system that does not require much labeling and is flexible to use for view-independent interaction analysis.

Original language	English
Title of host publication	ICMI 2023 Companion - Companion Publication of the 25th International Conference on Multimodal Interaction
Subtitle of host publication	Companion Publication of the 25th International Conference on Multimodal Interaction
Editors	Elisabeth André, Mohamed Chetouani
Publisher	Association for Computing Machinery
Pages	374–382
Number of pages	9
ISBN (Electronic)	9798400703218
ISBN (Print)	979-8-4007-0321-8
DOIs	https://doi.org/10.1145/3610661.3616234
Publication status	Published - 9 Oct 2023

Publication series

Name	ACM International Conference Proceeding Series

Bibliographical note

Publisher Copyright:
© 2023 ACM.

Funding

The work is partly funded by the China Scholarship Council (CSC).

Funders	Funder number
China Scholarship Council

Keywords

cognitive development
joint attention
mutual gaze
parent-child interaction

Access to Document

10.1145/3610661.3616234

3610661.3616234Final published version, 883 KBLicence: Taverne

Cite this

Li, P., Lu, H., Poppe, R. W., & Salah, A. A. (2023). Automated Detection of Joint Attention and Mutual Gaze in Free Play Parent-Child Interactions. In E. André, & M. Chetouani (Eds.), ICMI 2023 Companion - Companion Publication of the 25th International Conference on Multimodal Interaction: Companion Publication of the 25th International Conference on Multimodal Interaction (pp. 374–382). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3610661.3616234

Li, Peitong ; Lu, Hui ; Poppe, Ronald W. et al. / Automated Detection of Joint Attention and Mutual Gaze in Free Play Parent-Child Interactions. ICMI 2023 Companion - Companion Publication of the 25th International Conference on Multimodal Interaction: Companion Publication of the 25th International Conference on Multimodal Interaction. editor / Elisabeth André ; Mohamed Chetouani. Association for Computing Machinery, 2023. pp. 374–382 (ACM International Conference Proceeding Series).

@inproceedings{1ea423911b39403c94f1b1bb632cea62,

title = "Automated Detection of Joint Attention and Mutual Gaze in Free Play Parent-Child Interactions",

abstract = "Observing a child{\textquoteright}s interaction with their parents can provide us with important information about the child{\textquoteright}s cognitive development. Nonverbal cues such as joint attention and mutual gaze can indicate a child{\textquoteright}s engagement, and have diagnostic value. Since manual coding of gaze events during child-parent interactions is time-consuming and error-prone, there is a need for automatic assessment tools, capable of working with camera recordings without specialized eye-tracking equipment. There are few studies in this setting, and accessing naturalistic parent-child videos is difficult. In this paper, we investigate the feasibility of detecting joint attention and mutual gaze in videos. We test approach on challenging data of a child and a parent engaged in free play. By combining multiple off-the-shelf approaches, we manage to create a system that does not require much labeling and is flexible to use for view-independent interaction analysis.",

keywords = "cognitive development, joint attention, mutual gaze, parent-child interaction",

author = "Peitong Li and Hui Lu and Poppe, {Ronald W.} and Salah, {Albert Ali}",

note = "Publisher Copyright: {\textcopyright} 2023 ACM.",

year = "2023",

month = oct,

day = "9",

doi = "10.1145/3610661.3616234",

language = "English",

isbn = "979-8-4007-0321-8",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

pages = "374–382",

editor = "Elisabeth Andr{\'e} and Chetouani, {Mohamed }",

booktitle = "ICMI 2023 Companion - Companion Publication of the 25th International Conference on Multimodal Interaction",

address = "United States",

}

Li, P , Lu, H , Poppe, RW & Salah, AA 2023, Automated Detection of Joint Attention and Mutual Gaze in Free Play Parent-Child Interactions. in E André & M Chetouani (eds), ICMI 2023 Companion - Companion Publication of the 25th International Conference on Multimodal Interaction: Companion Publication of the 25th International Conference on Multimodal Interaction. ACM International Conference Proceeding Series, Association for Computing Machinery, pp. 374–382. https://doi.org/10.1145/3610661.3616234

Automated Detection of Joint Attention and Mutual Gaze in Free Play Parent-Child Interactions. / Li, Peitong ; Lu, Hui ; Poppe, Ronald W. et al.
ICMI 2023 Companion - Companion Publication of the 25th International Conference on Multimodal Interaction: Companion Publication of the 25th International Conference on Multimodal Interaction. ed. / Elisabeth André; Mohamed Chetouani. Association for Computing Machinery, 2023. p. 374–382 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Automated Detection of Joint Attention and Mutual Gaze in Free Play Parent-Child Interactions

AU - Li, Peitong

AU - Lu, Hui

AU - Poppe, Ronald W.

AU - Salah, Albert Ali

PY - 2023/10/9

Y1 - 2023/10/9

N2 - Observing a child’s interaction with their parents can provide us with important information about the child’s cognitive development. Nonverbal cues such as joint attention and mutual gaze can indicate a child’s engagement, and have diagnostic value. Since manual coding of gaze events during child-parent interactions is time-consuming and error-prone, there is a need for automatic assessment tools, capable of working with camera recordings without specialized eye-tracking equipment. There are few studies in this setting, and accessing naturalistic parent-child videos is difficult. In this paper, we investigate the feasibility of detecting joint attention and mutual gaze in videos. We test approach on challenging data of a child and a parent engaged in free play. By combining multiple off-the-shelf approaches, we manage to create a system that does not require much labeling and is flexible to use for view-independent interaction analysis.

AB - Observing a child’s interaction with their parents can provide us with important information about the child’s cognitive development. Nonverbal cues such as joint attention and mutual gaze can indicate a child’s engagement, and have diagnostic value. Since manual coding of gaze events during child-parent interactions is time-consuming and error-prone, there is a need for automatic assessment tools, capable of working with camera recordings without specialized eye-tracking equipment. There are few studies in this setting, and accessing naturalistic parent-child videos is difficult. In this paper, we investigate the feasibility of detecting joint attention and mutual gaze in videos. We test approach on challenging data of a child and a parent engaged in free play. By combining multiple off-the-shelf approaches, we manage to create a system that does not require much labeling and is flexible to use for view-independent interaction analysis.

KW - cognitive development

KW - joint attention

KW - mutual gaze

KW - parent-child interaction

UR - http://www.scopus.com/inward/record.url?scp=85175811187&partnerID=8YFLogxK

U2 - 10.1145/3610661.3616234

DO - 10.1145/3610661.3616234

M3 - Conference contribution

SN - 979-8-4007-0321-8

T3 - ACM International Conference Proceeding Series

SP - 374

EP - 382

BT - ICMI 2023 Companion - Companion Publication of the 25th International Conference on Multimodal Interaction

A2 - André, Elisabeth

A2 - Chetouani, Mohamed

PB - Association for Computing Machinery

ER -

Li P , Lu H , Poppe RW , Salah AA. Automated Detection of Joint Attention and Mutual Gaze in Free Play Parent-Child Interactions. In André E, Chetouani M, editors, ICMI 2023 Companion - Companion Publication of the 25th International Conference on Multimodal Interaction: Companion Publication of the 25th International Conference on Multimodal Interaction. Association for Computing Machinery. 2023. p. 374–382. (ACM International Conference Proceeding Series). doi: 10.1145/3610661.3616234

Automated Detection of Joint Attention and Mutual Gaze in Free Play Parent-Child Interactions

Abstract

Publication series

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this