Abstract
https://soundstart.sites.uu.nl/wp-content/uploads/sites/940/2023/04/Book_of_abstracts_MMSYM.pdf
Children learn language through interactions with their caregivers in a multimodal setting. Adults modify their speech and gestures when interacting with infants (for review see Crowe, 2016). Infants, in turn, are sensitive to such adaptations and make use of them (Hollich et al., 2005). Existing research on infant-adult interaction mainly focuses on acoustic properties of infant-directed speech (for review see de Boer, 2011), and their variation across different speakers (Broesch & Bryant, 2015; Ferjan Ramírez, 2022). Little is known about variation in co-speech gestures in infant-adult interaction.
This study therefore aims to answer the question of what the variation is like in infant-directed co-speech gestures related to prosodic phrasing. Live interaction between nine Dutch-speaking mothers and their 5- to 9-month-old infants were elicited in three daily activities: small talk, storytelling, and free play. Variation was operationalised as three types of differences concerning two articulators (i.e. hands, eyebrows) at both between- and within-speaker levels: phonological interconnectedness between intonational phrase (IP) boundaries and gesture types, temporal interconnectedness between IP boundaries and gesture peaks, and variation in gesture intensity peaks. Data annotation consisted of both manual annotation of hand gestures (Rohrer et al., 2021) and automatic annotation of eyebrow gestures using the facial behaviour analysis software OpenFace (Baltrušaitis et al., 2016).
Mixed-effect modelling (logistic regression and linear regression) was conducted on the association of gesture types and IP-final boundaries, distribution of gesture peaks and intensity of gesture peaks using the R package lme4 (Bates et al., 2015). Our analysis yielded evidence for similar phonological interconnectedness in infant-directed interaction and adult-directed interaction. That is, the same types of gestures tended to occur at IP-final boundaries in infant- and adult-directed speech. Moreover, temporal interconnectedness between IP boundaries and gesture peaks was different than previously reported for infant-directed gestures (De la Cruz-Pavía et al., 2020), i.e. gestures peaked closer to the end of an IP end than to its start. Finally, between-speaker variation in infant-directed gestures was observed for all three types of variation, whereas within-speaker variation was observed only for the temporal interconnectedness and variation in intensity peak of eyebrow gestures.
These results raise the question of how variation in infant-directed co-speech gesture influences early prosodic development. Follow-up research concerning the learning of prosodic phrasing in early infancy will be discussed in the presentation.
Keywords: infant-directed interaction; prosodic phrasing; co-speech gesture
Children learn language through interactions with their caregivers in a multimodal setting. Adults modify their speech and gestures when interacting with infants (for review see Crowe, 2016). Infants, in turn, are sensitive to such adaptations and make use of them (Hollich et al., 2005). Existing research on infant-adult interaction mainly focuses on acoustic properties of infant-directed speech (for review see de Boer, 2011), and their variation across different speakers (Broesch & Bryant, 2015; Ferjan Ramírez, 2022). Little is known about variation in co-speech gestures in infant-adult interaction.
This study therefore aims to answer the question of what the variation is like in infant-directed co-speech gestures related to prosodic phrasing. Live interaction between nine Dutch-speaking mothers and their 5- to 9-month-old infants were elicited in three daily activities: small talk, storytelling, and free play. Variation was operationalised as three types of differences concerning two articulators (i.e. hands, eyebrows) at both between- and within-speaker levels: phonological interconnectedness between intonational phrase (IP) boundaries and gesture types, temporal interconnectedness between IP boundaries and gesture peaks, and variation in gesture intensity peaks. Data annotation consisted of both manual annotation of hand gestures (Rohrer et al., 2021) and automatic annotation of eyebrow gestures using the facial behaviour analysis software OpenFace (Baltrušaitis et al., 2016).
Mixed-effect modelling (logistic regression and linear regression) was conducted on the association of gesture types and IP-final boundaries, distribution of gesture peaks and intensity of gesture peaks using the R package lme4 (Bates et al., 2015). Our analysis yielded evidence for similar phonological interconnectedness in infant-directed interaction and adult-directed interaction. That is, the same types of gestures tended to occur at IP-final boundaries in infant- and adult-directed speech. Moreover, temporal interconnectedness between IP boundaries and gesture peaks was different than previously reported for infant-directed gestures (De la Cruz-Pavía et al., 2020), i.e. gestures peaked closer to the end of an IP end than to its start. Finally, between-speaker variation in infant-directed gestures was observed for all three types of variation, whereas within-speaker variation was observed only for the temporal interconnectedness and variation in intensity peak of eyebrow gestures.
These results raise the question of how variation in infant-directed co-speech gesture influences early prosodic development. Follow-up research concerning the learning of prosodic phrasing in early infancy will be discussed in the presentation.
Keywords: infant-directed interaction; prosodic phrasing; co-speech gesture
| Original language | English |
|---|---|
| Pages | 71-72 |
| Number of pages | 2 |
| Publication status | Published - 26 Apr 2023 |
| Event | MMSYM 2023: The 1st International Multimodal Communication Symposium - Barcelona, Spain Duration: 26 Apr 2023 → 28 Apr 2023 |
Conference
| Conference | MMSYM 2023 |
|---|---|
| Country/Territory | Spain |
| City | Barcelona |
| Period | 26/04/23 → 28/04/23 |