Automatic Analysis of Bodily Social Signals

    Research output: Chapter in Book/Report/Conference proceedingChapterAcademicpeer-review

    Abstract

    The human body plays an important role in face-to-face interactions (Knapp & Hall, 2010; McNeill, 1992).We use our bodies to regulate turns, to display attitudes and to signal attention (Scheflen, 1964). Unconsciously, the body also reflects our affective and mental states (Ekman & Friesen, 1969). There is a long history of research into the bodily behaviors that correlate with the social and affective state of a person, in particular in interaction with others (Argyle, 2010; Dittmann, 1987; Mehrabian, 1968). We will refer to these behaviors as bodily social signals. These social and affective cues can be detected and interpreted by observing the human body’s posture and movement (Harrigan, 2008; Kleinsmith & Bianchi-Berthouze, 2013). Automatic observation and analysis has applications such as the detection of driver fatigue and deception, the analysis of interest and mood in interactions with robot companions, and in the interpretation of higher-level phenomena such as mimicry and turn-taking. In this chapter, we will discuss various bodily social signals, and how to analyze and recognize them automatically. Human motion can be studied on many levels, from the physical level involving muscles and joints, to the level of interpreting a person’s full-body actions and intentions (Poppe, 2007, 2010; Jiang et al., 2013). We will focus on automatically analyzing movements with a relatively short time scale, such as a gesture or posture shift. In the first section, we will discuss the different ways of measurement and coding, both from motion capture data and images and video. The recorded data can subsequently be interpreted in terms of social signals. In the second section, we address the automatic recognition of several bodily social signals. We will conclude the chapter with a discussion of challenges and directions of future work.
    Original languageEnglish
    Title of host publicationSocial Signal Processing
    EditorsJudee K. Burgoon, Nadia Magnenat-Thalman, Maja Pantic, Alessandro Vinciarelli
    PublisherCambridge University Press
    Pages155-167
    ISBN (Electronic)9781316676202
    ISBN (Print)978-1-107-16126-9
    DOIs
    Publication statusPublished - 2017

    Keywords

    • body motion
    • social signals
    • motion capture
    • human body
    • automated measurement
    • proxemics
    • kinesics

    Fingerprint

    Dive into the research topics of 'Automatic Analysis of Bodily Social Signals'. Together they form a unique fingerprint.

    Cite this