Mobile screen size limits multimodal synergy

Frans van der Sluis; E.L. van den Broek; Annemiek van Drunen; John G. Beerends

doi:10.1145/3232078.3232101

Mobile screen size limits multimodal synergy

Frans van der Sluis, E.L. van den Broek, Annemiek van Drunen, John G. Beerends

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Available bandwidth is still a limiting factor for mobile communication applications. Multisensory communication has already been identified as an possibility to moderate this limitation. One of the strengths of mobile communication lies in its combination of visual and auditory modalities. However, one of the most salient features of mobile devices have are their small screen size. This paper explores how the potential for multimodal synergy relates to the small screen size. In an experiment with 54 participants, the intelligibility was tested using a standardized video-listening test. The videos had a signal-to-noise ratio of -9dB and were presented on three different screen sizes, whilst keeping the video and auditory signals equal. Intelligibility was found to be significantly higher when using a large screen in comparison to using either of both smaller screens. We conclude that multisensory synergy is key to mobile applications, yet that screen size is a substantial constraint to this synergy. We argue that knowledge about human sensory processing can alleviate this constraint and maximize the potential quality of service of mobile video technology.

Original language	English
Title of host publication	Proceedings of the 30th European Conference on Cognitive Ergonomics (ECCE'18)
Editors	Egon L. van den Broek, Herre van Oostendorp, Françoise Détienne, Christian Stary
Place of Publication	New York, NY, USA
Publisher	Association for Computing Machinery
Number of pages	4
ISBN (Electronic)	978-1-4503-6449-2
DOIs	https://doi.org/10.1145/3232078.3232101
Publication status	Published - 5 Sept 2018

Publication series

Name	ICPS: ACM International Conference Proceeding Series
Publisher	ACM

Keywords

multimodal
multisensory
mobile
screen size
intelligibility
Quality of Service (QoS)
Field of View (FOV)
to- Noise Ratio (SNR)
Human-centered computing
Empirical studies in accessibility

Access to Document

10.1145/3232078.3232101Licence: Unspecified

vanderSluis_etal18-Mobile_screen_size_limits_multimodal_synergyFinal published version, 315 KBLicence: Taverne

Cite this

van der Sluis, F., van den Broek, E. L., van Drunen, A., & Beerends, J. G. (2018). Mobile screen size limits multimodal synergy. In E. L. van den Broek, H. van Oostendorp, F. Détienne, & C. Stary (Eds.), Proceedings of the 30th European Conference on Cognitive Ergonomics (ECCE'18) Article 16 (ICPS: ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3232078.3232101

van der Sluis, Frans ; van den Broek, E.L. ; van Drunen, Annemiek et al. / Mobile screen size limits multimodal synergy. Proceedings of the 30th European Conference on Cognitive Ergonomics (ECCE'18). editor / Egon L. van den Broek ; Herre van Oostendorp ; Françoise Détienne ; Christian Stary. New York, NY, USA : Association for Computing Machinery, 2018. (ICPS: ACM International Conference Proceeding Series).

@inproceedings{a1ce1967d7704911b652369cc2dbbfc3,

title = "Mobile screen size limits multimodal synergy",

abstract = "Available bandwidth is still a limiting factor for mobile communication applications. Multisensory communication has already been identified as an possibility to moderate this limitation. One of the strengths of mobile communication lies in its combination of visual and auditory modalities. However, one of the most salient features of mobile devices have are their small screen size. This paper explores how the potential for multimodal synergy relates to the small screen size. In an experiment with 54 participants, the intelligibility was tested using a standardized video-listening test. The videos had a signal-to-noise ratio of -9dB and were presented on three different screen sizes, whilst keeping the video and auditory signals equal. Intelligibility was found to be significantly higher when using a large screen in comparison to using either of both smaller screens. We conclude that multisensory synergy is key to mobile applications, yet that screen size is a substantial constraint to this synergy. We argue that knowledge about human sensory processing can alleviate this constraint and maximize the potential quality of service of mobile video technology.",

keywords = "multimodal, multisensory, mobile, screen size, intelligibility, Quality of Service (QoS), Field of View (FOV), to- Noise Ratio (SNR), Human-centered computing, Empirical studies in accessibility",

author = "{van der Sluis}, Frans and {van den Broek}, E.L. and {van Drunen}, Annemiek and Beerends, {John G.}",

year = "2018",

month = sep,

day = "5",

doi = "10.1145/3232078.3232101",

language = "English",

series = "ICPS: ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

editor = "{van den Broek}, {Egon L.} and {van Oostendorp}, Herre and Fran{\c c}oise D{\'e}tienne and Christian Stary",

booktitle = "Proceedings of the 30th European Conference on Cognitive Ergonomics (ECCE'18)",

address = "United States",

}

van der Sluis, F, van den Broek, EL, van Drunen, A & Beerends, JG 2018, Mobile screen size limits multimodal synergy. in EL van den Broek, H van Oostendorp, F Détienne & C Stary (eds), Proceedings of the 30th European Conference on Cognitive Ergonomics (ECCE'18)., 16, ICPS: ACM International Conference Proceeding Series, Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3232078.3232101

Mobile screen size limits multimodal synergy. / van der Sluis, Frans; van den Broek, E.L.; van Drunen, Annemiek et al.
Proceedings of the 30th European Conference on Cognitive Ergonomics (ECCE'18). ed. / Egon L. van den Broek; Herre van Oostendorp; Françoise Détienne; Christian Stary. New York, NY, USA: Association for Computing Machinery, 2018. 16 (ICPS: ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Mobile screen size limits multimodal synergy

AU - van der Sluis, Frans

AU - van den Broek, E.L.

AU - van Drunen, Annemiek

AU - Beerends, John G.

PY - 2018/9/5

Y1 - 2018/9/5

N2 - Available bandwidth is still a limiting factor for mobile communication applications. Multisensory communication has already been identified as an possibility to moderate this limitation. One of the strengths of mobile communication lies in its combination of visual and auditory modalities. However, one of the most salient features of mobile devices have are their small screen size. This paper explores how the potential for multimodal synergy relates to the small screen size. In an experiment with 54 participants, the intelligibility was tested using a standardized video-listening test. The videos had a signal-to-noise ratio of -9dB and were presented on three different screen sizes, whilst keeping the video and auditory signals equal. Intelligibility was found to be significantly higher when using a large screen in comparison to using either of both smaller screens. We conclude that multisensory synergy is key to mobile applications, yet that screen size is a substantial constraint to this synergy. We argue that knowledge about human sensory processing can alleviate this constraint and maximize the potential quality of service of mobile video technology.

AB - Available bandwidth is still a limiting factor for mobile communication applications. Multisensory communication has already been identified as an possibility to moderate this limitation. One of the strengths of mobile communication lies in its combination of visual and auditory modalities. However, one of the most salient features of mobile devices have are their small screen size. This paper explores how the potential for multimodal synergy relates to the small screen size. In an experiment with 54 participants, the intelligibility was tested using a standardized video-listening test. The videos had a signal-to-noise ratio of -9dB and were presented on three different screen sizes, whilst keeping the video and auditory signals equal. Intelligibility was found to be significantly higher when using a large screen in comparison to using either of both smaller screens. We conclude that multisensory synergy is key to mobile applications, yet that screen size is a substantial constraint to this synergy. We argue that knowledge about human sensory processing can alleviate this constraint and maximize the potential quality of service of mobile video technology.

KW - multimodal

KW - multisensory

KW - mobile

KW - screen size

KW - intelligibility

KW - Quality of Service (QoS)

KW - Field of View (FOV)

KW - to- Noise Ratio (SNR)

KW - Human-centered computing

KW - Empirical studies in accessibility

U2 - 10.1145/3232078.3232101

DO - 10.1145/3232078.3232101

M3 - Conference contribution

T3 - ICPS: ACM International Conference Proceeding Series

BT - Proceedings of the 30th European Conference on Cognitive Ergonomics (ECCE'18)

A2 - van den Broek, Egon L.

A2 - van Oostendorp, Herre

A2 - Détienne, Françoise

A2 - Stary, Christian

PB - Association for Computing Machinery

CY - New York, NY, USA

ER -

van der Sluis F, van den Broek EL, van Drunen A, Beerends JG. Mobile screen size limits multimodal synergy. In van den Broek EL, van Oostendorp H, Détienne F, Stary C, editors, Proceedings of the 30th European Conference on Cognitive Ergonomics (ECCE'18). New York, NY, USA: Association for Computing Machinery. 2018. 16. (ICPS: ACM International Conference Proceeding Series). doi: 10.1145/3232078.3232101

Mobile screen size limits multimodal synergy

Abstract

Publication series

Keywords

Access to Document

Fingerprint

Cite this