Principles of audiovisual integration in 3-D space

Previous research on multisensory integration (MSI) has mainly focused on multisensory processing in 2-D space. We investigated two principles that are often used to describe the circumstances under which multisensory integration is enhanced: the principle of inverse effectiveness, and that of the spatial alignment of the component stimuli. Importantly, we investigated whether these principles could be applied in a 3-D setting. Given that much of the behavioral findings related to IE are not consistent with findings from neurophysiological research, and that stimulus intensity usually changes as a function of the distance from an observer, we investigated whether and how decreases in intensity would influence MSI for stimuli presented at different distances. We observed that the amount of MSI was significantly larger when a decrease in stimulus intensity was accompanied by an increase in distance, but not when stimuli with the same decrease in intensity were presented at the same distance as the baseline condition (Figure 1, left panel). In the second study we investigated whether the spatial alignment of the component audiovisual stimuli in depth influenced the amount of MSI, just as the spatial alignment in azimuthal space has been shown to modulate MSI. The results indicated that when the location of stimuli in azimuthal space was task-relevant, but the location of stimuli in depth was not, only the spatial alignment of the stimuli in terms of azimuth influenced MSI (Figure 1, right panel). These results underline the importance of investigating multisensory interactions in depth by showing that factors that are known to modulate MSI in one plane of depth may have a different effect on MSI with variations of stimulus locations in depth.