The audio-visual integration and pre-preparation effects in emotion and timing discrimination tasks

YuChen Li

Journal of Psychological Science ›› 2023, Vol. 46 ›› Issue (1) : 3-10.

PDF(976 KB)
PDF(976 KB)
Journal of Psychological Science ›› 2023, Vol. 46 ›› Issue (1) : 3-10.

The audio-visual integration and pre-preparation effects in emotion and timing discrimination tasks

  • 1,YuChen Li2, 3, 4
Author information +
History +

Abstract

Emotional audio-visual integration is an effective way to transmit and decode emotional information, which is of great significance to human survival and evolution. In multisensory integration, it is generally understood that one sensory signal promotes or alters the perception of other sensory signals. Studies have shown that the cross-sensory pre-preparation effect between visual and auditory is a key influencing factor of emotional audio-visual integration. However, it has not received much attention in the pre-preparation effect, particularly with regard to the emotional dimension. And previous research did not distinguish the emotional pre-preparation effect from the cross-sensory pre-preparation effect. Thus the present two studies started from the perspective of these two pre-preparation effects to gain insight into the emotional audio-visual integration processing mechanism. Experiment 1 adopted the time order judgment and simultaneous judgment tasks. Participants were asked to determine the chronological sequence of the visual and auditory stimuli, or whether they were presented at the same time. Experiment 2 adopted the emotion discrimination tasks. Participants were required to determine the emotional valence of visual, auditory, or audio-visual stimuli. The purpose of these two studies was to examine the influence of pre-preparation effects on emotional audio-visual integration either in the time or emotional cognitive dimensions. In Experiment 1, the response time of the visual-leading condition was significantly slower than the auditory-leading condition, and the accuracy was significantly lower than the auditory-leading condition. However, the amount of relative multisensory response enhancement (rMRE) for each condition of audiovisual integration was negative. It meant that audio-visual integration did not have many advantages compared with the single visual or auditory sensory. In Experiment 2, the rMRE of the negative emotion in auditory-leading condition was significantly greater than that in the visual-leading condition; the rMRE of the positive emotion in visual-leading condition was significantly greater than auditory-leading condition; however, the difference between the visual- and the auditory-leading condition on the rMRE of the neutral emotion was not significant. These research results showed that in the temporal dimension, the mechanism of the audio-visual integration was manifested as the feature matching processing cross the visual and auditory, and a better pre-preparation effect for the presentation time was in auditory than visual sensory. In the emotional cognitive dimension, the integration of the emotional information among the visual and auditory sensory had occurred with the positive rMRE amount, and the better emotional preparation effect had been discovered in the visual sensory than the auditory. Moreover, auditory channels played a greater role when integrating negative emotions, while visual channels dominate when integrating positive emotions. In summary, present experimental evidence showed that the enhancement of the emotional audiovisual integration was based on the cognitive processing of emotional information, and the temporal discrimination inhibited the emotional audiovisual integration. Either the pre-preparation effect of cross-sensory matching or emotional integration promoted the processing of visual and auditory information respectively and depended on the different neural circuits under visual- or auditory-leading independently. At last, the advantage of audio-visual integration showed that the cross-sensory and emotional pre-preparation effect was mediated by the tasks, demonstrating a separation between them.

Key words

Cross-sensory pre-preparation effect / Emotional pre-preparation effect / Emotional audio-visual integration / auditory / visual

Cite this article

Download Citations
YuChen Li. The audio-visual integration and pre-preparation effects in emotion and timing discrimination tasks[J]. Journal of Psychological Science. 2023, 46(1): 3-10
PDF(976 KB)

Accesses

Citation

Detail

Sections
Recommended

/