Abstract: Emotional audio-visual integration is an effective way to transmit and decode emotional information, which is of great significance to human survival and evolution. In multisensory integration, it is generally understood that one sensory signal promotes or alters the perception of other sensory signals. Studies have shown that the cross-sensory pre-preparation effect between visual and auditory is a key influencing factor of emotional audio-visual integration. However, it has not received much attention in the pre-preparation effect, particularly with regard to the emotional dimension. And previous research did not distinguish the emotional pre-preparation effect from the cross-sensory pre-preparation effect. Thus the present two studies started from the perspective of these two pre-preparation effects to gain insight into the emotional audio-visual integration processing mechanism. Experiment 1 adopted the time order judgment and simultaneous judgment tasks. Participants were asked to determine the chronological sequence of the visual and auditory stimuli, or whether they were presented at the same time. Experiment 2 adopted the emotion discrimination tasks. Participants were required to determine the emotional valence of visual, auditory, or audio-visual stimuli. The purpose of these two studies was to examine the influence of pre-preparation effects on emotional audio-visual integration either in the time or emotional cognitive dimensions. In Experiment 1, the response time of the visual-leading condition was significantly slower than the auditory-leading condition, and the accuracy was significantly lower than the auditory-leading condition. However, the amount of relative multisensory response enhancement （rMRE） for each condition of audiovisual integration was negative. It meant that audio-visual integration did not have many advantages compared with the single visual or auditory sensory. In Experiment 2, the rMRE of the negative emotion in auditory-leading condition was significantly greater than that in the visual-leading condition; the rMRE of the positive emotion in visual-leading condition was significantly greater than auditory-leading condition; however, the difference between the visual- and the auditory-leading condition on the rMRE of the neutral emotion was not significant. These research results showed that in the temporal dimension, the mechanism of the audio-visual integration was manifested as the feature matching processing cross the visual and auditory, and a better pre-preparation effect for the presentation time was in auditory than visual sensory. In the emotional cognitive dimension, the integration of the emotional information among the visual and auditory sensory had occurred with the positive rMRE amount, and the better emotional preparation effect had been discovered in the visual sensory than the auditory. Moreover, auditory channels played a greater role when integrating negative emotions, while visual channels dominate when integrating positive emotions. In summary, present experimental evidence showed that the enhancement of the emotional audiovisual integration was based on the cognitive processing of emotional information, and the temporal discrimination inhibited the emotional audiovisual integration. Either the pre-preparation effect of cross-sensory matching or emotional integration promoted the processing of visual and auditory information respectively and depended on the different neural circuits under visual- or auditory-leading independently. At last, the advantage of audio-visual integration showed that the cross-sensory and emotional pre-preparation effect was mediated by the tasks, demonstrating a separation between them.

Key words: Cross-sensory pre-preparation effect, Emotional pre-preparation effect, Emotional audio-visual integration, auditory, visual

摘要： 本研究分别在时间和情绪认知维度上考察预先准备效应对情绪视听整合的影响。实验一采用时序判断任务，发现视觉引导显著慢于听觉引导，并且视听整合效应量为负值。实验二采用情绪辨别任务，发现视听整合效应量均为正值；在负性情绪整合中，听觉引导显著大于视觉引导；在正性情绪整合中，视觉引导显著大于听觉引导。研究结果表明，情绪视听整合是基于对情绪信息的认知加工，时间辨别任务会抑制视听情绪整合；并且无论是跨通道预先准备还是情绪预先准备都与引导感觉模态有关。

关键词: 跨通道预先准备效应, 情绪预先准备效应, 情绪视听整合, 视觉, 听觉