关键词: 工作记忆负载, 多感觉促进效应, 视觉, 听觉, 注意

Abstract: In daily lives, we often receive information from different modalities. Previous studies have found that multisensory stimuli can capture attention more effectively (i.e., multisensory enhancement) compared to unisensory stimuli. In the past decades, researchers have paid increasing attention to multisensory enhancement. However, it remains unclear whether the cognitive load affects the attention capture of multisensory stimuli. Some studies have shown that attention capture by multisensory stimuli is not influenced by the load, while others have demonstrated that attention capture by multisensory stimuli is modulated by the load. By adopting complex face and syllable stimuli, the present study aims to explore whether attention capture by audiovisual stimuli is modulated by the working memory load. Moreover, we are also interested in whether the multisensory enhancement for each modality is affected by the type of working memory load.
In present study, participants were required to perform a working memory task and a gender identification task, while verbal (in Experiment 1) or visual (in Experiment 2) working memory load was manipulated, respectively. Two within-participant variables were used: working memory load (low load vs. high load) and modality (visual, auditory vs. audiovisual). In Experiment 1, two or six numbers were presented at the center of the screen for 1000 ms, and participants were required to remember these numbers and their sequence. Then a visual (a face), auditory (a syllable), or a bimodal audiovisual target was presented for 700 ms, followed by a 1500 ms blank screen. Participants were instructed to judge the gender of the target as quickly and accurately as possible. At the end of each trial, a number in the memory set was presented and participants were asked to report the next number. In Experiment 2, spatial orientations were used in the working memory task instead of visual numbers, that is, participants had to remember the orientation of each line and the locations of the lines. Other experimental settings in Experiment 2 were the same as those in Experiment 1.
For the two experiments, a significant main effect of modality was found. Responses to multisensory targets were significantly faster than those to auditory targets, indicating that bimodal audiovisual stimuli could capture attention more effectively than unisensory stimuli. Our results support that, compared to unisensory auditory stimuli, bimodal audiovisual stimuli are more efficient for attention capture. Moreover, a significant interaction between working memory load and modality was found. That is, the crossmodal benefit was affected both by the working memory load and sensory modality. With the increase of verbal working memory load, the benefit of auditory stimuli from multisensory enhancement increased significantly, while the benefit of visual stimuli from multisensory enhancement remained unchanged. In contrast, with the increase of visual working memory load, the benefit of auditory stimuli from multisensory enhancement remained unchanged, while the benefit of visual stimuli from multisensory enhancement decreased significantly. These results suggest that working memory load affects the attention capture of multisensory stimuli, and the benefits of visual and auditory modalities from multisensory enhancement are modulated by the type of working memory load.

Key words: working memory load, multisensory enhancement, visual, auditory, attention