Abstract: MOT (Multiple Object Tracking) begins with an experimental paradigm developed by Pylyshyn and Storm (1988) and becomes an active and challenging research topic today. It is a widely used paradigm in the study of capacity-limit and object-based attention. Much like MOT, MIT (Multiple Identity Tracking) in which each object carries a unique identity is used to study more complicated cognitive processes, such as identity recognition. In previous studies, MOT and MIT tasks were usually presented in visual modality, for example, when marking the objects to be tracked during the cueing phase, the targets would flash a few times or there were squares occurred outside the targets, which were both by visual forms. However, in the real world, people usually receive various kinds of information from multiple sensory modalities. Especially when lacking visual information, people will rely more on the information input from auditory or other sensory modalities. The present study investigated the effect of visual marking and audiovisual cross-modality marking of targets on participants’ tracking performance. The present study included 2 experiments and compared the effects of four different marking type on people’s tracking performance. The four different marking types are visual-parallel marking, visual-sequence marking, auditory-sequence marking (audiovisual cross-modality marking) and audiovisual double-modality marking respectively. In the visual-parallel marking condition, objects to be tracked flashed a few times simultaneously, while in the visual-sequence marking condition, the targets flashed one by one. In the auditory-sequence marking condition, the observers first heard the sound of numbers (experiment 1) or characters (experiment 2) inside the objects, and then searched for the location of the targets according to the identity of the targets heard before. And in the audiovisual double-modality marking condition, the observers not only saw the flash of targets but also heard the identity of targets. Experiment 1 found that when there’re 4 to-be-tracked targets, people’s tracking performance in visual-sequence marking condition was the best among the four marking types. And when there’re 5 to-be-tracked targets, participants’ tracking performance in audiovisual double-modality marking condition was the best. Experiment 2 demonstrated that when there’re 4, 5, or 6 to-be-tracked targets, people’s tracking performances in visual-parallel marking condition and visual-sequence marking condition were better than that of auditory-sequence marking condition. The present study showed that tracking accuracy of visual-parallel marking and visual-sequence marking was significantly higher than that of audiovisual cross-modality marking. It suggested visual marking was more effective than audiovisual cross-modality marking in MOT, and parallel or serial processing didn’t affect the tracking performance.

Key words: Multiple Object Tracking, target marking, audiovisual cross-modality

摘要： 有关多目标追踪的研究多采用目标闪烁数次或目标周围出现方框等视觉方式对目标进行标记。在已有研究基础上，通过2个实验比较了不同目标数量下视觉平行标记、视觉序列标记、听觉序列标记(视听跨通道标记)和视听双通道序列标记4种目标标记方式对多目标追踪表现的影响。研究结果表明在多目标追踪任务中，线索阶段目标的视觉标记比视听跨通道标记更有效，且不受目标的序列标记或平行标记的影响，而视觉和听觉编码结合（视听双通道标记）可以促进任务难度较高时的追踪表现。

关键词: 多目标追踪, 目标标记方式, 视听跨通道