Inhibition of return (IOR) refers to an orientation mechanism that the detection speed and accuracy is impaired when an object is detected after the object is attended. Previous studies indicated that the phenomenon of inhibition of return was related to working memory (Bartolameo et al., 2001; Castel, Pratt & Craik, Chou & Yeh, 2008; 2003; Gilchrist & Harvey, 2000; Klein, 1988, 2000; Vaughan, 1984; Zhang et al., 2007). A secondary working memory task reduced the effect of return inhibition. For example, Chou and Yeh (2008) found that spatial working memory task influenced spatial inhibition of return, but not object inhibition of return, whereas object working memory task influenced object inhibition of return, but not spatial inhibition of return. These studies, however, only focused on the location-based and item-based inhibition of return. Thus, although a specific kind of inhibition of return is implied to rely on that kind of working memory, the generality of this finding is unclear. Forasmuch, in order to provide more evidences for the specific relations between inhibition of return and working memory, we tested the relation between color-based inhibition of return and working memory. A 3 working memory task (without working memory task, with color working memory task vs. with shape working memory task) X 2 color congruency of cue and target (congruent vs. incongruent) within subjects design was adopted. If color-based inhibition of return is related to color working memory, inhibition of return effects would be reduced when participants were required to complete a color working memory task at the same time. If color-based inhibition of return is related to shape working memory, inhibition of return effects would be reduced when participants were required to complete a shape working memory task at the same time. It was revealed that: 1)color-based inhibition of return effects reduced in color working memory condition (p < .01, Table 1, Figure 2); 2) color-based inhibition of return effects were not affected by the shape working memory task (p > .05, Table 1, Figure 2). In sum, the current study indicates that color-based inhibition of return relies on color working memory, which further confirms the generality of the specific relation between inhibition of return and working memory. Given the working memory span is limited, the interference between the color working memory task and the color inhibition of return task might be due to they occupy the same kind of working memory. In other words, visuo-spatial sketch pad of working memory (Baddeley, 1981, 1986) might be divided into several small units, each dealing with one aspect of visual information, such as location, color or shape. Resources competing might only arise within, but not between these small units.