It is generally believed that negative emotion can improve item memory, while impairing the association between items or between item-context picture pairs. However, recent research indicates that associative memory between two separate items also can be promoted, especially when the pair of items are unitized into a single unit. Both conceptual and perceptual manipulations can be used to unitize items, but few studies have compared these two methods of unitization within the same experiment directly. Therefore, this study aimed to investigate how different semantic and spatial relationships would influence the associative memory between negative items and neutral contexts. The present study adopted picture-pairs consisting of negative item-neutral context and neutral item-neutral context as experimental materials. The semantic relationship was manipulated by controlling the semantic relevance of item-context pairs, and the spatial relationship was manipulated by controlling the spatial distance of item-context pairs. This study used the study-recognition paradigm. During the study phase, participants were presented with 48 negative item-neutral context pairs and 48 neutral item-neutral context pairs, in which half of the item-context pairs were related or contiguous. After the presentation of all picture-pairs, a mathematical task was presented for 5 minutes. During the test phase, an item image was first presented and participants were asked to identify the item was old or new. If participants recognized old items correctly, he or she would then be shown three context scenes and were required to make a decision on which scene had been originally presented with the recognized item. The current study was a three-factor design, in which item emotional type, semantic relationship, and spatial distance were all within-participant variables. The dependent variables were the accuracy of studied items and their intact backgrounds. The results showed that negative items were always remembered better than neutral items. However, the effect of negative emotion on associative memory would be modulated by semantic relationship and spatial relationship between items and contexts. Specifically, related item-context pairs were remembered better than unrelated item-context pairs, spatially contiguous item-context pairs were also remembered better than spatially separated item-context pairs. In addition, when items and contexts were presented in a spatially separated manner, associative memory performance of negative items was significantly lower than that of neutral items in unrelated condition, while associative memory performance of negative items was significantly higher than that of neutral items in related condition; when items and contexts were presented in a spatially contiguous manner, no matter items and contexts were related or unrelated, there was no significant difference in associative memory performance between negative items and neutral items, indicating that the effect of the semantic relationship was more pronounced in the spatially separated condition. In conclusion, these findings suggest the effect of negative emotion on associative memory can be modulated by semantic and spatial relationships between items and contexts. Increasing the conceptual integration and perceptual integration can promote item-context associative memory, and the role of semantic relationship in the effect of negative emotion on associative memory will be further influenced by the spatial relationship.