The basic expressions view claims that there are basic facial expressions of emotions that are created by specific configurations of facial muscles. However, we often find that context played an important role in facial emotion perception in daily life. Moreover, a series of studies has shown that context often influences emotion perception unconsciously, but some studies have provided contradictory findings. In this paper, we provide a framework of how kinds of contexts effect on facial perception. Context effects on faces depend on the emotion seeds shared by the target expression and by the facial expression that would typically be associated with the emotional context. Words, bodies, visual scenes, and even voices shape how emotion is perceived in a face. Prior contradictory findings arise, in part, because of a lack of consideration of the perceptual similarity among facial expressions. The greater the perceptual similarity between the target face and the context-associated face, the easier it was to perceive the context emotion in the target face. On the other hand, words constitute a clear example of a perceiver-based context because they provide a top-down constraint in emotion perception, contributing information over and above the affective meaning available in structural information of a face. Besides, the evidence from eye movements while participants scanned expressive faces embedded in differing contexts showed that the information provided by the facial expression is combined with the context during the early stages of processing, and to some degree, is automatic. In addition, context effect is sensitive to cultural, age, and gender differences. Cultural context appears to influence how perceivers sample information from a face in a manner that is similar to the influence of situational context. And more important, there some findings suggest that cultural differences in reliance on context to interpret others’ emotions depend on perceptual integration processes that decline with age, leading to fewer cultural differences in perception among older adults than among younger adults. Furthermore, stress also influences background integration. A series of studies has shown that participants who were more stressed showed less of an influence of the background context on their ratings of the central face. At the end of the article, we point out that future investigations should examine the developmental timeline of face-context integration by focusing on children. And more emotions should be added in to create diverse combinations of facial expressions and contexts, and we can also promote validity of experimental result by adopting more natural and realistic examples.