Spatial representation of time can be found in most languages and cultures in the world. According to conceptual metaphor theory (Lakoff & Johnson, 1980), metaphor offers a way to understand an abstract domain (e.g. time) by utilizing corresponding knowledge from a better-understood, more concrete experience (e.g. space). Yet, the criticisms from scholars both within and outside cognitive linguistics have never stopped since the birth of CMT, especially for its psychological reality. In other words, the linguistic data (oral speech and written language) is always used as both the predictor of the conceptual mappings and the predicted result. In order to respond to the criticisms of CMT, an emerging synthesis of gesture and signed language has been proposed to address this issue. Human’s communication can be performed verbally and/or nonverbally. For a long time, gestures have been only considered to be meaningless hand waving, or to bear fixed form and formulaic meanings in the culture from a semiotic perspective. In recent years, gestures and speech are considered to be parts of a single psychological structure and share the same computational and psychological origin. When people talk about time, there are some accompanied hand movements, which are often called temporal gestures. A number of recent studies have focused on temporal gestures in different cultural contexts, showing that temporal gestures are strongly shaped by cultural practices and artifacts. For instance, Floyd(2008) noted that there is a celestial gesture system used by speakers of Nheenghatú, an indigenous language of the Brazilian Amazon, in which speakers can point or sweep to refer to punctate times of the day. For humans, the ability to communicate and use language is instantiated not only in the vocal modality but also in the visual modality. Sign languages are articulated in the three-dimensional space by deaf people. In the case of sign languages, different devices (e.g. lexical expressions, grammatical means and non-manual features) can be used in the expressions of temporal notions. Many studies in sign languages have lent support to the existence of time line and its important role in the creation of temporal expressions. It means that signers may automatically activate the mental timeline when processing temporal linguistic information. On the one hand, signers can use real space to create temporal references, in which the metaphorical association of locations in space represents different points of time. On the other hand, he or she can also directly use a lexical unit to refer to that temporal dimension without turning to deixis. Compelling evidence has shown that our concept of time is organized metaphorically in terms of space. Recent development brings a welcome widening of the perspective of time metaphor research and starts to reap the benefits of the broad range of methods available in cognitive science. If one wants to prove that conceptual metaphors are conceptual thinking rather than only linguistic decoration, alternative evidence beyond linguistic data for their psychological reality is required. Since both of sign and co-speech gesture exploit richness of the manual-visual modality to spatialize temporal concepts. The study of more signed languages, in relation to the co-speech temporal gestures, can provide rich psychological evidence to clarify the origins and transmission of spatial representations of time.