Native Mandarin Chinese listeners usually perceive lexical tones categorically—that is, the identification of one tone category shifts to another tone category sharply. On the other side, aging has negative effects on human listeners’ speech perception. In general, aging makes speech recognition more difficult. Although the categorical perception of Mandarin tones has been studied extensively for younger native Chinese listeners, little is known on how aging affects the categorical perception of Mandarin tones. Thus, aging may also degrade the ability of categorical perception of Mandarin tones. The previous studies have already detected the psychoacoustics factors of age-related changes, while some important factors, such as area-general fundamental cognitive function and area-specific ability of processing lexical tone category knowledge have not been taken into consideration. Thus, the purpose of the current study was to examine the aging effect on the categorical perception of Mandarin Chinese Tone 2 (rising F0 pitch contour) and Tone 4 (falling F0 pitch contour) as well as the factors contributing to aging. Two experiments of Mandarin tone perception were conducted for younger and older listeners with Mandarin Chinese as the native language. The 48 older participants were divided into three groups: 50 -to- 60 - year - old group，60 -to- 70 - year - old group，70 -to- 80 - year - old group. The first experiment was in one of the categorical perception paradigm: tone identification for a series of stimuli, the F0 contour of which systematically varied from Tone 2 to Tone 4. The participants were required to identify categories of the stimuli. In the second experiment, the sequence digital span test and the reversed sequence digital span test was designed to detect working memory function. Besides, in order to exclude the possibility that the aging effect simply reflects the hearing difference, the intensity of sound was set according to the elderly’ s hearing threshold. In the measures of categorical perception, all elderly groups showed significantly smaller category boundary width in the tone identification function than young listeners, but there was no significant difference among the three elderly groups. For older listeners, the category boundary width was significantly positively correlated with the scores of two working memory tests, but was not significantly correlated with age. Besides, the results indicated no significant difference in the slopes of the within-category tone identification function among the four groups, while the slopes in the across-category tone identification function of the three elderly groups were significantly more shallower than the young group. These results suggest that aging reduced degree of categorization perception of Mandarin tone and this deterioration was associated decay of working memory. Moreover, deterioration of the ability of invoking the category knowledge of lexical tone is another factor contributing to the aging of Mandarin tone categorical perception. Besides, when entering the elderly period, the aging procedures of different language function are not the same. To be more specific, age can not influence categorical perception of Mandarin tone directly.