手语语音加工及其神经机制研究述评*

doi:10.16719/j.cnki.1671-6981.202304031

摘要/Abstract

摘要： 语音加工在言语理解与产出中具有重要的作用。手语是基于视觉-手势模态的自然语言,也具有语音结构,但与口语的语音结构之间有很大的差异。本文总结和分析了近年来有关手语语音加工的研究,发现手语加工中存在基于音韵参数的语音信息激活和使用,且手语的语音加工神经机制与口语具有相似之处,尤其在双侧颞叶和左侧额下回等部分脑区存在重合,同时在左侧枕叶和缘上回等脑区具有一定的特异性。最后,文章从语音加工单元和手语语种等方面展望了未来的研究方向。

关键词: 手语, 语音加工, 音韵参数, 神经机制

Abstract: Phonological processing concerns access to and use of mental phonological representation, which is essential to language comprehension and production. However, most findings and theories on phonological processing are based on spoken or written languages, with relatively little evidence from sign languages. As natural human languages, sign languages can also be analyzed at a phonological level. But they are articulated in a visual-gestural modality and have some unique features different from spoken languages. For example, 1) the manual and largely simultaneous articulation of phonological units or parameters (e.g., location, handshape, and movement), and 2) a tight relation between phonology and semantics due to the prevalence of iconicity, i.e., sign forms visually resemble their meanings. These features may result in some phonological processing mechanisms different from those found in spoken languages, hence posing challenges to current language processing theories. To reveal what is fundamental and what is modality-specific about language processing, this article attempts to review recent works on phonological processing in sign languages, with a focus on its empirical evidence and neurological mechanisms.
On the one hand, studies from different paradigms and techniques show that parameter-based phonological information is activated and used for sign recognition. First, signs with higher phonological neighborhood density are recognized slower than those with low density, indicating that phonological competitors are activated and compete for identification. Priming studies have found that prime-target sign pairs with one- or two-parameter overlap are responded to at a different speed or elicit different amplitude N400s from those unrelated pairs. In addition, eye-tracking studies using the visual world paradigm show that, compared to unrelated pictures, participants spend more time looking at competing pictures whose corresponding signs share one or two parameters with the target signs. These findings indicate that phonological processing in sign languages is psychologically real, and that the processing units may involve both individual parameters and two-parameter combinations. However, results are mixed regarding the actual effects of the units, with some showing facilitative effects of location overlap, some showing inhibitory effects, while others showing no effects of location but facilitative effects of handshape and location-handshape overlap.
On the other hand, brain image studies using PET, fMRI and TMS have showed that sign languages share similar neurological mechanisms of phonological processing with spoken languages. In sign perception and comprehension tasks, signers tend to activate the superior temporal cortex bilaterally when viewing signs or sign-phonetic and syllabic units. In explicit phonological judgement tasks, a left-lateralised network is engaged, including the left inferior frontal cortex, superior marginal gyrus, and the superior parietal lobule. These processing regions have also been found in speech sound perception and word rhyme judgement, indicating that there are some similar phonological processing mechanisms in the two language modalities. Differences also exist, though, at the whole brain level, significant activation is found in the left occipital lobe in handshape judgement but not in the frontal-parietal network and the cerebellum bilaterally in speech rhyme judgement. Yet it is unclear whether these differences are related to modality-specific processing mechanisms or the sensory properties of phonological parameters.
To sum up, research available has provided some evidence for the psychological reality and neural mechanisms of phonological representation and processing in sign languages, but there is still a lack of consistency about the processing units and their roles as well as evidence on the modality-specific processing mechanisms. Thus some suggestions are given for future studies: (1)Future investigation may look at the roles of phonological parameters; (2) Further exploration of the neural mechanisms of sign phonology processing, especially on the brain activities in specific parameter perception and comparison of mechanisms may invoke in different tasks; (3)Enrichment of studies on various kinds of sign languages, as different languages may have their own unique phonological, grammatical or syntactical properties, which may result in cross-linguistic differences in processing mechanisms.

Key words: sign languages, phonological processing, phonological parameters, neural mechanisms

张小红, 李红. 手语语音加工及其神经机制研究述评^*[J]. 心理科学, 2023, 46(4): 1017-1023.

Zhang Xiaohong, Li Hong. Review of Phonological Processing in Sign Languages and Its Neural Mechanisms[J]. Journal of Psychological Science, 2023, 46(4): 1017-1023.

参考文献

[1] 龚群虎. (2005). 双语聋教育的理论和实践. 华夏出版社..
[2] 顾笙韵, 张吉生. (2017). 手语音系研究及其理论模型. 外国语, 40(1), 52-65.
[3] 李俊宏, 丁国盛. (2013). 手语和口语理解及产生的脑机制对比. 心理科学进展, 21(9), 1560-1569.
[4] 祁志强, 彭聃龄. (2010). 语音加工的脑机制研究: 现状、困惑及展望. 北京师范大学学报(社会科学版), 4, 40-47.
[5] 邱云峰, 姚登峰, 李荣, 刘春达. (2018). 中国手语语言学概论. 中国国际广播出版社..
[6] Andin J., Fransson P., Dahlström Ö., Rönnberg J., & Rudner M. (2019). The neural basis of arithmetic and phonology in deaf signing individuals. Language, Cognition and Neuroscience, 34(7), 813-825.
[7] Andin J., Fransson P., Rönnberg J., & Rudner M. (2015). Phonology and arithmetic in the language-calculation network. Brain and Language, 143, 97-105.
[8] Anthony J. L., Williams J. M., Aghara R. G., Dunkelberger M., Novak B., & Mukherjee A. D. (2010). Assessment of individual differences in phonological representation. Reading and Writing, 23(8), 969-994.
[9] Banaszkiewicz A., Bola Ł., Matuszewski J., Szczepanik M., Kossowski B., Mostowski P., Rutkowski P., Sliwinska M., Jednorog K., Emmorey K., & Marchewka A. (2021). The role of the superior parietal lobule in lexical processing of sign language: Insights from fMRI and TMS. Cortex, 135, 240-254.
[10] Brentari, D. (2012). Phonology. In R.Pfau, M. Steinbach, & B. Woll (Eds.), Sign language: An international handbook (pp. 21-54). De Gruyter Mouton.
[11] Brentari, D. (2019). Sign language phonology. Cambridge University Press..
[12] Cardin V., Orfanidou E., Kästner L., Rönnberg J., Woll B., Capek C. M., & Rudner M. (2016). Monitoring different phonological parameters of sign language engages the same cortical language network but distinctive perceptual ones. Journal of Cognitive Neuroscience, 28(1), 20-40.
[13] Cardin V., Orfanidou E., Rönnberg J., Capek C. M., Rudner M., & Woll B. (2013). Dissociating cognitive and sensory neural plasticity in human superior temporal cortex. Nature Communications, 4(1), Article 1473.
[14] Carreiras M., Gutiérrez-Sigut E., Baquero S., & Corina D., (2008). Lexical processing in Spanish Sign Language (LSE). Journal of Memory and Language, 58(1), 100-122.
[15] Caselli N. K., Emmorey K., & Cohen-Goldberg A. M. (2021). The signed mental lexicon: Effects of phonological neighborhood density, iconicity, and childhood language experience. Journal of Memory and Language, 121, Article 104282.
[16] Dye, M. W. G., & Shih, S. I. (2006). Phonological priming in British Sign Language. In L. M. Goldstein, D. H. Whalen, & C. T. Best (Eds.), Laboratory phonology 8 (pp. 241-264). Mouton de Gruyter.
[17] Emmorey K., Xu J., & Braun A. (2011). Neural responses to meaningless pseudosigns: Evidence for sign-based phonetic processing in superior temporal cortex. Brain and Language, 117(1), 34-38.
[18] Fenlon J., Cormier K., & Brentari D. (2018). The phonology of sign languages. In S. J. Hannahs & A. R. K. Bosch (Eds.), The Routledge handbook of phonological theory (pp. 453-475). Routledge.
[19] Grushkin, D. A. (2017). Writing signed languages: What for? What form? American Annals of the Deaf, 161(5), 509-527.
[20] Gutiérrez E., Müller O., Baus C., & Carreiras M. (2012). Electrophysiological evidence for phonological priming in Spanish Sign Language lexical access. Neuropsychologia, 50(7), 1335-1346.
[21] Hickok, G., & Poeppel, D. (2007). The cortical organization of speech processing. Nature Reviews Neuroscience, 8(5), 393-402.
[22] Hu Z. G., Wang W. J., Liu H. Y., Peng D. L., Yang Y. H., Li K. C., & Ding G. S. (2011). Brain activations associated with sign production using word and picture inputs in deaf signers. Brain and Language, 116(2), 64-70.
[23] Liddell, S. K., & Johnson, R. E. (1989). American Sign Language: The phonological base. Sign Language Studies, 64(1), 195-278.
[24] Lieberman, A. M., & Borovsky, A. (2020). Lexical recognition in deaf children learning American Sign Language: Activation of semantic and phonological features of signs. Language Learning, 70(4), 935-973.
[25] Liu L. F., Yan X., Liu J., Xia M. R., Lu C. M., Emmorey K., Chu M., & Ding G. S. (2017). Graph theoretical analysis of functional network for comprehension of sign language. Brain Research, 1671, 55-66.
[26] MacSweeney M., Capek C. M., Campbell R., & Woll B. (2008). The signing brain: The neurobiology of sign language. Trends in Cognitive Sciences, 12(11), 432-440.
[27] MacSweeney, M., & Emmorey, K. (2020). The neurobiology of sign language processing. In D. Poeppel, G. R. Mangun, & M. S. Gazzaniga (Eds.), The cognitive neurosciences (pp. 849-857). The MIT Press.
[28] MacSweeney M., Waters D., Brammer M. J., Woll B., & Goswami U. (2008). Phonological processing in deaf signers and the impact of age of first language acquisition. NeuroImage, 40(3), 1369-1379.
[29] Meade G., Lee B., Massa N., Holcomb P. J., Midgley K. E., & Emmorey K. (2021). The organization of the American Sign Language lexicon: Comparing one- and two-parameter ERP phonological priming effects across tasks. Brain and Language, 218, Article 104960.
[30] Newman A. J., Supalla T., Fernandez N., Newport E. L., & Bavelier D. (2015). Neural systems supporting linguistic structure, linguistic experience, and symbolic communication in sign language and gesture. Proceedings of the National Academy of Sciences of the United States of America, 112(37), 11684-11689.
[31] Ortega, G. (2017). Iconicity and sign lexical acquisition: A review. Frontiers in Psychology, 8, Article 1280.
[32] Perlman M., Little H., Thompson B., & Thompson R. L. (2018). Iconicity in signed and spoken vocabulary: A comparison between American Sign Language, British Sign Language, English, and Spanish. Frontiers in Psychology, 9, Article 1433.
[33] Petitto L. A., Zatorre R. J., Gauna K., Nikelske E. J., Dostie D., & Evans A. C. (2000). Speech-like cerebral activity in profoundly deaf people processing signed languages: Implications for the neural basis of human language. Proceedings of the National Academy of Sciences of the United States of America, 97(25), 13961-13966.
[34] Price, C. J. (2012). A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language and reading. NeuroImage, 62(2), 816-847.
[35] Rudner M., Karlsson T., Gunnarsson J., & Rönnberg J. (2013). Levels of processing and language modality specificity in working memory. Neuropsychologia, 51(4), 656-666.
[36] Rudner M., Orfanidou E., Kästner L., Cardin V., Woll B., Capek C., & Rönnberg J. (2019). Neural networks supporting phoneme monitoring are modulated by phonology but not lexicality or iconicity: Evidence from British and Swedish Sign Language. Frontiers in Human Neuroscience, 13, Article 374.
[37] Sandler, W. (2017). The challenge of sign language phonology. Annual Review of Linguistics, 3, 43-63.
[38] Thierfelder P., Wigglesworth G., & Tang G. (2020). Sign phonological parameters modulate parafoveal preview effects in deaf readers. Cognition, 201, Article 104286.
[39] Thompson R. L., Vinson D. P., Fox N., & Vigliocco G. (2013). Is lexical access driven by temporal order or perceptual salience? Evidence from British Sign Language. Proceedings of the 35th annual meeting of the cognitive science society, Berlin, Germany.
[40] van der Kooij, E. (2002). Phonological categories in Sign Language of the Netherlands: The role of phonetic implementation and iconicity (Unpublished Doctorial dissertation). Leiden University.
[41] Villameriel S., Costello B., Dias P., Giezen M., & Carreiras M. (2019). Language modality shapes the dynamics of word and sign recognition. Cognition, 191, Article 103979.
[42] Wienholz A., Nuhbalaoglu D., Steinbach M., Herrmann A., & Mani N. (2021). Phonological priming in German Sign Language: An eye tracking study using the Visual World Paradigm. Sign Language & Linguistics, 24(1), 4-35.
[43] Winsler K., Midgley K. J., Grainger J., & Holcomb P. J. (2018). An electrophysiological megastudy of spoken word recognition. Language, Cognition and Neuroscience, 33(8), 1063-1082.

手语语音加工及其神经机制研究述评^*

Review of Phonological Processing in Sign Languages and Its Neural Mechanisms

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	薛靖华, 朱睿达, 刘超. 从经济领域到社会领域：决策中的不确定性^*[J]. 心理科学, 2024, 47(2): 384-392.
[2]	刘欣, 梅颖, 吴奇, 雷怡. 恐惧反转学习：一种新的恐惧调节手段^*[J]. 心理科学, 2024, 47(2): 494-501.
[3]	张玥赵光何蔚祺胡中华. 视觉空间伪忽视与表征伪忽视神经机制的异同[J]. 心理科学, 2022, 45(6): 1500-1507.
[4]	张思敏王允宏陈飞董亓易如赛李阳丁雪辰. 羞怯的认知神经机制[J]. 心理科学, 2022, 45(2): 386-393.
[5]	王海霞雷怡郑文瑜李红王超. rtfMRI-NF技术在抑郁症干预中的应用[J]. 心理科学, 2022, 45(2): 498-505.
[6]	杨晓娟刘建平李富洪来松海. 特质自我控制个体在多源干扰任务下的认知神经机制[J]. 心理科学, 2021, 44(6): 1322-1327.
[7]	刘佳宁吴奇谢涛俄眉王金霞雷怡. 应激激素对恐惧消退作用的神经生理机制[J]. 心理科学, 2021, 44(3): 559-566.
[8]	曹宇李恒. 象似性和手语学习经验对中国手语-汉语跨模态语义启动效应的影响[J]. , 2021, 44(1): 67-73.
[9]	胡艺馨邵韵如张晓燕王大伟. 夜间进食综合征的研究进展[J]. , 2020, 43(6): 1514-1521.
[10]	康冠兰罗霄骁. 视听跨通道信息的整合与冲突控制[J]. , 2020, 43(5): 1072-1078.
[11]	聂衍刚利振华窦凯. 预先承诺的理论解释及其神经机制[J]. , 2019, 42(5): 1202-1208.
[12]	阮怡君刘翔平. 发展性阅读障碍的视听时间敏感性缺陷及其脑神经机制[J]. 心理科学, 2019, 42(1): 36-42.
[13]	宋有明丁凤琴石文典陈晓惠. 文化差异视角下个体合作行为的产生、影响机制建构及认知神经基础[J]. 心理科学, 2018, 41(5): 1227-1232.
[14]	李海江卢家楣. 宽恕心理的行为测量及认知神经机制[J]. 心理科学, 2018, 41(3): 700-705.
[15]	蔺义芹谢慧夏礼胜张丹丹. 正常人以及精神障碍人群在合作与竞争中的表现[J]. 心理科学, 2018, 41(1): 225-230.