Abstract: In today's rapidly changing environment, cognitive and behavioral flexibility are becoming increasingly crucial. Reversal learning refers to the ability to adjust previously learned responses or strategies in the face of environmental changes or new rules, reflecting an individual's cognitive or behavioral flexibility. The reversal learning paradigm, originally applied in animal studies and later extended to human research, is widely used to assess cognitive flexibility. In a classic reversal learning paradigm, participants select between two stimuli to receive a reward; after a reversal of outcomes, they must adjust their choice. This process can be further complicated by probabilistic reversal learning to probe adaptability to changes. Despite numerous studies exploring the cognitive mechanisms, neural bases, and applications of reversal learning in the field of psychopathology, systematic reviews focusing on the cognitive and neural foundations of reversal learning and its applications remain scarce. This study aims to combine cognitive computational modeling and MRI research to comprehensively examine the cognitive processing models and neural mechanisms underlying reversal learning. It further analyzes the applications of reversal learning in psychopathology, with the goal of promoting the flexible use of reversal learning in future research and providing theoretical support for psychopathological studies.
The reinforcement learning models (RLM) allow for a nuanced analysis of the cognitive processes involved in reversal learning. These models divide the reversal learning process into decision-making, feedback reception, and learning stages, and provide corresponding computational metrics for each stage, such as value estimation (Q value), decision bias (P value), and decision stability (β value) during the decision-making stage; feedback sensitivity (ρ ), feedback strength (R ), feedback valence, and prediction error (PE) during feedback reception; and learning rate (α) in the learning stage.
The decision-making process in reversal learning is primarily influenced by the fronto-parietal network, including the ventromedial prefrontal cortex (vmPFC), dorsomedial prefrontal cortex (dmPFC), and parietal cortex. The processing of feedback intensity during the feedback phase is mainly associated with bilateral orbitofrontal cortex (OFC). Positive feedback processing primarily involves the medial OFC, ventral striatum (VS), anterior cingulate cortex (ACC), amygdala, and other brain regions associated with reward processing and emotional responses. In contrast, the processing of negative feedback and reward prediction errors (PE) relies on the fronto-parietal control network (dlPFC, vlPFC, IFG, and superior parietal lobule), the salience network (inferior frontal cortex, insula, and dACC), the emotion processing network (OFC and amygdala), the reward and motivation system (including the striatum and its specific regions such as dorsolateral striatum, ventral pallidum, ventral striatum, ventromedial prefrontal cortex, medial OFC, anterior insula, and dorsal anterior cingulate cortex), and the default mode network (mPFC and dPCC). This highlights a clear functional dissociation between the brain regions involved in processing positive and negative feedback. Finally, the learning phase predominantly engages the coordinated activity of the mPFC and dACC. These brain regions support successful completion of reversal learning tasks by integrating feedback information, monitoring errors, and adapting behavior.
Reversal learning paradigms have been widely used in neuropsychological research, such as in attention deficit hyperactivity disorder (ADHD), autism spectrum disorder (ASD), addictive behaviors, obsessive-compulsive disorder (OCD), depression and schizophrenia. For instance, individuals with ADHD may exhibit altered learning rates, leading to suboptimal responses to feedback, while those with OCD might demonstrate excessive sensitivity to negative feedback, resulting in rigid and perseverative behaviors. By integrating computational models and MRI techniques, reversal learning research not only reveals the dynamic characteristics of cognitive processing but also provides new perspectives for understanding cognitive flexibility deficits in various neuropsychiatric disorders.
According to the analysis of the existing research, there are three potential directions for future research. First, refine cognitive computational models to analyze more precisely the cognitive processing mechanisms underlying reversal learning. Second, update neural computational models to gain deeper insights into the neural basis of reversal learning. Finally, when applying reversal learning to psychopathology, individual differences should be fully considered.

Key words: reversal learning, flexibility, reinforcement learning model, magnetic resonance imaging (mri)

摘要： 反转学习是一种反映认知灵活性的关键认知能力。当前研究结合强化学习模型和脑影像研究,探讨了反转学习的认知神经基础及临床应用。研究发现,强化学习模型将反转学习分解为决策、反馈和学习（即根据反馈来调整后续行为）三个认知过程,并提供了精细化的认知计算模型。脑影像研究表明,决策过程由额顶控制网络主导;正反馈主要激活奖赏系统,负反馈则激活额顶控制网络（认知控制和注意调节）及情感加工网络等;学习过程涉及前额叶-扣带回网络。反转学习广泛应用于精神病理学研究,为理解注意缺陷多动障碍、抑郁症、强迫症等疾病提供了新视角。未来研究可通过完善认知计算模型和神经计算模型,以进一步提升反转学习的理论深度和应用广度。

关键词: 反转学习, 灵活性, 强化学习模型, 磁共振成像（MRI）