关键词: 反转学习, 灵活性, 强化学习模型, 磁共振成像（MRI）

Abstract: In today's rapidly changing environment, cognitive and behavioral flexibility are becoming increasingly crucial. Reversal learning refers to the ability to adjust previously learned responses or strategies in the face of environmental changes or new rules, reflecting an individual's cognitive or behavioral flexibility. The reversal learning paradigm, originally applied in animal studies and later extended to human research, is widely used to assess cognitive flexibility. In a classic reversal learning paradigm, participants select between two stimuli to receive a reward; after a reversal of outcomes, they must adjust their choice. This process can be further complicated by probabilistic reversal learning to probe adaptability to changes. Despite numerous studies exploring the cognitive mechanisms, neural bases, and applications of reversal learning in the field of psychopathology, systematic reviews focusing on the cognitive and neural foundations of reversal learning and its applications remain scarce. This study aims to combine cognitive computational modeling and MRI research to comprehensively examine the cognitive processing models and neural mechanisms underlying reversal learning. It further analyzes the applications of reversal learning in psychopathology, with the goal of promoting the flexible use of reversal learning in future research and providing theoretical support for psychopathological studies.
The reinforcement learning models (RLM) allow for a nuanced analysis of the cognitive processes involved in reversal learning. These models divide the reversal learning process into decision-making, feedback reception, and learning stages, and provide corresponding computational metrics for each stage, such as value estimation (Q value), decision bias (P value), and decision stability (β value) during the decision-making stage; feedback sensitivity (ρ ), feedback strength (R ), feedback valence, and prediction error (PE) during feedback reception; and learning rate (α) in the learning stage.
The decision-making process in reversal learning is primarily influenced by the fronto-parietal network, including the ventromedial prefrontal cortex (vmPFC), dorsomedial prefrontal cortex (dmPFC), and parietal cortex. The processing of feedback intensity during the feedback phase is mainly associated with bilateral orbitofrontal cortex (OFC). Positive feedback processing primarily involves the medial OFC, ventral striatum (VS), anterior cingulate cortex (ACC), amygdala, and other brain regions associated with reward processing and emotional responses. In contrast, the processing of negative feedback and reward prediction errors (PE) relies on the fronto-parietal control network (dlPFC, vlPFC, IFG, and superior parietal lobule), the salience network (inferior frontal cortex, insula, and dACC), the emotion processing network (OFC and amygdala), the reward and motivation system (including the striatum and its specific regions such as dorsolateral striatum, ventral pallidum, ventral striatum, ventromedial prefrontal cortex, medial OFC, anterior insula, and dorsal anterior cingulate cortex), and the default mode network (mPFC and dPCC). This highlights a clear functional dissociation between the brain regions involved in processing positive and negative feedback. Finally, the learning phase predominantly engages the coordinated activity of the mPFC and dACC. These brain regions support successful completion of reversal learning tasks by integrating feedback information, monitoring errors, and adapting behavior.
Reversal learning paradigms have been widely used in neuropsychological research, such as in attention deficit hyperactivity disorder (ADHD), autism spectrum disorder (ASD), addictive behaviors, obsessive-compulsive disorder (OCD), depression and schizophrenia. For instance, individuals with ADHD may exhibit altered learning rates, leading to suboptimal responses to feedback, while those with OCD might demonstrate excessive sensitivity to negative feedback, resulting in rigid and perseverative behaviors. By integrating computational models and MRI techniques, reversal learning research not only reveals the dynamic characteristics of cognitive processing but also provides new perspectives for understanding cognitive flexibility deficits in various neuropsychiatric disorders.
According to the analysis of the existing research, there are three potential directions for future research. First, refine cognitive computational models to analyze more precisely the cognitive processing mechanisms underlying reversal learning. Second, update neural computational models to gain deeper insights into the neural basis of reversal learning. Finally, when applying reversal learning to psychopathology, individual differences should be fully considered.

Key words: reversal learning, flexibility, reinforcement learning model, magnetic resonance imaging (mri)