Journal of Psychological Science ›› 2023, Vol. 46 ›› Issue (2): 470-477.

Previous Articles     Next Articles

An Application of the Odds Ratios Method in Differential Item Functioning: An Illustration with an English Test

Liu Chutong1, Jin Ruyi1, He Ying1, Zhang Minqiang1,2,3,4, Gao Fangxin1   

  1. 1School of Psychology, South China Normal University, Guangzhou, 510631 2Key Laboratory of Brain, Cognition and Education Sciences, South China Normal University, Ministry of Education, Guangzhou, 510631 3Center for Studies of Psychological Application, South China Normal University, Guangzhou, 510631 4Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University, Guangzhou, 510631
  • Received:2020-11-08 Revised:2021-08-05 Online:2023-03-20 Published:2023-03-20

OR法在DIF检验中的应用——以英语学业能力测验为例

刘楚铜1 金如意1 何 颖1 张敏强1,2,3,4 高方昕1   

  1. 1 华南师范大学心理学院,广州,510631 2 华南师范大学脑认知与教育科学教育部重点实验室,广州,510631 3 华南师范大学心理应用研究中心,广州,510631 4 华南师范大学广东省心理健康与认知科学重点实验室,广州,510631
  • 通讯作者: 张敏强

Abstract: The detection of differential item functioning (DIF) is an essential step for increasing the validity of a test among groups. A common feature of the conventional DIF approaches is that different groups of examinees are placed on the same metric based on a matching variable. However, contaminated matching variable would make the DIF detection biased. Some investigators developed the odds ratios (OR) method, which does not require matching variables. The OR method is easy to understand and simple for practitioners. However, all previous studies about the OR method are simulation studies, and the application of OR method in the DIF detecting of the academic achievement test needs further investigation. Previous studies indicated that the IRT-based approaches performed well in the DIF detecting of the academic achievement test. The Wald χ^2 test and likelihood ratio test (LRT) are widely used in DIF detection as IRT-based approaches. It is helpful to compare the performance and the operating procedure of OR method with the Wald χ^2 test and the LRT method. The current study aimed to introduce the operating procedure of the OR methods in the DIF detecting of the academic achievement test, and demonstrate that the OR method was a better method in DIF detecting by comparing the performance and the operating procedure among the OR method, the Wald χ^2 test, and the LRT method. An empirical study was conducted, and the data was selected from an English test. Valid data from 3241 senior high school students, including 1427 males (44.03%) were analyzed. The DIF detection was conducted on 35 dichotomous items from the English examination. The difficulty of the 35 dichotomous items were acceptable (from .20 to .80). Besides, the Cronbach’s alphas of the 35 items were .86. The DIF detection results of the OR method, the Wald χ^2 test, and the LRT method on English test would be testified compared. The empirical study found that 16 items were detected as DIF items by the OR method, including both 8 items favoring females and males respectively. And 14 items were detected as DIF items by the Wald χ2 test, including 10 items favoring females and 4 items favoring males. In addition, there were 14 items detected as DIF items by the LRT method, including 8 items favoring females and 6 items favoring males. The DIF detection results of the OR method and the LRT method were consistent for 31 items, and inconsistent for 4 items. The DIF detection results of the Wald χ2 test and the LRT method were consistent for 19 items, and inconsistent for 16 items. Previous studies had provided evidence for the application of the LRT method in the DIF detecting of the academic achievement test. And this study indicated that performance of the OR method was similar to the LRT method, but different to the Wald χ2 test. Thus, the present study also provided evidence for the application of OR method in the DIF detection of the academic achievement test, which was of great empirical value. Besides, concerning the operating procedure, the OR method is easy and simple for practitioners, and the computation task in the OR method is minimized. It is obvious that the OR method is more efficient. Generally speaking, the OR method is practicable and efficient in DIF detection.

Key words: odds ratios method, differential item functioning, method applicating

摘要: 本文将比值比方法(Odds Ratios Method, OR)这一DIF检验方法应用到某英语测验的DIF检验中,介绍OR法进行DIF检验的完整流程,并将其检验结果与应用较为广泛的Wald χ^2检验和似然比检验(Likelihood Ratio Test, LRT)的检验结果进行比较,以进一步验证OR法在实证研究中的适用性。研究结果显示,OR法与LRT法结果差异较小且OR法步骤更简便,说明OR法与LRT法在DIF检验中的表现相似,又LRT法适用于实证研究中,因此OR法同样适用于实证研究的DIF检验。

关键词: 比值比方法, 项目功能差异, 方法应用