OR法在DIF检验中的应用——以英语学业能力测验为例

刘楚铜 金如意 何颖 张敏强 高方昕

心理科学 ›› 2023, Vol. 46 ›› Issue (2) : 470-477.

PDF(919 KB)
中文  |  English
PDF(919 KB)
心理科学 ›› 2023, Vol. 46 ›› Issue (2) : 470-477.
统计、测量与方法

OR法在DIF检验中的应用——以英语学业能力测验为例

  • 刘楚铜1 金如意1 何 颖1 张敏强1,2,3,4 高方昕1
作者信息 +

An Application of the Odds Ratios Method in Differential Item Functioning: An Illustration with an English Test

  • Liu Chutong1, Jin Ruyi1, He Ying1, Zhang Minqiang1,2,3,4, Gao Fangxin1
Author information +
文章历史 +

摘要

本文将比值比方法(Odds Ratios Method, OR)这一DIF检验方法应用到某英语测验的DIF检验中,介绍OR法进行DIF检验的完整流程,并将其检验结果与应用较为广泛的Wald χ^2检验和似然比检验(Likelihood Ratio Test, LRT)的检验结果进行比较,以进一步验证OR法在实证研究中的适用性。研究结果显示,OR法与LRT法结果差异较小且OR法步骤更简便,说明OR法与LRT法在DIF检验中的表现相似,又LRT法适用于实证研究中,因此OR法同样适用于实证研究的DIF检验。

Abstract

The detection of differential item functioning (DIF) is an essential step for increasing the validity of a test among groups. A common feature of the conventional DIF approaches is that different groups of examinees are placed on the same metric based on a matching variable. However, contaminated matching variable would make the DIF detection biased. Some investigators developed the odds ratios (OR) method, which does not require matching variables. The OR method is easy to understand and simple for practitioners. However, all previous studies about the OR method are simulation studies, and the application of OR method in the DIF detecting of the academic achievement test needs further investigation. Previous studies indicated that the IRT-based approaches performed well in the DIF detecting of the academic achievement test. The Wald χ^2 test and likelihood ratio test (LRT) are widely used in DIF detection as IRT-based approaches. It is helpful to compare the performance and the operating procedure of OR method with the Wald χ^2 test and the LRT method. The current study aimed to introduce the operating procedure of the OR methods in the DIF detecting of the academic achievement test, and demonstrate that the OR method was a better method in DIF detecting by comparing the performance and the operating procedure among the OR method, the Wald χ^2 test, and the LRT method. An empirical study was conducted, and the data was selected from an English test. Valid data from 3241 senior high school students, including 1427 males (44.03%) were analyzed. The DIF detection was conducted on 35 dichotomous items from the English examination. The difficulty of the 35 dichotomous items were acceptable (from .20 to .80). Besides, the Cronbach’s alphas of the 35 items were .86. The DIF detection results of the OR method, the Wald χ^2 test, and the LRT method on English test would be testified compared. The empirical study found that 16 items were detected as DIF items by the OR method, including both 8 items favoring females and males respectively. And 14 items were detected as DIF items by the Wald χ2 test, including 10 items favoring females and 4 items favoring males. In addition, there were 14 items detected as DIF items by the LRT method, including 8 items favoring females and 6 items favoring males. The DIF detection results of the OR method and the LRT method were consistent for 31 items, and inconsistent for 4 items. The DIF detection results of the Wald χ2 test and the LRT method were consistent for 19 items, and inconsistent for 16 items. Previous studies had provided evidence for the application of the LRT method in the DIF detecting of the academic achievement test. And this study indicated that performance of the OR method was similar to the LRT method, but different to the Wald χ2 test. Thus, the present study also provided evidence for the application of OR method in the DIF detection of the academic achievement test, which was of great empirical value. Besides, concerning the operating procedure, the OR method is easy and simple for practitioners, and the computation task in the OR method is minimized. It is obvious that the OR method is more efficient. Generally speaking, the OR method is practicable and efficient in DIF detection.

关键词

比值比方法 / 项目功能差异 / 方法应用

Key words

odds ratios method / differential item functioning / method applicating

引用本文

导出引用
刘楚铜 金如意 何颖 张敏强 高方昕. OR法在DIF检验中的应用——以英语学业能力测验为例[J]. 心理科学. 2023, 46(2): 470-477
Liu Chutong, Jin Ruyi, He Ying, Zhang Minqiang, Gao Fangxin. An Application of the Odds Ratios Method in Differential Item Functioning: An Illustration with an English Test[J]. Journal of Psychological Science. 2023, 46(2): 470-477

PDF(919 KB)

Accesses

Citation

Detail

段落导航
相关文章

/