心理科学

心理科学 ›› 2023, Vol. 46 ›› Issue (6): 1486-1494.DOI: 10.16719/j.cnki.1671-6981.20230627

• 统计、测量与方法 • 上一篇    下一篇

多分属性的非参数诊断分类：18种距离判别法的对比 *

徐慧颖1, 陈琦鹏1, 刘耀辉1, 詹沛达**1,2   

  1. 1浙江师范大学心理学院, 金华, 321004;
    2浙江省智能教育技术与应用重点实验室, 金华, 321004
  • 出版日期:2023-11-20 发布日期:2023-12-19
  • 通讯作者: ** 詹沛达, E-mail: pdzhan@gmail.com
  • 基金资助:
    *本研究得到国家自然科学基金青年基金项目（31900795）和浙江省哲学社会科学规划“之江青年理论与调研专项课题”（22ZJQN38YB）的资助

Nonparametric Diagnostic Classification for Polytomous Attributes: A Comparison of 18 Distance Discriminant Methods

Xu Huiying1, Chen Qipeng1, Liu Yaohui1, Zhan Peida1,2   

  1. 1School of Psychology, College of Teacher Education, Zhejiang Normal University, Jinhua, 321004;
    2Key Laboratory of Intelligent Education Technology and Application of Zhejiang Province, Zhejiang Normal University, Jinhua, 321004
  • Online:2023-11-20 Published:2023-12-19

PDF

摘要： 本文通过两个模拟研究和一个实证研究对比探究了18种非参数距离判别法在多分属性诊断测验中的心理计量学表现。模拟研究结果表明：(1)在多分属性诊断测验中非参数距离判别法的分类准确性不受样本量的影响;(2)随题目质量的升高或测验长度的增加而增加,随着属性个数或属性水平数的增加而减少;(3)在不同测验情境下的表现优劣具有稳健性,其中坎贝拉距离、曼哈顿距离、欧式距离、标准欧式距离、平方欧式距离、闵氏距离、汉明距离和索卡尔米切纳差异这8种方法表现接近且相对较好。结合实证研究结果发现,无论是分析模拟数据还是分析实证数据,非参数距离判别法的表现具有稳健性;且表现较好的8种方法与参数模型的分类结果具有较高的一致性。

关键词: 认知诊断, 多分属性, 非参数认知诊断, 距离判别法, 汉明距离, 欧氏距离, 曼哈顿距离

Abstract:

In the past decades, there has been increasing interests in cognitive diagnostic assessment (CDA) that allows identifying the mastery or non-mastery of specific fine-grained attributes required for solving problems in educational and psychological assessments. In the field of cognitive diagnosis, researchers have proposed a variety of methods to classify respondents into several classes according to their attribute patterns. In general, existing methods can be classified into two categories, including parametric and non-parametric diagnostic methods. Parametric diagnostic methods are based on psychometric models. For different test situations, the theoretical relationship between observed response pattern (ORP) and latent attribute vector is described by cognitive diagnosis models (CDM), such as the DINA model and its generalized models. In contrast, the distance discrimination method in non-parametric diagnostic methods generally assigns respondents directly to a latent category by minimizing the distance between the ORP and the ideal response pattern (IRP). The most important feature of non-parametric diagnostic methods is that they do not involve any CDM and can be computed at arbitrary sample sizes. Examples include the Hamming distance discriminant and the Manhattan distance discriminant.

However, the majority of current methods (both parametric and non-parametric diagnostic methods) assumes that attributes are binary variables (e.g., "0" for "non-mastery" and "1" for "mastery"). This "black-and-white" classification is too coarse and may not be able to meet the needs of refined measurement in practical scenarios. With the increasing demand for refined diagnosis, several CDMs with polytomous attributes have been proposed in recent years. However, non-parametric diagnostic methods have not yet touched on polytomous attributes, which leaves a gap for researchers and practitioners to understand the performance of non-parametric diagnostic methods in the diagnostic assessments with polytomous attributes.

To investigate the performance of non-parametric diagnostic methods in the diagnostic assessments with polytomous attributes, two simulation studies were conducted to compare the diagnostic classification accuracy of 20 non-parametric distance discrimination methods under different test conditions consisting of 5 independent variables, including sample size, item quality, test length, number of attribute levels, and number of attributes, and to compare them with a CDM with polytomous attributes. In simulation study 1, three independent variables were manipulated, including sample size (N = 30, 50, and 100), test length (I = 25 and 50), and item quality (IQ = high; i.e., the mean value of guessing and slipping is around. (1) And low (i.e., the mean value of guessing and slipping is around .(2) All 18 non-parametric methods were implemented using Python's SciPy package; the CDM with polytomous attributes was implemented using the full Bayesian MCMC algorithm. The weighted- and exact-attribute pattern correct classification rates were used to evaluate the classification accuracy. In simulation study 2, two independent variables were manipulated: the number of attributes (K = 3, 5, and 7) and the number of attribute levels (Lk = 3 and 5). The sample size was fixed as 100, the test length was fixed as 50, and the item quality was fixed as high, respectively. All other conditions were consistent with simulation study 1.

The results of studies indicated that: (1) The effect of sample size on the classification accuracy of all non-parametric methods was small; (2)The classification accuracy of non-parametric methods increased with increasing item quality and test length, but decreased with the increasing number of attributes and number of attribute levels; and (3) In two simulation studies, the performance of the 18 non-parametric distance discrimination methods was robust across all test conditions, with the 8 distances of Canberra, Manhattan, Euclidean, Seuclidean, Sqeuclidean, Minkowski, Hamming, and Sokal-Michener dissimilarity distance discrimination methods performing relatively better. (4) In empirical study, the classification findings of the majority of nonparametric distance discriminant approaches match well with the RPa-DINA model.

In conclusion, this study is the first attempt to explore the performance of non-parametric diagnostic methods in the diagnostic assessments with polytomous attributes, which expands the application of non-parametric diagnostic methods and enriches the data analysis methods for polytomous attributes.

Key words: cognitive diagnosis, polytomous attributes, non-parametric cognitive diagnosis, distance discrimination, hamming distance, euclidean distance, Manhattan distance