Psychological Science ›› 2018, Vol. 41 ›› Issue (1): 180-188.

Previous Articles     Next Articles

A simple and effective Q-matrix Estimation method:from non-parametric perspective

1, Yu-Ting HAN1,3,Tu Dong-Bo   

  • Received:2016-11-29 Revised:2017-06-16 Online:2018-01-20 Published:2018-01-20
  • Contact: Tu Dong-Bo

一种简单有效的Q矩阵估计方法开发:基于非参数化方法视角

汪大勋1,高旭亮2,韩雨婷1,3,涂冬波1   

  1. 1. 江西师范大学
    2. 江西师范大学心理学院
    3. 北京师范大学
  • 通讯作者: 涂冬波

Abstract: Abstract Nowadays, we are not satisfied with a total score from measurement, but hope to get a informative report. As the core of new generation test theory, cognitive diagnosis(CD) attracts more and more people's attention. Since it can reveal the result form a microscopic perspective, such as individuals’ knowledge structures, processing skills and cognitive procedure etc, it would help us to take individualized teaching and promote students ' development. Cognitive diagnosis assessments infer the attribute mastery pattern of respondents by item responses based on Q-matrix. The Q-matrix plays the role of a bridge between items and respondents. Many studies have shown that misspecification of the Q-matrix can affect the accuracy of model parameters and result in the misclassification of respondents. In practice, Q_matrix is established by experts. However ,with the application of cognitive diagnosis ,more and more researchers found that specification of Q-matrix was very hard. Different experts may provided different Q_matrices. To avoid the subjectivity from experts in Q-matrix specification and ensure the correct of Q_matrix, researchers are trying to look for objective methods. Many researchers have found a number of methods to estimate and validate the Q_matrix. Nevertheless,existing methods need information from parameter and a large amount of computation. To simplify the method of Q-matrix estimation, this article introduces a new Q-matrix estimation method based on Hamming Distance(HD) which is simple and non-parametric. The process of the method as follow: Firstly , we infer the attribute mastery pattern of respondents by Hamming Distance. Secondly, we can establish a Expected Response Pattern(ERP) matrix by the relationship between attribute mastery pattern of all respondents and each measurement pattern. Finally, the method measures the distance between all respondents’ Observed Response Pattern(ORP)and Expected Response Pattern(ERP)in each measurement pattern, and choose a measurement pattern with the minimum Hamming Distance to items. In this way, we can infer the measurement pattern of items. When there are more than one measurement patterns which are the same minimum Hamming Distance, we take random choose. This method based on some items which were assumed correctly pre-specified. In order to explore the effect of the method, we considered different number of participants, different number of base items and different Q-matrix whose attribute number is different. The item parameters and attribute mastery pattern of respondents are obeyed a uniform distribution. The Monte Carlo simulation study and real data study showed that: generally, the Hamming Distance method can recover the real Q-matrix with a high rate of success, especially when item attributes is 3 and the number of base items is more than 10. When attributes is 3, no matter how many base items and participants is, the rate of success of the method can reach at least 97%. When the number of base items is more than 10, no matter how many participants is, the rate of success can reach 90% in 3 Q-matrix. Relative to the sample size, the number of base items is more important. Furthermore, the method is easier to understand and needs less computation. For example, the time taken by program is not more than 30 second under 3 attributes and 8 base items condition. The real data study also showed that the Hamming Distance method can estimate the Q_matrix with a high success rate. The results of this study demonstrate that Hamming Distance method is an easy and preferable method in Q-matrix Estimation. Compared to the existing methods, Hamming Distance method is faster and superior. Besides, without the needs of parameters estimation, the method is not affected by the deviation caused by the misfit between model and data. In a word, the Hamming Distance method is simple and effective in Q-matrix Estimation, what is meaningful to the simplification of cognitive diagnosis.

Key words: Key words cognitive diagnosis, Q-matrix, Hamming Distance, DINA model

摘要: 摘要:Q矩阵是认知诊断的基础,错误的Q矩阵会影响参数估计和被试诊断正确率,开发一种简单而有效的Q矩阵估计方法有助于Q矩阵的正确界定。相对于参数化的Q矩阵估计方法,本研究将海明距离(Hamming Distance,HD)用于Q矩阵估计,开发出一种简单有效的非参数化的Q矩阵估计方法。采用Monte Carlo模拟方法与实证研究相结合的研究范式,对该方法的科学性与合理性及其效果进行研究,研究结果发现(1)基于海明距离的Q矩阵估计法具有较高的估计正确率,并且该方法不受被试样本容量影响。(2)该方法简单易懂,运算时间短,是一种简单而有效的Q矩阵估计方法。(3)新方法对于Tatsuka(1990)分数减法测验的Q矩阵的估计准确率尚可,说明新方法在实践中具有较好的潜在应用前景与应用价值。

关键词: 关键词:认知诊断, Q矩阵, 海明距离, DINA模型