Comparison and Application of Polytomous Cognitive Diagnosis Models with Different Link Functions: Based on category-level Q-matrix

Journal of Psychological Science ›› 2019, Vol. 42 ›› Issue (2) : 437-445.

PDF(1596 KB)
PDF(1596 KB)
Journal of Psychological Science ›› 2019, Vol. 42 ›› Issue (2) : 437-445.

Comparison and Application of Polytomous Cognitive Diagnosis Models with Different Link Functions: Based on category-level Q-matrix

Author information +
History +

Abstract

Cognitive Diagnosis Assessment (CDA) has recently received increasing attention due to the diagnostic function that is not available in traditional measurement theory. At present, a variety of CDMs have been proposed, but these models are mostly 0-1 score, cannot be used for polytomously scored items, and that will greatly limit their application and development in cognitive diagnosis. Because polytomous scored items are very common in practical applications, they have higher efficacy than dichotomous scored items. To deal with polytomous items more appropriately, a few polytomous CDMs have been developed. In view of the current research, researchers often choose different link functions to develop polytomous CDMs:Dongbo Tu developed the polytomous P-DINA model based on the Global or Cumulative link function in 2010, and Ma & de la Torre developed the Seq-GDINA model based on the Continuation Ration link function in 2016. In the Seq-GDINA model, the author defines a property definition based on the item category, modifying the traditional Q matrix to a category-level Q-matrix (QC-matrix). The traditional Q-matrix is based on the definition of the project level, ignoring the different manifestations of the attributes in the category, which will result in the loss of the existing information. In this paper a new polytomous model LC-DINA based on Local or Adjacent Categories Link Function was developed, and the MCMC algorithm was used to realize the estimation of the parameters and the properties of the new model. Then, the proposed model LC-DINA was compared with the existing model Seq-DINA, P-DINA under the QC setting by Simulation study and using the data from the Trends in International Mathematics and Science Study 2007(TIMMS) assessment. In conclusion, the Monte Carlo simulation and empirical application research are used to study the logical structure and applicable conditions of the three cognitive recognition models under three different link functions, providing a practical reference for users. The results of the paper showed that:(1)The good precision of ability parameters indicated that MCMC algorithm method and new polytomous cognitive diagnosis model was feasible. (2)The comparison of LC-DINA, P-DINA and Seq-DINA showed that the estimated precision of person parameters (MMR and PRM) is influenced by the quality of the item. The attribute match ration (MMR & PMR) decreased with the increase of the slipping and guessing parameter. (3)When the real model is consistent with the fitting model, the attribute match ration is the highest. In this condition, the LC-DINA model is the best when the item quality is best and the P-DINA model is the best when the item quality is poor, and the Seq-DINA model is the worst in all the conditions. (4)In the application of using the data from the TIMSS 2007 assessment show that: LC-DINA is the most suitable model based on the Deviance Information Criterion (DIC) and the correlation coefficient between the expected score and the observed score. In the Seq-GDINA model, the author defines a property definition based on the item category, modifying the traditional Q matrix to a category-level Q-matrix (QC-matrix). Take √(7.5?0.3-16)=?as an example, to solve this item, three steps may be involved: first calculate 7.5 / 0.3 = 25, need to use the "division" (A1), Then calculate 25-16 = 9, use "subtraction" (A2), and finally calculate √9=3, use "open square" (A3). In the example, you can see that the different categories of the project measure different attributes. The QC-matrix is defined as follows: item j is assumed to have H j+1 categories (i.e., 0; 1; ...; Hj ),the QC-matrix is a H j*K binary matrix, each of Hj rows has K elements indicating which attributes are required by the category. Therefore, the QC matrix of the above example is [■(1&0&0@0&1&0@0&0&1)], While the traditional Q matrix is [1,1,1]. The traditional Q-matrix is based on the definition of the project level, ignoring the different manifestations of the attributes in the category, which will result in the loss of the existing information. In this paper a new polytomous model LC-DINA based on Local or Adjacent Categories Link Function was developed, and the MCMC algorithm was used to realize the estimation of the parameters and the properties of the new model. Then, The proposed model LC-DINA was compared with the existing model SEQ-DINA, P-DINA under the QC setting by Simulation study and using the data from the Trends in International Mathematics and Science Study 2007(TIMMS) assessment. In conclusion, the Monte Carlo simulation and empirical application research are used to study the logical structure and applicable conditions of the three cognitive recognition models under three different link functions, which provides a reference for practical users. The results of the paper showed that:(1)The good precision of ability parameters indicated that MCMC algorithm method and new polytomous cognitive diagnosis model was feasible. (2) The comparison of LC-DINA, P-DINA and SEQ-DINA showed that the estimated precision of person parameters (MMR and PRM) is influenced by the quality of the item. The attribute match ration (MMR & PMR) decreased with the increase of the slipping and guessing parameter.(3) When the real model is consistent with the fitting model, the attribute match ration is the highest. In this condition, the LC-DINA model is the best when the item quality is best and the P-DINA model is the best when the item quality is poor, and the Seq-DINA model is the worst in all the conditions.(4)In the application of using the data from the TIMSS 2007 assessment show that: LC-DINA is the most suitable model based on the Deviance Information Criterion(DIC) and the correlation coefficient between the expected score and the observed score.

Key words

cognitive diagnosis model / link function / Polytomous model / MCMC algorithm

Cite this article

Download Citations
Comparison and Application of Polytomous Cognitive Diagnosis Models with Different Link Functions: Based on category-level Q-matrix[J]. Journal of Psychological Science. 2019, 42(2): 437-445
PDF(1596 KB)

Accesses

Citation

Detail

Sections
Recommended

/