A Probabilistic Representation Approach for the Nonparametric Classification Method to Cognitive Diagnosis

Journal of Psychological Science ›› 2021, Vol. 44 ›› Issue (5) : 1249-1258.

PDF(1250 KB)
PDF(1250 KB)
Journal of Psychological Science ›› 2021, Vol. 44 ›› Issue (5) : 1249-1258.

A Probabilistic Representation Approach for the Nonparametric Classification Method to Cognitive Diagnosis

Author information +
History +

Abstract

Cognitive diagnostic assessment (CDA) can be regarded as a kind of formative assessments because it is intended to promote assessment for learning in classrooms by providing the diagnostic information about students’ cognitive strengths and weaknesses. CDA has received increasing attention in recent years. Statistical pattern recognition and classification methodology are two critical parts in CDA. Q-matrix corresponding to feature generation is of paramount importance in CDA. Therefore, Q-matrix becomes a popular research area in the field of CDA. Beside of the study of Q-matrix, many of the studies have focused on cognitive diagnostic models (CDMs). CDMs required a large sample size for calibrating item parameters. Therefore, the nonparametric classification method has become a popular method for classroom assessments. It is very suitable for classroom assessments, because it does rely on a large sample size for the calibration of item parameters. However, classification results for nonparametric classification method are often expressed as the form of 0-1 vector. The classification results lack a probabilistic representation, and cannot finely distinguish the differences in the degree of mastery of attributes among the subjects. At the same time, it lacks the reliability and validity indicators for evaluating the quality of classification results. To solve this problem, a probabilistic representation based approach for the nonparametric classification method to cognitive diagnosis is proposed based on binomial distribution and Boltzmann distribution. The probabilistic representation method handles attributes mastery status at a probability level. A simulation study was conducted to investigate the performance of the probabilistic representation method under three factors (slip parameter, guessing parameter, and two CDMs). Five independent attributes and the reduced Q-matrix as a test Q-matrix were considered in the simulation study. Item response data was generated by the deterministic inputs, noisy “and” gate (DINA) model or the deterministic inputs, noisy “or” gate (DINO) model. Simulation results showed that the performance of the new method is promising in terms of pattern matched rates and attribute matched rates. Results showed that the 0-1 classification results transformed from the probabilistic representation are highly consistent with those of the nonparametric classification method; the attribute posterior probability obtained from the probabilistic representation method is very closed with that of DINA or DINA model; the marginal posterior probability of attributes obtained from the new method can be used to evaluate the validity of classification results. From the simulation and real data study, the major findings and implications are the following: (a) compared with the DINA or DINO model and the conjunctive or disjunctive nonparametric classification method, the accuracy of the nonparametric method is slightly lower than that of the DINA or DINO model when the slip and guessing parameters of test items are quite different; (b) under most conditions, binomial distribution and Boltzmann distribution can be applied to accurately estimate the attribute mastery probability for the nonparametric classification method; (c) classification consistency and accuracy in CDMs and the probabilistic representation can be combined to construct the reliability and validity index for the evaluation of the classification quality of the nonparametric classification method.

Key words

cognitive diagnosis / nonparametric classification method / attributes mastery probability / classroom assessment / classification accuracy / classification consistency

Cite this article

Download Citations
A Probabilistic Representation Approach for the Nonparametric Classification Method to Cognitive Diagnosis[J]. Journal of Psychological Science. 2021, 44(5): 1249-1258
PDF(1250 KB)

Accesses

Citation

Detail

Sections
Recommended

/