Abstract
Cognitive Diagnostic Assessment is based on the incidence Q-matrix (Tatsuoka, 2009). The entries of Q-matrix indicate which skills and knowledge are involved in the solution of each item. In real situations, whether the items have or have not been identified attributes before its construction, it will cost a lot of money, require more efforts to identify attributes through specialists according the special procedure and yet can’t completely assume the correctness due to the subjectivity. On-line item attributes identification is a new field and study of the impact of item bank hasn’t been found in the literature. So this study is concerned with the impact of item bank on on-line item attributes identification in cognitive diagnostic computerized adaptive testing (CD-CAT), especially when the item bank doesn’t include the whole reachability matrix.
The study describes the impact of knowledge states’ equivalent classes on the item attributes vectors’ equivalent classes. Some of those are called the discriminate item attributes vector when the item attributes vectors’ equivalent classes only include one item attributes vector; the others are called indiscriminate item attributes vector. Moreover, the study introduces Marginal Maximum Likelihood Estimation (MMLE) for on-line item attributes identification, which integrates the uncertainty of estimate knowledge states in the procedure of identification.To explore whether the accurary of discriminate item attributes vectors is better than that of indiscriminate item attributes vectors, and whether the columns of reduced Q matrix except the columns of reachability matrix can provide the reasonable accurary of attributes identification. Considering six attributes under the unstructured condition, two simulation experiments are conducted using deterministic inputs,noisy “and” gate model (DINA).
The simulation results show that log odds ratios are almost all above zero. It indicates that the correct classification rates (the proportion of times a item is correctly classified on a attributes vector) of the discriminate item attributes vector is significantly better than indiscriminate item attributes vector. The more number of the items in reduced Q matrix except the whole reachability matrix counld compensate the insufficient item bank to some extent. It also demonstrates that the reachability matrix is important for item bank designed for cognitive diagnostic computerized adaptive testing. Other areas of applications of the reachability matrix , including test construction and test equating, are worth further consideration.
Key words
the reachability matrix /
CD-CAT /
On-Line identification /
MMLE /
DINA
Cite this article
Download Citations
The Study of the Impact of the Structure of Item Bank on On-Line Raw Item Attributes Identification Accuracy[J]. Journal of Psychological Science. 2012, 35(2): 452-456
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}