Abstract
Cognitive diagnostic computerized adaptive testing (CD-CAT) has become an increasingly important testing mode. Item selection methods are currently one of the most pressing issues in the field of CD-CAT research. For some tasks, not only a participant’s strategy and knowledge state (KS, i.e., attribute mastery pattern) need to be diagnosed, but also his/her macro ability needs to be assessed in CD-CAT. For the sake of high efficient, item selection methods for this type of CD-CAT have to consider the above three purposes.
The Multiple-Strategy Shannon Entropy (MSSHE) index (Dai et al., 2015) and Fisher information index were combined to produce two new synthetic item selection methods in this paper. The first method was the “multiple-strategy dapperness with information” (MSDWI) index method, in which the MSDWI index was the Fisher information index Multiplied by the reciprocal of the MSSHE index. The second method was the “two-step item selection method”, in which items were first selected using the MSSHE index and then selected using the Fisher information index when the test length reached a certain value.
Based on the multiple-strategy reduced reparameterized unified model (MS-RRUM) and independence attributes, under different lengths of fixed-length CD-CAT and with various numbers of knowledge attributes, three selection item methods were systematically compared in the present study: the above two methods and random method. The strategy, KS and macro ability recoveries were considered as evaluation indices. Experiments were repeated 20 times using the MATLAB 2012b software package. Groups of examinees were simulated, and each group was composed of 1000 examinees. In the simulation, examinees answered the items that were selected by a certain item selection method, and their strategies, KSs and abilities were estimated. Then, the estimated values were compared with the true values.
Results showed that with respect to strategy, KS and ability estimations, the two-step method always yielded the best performance when the attribute number was 4 or 6. When the attribute number was 8, the two-step method yielded the best performance with respect to strategy and KS estimations; however, the random method yielded the best performance with respect to ability estimation.
In single-strategy CD-CAT which considering KS and ability estimations meanwhile, the “dapperness with information” (DWI) index method which was “in one step” was proper (Dai, Zhang, & Li, 2016). However, according to the current study, in multiple-strategy CD-CAT which considering strategy, KS and ability estimations meanwhile, the MSDWI method which was “in one step” was inappropriate. Instead, the two-step method was better. Besides, for this context, it is not proper to involve too much attributes in a single test.
This study performs the CD-CAT which considering three purposes meanwhile, and provides a theoretical foundation for more complicated cognitive diagnostic modeling in future studies. A proper item selection method in this CD-CAT context has been proposed, making item selection efficient and saving item bank.
Key words
CD-CAT /
multiple-strategy cognitive diagnosis /
multiple-strategy reduced reparameterized unified model (MS-RRUM) /
two-step item selection method
Cite this article
Download Citations
Guang MingLI Xin GuangWANG shan HU.
A CD-CAT Item Selection Method Containing Strategy, Knowledge State and Ability[J]. Journal of Psychological Science. 2018, 41(2): 459-465
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}