关键词: 认知诊断, 计算机化自适应测验, 项目区分度, 选题策略, 分层选题

Abstract: The dual-objective cognitive diagnostic computerized adaptive testing (CD-CAT) can simultaneously evaluate the knowledge state and ability of the participant, which exceedingly enrich the test information. Therefore, it is of great practical significance. The discrimination reflects items characteristics. It is an important factor affecting the quality of measurement. In addition to construct the information index of the item in CD-CAT, it can improve the relevance of the selected item by weighting the information index through the characteristics of the item or the participant. Hence, to improve the measurement accuracy of the dual-objective CD-CAT. Firstly, this paper proposed six selection methods by combining the six kinds of items discrimination (The discrimination index D, general model discrimination index GDI, odds ratio OR, the discrimination parameters a, attribute discrimination index ADI and cognitive diagnostic index CDI) with the information product approach (IPA). The new methods were D-WIPA, GDI-WIPA, OR-WIPA, a-WIPA, CDI-WIPA and ADI-WIPA, respectively. Secondly, to solve the problem of uneven item exposure in the existing item selection strategy of the dual-objective CD-CAT, This study draws lessons from traditional CAT stratification method. Six kinds of item discrimination indexes (D, GDI, OR, a, ADI and CDI) were used as stratification indexes of item bank respectively. To investigate the performance of different discrimination indexes in measuring accuracy and improving the uniformity of item exposure. The Monte Carlo simulation experiment was used in this study. Specific experimental design is as follows: (1) The item banks measured 6 attributes consisting of 400 items in total, and each item measured at least one attribute. (2) The true knowledge state of each participant was generated by HO-DINA. Among them, the slope parametersof the HO-DINA followed the lognormal distribution and the intercept parametersfollowed the standard normal distribution. (3) The responses were obtained by DINA model. Specifically, CDM parameters of slipping and guessing randomly selected from the uniform distribution in a range from 0.05 to 0.25. (4) Using 2PL to estimate the ability of each participant and item feature parameters based on the mirt package in R. (5) The D-WIPA, GDI-WIPA, OR-WIPA, a-WIPA, CDI-WIPA and ADI-WIPA were used to select item. (6) The test length was set to 15, 25 and 40. (7) Four indexs, namely the pattern measurement rates, root mean square error of latent trait, chi-square value and test overlap rate, ware adopted to compare the efficiency of different item selection methods. The same experiment was repeated 30 times. Simulation results indicated that: (1) The new methods proposed in this study can significantly improve the estimation accuracy of the knowledge state in short tests. With the increase of test length, the advantage of the new methods in the estimation accuracy of ability becomes more obvious. (2) OR-WIPA method performs best in estimating knowledge state and ability compare with D-WIPA and a-WIPA methods. The advantage of GDI-WIPA and CDI-WIPA methods in knowledge state are more obvious in short tests. On the contrary, the advantage of ability estimation are more obvious in long tests. (3) The stratification selection strategy can help each method to reduce the test overlap rate and chi-square value and imporove the uniformity of item exposure. Specially, the stratification selection according to the OR and D index can also maintain high measurement accuracy on the basis of controlling the exposure uniformity of the item. What’s more, stratification by D,CDI and GDI can also control item exposure well. Stratification by a or ADI index can control item exposure to some extent, but the performance is not good as other stratification indexs. To summarize, OR weighting method can significantly improve the accuracy of knowledge state and ability estimation. OR stratification selection also can greatly improve the item exposure uniformity under the condition of ensuring measurement accuryacy.

Key words: cognitive diagnostic, computerized adaptive testing, item discrimination, item selective strategy, stratification selection strategy