关键词: 认知诊断计算机化自适应测验, 题目曝光控制, 分类准确率, 曝光率

Abstract: Cognitive diagnostic assessment (CDA) has recently been attracted more and more attention in educational and psychological measurement. CDA can provide detailed information about the strengths and weakness of individuals for a specific content domain. At the same time, the computerized adaptive testing (CAT) can lead to as well as or even better measure accuracy of individuals’ latent skills than the traditional paper and pencil (P&P) tests. To take advantage of both CDA and CAT, these two technologies have been combined to form the cognitive diagnostic computerized adaptive testing (CD-CAT). The major concern of the CD-CAT is the measurement accuracy, therefore, a bunch of item selection methods were developed to achieve this goal. Worth noting that these methods mainly focused on maximizing the measurement accuracy without concerned for item exposure control, as a consequence, a highly uneven distribution of item bank usage was raised. To find a balance between measurement accuracy and item exposure control, some new item selection methods were proposed, for instance restrictive progressive (RP), restrictive threshold (RT), stratified multistage (SM), stratified multistage & item eligibility (SMIE), randomization halving algorithm (RHA), and stratified dynamic binary searching (SDBS) methods. However, the model that used in these studies was either the deterministic input, noisy ‘and’ gate (DINA) or the reduced reparameterized unified model (RRUM). Moreover, some of them tended to produce high item exposure rate, and some of them are lack of flexibility, such as the SDBS method. In order to cope with these shortages above mentioned, two new methods are proposed to balance the measurement accuracy and item exposure rate, the new methods combine the RHA with RP and RT methods, and namely, the halving inverse RP (HIRP) and halving inverse RT (HIRT) methods. A Monte Carlo simulation is conducted to examine the performance of HIRP, HIRT and existed item exposure control methods. Four factors are manipulated in the study: model type, number of attributes, test length, and item selection method. Specifically, the model type includes the DINA model and RRUM; four and six number of attributes are used; the test length are 20 and 30, respectively; and 11 item selection methods—— Random, PWKL, RP, RT, SM, IE, SMIE, RHA, SDBS, HIRP, and HIRT—— are adopted in the study. At the same time, the covariates include: the size of the item bank, item parameters of the model, the sample size, and the maximum item exposure rate. The main results show that: (1) The HIRT method produces slightly worse item exposure rate but with significant higher measurement accuracy than RP, SM, SMIE, RHA, and SDBS; meanwhile, HIRT has slightly lower measurement accuracy than PWKL, RT, and IE, but has more even distribution of item exposure. (2) The measurement accuracy of HIRP is lower than PWKL, RP, RT, IE, and HIRT, but the item exposure rate, in general, is better than these methods; in addition, HIRP produces higher measurement accuracy and better item exposure rate than SM, SMIE, and SDBS. In conclusion, both HIRP and HIRT, in some degree, can balance the trade-off between measurement accuracy and item exposure rate.

Key words: cognitive diagnostic computerized adaptive testing, item exposure control, measurement accuracy, item exposure rate