关键词: CD-CAT, 多级计分, 选题方法

Abstract: The cognitive diagnostic computer adaptive test (CD-CAT) combines the advantages of both cognitive diagnosis and CAT to make adaptive diagnosis of different individuals, providing examinees with a more detailed evaluation of the advantages and disadvantages. Until recently, most research and applications of CD-CAT focused on dichotomous items, and only a few studies investigated CD-CAT with polytomous items, as well as more specific polytomous item selection methods. There are large polytomous item/data in the practical application of education and psychological tests. Both dichotomous and polytomous items are used in many standardized tests. More importantly, polytomous item has more advantages over dichotomous items in some aspects. For example, polytomous items can provide more information about examinee and fewer polytomous items can achieve the same precision compared with dichotomous items, and some features are easier to measure with a rating scale (such as attitude, interest and so on). One of the crucial elements of CD-CAT is the item selection method. By choosing more efficient item selection method, examinees’ ability or cognitive patterns can be better estimated. The main purpose of this study is to develop two new item selection methods for polytomous CD-CAT (PCD-AT), namely maximum expected posterior weighted KL method (MEPWKL), and maximum expected hybrid KL method (MEHKL). Two simulation studies were carried out to compare the MEPWKL and MEHKL item selection algorithm with the current three item selection methods. The pattern correct classification rate (PCCR), attribute correct classification rate (ACCR), and a few descriptive statistics (i.e., minimum, maximum, mean, and variance), of the test lengths were calculated based on the termination rules to compare the efficiency of the item selection methods. The first simulation study used a fixed test length termination rule, taking into account three factors, namely test length (L =5, 10 and 15), item selection methods (KL, PS-PWKL, HKL, MEPWKL and MEHKL), and attribute pattern distribution (uniform vs. higher-order). The results of Study 1 showed that the two new selection methods MEPWKL and MEHKL had similar the pattern correct classification rate (PCCR). However, when the test length was shorter, the PCCR rate of MEHKL was slightly higher than that of MEPWKL. Under the same conditions, the PCCR rates of the two new item selection methods were substantially higher than those of KL, PS-PWKL and PS-HKL. Study 2 seeks to investigate the efficiency of the two proposed algorithms against the existing item selection methods in a variable-length test. In Study 2, the variable length test stopped when the probability of the cognitive pattern with the highest probability reached predetermined values (such as 0.6, 0.7, 0.8, and 0.9). The results of Study 2 showed that, under all conditions, the average test length of the two proposed methods was shorter than that of other item selection methods, that is, the test efficiency of the two new methods was the highest. However, the average test length of MEHKL test was shorter than that of MEPWKL, especially when the attribute profiles were generated from a higher-order distribution. In conclusion, this study proposed two new item selection methods for polytomous CD-CAT. Simulation results showed that the new item selection methods were obviously superior to other traditional methods in terms of classification accuracy and test length.

Key words: CD-CAT, polytomous items, item selection methods