Journal of Psychological Science ›› 2021, Vol. 44 ›› Issue (2): 440-448.

Previous Articles     Next Articles

Item Selection Strategy Based on Gini index for Cognitive Diagnostic Computerized Adaptive Testing

  

  • Received:2019-02-24 Revised:2020-04-25 Online:2021-03-20 Published:2021-03-20
  • Contact: Tu Dong-Bo

基于Gini指数的认知诊断计算机化自适应选题策略

罗芬,王晓庆,蔡艳,涂冬波   

  1. 江西师范大学
  • 通讯作者: 涂冬波

Abstract: Gini index which describes the impurity of data has been widely used in decision tree algorithms. In this paper, A new selection strategy based on Gini index for cognitive diagnositic computerized adaptive testing is developed, it is expected to ensure high test efficiency and uniformity of item bank usage. Two simulation studies aimed to investigate the efficiency of the Gini compared with SHE, PWKL, MPWKL, GDI, KL, Random select stratgies considering a variety of factors, namely, cognitive diagnostic model, and test termination rule. The pattern measurement rates(PMR), , test overlap rate(TOE), test average comsumed time(TC), test average length(TL) and its standard deviation were calculated based on the termination rules to compare the efficiency of the item selection indices. In the first simulation search, the indexes were computed using fixed-test lengths 20 and variable length test as a stopping rule for GDINA, DINA, DINO, RRUM, ACDM, LLM models on five attributes. In the second simulation search, the indexes were computed as same as the first one but only for GDINA on eight attributes. Some conclusions are concluded. (1) The four selection strategies of Gini, SHE, MPWKL and GDI have high measurement accuracy and have a little change under different CDM, so they are not sensitive to the cognitive diagnostic model and can be applied to the item banks of different mixed CDM in the actual test.(2) Compared with the PMR of SHE, MPWKL and GDI, those of Gini has little difference, but the uniformity of item bank usage of Gini is better than the three of them. Overall, Gini is more conducive to both measurement accuracy and the uniformity of item bank usage. (3) PWKL under different CDM, the fluctuation range of PMR is big, PMR of the PWKL with DINA model is as high as PMR of the Gini, SHE, MPWKL and GDI strategies, but under other models, PMR of the PWKL dropped about 5%, therefore, in practice ,adopting PWKL strategies should take the test of Goodness for Fit, besides DINA model, it is not recommended PWKL strategy. However, the utilization index of PWKL strategy in item bank is the best among the five strategies. With the DINA model, the index of Gini and PWKL strategy in item bank are basically the same, but the selection speed of Gini in item bank is about 37% faster than that of PWKL strategy. Therefore, if the actual data conforms to the DINA model, it is suggested to use Gini strategy instead of PWKL strategy.(5) although the measurement accuracy of MPWKL strategy is very high, it takes too much time to select next item, and the usage of item bank is the most uneven. Therefore, it is not recommended to use in the actual test.(6) when the number of attributes increases to 8, the measurement accuracy of each selection strategy declines rapidly, especially PWKL, test length is up to 30, PMR is less than 80%. Therefore, in CD-CAT, it is not recommended to measure too many attributes, but generally recommended about 6 attributes.

Key words: CD-CAT, item selection strategy, Gini index, classification accuracy, uniformity of item bank usage

摘要: Gini指数可用来描述分布的不均匀性,已广泛应用于决策树算法,本文开发了基于Gini指数的认知诊断计算机化自适应选题策略,并在饱和模型和缩减模型下与SHE, MPWKL,GDI,PWKL选题策略进行比较。模拟研究表明,基于Gini指数的选题策略与SHE,MPWKL,GDI相比,分类精度相近并提高了题库的利用率;与PWKL相比,提高了分类的精度和选题速度,综合来看,基于Gini指数的选题策略能够兼顾分类精度和题库使用均匀性。

关键词: CD-CAT, 选题策略, Gini指数, 分类精度, 题库使用均匀性