关键词: 属性分类一致性信度, 区间估计, Bootstrap法, 认知诊断模型误设

Abstract: In the field of cognitive diagnosis, the research about reliability is a relatively new area. In the field of psychology measurement, model-data fit test of cognitive diagnosis test is an important theme. The present methods about calculating the attribute reliability of cognitive diagnosis are only point estimation, and they can't provide the accuracy of the estimation. Interval estimation not only gives a range of estimation, but also gives the accuracy. Therefore, it is important to report confidence interval of reliability of cognitive diagnosis test. Model-data fit test of cognitive diagnosis test is mainly to study the validity and sensitivity of the statistics. In practice, practitioners don't know the model that really fits the data. When choosing a model, it is necessary to determine which model is more appropriate by using the model-data fit test. This study explored the performance about estimating average of reliability and standard error by bootstrap method in model misspecification. Markon and Chmielewski (2013) studied the relationship between the error designation and the reliability of the general response model of psychology. The study found that using the wrong model in the process of data analysis would reduce the reliability of the test. Then, can we study the overall fitting of model data from other angles? Based on this, The purpose of this study was to study the overall fitness of cognitive diagnosis model data from the perspective of reliability. Using the three cognitive models of DINA, GDINA and RRUM, from the perspective of reliability, we compared the average of attribute-level classification consistency reliability, standard error and other indicators in the Bootstrap interval estimation under different sample sizes, number of questions and number of attributes. The study tested -2LL, AIC, BIC selection rate of the correct model, and their effect in the model misunderstanding. The results indicated that the attribute classification consistency and the standard error performed well in the whole study by bootstrap method estimating. The correct selection rate of bootstrap method was equal to that of AIC and BIC in which DINA generating model was. When generating model was GDINA, correct selection rate of bootstrap method was the same as that of BIC, but it was better than that of AIC. When generating model was RRUM, correct selection rate was better than that of BIC, but it was not as good as that of AIC. The correct selection rate of bootstrap method increased with the increase of the number of subjects. In this study, the attribute classification average reliability and standard error estimated by Bootstrap interval from the perspective of reliability, were used as indicators to investigate their performance in model misunderstanding. In summary, the average and standard error of attribute-level classification consistency were applied in the misunderstanding of cognitive diagnosis model, and it was compared with other commonly used overall model data. Under the conditions, the effects of their misunderstanding in the model were investigated. This research would provide new ideas for model data fitting research.

Key words: attribute-level classification consistency reliability, interval estimation, bootstrap method, cognitive diagnostic model