Abstract: In the field of cognitive diagnosis, the research about reliability is a relatively new area. In the field of psychology measurement, model-data fit test of cognitive diagnosis test is an important theme. The present methods about calculating the attribute reliability of cognitive diagnosis are only point estimation, and they can't provide the accuracy of the estimation. Interval estimation not only gives a range of estimation, but also gives the accuracy. Therefore, it is important to report confidence interval of reliability of cognitive diagnosis test. Model-data fit test of cognitive diagnosis test is mainly to study the validity and sensitivity of the statistics. In practice, practitioners don't know the model that really fits the data. When choosing a model, it is necessary to determine which model is more appropriate by using the model-data fit test. This study explored the performance about estimating average of reliability and standard error by bootstrap method in model misspecification. Markon and Chmielewski (2013) studied the relationship between the error designation and the reliability of the general response model of psychology. The study found that using the wrong model in the process of data analysis would reduce the reliability of the test. Then, can we study the overall fitting of model data from other angles? Based on this, The purpose of this study was to study the overall fitness of cognitive diagnosis model data from the perspective of reliability. Using the three cognitive models of DINA, GDINA and RRUM, from the perspective of reliability, we compared the average of attribute-level classification consistency reliability, standard error and other indicators in the Bootstrap interval estimation under different sample sizes, number of questions and number of attributes. The study tested -2LL, AIC, BIC selection rate of the correct model, and their effect in the model misunderstanding. The results indicated that the attribute classification consistency and the standard error performed well in the whole study by bootstrap method estimating. The correct selection rate of bootstrap method was equal to that of AIC and BIC in which DINA generating model was. When generating model was GDINA, correct selection rate of bootstrap method was the same as that of BIC, but it was better than that of AIC. When generating model was RRUM, correct selection rate was better than that of BIC, but it was not as good as that of AIC. The correct selection rate of bootstrap method increased with the increase of the number of subjects. In this study, the attribute classification average reliability and standard error estimated by Bootstrap interval from the perspective of reliability, were used as indicators to investigate their performance in model misunderstanding. In summary, the average and standard error of attribute-level classification consistency were applied in the misunderstanding of cognitive diagnosis model, and it was compared with other commonly used overall model data. Under the conditions, the effects of their misunderstanding in the model were investigated. This research would provide new ideas for model data fitting research.

Key words: attribute-level classification consistency reliability, interval estimation, bootstrap method, cognitive diagnostic model

摘要： 认知诊断模型选择是认知诊断评估中重要研究问题之一。在实际应用中实践者并不知道真正拟合数据的模型，通常会用模型拟合指标检验模型与数据的拟合程度。从测量结果质量来看，除保证模型与数据拟合之外，还需要重点评价模型诊断结果的信度和效度等。考虑到以往研究大都采用基于信息量的拟合指标去判定模型与数据的匹配性，本研究提出综合考虑模型拟合指标与信度指标用于模型选择或评价模型误设。考虑实验因素为真实模型或分析模型（DINA模型、G-DINA模型、R-RUM模型）、样本量、题量和属性个数，在五因素（3×3×2×2×2）实验设计条件下，比较Bootstrap区间估计的属性分类一致性信度平均数与标准误和常用的拟合统计量-2LL、AIC、BIC对正确模型的选择率。结果表明：-2LL在题目数量多的情况下表现较好，而AIC、BIC在被试量较大的情况下表现较好，在不同的研究条件下，-2LL、AIC、BIC的模型选择率很不稳定，而用Bootstrap法估计的属性分类一致性信度平均数和标准误在不同研究条件的模型选择率较稳定，总体表现较好。

关键词: 属性分类一致性信度, 区间估计, Bootstrap法, 认知诊断模型误设