关键词: 关键词：属性分类一致性信度, 区间估计, Bootstrap法, 平行测验法, 平行测验配对法

Abstract: Abstract: As is well known, point estimate contains limited information about a population parameter and could not give how far it could be from the population parameter. The confidence interval of the parameter could provide more information about the precision of estimated parameters. In evaluating the quality of a test, the confidence interval of composite reliability has received more and more attention in recent years. However, there is no research on the estimation of the reliability interval of the cognitive diagnostic assessment. The researchers often only report the reliability point estimation, and no one is concerned about the confidence interval in the report. It is far from enough to report the reliability of the cognitive diagnostic assessment. The confidence interval of reliability in cognitive diagnostic assessment should be taken into account. We can estimate error range from the confidence interval. This study compares three reliability interval estimation methods of the cognitive diagnostic assessment based on DINA model and attribute-level classification consistency (Wang, et al., 2015). There are three approaches to estimate the confidence interval of the cognitive diagnostic assessment: Bootstrap method, Parallel test method, and Parallel test pairing method. Each of the three approaches produces the standard error, average and confidence interval about attribute-level classification consistency reliability. These factors were considered in the simulation design: (a) the number of test attributes(k=5, and 7); (b) the number of test items (5 attributes is t=15, and 31; 7 attributes is t=21, and 42); (c) the quality of test items [U(0.05,0.25), and U(0.25,0.45)]; (d) the number of sample size (n=500, 1000 and 1500); (e)the method for calculating the confidence intervals of attribute-level classification consistency reliability (Bootstrap, parallel test method, and parallel test pairing method). Totally, 72 treatment conditions were generated in terms of the above 5-factor simulation design (i.e., 72=2×3×2×2×3). The simulation results indicated: (1) Whether tests contain 5 or 7 attributes of the independent attribute hierarchical relationship, the standard error and reliability interval which are obtained by three interval estimation methods estimated (Bootstrap method, parallel test method and parallel test matching method) are all affected by the quality of test, sample size or test length. With the increase of the number of test items and subjects, the standard error and the length of confidence interval tend to decrease. As the quality of the subject decreases, the standard error and the length of confidence interval increase; (2) Whether tests contain 5 or 7 attributes of the independent attribute hierarchical relationship, the average of the reliability estimating by three interval estimation methods is affected by the quality of test, sample size or test length. With the increase of the amount of test, the average of the reliability shows a larger increase. With the increase of the number of subjects, the change of the average of the reliability is small; as the quality of the subject declines, the average of the reliability shows a larger decline; (3) The parallel test method and the Bootstrap method are close to the standard errors and confidence intervals estimated when test length and sample size is small. However, with the increase of the number of the subjects, the estimation accuracy of the Bootstrap method is improved rapidly. When a large amount of test items and the number of subjects was large, the result was basically close to the parallel test matching method. Bootstrap method requires the least time, parallel test matching method in practice is difficult to achieve. Therefore, recommended Bootstrap method when estimate the confidence interval of cognitive diagnosis.

Key words: Key words: attribute-level classification consistency, interval estimation, bootstrap method, parallel test method, parallel test pairing method