Abstract: Cognitive diagnostic assessment has two phases, like a statistical pattern recognition and classification methodology. The first phase is feature generation, followed by classification stage. Q-matrix corresponds to the feature generation phase in statistical pattern recognition. Feature generation is of paramount importance in any pattern recognition task. Therefore, Q-matrix plays a very important role in establishing a relation between latent attribute patterns and ideal response patterns. In practice, a Q-matrix is difficult to specify correctly in cognitive diagnostic assessment and misspecification of the Q-matrix can seriously affect the accuracy of both item parameter estimates and the classification of examinees. The existing methods including the δ method, the γ method, the Q-matrix refinement method, and maximum likelihood estimation method relies on estimates of examinees’ attribute patterns and its classification accuracy. It is not suitable for the case of a test with a short test length because the short test is seldom used to obtain high classification accuracy. The purpose of this study is to propose a method for Q-matrix specification. We assume that a cognitive requirement for multiple skills within an item is conjunctive - that is, answering the item correctly requires mastery of all the skills required by that item. We consider two expected or ideal response matrices, denoted by ERMQ and ERMR. ERMQ or ERMR can be generated from a reduced Q-matrix or a reachability matrix and the universal set of attribute patterns under the conjunctive assumption. Then any column of the ERMQ can be expressed by the columns of the ERMR under the logical AND operation. This is because the augment algorithm in the generalized Q-matrix theory provides the useful fact that any column of the reduced Q-matrix can be expressed by the columns of the reachability matrix under the logical OR operation. A simulation study was conducted to investigate the performance of the new method under three factors (sample size, item parameters in the reachability matrix, and item parameters for the raw items with unknown q-vector) under the deterministic inputs, noisy “and” gate (DINA) model. Simulation results show that the performance of the new method is promising in terms of correct recovery rates of q-entries and correct classification rates of examinees’ attributes. There listed some major results: (a) the average correct recovery rates of q-entries is above 0.90, when guessing and slipping parameters of items in the reachability matrix and raw items are less than 0.20 and 0.3, respectively, (b)the average difference is very small between the correct classification rates of attributes obtained from the nonparametric classification approach based on the true or simulated Q-matrix and the estimated Q-matrix, (c)for an independent structure with 5 attributes, a relatively small sample size of 120 is required through random sampling that is often easy to attain, (d)the new method only needs subject matter experts to specify a Q-matrix for a part of test items which corresponds to the reachability matrix. One conclusion of this study is that the new method will play a very important role in assisting subject matter experts in Q-matrix specification.

Key words: cognitive diagnostic assessment, the generalized Q-matrix theory, the argument algorithm, Q-matrix specification, the reachability matrix

摘要： Q矩阵标定是实施认知诊断评估的前提，已有Q矩阵修正方法并不太适合测验中已知属性向量的题目数较少的情形。根据拓展Q矩阵理论中可达阵R列与简化Q阵列存在布尔“或”关系，在一定认知假设下，率先提出可达阵R与简化Q阵的潜在反应列存在布尔“与”关系，并由此提出基于可达阵的Q矩阵标定方法。研究显示：在已知一个可达阵下，当可达阵项目的猜测或失误参数在.20以下且待标定项目的项目参数约在.30以下时，新方法所得Q矩阵元素返真率基本在.90以上，并且真实Q矩阵与估计Q矩阵下被试分类准确率差异很小；对于含5个属性的独立结构，新方法要求的随机样本的样本量较小；实证研究也印证了模拟研究的结论。新方法只需专家标定少量题目的Q矩阵，即已经标定的Q矩阵对应属性层级结构的可达阵。

关键词: 认知诊断评估, 拓展Q矩阵理论, 可达阵, 扩张算法, Q矩阵标定方法