Abstract
CD(Cognitive Diagnosis) can provide personalized diagnostic results for subjects. The selection strategy of CD and CD-CAT (Cognitive Diagnostic Computerized Adaptive Testing) is also a test design in essence, which can provide references for the design of cognitive diagnostic tests. Some studies believe that a high accuracy rate can be obtained only when each attribute is fully measured. Therefore, in CD-CAT, the Attribute Balance(AB) strategy was proposed. Combined with the attribute balance index, MMGDI (Modified Maximum Global Discrimination Index) had a high attribute and pattern accuracy rate. However, when all attributes in projects were constrained by the hierarchical relationship, and the attribute balance strategy was difficult to implement. Therefore, it is more reasonable to consider the item attribute vector balance strategy.
It is called the Item Attribute Vector Balance (IAVB) matrix if the attribute hierarchy relationship contained in the test Q matrix which is consistent with the theoretical attribute and hierarchy relationship, and the test Q matrix is composed of different item attribute vectors with the same number of uses.IAVBwas the extension of the AB. In this paper, the study was conducted in two sub-studies. The comparison of IAVB and ABunder the conditions of 3 and 5 attributes with independent structure and 5 attributes with unstructured. The total number of participants in each study was 30 times of theknowledge state categories. The error and guessing parameters of DINA model were fixed at 0.05. Based on the expected response mode, for each score x, a random number r which follows the U (0,1) distribution was generated. If r>0.05, the score was still x; otherwise, the score was 1-x..
The following conclusions could be obtained through the simulation studys : (1) when the numbers of item attributes examined were approximately the same, PMR(Pattern Match Ratio)and other indicators of theIAVB tests were higher than others. Compared with other tests, PMR of the IAVB tests in Study 1 increased by more than 17%. PMR of the IAVB tests in study 2 increased by more than 15%. (2) When the numbers of attributes were different between items, the less the number of attributes was, the higher PMR and other indicators were; The more attributes the item examined, the less information it provided; (3) With independent structure, when the numbers of attributes examied in each item were the same and the times of each attribute examied in a test were the same, each index of the IAVB tests were higher than the AB tests, and the indicators of the AB testswere higher than other tests. For example, in Study 1, compared with the AB tests, PMR of the IAVB test Q1 increased by 14.70% and PMR of Q4 increased by 23.54%. Compared with other tests , PMR of the AB tests increased by 3.98%. In the second study, compared with the AB test Q51, PMR of the IAVB test Q5 increasedby 24.08%, and PMR of Q51 was higher than other tests of Q52 and Q53 30.19%.
In conclusion, IAVB strategy, which considers both the attribute hierarchy and the sufficient measurement of each attribute, is the extension of AB strategy. In the design of cognitive diagnostic tests, more items should be selected which have fewer attributes and meet the condition of IAVB. This strategy can be applied not only to small tests, but also to large tests (such as PISA). The design of large test can include as many items as possible with fewer attributes (reusable), and satisfy the item attribute vector balance.
Key words
Cognitive diagnostic test design /
IAVB /
AB /
PMR
Cite this article
Download Citations
Xiao-Juan TANG Shu-Liang DING Zonghuo YU.
Item Attribute Vector Balance Strategy in Cognitive Diagnostic Test Design[J]. Journal of Psychological Science. 2022, 45(6): 1466-1474
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}