Abstract
Longitudinal cognitive diagnostics can assess students’ strengths and weaknesses over time, profile students’ developmental trajectories, and can be used to evaluate the effectiveness of teaching methods and optimize the teaching process. Researchers have proposed different longitudinal diagnostic classification models, which provide methodological support for the analysis of longitudinal cognitive diagnostic data. Although these parametric longitudinal cognitive diagnostic models can effectively assess students’ growth trajectories, their requirements for coding ability and sample size hinder their application among frontline educators, and they are time-consuming and not conducive to providing timely feedback. On the one hand, the nonparametric approach is easy to calculate, efficient to apply, and provides timely feedback; on the other hand, it is free from the dependence on sample size and is particularly suitable for analyzing assessment data at the classroom or school level. Therefore, this study attempts to apply nonparametric method to longitudinal cognitive diagnostic assessments for tracking student’s learning trajectories.
This study extended the longitudinal Hamming distance discriminant (Long-HDD) based on the Hamming distance discriminant (HDD), which uses the Hamming distance to represent the dependence between attribute mastery patterns of the same student at adjacent time points. To explore the performance of Long-HDD in longitudinal cognitive diagnostic data, we conducted three simulation studies and an empirical study and compared the classification accuracy of the HDD, Long-HDD, and Long-DINA models. The purpose of simulation study 1 was to compare the performance of performance of three methods under different simulation conditions. Simulation study 2 focused on the classification accuracy of the three methods at moderate attributes transfer probability level (p(0→1)=0.5, p(1→0)=0.05) and high attributes transfer probability level (p(0→1)=0.8, p(1→0)=0.05). To further highlight the advantages of the Long-HDD in small-scale assessments, the Long-DINA model was used as the data generation model in Study 3. At this point, if Long-HDD still outperforms or does not lose out to Long-DINA model's, the relative advantage of using Long-HDD in a small-scale assessments can be further highlighted. Furthermore, an empirical study was conducted to illustrate the application of the Long-HDD.
Under the comparison of the three methods, the results of the simulation studies showed that (1) Long-HDD had higher classification accuracy in longitudinal diagnostic data analysis; (2) Long-HDD performed almost independently of sample size and performed better with a smaller sample size compared to Long-DINA; and (3) Long-HDD consumed much less computational time than Long-DINA. In addition, the results of the empirical study showed that there was good consistency between the results of the Long-HDD and the Long-DINA model?in tracking changes in attribute development. The percentage of mastery of each attribute increased with the increase of time points.
In summary, the long-HDD proposed in this study extends the application of nonparametric methods to longitudinal cognitive diagnostic data and can provide high classification accuracy. Compared with parameterized longitudinal DCM, it can provide timely diagnostic feedback due to the fact that it is not affected by sample size, simple calculation, and less time-consuming. It is more suitable for small-scale longitudinal assessments such as class and school level.
Key words
cognitive diagnosis /
nonparametric classification /
longitudinal data analysis /
Hamming distance
Cite this article
Download Citations
Liu Yaohui, Chen Qipeng, Xu Huiying, Zhan Peida,.
Longitudinal Hamming Distance Discrimination: Developmental Tracking of Latent Attributes[J]. Journal of Psychological Science. 2023, 46(3): 742-751
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}