Modeling of the effect on multidimensional latent speeds in the between-item multidimensional response time

GUO Xiao-Jun; LUO Zhao-Sheng; YAN Juan

PDF(948 KB)

Journal of Psychological Science ›› 2022, Vol. 45 ›› Issue (5) : 1222-1229.

Modeling of the effect on multidimensional latent speeds in the between-item multidimensional response time

¹,³, ², ¹,³

Author information +

History +

Abstract

With the popularity of computer-based testings, it is easier to collect item response times (RTs) in psychological and educational assessments. RTs can provide an important source of information for respondents and tests. Accordingly, RTs can help in evaluating the speed of respondents, detecting cheating behaviors and designing better tests. RTs can also be used for improving the accuracy of parameter estimation and others. To full use of RTs, researchers have invested substantial effort in developing statistical models of RTs. Most of the proposed models posit a unidimensional latent speed to account for RTs in tests. However, there are many multidimensional tests in psychological and educational assessments. Based on the assumption that each latent speed should be paired with a specific latent ability in multidimensional tests, a multidimensional lognormal response time model (MLRT) model was proposed with extended the unidimensional lognormal response time model (ULRTM). In multidimensional tests, there are between-item and within-item multidimensionality. There may be effects between different latent speeds in the between-item multidimensionality. MLRTM may not be appropriate for this situation. To capture the effect between different latent speeds, this study proposed higher-order lognormal response time model (HO-LRTM) and bifactor lognormal response time model (Bi-LRTM) based on the corresponding response model. Model parameters in the HO-LRTM and Bi-LRTM can be estimated via maximum likelihood estimation in Mplus. In the simulation study, the results showed that the parameters of HO-LRTM and Bi-LRTM can be accurately estimated. In empirical data, three sets (A, C and D) were chose from the Raven’s Standard Progressive Matrices. Each set has 12 items. Firstly, RTs were explored the structure of latent speed by the empirical kaiser criterion (EKC) and exploratory factor analysis (EFA). The results of the EKC and EFA indicated that the latent structure of RTs is a three-dimensional structure. Secondly, according to the different fit indices, the Bi-LRTM fits better than other models. Furthermore, it is necessary to free speed-slope parameters in the response time models by comparing the fitting effect of fixed and free the parameters. Finally, this study assessed unidimensionality of Bi-LRTM based on some statistical indices. These statistical indices showed the necessity of general and specific latent speed in the Bi-LRTM. Overall, the proposed Bi-LRTM works well in simulation study and empirical data. That is, considering the effect on multidimensional latent speeds meets the need for analyzing the between-item multidimensional response time.

Key words

response time / the between-item multidimensionality / higher-order lognormal response time model / bifactor lognormal response time model

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

Modeling of the effect on multidimensional latent speeds in the between-item multidimensional response time[J]. Journal of Psychological Science. 2022, 45(5): 1222-1229

References

陈飞鹏, 詹沛达, 王立君, 陈春晓, 蔡毛. (2015). 高阶项目反应模型的发展与应用. 心理科学进展, 23(1), 150–157. 郭磊, 尚鹏丽, 夏凌翔. (2017). 心理与教育测验中反应时模型应用的优势与举例. 心理科学进展, 25(4), 701–712. 顾红磊, 温忠麟. (2017). 多维测验分数的报告与解释: 基于双因子模型的视角. 心理发展与教育, 33(4), 504–512. 侯杰泰, 温忠麟, 成子娟. (2004). 结构方程模型及其应用. 北京: 教育科学出版社. 毛秀珍, 夏梦连, 辛涛. (2018). 全信息项目双因子分析：模型、参数估计及其应用. 心理科学进展, 26(2), 358–367. 孟祥斌. (2016). 项目反应时间的对数偏正态模型. 心理科学, 39(3), 727–734. 温忠麟, 汤丹丹, 顾红磊. (2019). 预测视角下双因子模型与高阶因子模型的一般性模拟比较. 心理学报, 51(3), 383–391. 徐霜雪, 俞宗火, 李月梅. (2017). 预测视角下双因子模型与高阶模型的模拟比较. 心理学报, 49(8), 1125–1136. 詹沛达, Hong Jiao, Kaiwen Man. (2020). 多维对数正态作答时间模型：对潜在加工速度多维性的探究.心理学报, 52(9), 1132–1142. 张厚粲, 王晓平. (1989). 瑞文标准推理测验在我国的修订. 心理学报, 2, 113–121. Adams, R. J., Wilson, M., & Wang, W. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21(1), 1–23. Bolsinova, M., & Tijmstra, J. (2018). Improving precision of ability estimation: Getting more from response times. British Journal of Mathematical and Statistical Psychology, 71(1), 13–38. Bonifay, W. E., Reise, S. P., Scheines, R. S., & Meijer, R. R. (2015). When Are Multidimensional Data Unidimensional Enough for Structural Equation Modeling? An Evaluation of the DETECT Multidimensionality Index. Structural Equation Modeling: A Multidisciplinary Journal, 22(4), 1–13. Braeken, J., & van Assen, M. A. L. M. (2017). An empirical Kaiser criterion. Psychological Methods, 22(3), 450–466. Cai, L., Yang, S., & Hansen, M. (2011). Generalized Full-Information Item Bifactor Analysis．Psychological Methods, 16(3), 221–248． de la Torre, J., & Song, H. (2009). Simultaneous estimation of overall and domain abilities: A higher-order IRT model approach. Applied Psychological Measurement, 33(8), 620–639. Fox, J-P., & Marianti, S. (2017). Person-Fit Statistics for joint models for accuracy and speed. Journal of Educational Measurement, 54(2), 243–262. Huang, H. Y., Wang, W. C., Chen, P. H., & Su, C. M. (2013). Higher-order item response models for hierarchical latent traits. Applied Psychological Measurement, 37(8), 619–637. Klein Entink, R. H., van der Linden, W. J., & Fox, J. P. (2009). A Box-Cox normal model for response times. British Journal of Mathematical and Statistical Psychology, 62(3), 621–640. Muthén, L. K., & Muthén, B. (2019). Mplus. The comprehensive modelling program for applied researchers: user’s guide, 5. Ranger, J., & Kuhn. J. T. (2012). A flexible latent trait model for response times in tests. Psychometrika, 77(1), 31–47. Ranger, J., & Kuhn, J-T. (2015). A mixture proportional hazards model with random effects for response times in tests. Educational and Psychological Measurement, 76(4), 1–25. Reise, S. P. (2012 ). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696． Reise, S. P., Moore, T. M., & Haviland, M. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92(6), 544–559. Reise, S. P., Scheines, R., Widaman, K., & Haviland, M. G. (2013). Multidimensionality and Structural Coefficient Bias in Structural Equation Modeling A Bifactor Perspective. Educational and Psychological Measurement, 73(1), 5–26. Stucky, B. D., & Edelen, M. O. (2015). Using hierarchical IRT models to create unidimensional measures from multidimensional data. In S. P. Reise & D. A. Revicki (Eds.), Handbook of item response theory modeling: Applications to typical performance assessment (pp.183-206). New York: Routledge. van der Linden, W. J. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31(2), 181–204. van der Linden, W. J., & Xin hui Xiong. (2013). Speededness and Adaptive Testing. Journal of Educational and Behavioral Statistics, 38(4), 418–438.