Simulation study of the detection of measurement invariance and its influential factors for ordinal categorical data

Abstract

Abstract:

The study introduces a new method of model construction and parameters estimation for ordinal categorical data，with the method of exploring measurement invariance based on this method. simulation study explores the effect of parameters recovery, difference between estimated value and known population parameters value ,empirical power under various total sample size, the ratio of reference group to focal group size, the difference of parameters between two groups, test length. Results indicate that the parameters recovery well when robust weighted least squares estimator using a diagonal weight matrix(WLSMV) are used. The type Ⅰerror of DIFFTEST consistently adhered closely to the nominal alpha level of 0.05.Under the condition of large sample size, reference and focal group size are equal, the difference of parameters is bigger, the power of DIFFTEST show better. Control the number of items whose measurement invariance are violated, the power of DIFFTEST become lower when the test length become longer.

Key words: Ordinal Data, DIFFTEST, Measurement Invariance, Simulation Study

摘要：

研究介绍了针对等级数据的模型建构（LRV，潜在反应变量模型）和参数估计（WLSMV）方法，以及在此基础上的测量不变性检验（DIFFTEST）方法，同时采用蒙特卡洛模拟研究方法，考察样本总量大小、组间样本量对比情况、阈值差异程度、量表长度等因素，对DIFTEST进行针对等级数据的测量不变性检验效果的影响情况，以及WLSMV估计方法下的参数复原情况。研究结果发现WLSMV估计方法参数的复原效果很好；DIFFTEST的一类错误概率达到可接受水平，在大样本情况下、组间样本量基本相等、阈值差异程度较大时，DIFFTEST检测力较好。在控制测量不变性遭受破坏的项目个数情况下，随着测验长度的增加，DIFFTEST的检测力下降。

关键词: 等级数据, DIFFTEST, 测量不变性, 模拟研究

Chong LI. Simulation study of the detection of measurement invariance and its influential factors for ordinal categorical data[J]. Psychological Science, 2011, 34(6): 1482-1487.

李冲刘红云. 等级数据的测量不变性检验及影响因素模拟研究[J]. 心理科学, 2011, 34(6): 1482-1487.

References

白新文，陈毅文.(2004). 测量等价性的概念及其判定条件. 心理科学进展,12(2): 231–239. 蔡华俭，林永佳，伍秋萍，严乐，黄玄凤.(2008). 网络测验和和纸笔测验的测量不变性研究—以生活满意度量表为例. 心理学报, 40 ( 2) :228–239. Asparouhov,T., &Muthen,B.(2006). Robust Chi-Square Difference Testing with Mean and Variance Adjusted Test Statistics. Mplus Web Notes: No. 10 http://www.statmodel.com/examples/webnote.shtml Babakus, E.CE Ferguson Jr.(1987).The sensitivity of confirmatory maximum likelihood factor analysis to violations of measurement scale and distributional assumptions. Journal of Marketing Research, Structural Equation Modeling: A Multidisciplinary Journal,3,105-175. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley. Byrne B M, Shavelson R J, &Muthen B（1989）. Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychological Bulletin,105: 456~466 Dolan, &Conor V. (1994).Factor analysis of variables with 2, 3, 5 and 7 response categories: A comparison of categorical variable estimators using simulated data. British Journal of Mathematical and Statistical Psychology, 47, 309-326. Drasgow, F.（1984）. Scrutinizing psychological tests: measurement equivalence and equivalent relations with external variables are the central issues. Psychological Bulletin, 95: 134~135 Flora,D.B., &Curran,P.J.(2004).An Empirical Evaluation of Alternative Methods of Estimation for Confirmatory Factor Analysis With Ordinal Data. Psychological Methods,4, 466–491 Green,S.B.,Akey,T.M.,Fleming,K.K., Hershberger, S.L.& ; Marquis, J. G. (1997). Effect of number of scale points on chi-square fit indices in confirmatory factor analysis. Structural Equation Modeling: A Multidisciplinary Journal,4,108-120. Hutchinson, SR. Olmos, A.(1998). Behavior of descriptive fit indexes in confirmatory factor analysis using ordered categorical data. Structural Equation Modeling: A Multidisciplinary Journal,5,344-364. Horn, J. L., & McArdle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18, 117-144. J?reskog,K.&S?rbom,D.(1996).PRELIS2:User’s reference guide. Chicago: scientific software international. Kaplan, D.(2000).structural equation model: foundation and extensions. Thousand Oaks, CA: sage. Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49, 115-132 Muthén, B. (1993). Goodness of fit with categorical and other non-normal variables. In K. A. Bollen, & J. S. Long (Eds.), Testing Structural Equation Models (pp. 205-243). Newbury Park, CA: Sage. Muthén, B., & Kaplan D. (1985). A comparison of some methodologies for the factor analysis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38, 171-189. Muthén, B., du Toit, S.H.C., & Spisic, D. (1997). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. http://gseis.ucla.edu/faculty/muthen/full_paper_list.htm 75） Muthén, B., &Muthén, T.(2007). Mplus User Guide (version 5). Reise, S. P.,Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin,114, 552-566. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton-Mifflin. Vandenberg, R. J.,&Lance, C. E. (2000).A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69.

[1]	Qiuheng Shi. Investigating the Acceptable Noninvariance Rate of Alignment – A Monte Carlo Simulation Study [J]. Journal of Psychological Science, 2022, 45(5): 1230-1242.
[2]	Xian-Liang ZHENG Gu Haigen Zhao Bi-Hua. Testing Measurement Invariance of Second-Order Factor Models: taking Internet Altruistic Behavior Scale of Undergraduates as an example [J]. Psychological Science, 2011, 34(5): 1195-1200.