Investigating the Acceptable Noninvariance Rate of Alignment – A Monte Carlo Simulation Study

Qiuheng Shi

Journal of Psychological Science ›› 2022, Vol. 45 ›› Issue (5) : 1230-1242.

PDF(2746 KB)
PDF(2746 KB)
Journal of Psychological Science ›› 2022, Vol. 45 ›› Issue (5) : 1230-1242.

Investigating the Acceptable Noninvariance Rate of Alignment – A Monte Carlo Simulation Study

  • 1,Qiuheng Shi2
Author information +
History +

Abstract

Traditional multiple-group confirmatory factor analysis (multiple-group CFA) is usually criticized for having a too restrictive model assumption, namely the exact scalar measurement invariance. A new multiple-group analysis methodology, alignment, evaluates approximate measurement invariance of the model parameters and, more importantly, permits factor mean comparisons without imposing scalar invariance which is usually required in multiple-group CFA. Previous simulation studies – of Asparouhov and Muthén, as well as of Flake and McCoach – chose specific noninvariance rates only to justify the performance of alignment under certain specific conditions. In contrast, this current simulation study aimed to investigate the noninvariance rate ranges of alignment more broadly in both one-factor and three-factor models. By gradually increasing or decreasing the noninvariant model parameters, the results show that the acceptable noninvariance rate of alignment in one-factor models can attain 100% when the average group size is large enough and the identification option is fixed alignment. Meanwhile, in three-factor models, the acceptable noninvariance rate of alignment ranges from 20% to 30%. Alignment is able to obtain accurate parameter estimates when the magnitude of noninvariance is large and when there are three groups involved with either 10% noninvariant intercepts + 20% noninvariant factor loadings, or 17% noninvariant intercepts + 10% noninvariant factor loadings. However, when there are 20% noninvariant intercepts + 10% noninvariant factor loadings, the results fail to meet the four standards of accurate parameter estimation as proposed in this study. Therefore, in this condition, acceptable noninvariance rate of alignment ranges from 27% to 30% (i.e., 17% noninvariant intercepts + 10% noninvariant factor loadings to 10% noninvariant intercepts + 20% noninvariant factor loadings). Using the same procedure, we found that alignment obtains accurate parameter estimates when the magnitude of noninvariance is large and the amount of groups is nine, with 17% noninvariant intercepts + 10% noninvariant factor loadings. Thus, in this condition, the acceptable noninvariance rate is below 27% (i.e., 17% noninvariant intercepts + 10% noninvariant factor loadings). When the magnitude of noninvariance is large and the amount of groups is 15, using the same previous procedures, we found that the acceptable noninvariance rate is below 27% (17% noninvariant intercepts + 10% noninvariant factor loadings). When the magnitude of noninvariance is large and the amount of groups is 30, the acceptable noninvariance rate is below 20% (10% noninvariant intercepts + 10% noninvariant factor loadings). In three-factor models, when the magnitude of noninvariance is small and there are three groups, by once again using the same procedures, we concluded that the acceptable noninvariance rate is below 23% (13% noninvariant intercepts + 10% noninvariant factor loadings). When the magnitude of noninvariance is small and there are nine groups, we conclude that the acceptable noninvariance rate is below 20% (10% noninvariant intercepts + 10% noninvariant factor loadings). When the magnitude of noninvariance is small and the amount of groups is 15, the acceptable noninvariance rate is also below 20% (10% noninvariant intercepts + 10% noninvariant factor loadings). When the magnitude of noninvariance is small and there are 30 groups, the acceptable noninvariance rate is below 20% (10% noninvariant intercepts + 10% noninvariant factor loadings). In summary, within the noninvariance rate ranges of 20% to 30% in alignment three-factor models, the noninvariant parameters with a large magnitude of noninvariance demonstrate a clearer pattern of noninvariance and are more easily identified, signifying that alignment allows for more large noninvariant parameters. When the amount of groups decreases from 30, 15 or nine to three, alignment functions with more noninvariant parameters. Previous simulation studies on alignment have not investigated noninvariance rate ranges under different simulation conditions, instead only considering certain specific noninvariance rates. This current study, then, adds to the literature by investigating a broader range of noninvariance rate ranges.

Key words

Monte Carlo simulation study / alignment / measurement invariance / noninvariance rate

Cite this article

Download Citations
Qiuheng Shi. Investigating the Acceptable Noninvariance Rate of Alignment – A Monte Carlo Simulation Study[J]. Journal of Psychological Science. 2022, 45(5): 1230-1242
PDF(2746 KB)

Accesses

Citation

Detail

Sections
Recommended

/