Abstract
Because few sampling distributions of mediating effect are normally distributed, in recent years, some asymmetric interval methods such as parametric residual bootstrap, Monte Carlo methods, and Bayesian methods have been developed and proposed for analyzing multilevel mediation. These approaches do not impose the assumption of normality of the sampling distribution of mediating effects. However, little is known about how these methods perform relative to each other.
This study conducts a simulation using R software. This simulation examines several approaches for testing 2-1-1 multilevel mediation with fixed slope. Four factors were considered in the simulation design: (a) sample size of level two ( =10, 20, 30, 50, 100); (b) sample size of level one ( =10, 20); (c) parameter combinations (a=b=0, a=.39 and b=0, a=0 and b=.59, a=b=.14, .39, .59); (d) method for testing multilevel mediation (Monte Carlo method, parametric percentile residual Bootstrap method, bias-corrected parametric percentile residual Bootstrap method, Bayesian method with informative prior and Bayesian method with non-informative prior). A total of 60 treatment conditions were designed in the 4-factor simulation. 500 replications were generated for each treatment condition. For the Bootstrap method, 1,000 bootstrap samples were drawn in each replication. For the Monte Carlo method, 5,000 samples were drawn in each parameter with normal distribution. For the Bayesian methods, 11,000 Gibbs iteration were implemented in each replication, 10,000 posterior samples of the model parameters were recorded after 1,000 burn-in iterations. The methods were compared in terms of (a) Relative mean square error, (b) TypeⅠerror rate, (c) Power, (d) Interval width, (e) Interval imbalance.
The simulation study found the following results: 1) the performance of Bayesian method with informative prior were superior to that of the other methods in terms of Relative mean square error. 2) The Power of the Bayesian method with informative prior was the highest among all the methods. However, extra power comes at the cost of underestimation of Type I error. Power of bias-corrected parametric percentile residual Bootstrap method was the second greatest, with elevated Type I error in some conditions. 3) The performance of Monte Carlo method was superior to that of the other methods for Type I error. 4) Interval width of Bayesian method with informative prior is the smallest among different methods. Interval width of Monte Carlo method was the second smallest. 5) Interval imbalance of Bayesian method with informative prior is smallest among different methods.
The simulation results indicated that 1) when informative prior was available, Bayesian method was recommended to analyze mediation. 2) If informative prior was not available, Monte Carlo method should be adopted to analyze mediation.
Key words
multilevel mediation /
Bayesian method /
Monte Carlo method /
parametric bootstrap method /
prior information
Cite this article
Download Citations
Fang Jie Zhong-Lin WEN.
A Comparison of Three Methods for testing Multilevel Mediation[J]. Journal of Psychological Science. 2018, 41(4): 962-967
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}