Abstract: In the studies of psychology, education and management, we often face data with a two-level structure. For example, students are nested within schools, and employees are nested within enterprises. In such two-level studies, subjects (e. g., students, employees) do not perform independently. Subjects within the same group are usually correlated each other. The independence assumption on individuals in such two-level studies is usually not true. Estimating test reliability is an important step in data analysis. If test reliability is overestimated, the statistical results based on the test are misleading. Reliability is not an intrinsic property of a test, rather, it varies depending on the population in which it is used. Previous research showed that test reliability would be overestimated if the nested relationship was not considered. Hence, test reliability estimation methods that were proposed under the frame of single-level designs are not appropriate for the two-level designs. Raykov and du Toit (2005) deduced a formula for estimating reliability of unidimensional test in the two-level designs based on a two-level confirmatory factor analysis model in which factor loadings of the beween-group part were constrained to be equal to their counterparts of the within-group part. Their formula is only suitable for a rather special situation when the above constraints are correct for the model. Morerover, their method is difficult to be understood, and their program is complicated to be imitated. Till now, most empirical researchers still estimate test reliability as in a single-level design even if the study is a two-level design. So it is necessary to study how to estimate reliability in two-level design and propose a simpler program for computation. We deduced a new formula to estimate the test reliability of the unidimensional test in two-level designs based on two-level confirmatory factor analysis. Whether factor loadings of the beween-group part are fixed or not, the formula is appropriate to estimate the test reliability. An example was illustrated how to estimate test reliability in two-level designs by using our proposed formula with a simple Mplus program. For the purpose of comparison, we also estimated test reliability by using Raykov and du Toit’s (2005) method and simply calculate the composite reliability by treating the data set as if it came from a single-level design. Both of the latter results overestimated the test reliability. Therefore, our new method was recommended. Before we use the new method to compute the reliability of unidimensional test in two-level design, we should analyze a two-level confirmatory factor analysis model and test the goodness of fit. The estimated test reliability is meaningful only when the model is accepted.

Key words: two-level study, unidimensional test, reliability, confirmatory factor analysis

摘要： 在心理、教育和管理等研究领域中，经常会碰到两水平（两层）的数据结构，如学生嵌套在班级中，员工嵌套在企业中。在两水平研究中，被试通常不是独立的，如果直接用单水平信度公式进行估计，会高估测验信度。文献上已有研究讨论如何更准确地估计两水平研究中单维测验的信度。本研究指出了现有的估计公式的不足之处，用两水平验证性因子分析推导出一个新的信度公式，举例演示如何计算，并给出简单的计算程序。

关键词: 两水平研究, 单维测验, 信度, 验证性因子分析