The validity in all applications of CDT (cognitive diagnosis theory) depends on the extent to which the selected model accurately reflects the real data. Only when the CDMs (cognitive diagnosis models) fit the data well, can the advantages and functions of CDT emerge. Selection of the wrong model would lead to relatively large error in parameter estimation, test equating, the analysis of differential item functioning and so on, which would result in adverse effect. Therefore, it is required to evaluate model-data fit before applying CDT. Common fit statistics in the field of CDT consist of test fit statistics and item fit statistics; the former evaluate fit from test angle, the latter evaluate fit from item angle, which can guide the selection of items. Test fit statistics usually are relative fit statistics and item fit statistics usually are absolute fit statistics. In fact, it is very tedious and blind to use item fit statistic to assess model-data fit for all items, and absolute fit assessment is quite difficult. Thus, many researchers only consider relative fit assessment, which can give a direction to model selection. Relative fit statistics can be expounded and compared from model misspecification and Q-matrix misspecification. Traditional relative fit statistics such as, -2LL, AIC, AICc, BIC and DIC4 have been used to test the validity of CD models. However, CVLL has never been applied in the real application. Given that CVLL is a reliable index in item response theory, it is worthy to investigate how well it performed in detecting the validity in CD models. The aim of this study is to evaluate these different statistics in terms of the proportion of times each fitted model was selected out of the 30 iterations. Four conditions were considered in the simulation study, including sample sizes, test lengths, numbers of attributes, and Q-matrix misspecifications. Three models were used to generate the data: the DINA model, the NIDA model, and R-RUM. An empirical example involving real data was used to illustrate how the different fit statistics can be employed to identify misspecifications. This research discovered that : a) CVLL performed best, b) AIC and BIC performed better than AICc, c) AICc and BIC tended to select the reduced model than the saturated model in small sample size and long test length, d) When the number of attributes became 9, the correct rates of these statistics decreased, meanwhile, CVLL performed best, e) If the true model was R-RUM, the selection rates of correct Q-matrix of these statistics decreased except CVLL, f) The empirical example proved that DINA, NIDA and R-RUM neither fit the fraction data adequately, but R-RUM is the best-fitting model among a set of competing models. Although the results of this work are encouraging, additional work is needed to further understand model-data fit evaluation under the CDM context and to broaden the generalizability of the current findings. First, this study only covered three models. Other types of CDMs – such as multiple-strategies models or high-order models – need to be investigated. Second, due to the important role the Q-matrix played in cognitive diagnosis modeling, future research should systematically examine the impact of not only the degree but also the type of Q-matrix misspecifications on the different fit statistics.