A Simulation Study on the Interaction Effects of Underfactoring and Nuisance Parameters on Model Fit Indices

Latent measurement models pose the challenge of specifying the correct number of latent factors. Researchers may underestimate data complexity and neglect factors in confirmatory factor analysis, which is typically evaluated using fit indices, such as CFI, RMSEA, and SRMR. Little is known about these indices' behavior in the case of underfactoring and its interaction with various nuisance parameters. Therefore, our extensive simulation study examined the sensitivity of the CFI, RMSEA, and SRMR to underfactoring based on up to 428,433 datasets with sample sizes from 150 to 1,350 covering 72 different configurations of multi-factorial model conditions. To summarize, results showed that underfactoring affected all three fit indices, with the effect diminishing as the number of latent variables increased. Additionally, a decreasing mean loading size reduced the impact of underfactoring on RMSEA and SRMR. Consequently, common cut-offs, such as those by Hu and Bentler (1999), often led to false negative results, particularly with RMSEA and SRMR.[AQ]