Detecting model misfit in structural equation modeling with machine learning—a proof of concept

Despite the popularity of structural equation modeling in psychological research, accurately evaluating the fit of these models to data is still challenging. Using fixed fit index cutoffs is error-prone due to the fit indices’ dependence on various features of the model and data (“nuisance parameters”). Nonetheless, applied researchers mostly rely on fixed fit index cutoffs, neglecting the risk of falsely accepting (or rejecting) their model. With the goal of developing a broadly applicable method that is almost independent of nuisance parameters, we introduce a machine learning (ML)-based approach to evaluate the fit of multi-factorial measurement models. We trained an ML model based on 173 model and data features that we extracted from 1,323,866 simulated data sets and models fitted by means of confirmatory factor analysis (i.e., training observations). We evaluated the performance of the ML model based on 1,659,386 test observations unseen during model training. The ML model performed very well in detecting model (mis-)fit in the majority of conditions, thereby outperforming the fixed fit index cutoffs by Hu and Bentler (1999) across the board. Only minor misspecifications—single neglected residual correlations (and cross-loadings), in particular—proved to be challenging to detect. From this proof-of-concept study we conclude that ML is very promising in the context of model fit evaluation.