Gang Fang's Blog

What are the differences between SEM, Regression Analysis and ML? They all seem to estimate and predict the relationship between dependent variables and independent variables.

Answer: They differ in multiple important aspects, such as scope, purpose, assumptions and output interpretation.

Scope: SEM is a general modeling framework encompassing multiple multivariate techniques, while Regression Analysis is a single statistical method. Compared to these, ML’s scope is much bigger involving algorithms ranging from simple to extremely complex.

Purpose: SEM aims to understand relationships between observed variables and latent constructs, while Regression Analysis focuses on estimating the relationship between a single dependent variable and one or multiple dependent variables. ML on the other hand, focuses less on understanding variable relationships and more on predictions and a whole array of application objectives.

Assumptions: SEM and Regression Analysis both make statistical assumptions when SEM’s are more stringent. ML models vary widely re this. Some require little to no formal assumptions.

Output interpretation: for regression analysis and SEM, the outputs include estimates of the relationships (coefficients), model fit indicators and error estimates and significance tests. For ML, the focus is often the predictions and applications.

Critique: this is a horizontal comparison supported by basic reading. It’s considered accurate.

What is the difference between error estimates and significance tests?

Answer: They serve different purposes. Error estimates serve to tell us how large the discrepancies are between observed data and predicted values. Significance tests tell us to what extent the patterns observed in the sample data can be generalized to the broader population.

Critique: Basic reading. Considered accurate.