90% of Regression Models Could Improve with RMSE: Here's Why!

In regression analysis, the choice between Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) significantly impacts model performance. Statistics show that RMSE is minimized by the conditional expectation, or mean, represented as E(Y | X). In contrast, MAE is minimized by the conditional median, denoted as Median(Y | X). This distinction is crucial because it determines what aspect of the target variable the model aims to predict. Most regression models are designed to predict the expected value of the target, which aligns better with RMSE. Therefore, using RMSE could potentially enhance the accuracy of up to 90% of regression models. This insight suggests that analysts should consider RMSE over MAE for more effective model training, hyperparameter searches, and evaluations.

Source: www.reddit.com