Simple Linear Regression Analysis with Non-Normal Errors

Abstract

We consider a simple linear regression with non-normal errors; specifically, errors follow the double exponential distribution and uniform distribution. Maximum likelihood estimates for parameters (intercept and slope) are investigated. Although they do not have closed mathematical forms, they can be derived uniquely by numerical methods. Through Monte Carlo simulation, ordinary least square estimates (LSE) and maximum likelihood estimates (MLE) are derived and compared through their biases, variances and mean squared errors. Simulation studies show that both MLE and LSE follow normal distributions and MLE has smaller mean squared errors compared to LSE. This paper suggests or confirms that one prefers to use MLE to estimate unknown parameters if there exists strong evidence that errors do not follow a normal distribution.