LS and Maximum Likelihood estimation (MLE) overfit when the dimension of the model is not small relative to the sample size. This happens almost always in high-dimensions. Regularziation often works by adding a penalty to the fitting criterion as in classical model selection methods such as AIC or BIC and L1-penalized LS called Lasso. We will also introduce Cross-validation (CV) for regularization parameter selection.