Lasso

updated: 2023-02-06
#machine-learning/lasso

Lasso in machine learning is a model regularization to learn sparse models and sparse data representations through regularization, i.e.,

where is the objective for a model with parameters and data , and is the regularized objective. Parameter is the lasso penalty, which controls the sparsity of the model.

Lasso stands for the least absolute shrinkage and selection operator proposed in 1996 by Robert Tibshirani.

• Regression Shrinkage and Selection via the Lasso on JSTOR

Lasso quickly becomes one of the fundamental methods used in a variety of machine learning methods.

Why Lasso?

• Preventing overfitting
• Colinearity problem
• Interpretability

Why it works?

• Intuition
• Lasso from laglagian perspective
• Gradient

Adaptive lasso

• Bias and variance

Lasso in neural networks

• LassoNet

Sparse coding

• Sparse PCA (dictionary & atom)

References

• “Lasso & Ridge Regression” in 200 Words - Data Science