Blockwise.
Back to learning
Model guide

Elastic Net

Mix two ways to restrain a linear model: shrink weights smoothly and allow some to become exactly zero.

Start with an example

For related measurements such as area and room count, the L2 part can make the fit steadier while the L1 part still encourages a compact coefficient table.

How it learns

An input feature is a measurement about an example. The target is its known answer: a number for regression, or a category for classification. Training means learning from examples whose answers are already known.

  1. A linear model gives each input a weight. Elastic Net combines a cost for absolute weight size (L1) with a cost for squared weight size (L2). The first can switch weights off; the second encourages gradual shrinkage.
  2. alpha sets the total penalty strength. l1_ratio sets the mixture: 1 uses the L1 end and 0 uses the L2 end. Compare mixtures using separate evaluation examples rather than assuming one is always best.

Try it yourself

The example above gives the idea. The experiment below uses a separate, made-up dataset with unitless measurements so you can focus on how the model works. Read the graph guide first, then change one thing at a time.

1. Understand what you are looking at

See how a penalty discourages a line from relying on a large slope. This graph fits one straight line, even when the example above discusses a more complex curve.

Across: the horizontal axis
The input measurement x, from 0 to 10. In a delivery task, an input could be distance; here it has no unit.
Up: the vertical axis
The target: the numerical answer being predicted. In a delivery task, it could be time; here the target has no unit.

Each dark dot pairs one input with its observed target. The green line gives the model's predicted target at each input. A vertical gap from a dot to the line is a prediction error.

How to read one result: Start at x on the bottom axis and follow the vertical dashed line to the outlined diamond on the curve. Read its height on the left: that is the predicted target for this input.

2. Predict, change, observe

  1. Set Penalty strength (alpha) to 0 and compare the prediction at x = 0 with the one at x = 10.
  2. Predict what will happen to that difference when alpha rises to 20. Increase it and watch the line become flatter; the fitting procedure recomputes the line for each setting.
Interactive Elastic Net illustrationEach dark dot pairs one input with its observed target. The green line gives the model's predicted target at each input. A vertical gap from a dot to the line is a prediction error.02.557.5100246810Target valueInput feature (x)
  • Observations
  • Model prediction
  • Inspected result (diamond)
Prediction at x = 6.0: 4.912

Mean squared training error: 0.893

Mean squared training error averages the squared vertical gaps from the observed dots to the line. Smaller means a closer fit to these training examples, not necessarily better predictions on new examples.

3. Explain it in your own words

If the line becomes flatter as the penalty grows, has the input become less important to this fitted model, or has it been proved irrelevant in the real world?

Compare your reasoning

The fitted model relies less on the input. This does not prove real-world irrelevance. The penalty trades a closer fit for smaller weights. If unsure, compare alpha = 0 and alpha = 20 and describe the change in predictions without making a claim about cause and effect.

Ready to continue when you can explain the result and why it changes. If the explanation is still unclear, reset and repeat the two steps above with one control at a time.

What this experiment simplifies

This one-feature fit minimizes mean squared error plus a penalty on the slope. Increasing alpha shrinks the slope; L1 can make it exactly zero. The intercept is not penalized.

These are small teaching implementations, not scikit-learn runs. Training error on these examples does not estimate performance on new data.

View the plotted examples
Input feature (x)Observed target
0.302.78
0.922.97
1.544.17
2.164.08
2.784.33
3.403.24
4.023.41
4.642.87
5.263.37
5.883.16
6.504.58
7.125.27
7.746.6
8.366.59
8.987.51
9.607.14

The math, one step at a time

Read the symbols after the idea makes sense. You do not need to memorize the formula.

min⁡b,wSSE2n+αr∑j∣wj∣+α(1−r)2∑jwj2\min_{b,w}\frac{\mathrm{SSE}}{2n}+\alpha r\sum_j|w_j|+\frac{\alpha(1-r)}{2}\sum_j w_j^2Read it like thisAdd the scaled squared-error total to a mixture of absolute-weight and squared-weight penalties. Alpha controls total strength; r chooses the mixture.What the symbols meanb,  wjb,\;w_jb is the intercept, the starting prediction before adding input contributions. w is a learned weight; the lower j selects the weight for input j. A weight tells how strongly that input affects the prediction.SSE\mathrm{SSE}Sum of squared errors: square each prediction's difference from its observed answer, then add those squares.n,  in,\;in is the number of training examples and i selects one example. Dividing their total by n gives their average.α\alphaAlpha: the penalty strength you choose. Larger alpha makes large weights more costly; it is not a prediction or an input measurement.rrr is the chosen L1 share, called l1_ratio in code. It lies between zero and one; 1-r is the remaining L2 share.×,  ⋅\times,\;\cdotBoth marks mean multiply. Letters written next to each other also multiply: wx means w times x. For vectors, a dot means multiply matching entries and add the results.++Plus: add the values together.−-Minus: subtract the next value. Before a value, it changes that value's sign.ab\frac{a}{b}A fraction means divide the entire top by the entire bottom. For example, 6 over 3 means 6 divided by 3, which is 2. The bottom cannot be zero.∑\sumCapital sigma: add one term for every indicated item. The symbol below says which item to count; limits such as j = 1 to p mean start at 1 and stop at p. For example, adding 2, 4, and 6 gives 12.min⁡\minMinimize: choose the listed parameters to make the following error or cost as small as possible.x2x^2Squared: multiply a value by itself. For example, 3 squared is 9. Squaring an error removes its sign and gives larger mistakes more influence.∣w∣|w|Absolute value: the size of a number without its sign. Both -3 and 3 have absolute value 3.

SSE is the sum of squared prediction errors, n counts training rows, and r is l1_ratio between zero and one. α sets total penalty strength.

What to watch for

  • The mixture adds another setting to validate and can still underfit.
  • Coefficients depend on scaling and correlated inputs; they do not by themselves explain causes.
RidgeLasso

Further reading

The official documentation goes further when you are ready.