Blockwise.
Back to learning
Model guide

Logistic regression

Turn a weighted input score into a probability between zero and one. Despite its name, this model is commonly used for classification.

Start with an example

A parcel's measurements produce a review probability of 0.7. A threshold of 0.5 sends it for review; a threshold of 0.8 does not.

How it learns

An input feature is a measurement about an example. The target is its known answer: a number for regression, or a category for classification. Training means learning from examples whose answers are already known.

  1. Multiply inputs by learned weights and add a starting value to make a score. An S-shaped conversion, called the sigmoid, turns that score into an estimated probability between zero and one.
  2. Training adjusts the weights to give known class labels higher probability. A separate decision threshold turns the estimate into an action: for example, a review probability of at least 0.5 could trigger review. Moving that threshold does not change the probability itself.

Try it yourself

The example above gives the idea. The experiment below uses a separate, made-up dataset with unitless measurements so you can focus on how the model works. Read the graph guide first, then change one thing at a time.

1. Understand what you are looking at

See how an input becomes an estimate of which of two classes it belongs to. Here class 1 could mean a parcel needs review, and class 0 could mean it does not.

Across: the horizontal axis
The input measurement x, from 0 to 10. Moving right means a larger input.
Up: the vertical axis
The estimated probability of class 1, from 0 to 1. A curve height of 0.8 means an estimated 80% chance, not an observed class of 0.8.

Dark dots are observed answers: class 0 at the bottom or class 1 at the top. The green line is the model's estimate, which can lie between 0 and 1.

How to read one result: Start at x on the bottom axis and follow the vertical dashed line to the outlined diamond. Its height is the estimated probability of class 1 at that input; it is not a new observed example.

2. Predict, change, observe

  1. Reset and inspect x = 5. The curve is halfway up: 0.5 means an estimated 50% chance of class 1. Predict what happens there if Midpoint moves to 7.
  2. Move Midpoint to 7 and check the lower estimate at x = 5. Then reset and raise Slope: compare how quickly the curve moves from low to high probability.
Interactive Logistic regression illustrationDark dots are observed answers: class 0 at the bottom or class 1 at the top. The green line is the model's estimate, which can lie between 0 and 1.00.250.50.7510246810Class probabilityInput feature (x)
  • Observations
  • Model prediction
  • Inspected result (diamond)
Prediction at x = 6.0: 0.769

Training errors: 3 of 16.

Training errors count observed labels that disagree with the decision: estimates of at least 0.5 choose class 1; lower estimates choose class 0.

3. Explain it in your own words

Does a curve height of 0.8 mean the observed class is 0.8, or that class 1 has an estimated probability of 80%?

Compare your reasoning

It is an estimated probability of 80%. An observed example still has class 0 or class 1. In this experiment, estimates of at least 0.5 are assigned to class 1 when counting training errors. Revisit the dots at 0 and 1 to separate observed answers from estimates.

Ready to continue when you can explain the result and why it changes. If the explanation is still unclear, reset and repeat the two steps above with one control at a time.

What this experiment simplifies

The slope and midpoint are model parameters, adjusted by hand here; training would learn them from data. The curve shows the probability of class 1.

These are small teaching implementations, not scikit-learn runs. Training error on these examples does not estimate performance on new data.

View the plotted examples
Input feature (x)Observed class
0.300
0.920
1.540
2.161
2.780
3.400
4.020
4.641
5.261
5.881
6.501
7.120
7.741
8.361
8.981
9.601

The math, one step at a time

Read the symbols after the idea makes sense. You do not need to memorize the formula.

z=b+w⋅x,p(y=1∣x)=11+exp⁡(−z)z=b+w\cdot x,\qquad p(y=1\mid x)=\frac{1}{1+\exp(-z)}Read it like thisFirst add weighted inputs to the intercept to get a raw score. Then convert that score into a probability: one divided by one plus exp of the negative score.What the symbols meanx,  xjx,\;x_jx is the input: measurements supplied to the model. A small lower j selects input number j, such as distance from several measurements.b,  wjb,\;w_jb is the intercept, the starting prediction before adding input contributions. w is a learned weight; the lower j selects the weight for input j. A weight tells how strongly that input affects the prediction.p(y=1∣x),  zp(y=1\mid x),\;zRead this as the estimated probability that the answer is 1, given input x. The vertical bar means given. z is the weighted score before converting it into a probability.==Equals: the expressions on both sides have the same value.++Plus: add the values together.−-Minus: subtract the next value. Before a value, it changes that value's sign.ab\frac{a}{b}A fraction means divide the entire top by the entire bottom. For example, 6 over 3 means 6 divided by 3, which is 2. The bottom cannot be zero.×,  ⋅\times,\;\cdotBoth marks mean multiply. Letters written next to each other also multiply: wx means w times x. For vectors, a dot means multiply matching entries and add the results.exp⁡(z)\exp(z)Exponential: raise the fixed number e, about 2.718, to the power z. For example, exp(0) is 1. It always returns a positive value.

z is a weighted score, b is the intercept, and exp is the exponential function. p is the model's estimated probability of class 1, not a guarantee.

What to watch for

  • A linear score cannot express every boundary without transformed features.
  • Estimated probabilities can be poorly calibrated; threshold choice needs task-specific validation.
Softmax regressionSGD linear modelsRidge

Further reading

The official documentation goes further when you are ready.