Blockwise.
Back to learning
Model guide

One-versus-rest

Give each class its own yes-or-no detector. Each detector asks whether an example belongs to its class rather than any of the others.

Start with an example

For routes A, B, and C, train A versus the rest, B versus the rest, and C versus the rest. Compare their scores to choose one route.

How it learns

An input feature is a measurement about an example. The target is its known answer: a number for regression, or a category for classification. Training means learning from examples whose answers are already known.

  1. For each class, turn the training labels into a yes-or-no question: does this example belong to that class? Train a separate detector for each question, using all the classes' examples.
  2. For a task with one correct class, compare the detectors' scores at the same input and choose the largest. For a task where several labels can apply, make a separate yes-or-no decision for each label.

Try it yourself

The example above gives the idea. The experiment below uses a separate, made-up dataset with unitless measurements so you can focus on how the model works. Read the graph guide first, then change one thing at a time.

1. Understand what you are looking at

Compare three separate yes-or-no detectors, one per class. Each line estimates that class against the rest; these independent estimates do not have to add to 1.

Across: the horizontal axis
The input measurement x, from 0 to 10. Compare all class lines at the same horizontal position.
Up: the vertical axis
Each class's estimated probability, from 0 to 1. A height of 0.8 means an estimate of 80% for that class.

Green, orange, and purple lines belong to classes A, B, and C. There are no observed training dots here: these class scores or contests are constructed for illustration.

How to read one result: Start at x on the bottom axis and follow the vertical dashed line. Compare the three curve heights there: the highest identifies the winning class. The outlined diamond marks that highest value.

2. Predict, change, observe

  1. Inspect x = 5 and read all three class lines at that same position. The highest estimate identifies the winning class in this illustration.
  2. Predict which line will move when Center of class B changes from 5 to 7. Try it: B's detector moves, while A and C keep their original curves.
Interactive One-versus-rest illustrationGreen, orange, and purple lines belong to classes A, B, and C. There are no observed training dots here: these class scores or contests are constructed for illustration.00.250.50.7510246810Class probabilityInput feature (x)
  • Class A
  • Class B
  • Class C
  • Inspected result (diamond)
Highest class probability at x = 6.0: 0.690

Compare class lines at this input. This readout is the highest value, not the sum of the class values.

3. Explain it in your own words

Must these three independent detector estimates add to 1, as softmax probabilities do?

Compare your reasoning

No. Each detector makes a separate estimate. This illustration compares those estimates directly; a library may normalize probabilities for single-label output. If unsure, compare all three heights at one x and revisit the difference between separate detectors and a shared probability budget.

Ready to continue when you can explain the result and why it changes. If the explanation is still unclear, reset and repeat the two steps above with one control at a time.

What this experiment simplifies

Three fixed binary classifiers each score one class against the rest. Their independent probabilities need not add to 1. The toy score curves are chosen by hand; a real wrapper trains the classifiers.

These are small teaching implementations, not scikit-learn runs. Training error on these examples does not estimate performance on new data.

The math, one step at a time

Read the symbols after the idea makes sense. You do not need to memorize the formula.

Ndetectors=K,c^=arg⁡max⁡c  sc(x)N_{\mathrm{detectors}}=K,\qquad \hat c=\underset{c}{\arg\max}\;s_c(x)Read it like thisTrain one binary detector for each possible class. For a new input, compare the class scores and choose the class with the largest score.What the symbols meanNdetectors,  KN_{\mathrm{detectors}},\;KN with lower detectors counts the binary classifiers to train. K is the number of possible classes, such as three different routes.sc(x)s_c(x)The score given to class c for input x. These separate scores need not form probabilities that add to one.x,  xjx,\;x_jx is the input: measurements supplied to the model. A small lower j selects input number j, such as distance from several measurements.c,  c^c,\;\hat cc is one possible class, such as ordinary or urgent. c with a hat is the class the model chooses.==Equals: the expressions on both sides have the same value.arg⁡max⁡c\underset{c}{\arg\max}Arg max: return the class with the largest score, rather than the score itself. If scores for A and B are 0.2 and 0.8, choose B.

K is the number of classes. argmax means the class with the largest score. In the special binary case, scikit-learn uses one fitted detector.

What to watch for

  • Separate detectors do not automatically produce a mutually calibrated set of scores.
  • Training work grows with class count, and rare classes may have highly imbalanced binary training problems.
One-versus-oneSoftmax regressionLogistic regression

Further reading

The official documentation goes further when you are ready.