Model guide

PCA

Turn the measurement axes toward directions where the data spreads most. Keeping fewer directions gives a smaller representation of each example.

Start with an example

If parcel length and width tend to increase together, one diagonal direction may summarize much of their shared variation. Keeping both directions rotates the coordinates without reducing their count.

How it learns

An input feature is a measurement about an example. PCA learns from those measurements alone; it does not need an answer label.

Move the origin to the average input measurements, then find the direction along which the examples spread most. This first principal component combines the original measurements into one coordinate along that direction.
Further components capture remaining spread in directions at right angles to those already chosen. Keep fewer directions to compress the examples, losing variation in discarded directions. PCA uses the input measurements alone, without known target answers.

Try it yourself

The example above gives the idea. The experiment below uses a separate, made-up dataset with unitless measurements so you can focus on how the model works. Read the graph guide first, then change one thing at a time.

1. Understand what you are looking at

Summarize two measurements with fewer directions. Here both axes are input features: there is no target to predict and no class label.

Across: the horizontal axis: The first measured feature. Moving right means this measurement is larger.
Up: the vertical axis: The second measured feature, not an answer to predict. Moving up means this measurement is larger.

Dark dots are original measurements. Orange dots are reconstructions using the retained directions; faint dashed connectors show what was lost. The green line is the first principal axis.

How to read one result: With one component, follow an original dot along its dashed connector to the green line. The orange endpoint is the closest reconstruction using that direction. Shorter connectors mean less information lost.

2. Predict, change, observe

Keep Retained components at 1. Follow a faint dashed connector from an original dot to its orange reconstruction on the green principal axis, the direction with the most variation.
Predict what will happen to those connector lengths when you keep 2 components. Change the control to 2: reconstructions return to the original positions and reconstruction error becomes zero.

Retained components 1How many directions represent each example. One compresses two measurements into one direction; two preserve both.Noise in the example data 1How far the invented measurements scatter. This changes the data and recomputes PCA; it is not a PCA setting.

Observations
First principal axis
Reconstructed observations

Variance retained: 98.7%.

Mean squared reconstruction distance: 0.150

Variance retained measures how much of the original spread is preserved. Reconstruction error averages the squared distances between original and reconstructed dots; smaller means less was lost.

3. Explain it in your own words

If keeping one direction preserves most variation, does that prove it preserves the information needed to predict a future target?

Compare your reasoning

No. PCA chooses directions using the spread of the input measurements, without a target. A low-variation direction could still matter for a prediction task. If unsure, return to 1 component and describe what is discarded: distances away from the retained axis.

Ready to continue when you can explain the result and why it changes. If the explanation is still unclear, reset and repeat the two steps above with one control at a time.

What this experiment simplifies

PCA learns the direction with the most variation. With one retained component, observations project onto that line; two components preserve both directions. Noise changes the example data, not a PCA hyperparameter.

These are small teaching implementations, not scikit-learn runs. Training error on these examples does not estimate performance on new data.

View the plotted examples

First feature	Second feature
0.60	1.9
1.18	1.06
1.76	2.32
2.34	2.28
2.92	3.74
3.50	2.9
4.08	4.16
4.66	4.12
5.24	5.58
5.82	4.74
6.40	6
6.98	5.96
7.56	7.42
8.14	6.58
8.72	7.84
9.30	7.8

The math, one step at a time

Read the symbols after the idea makes sense. You do not need to memorize the formula.

z=(x-\mu)W

μ is the training mean and W contains the retained component directions. z is the transformed row. n_components sets how many directions are kept.

What to watch for

Large input variation is not necessarily useful for predicting the target.
Discarding components loses information. Even without labels, fitting PCA on validation or test rows leaks information about their input distribution.

PCA