12.1 Distance
Everything depends on what “close” means, and that choice is made by the analyst before any algorithm runs.
Definition 12.1 (Common distances). For observations \(\underline {x}\) and \(\underline {y}\) in \(\mathbb {R}^{p}\):
- (i)
- Euclidean: \(d = \sqrt {\left (\underline {x}-\underline {y}\right )'\left (\underline {x}-\underline {y}\right )}\);
- (ii)
- Mahalanobis: \(d = \sqrt {\left (\underline {x}-\underline {y}\right )'S^{-1}\left (\underline {x}-\underline {y}\right )}\);
- (iii)
- Manhattan: \(d = \sum ^{p}_{i=1}\left |x_i-y_i\right |\).
Note 12.2. Euclidean distance is not scale-invariant, so a variable measured in small units dominates it — the same difficulty met in Section 4.3, and with the same remedy of standardising first. Mahalanobis distance handles both scale and correlation automatically, since \(S^{-1}\) discounts directions in which the data already vary a great deal; it is the distance the ellipsoids of Section 5 are built from. Its drawback is that it requires estimating \(S\), which is unstable when \(n\) is small relative to \(p\).
Questions on this section
Stuck on something here? Ask below and it stays attached to this topic.