A nearest neighbor is relative to a corpus, representation, and metric.
The team compares a query against help-center article vectors. Their old index ranked a small set of relevant articles clearly. After the migration, exact search gives several candidates with similar distances, while approximate search returns a different top five. The phrase “high-dimensional curse” is tempting, but it is not a diagnosis.
Embeddings may have hundreds or thousands of coordinates. An individual coordinate is not usually a stable human-readable feature. Geometry is still useful: vector norms, angles, distances, and ranking margins can reveal a change. But semantic relevance must be checked with labeled queries, and search behavior also depends on corpus membership, filtering, model version, and index settings.
- Observed
- Top-k results changed after a model and index migration.
- Competing causes
- Embedding space, normalization, corpus, filters, metric, or approximate-index recall.
- Baseline
- Same frozen query set, same corpus snapshot, exact neighbors and judged relevance.
- Decision
- Do not add dimensions or tune a threshold until the changed stage is isolated.
In many random spaces, distances bunch toward a typical scale.
Imagine vectors whose coordinates are drawn independently from a similar, zero-centered distribution. Squared Euclidean distance adds coordinate-wise squared differences. As coordinates accumulate, its total grows roughly with dimension, while relative fluctuations often shrink. Distances can cluster around a typical value, so the nearest and farthest candidates may differ less as a fraction of that scale.
For unit vectors, cosine similarity is the dot product. Under an idealized isotropic random model, unrelated directions tend to have cosine near zero, with spread that typically narrows as dimension grows. Real embeddings are not guaranteed to be isotropic, independent, or random. Training intentionally shapes their neighborhoods; anisotropy, clusters, norms, and task structure can dominate.
More coordinates contribute to squared distance.
Nearest/farthest ratio depends on data distribution.
Useful only with a compatible representation and policy.
Inspect norms, score histograms, and labeled neighbor quality.
Cosine and Euclidean answer related but distinct questions.
Euclidean distance includes both orientation and vector length. Cosine similarity compares
direction after dividing by magnitudes. If all nonzero vectors are normalized to unit
length, then ‖a−b‖² = 2(1−cos(a,b)), so cosine ranking and Euclidean ranking
agree. Without that normalization, magnitude can alter Euclidean distance while cosine
stays unchanged for positive scaling.
Even with a chosen metric, “nearest” only means nearest under that metric in the indexed representation. It does not mean relevant, correct, safe, or useful. Check whether the vector model and corpus share one coordinate space and whether query and document preprocessing are compatible.
Cosine similarity or Euclidean distance, recorded with the index.
Apply consistently if metric equivalence is expected.
Evaluate relevant candidates separately from numeric closeness.
Inspect recall, duplicates, and score margins.
Distance concentration is a lens for experiments, not a verdict on embeddings.
“Curse of dimensionality” describes several related difficulties: volume grows rapidly, finite samples sparsely cover a space, and some distance-based algorithms lose useful contrast or cost. It does not say all high-dimensional learning fails. A learned representation can put task-relevant examples into useful neighborhoods even when raw ambient dimension is large.
For diagnosis, compare distributions rather than one nearest score: norms, pairwise distances or cosines, nearest-neighbor margins, duplicate rates, and relevance by query segment. Compare against simple baselines such as lexical retrieval and against exact vector search. If query behavior differs by language, document length, or category, aggregate metrics can mask the failing slice.
Watch one synthetic neighborhood, then ask what evidence it does not provide.
Euclidean distance: min 0.37, mean 2.79, max 5.40; nearest/farthest ratio 0.069.
Cosine: min -0.935, mean -0.006, max 0.986.
These fixed-seed, roughly centered synthetic vectors help expose a distribution effect. They are not trained embeddings, and a sample of 32 candidates is far too small to predict production retrieval quality.
Approximate nearest neighbors trade exactness for operational cost.
Approximate nearest-neighbor (ANN) indexes reduce query work by searching a subset or compressed representation. They can make large retrieval workloads practical, but the useful question is not simply “is ANN faster?” Compare approximate top-k with exact top-k over the same vectors and queries. Recall@k can be defined as the proportion of exact top-k items also returned by ANN; also measure judged relevance because exact nearest vectors may themselves be semantically poor.
Record latency distributions, memory, index-build time, update cost, and recall as you vary the index's search effort. A setting that improves recall may raise latency or memory. Compare across realistic query types and corpus growth. Index parameters and tradeoffs are implementation-specific; use the chosen index's documentation for its controls and definitions.
Same metric, vectors, filters, and corpus snapshot.
Not the same as human relevance.
Include p95, build, and update behavior.
Make quality and operational limits explicit.
Vector helpers should enforce the comparison contract.
The examples compare cosine and Euclidean distance for equal, nonempty vectors; they reject mismatched dimensions, non-finite coordinates, and zero-vector cosine. They are exact calculations over a pair of vectors, not an ANN index or relevance evaluator.
package main
import (
"errors"
"math"
)
func Cosine(a, b []float64) (float64, error) {
if len(a) == 0 || len(a) != len(b) {
return 0, errors.New("vectors must have equal nonzero dimensions")
}
dot, aa, bb := 0.0, 0.0, 0.0
for i, x := range a {
y := b[i]
if math.IsNaN(x) || math.IsInf(x, 0) || math.IsNaN(y) || math.IsInf(y, 0) {
return 0, errors.New("coordinates must be finite")
}
dot += x * y
aa += x * x
bb += y * y
}
if aa == 0 || bb == 0 {
return 0, errors.New("cosine is undefined for a zero vector")
}
return dot / math.Sqrt(aa*bb), nil
}
func Euclidean(a, b []float64) (float64, error) {
if len(a) == 0 || len(a) != len(b) {
return 0, errors.New("vectors must have equal nonzero dimensions")
}
sum := 0.0
for i, x := range a {
y := b[i]
if math.IsNaN(x) || math.IsInf(x, 0) || math.IsNaN(y) || math.IsInf(y, 0) {
return 0, errors.New("coordinates must be finite")
}
d := x - y
sum += d * d
}
return math.Sqrt(sum), nil
}
These choices apply to the comparisons throughout this story.