Similarity Metrics and Distance Functions

A team ships a product search over 400,000 items. It works, mostly. Then someone notices that the query "lightweight running shoe" returns, at rank one, a 4,000-word buyer's guide called "The Complete Guide to Athletic Footwear" — and returns the actual lightweight running shoe at rank fourteen. This happens for every query. Long documents always win.

The embedding model is fine. The index is fine. The bug is one line in the index configuration: the collection was created with distance: Dot instead of distance: Cosine, and the vectors were never normalised. The buyer's guide has a vector twice as long as the shoe's, so it scores twice as high on every query regardless of what the query is about.

That is a one-word fix worth several weeks of confused debugging, and it is representative. The similarity function is the least glamorous decision in a vector search system and one of the few that can silently invert your rankings. This lesson works through the functions that matter, the arithmetic behind each, when each is correct, and how to compute them fast enough to matter at scale.

A running example

Every metric below is computed on the same two vectors, so you can compare the numbers directly rather than trusting a description.

a = [3, 0, 4]b = [0, 4, 3]componentwise difference a - b = [ 3, -4,  1]

Useful facts to have ready: $\lVert a\rVert = \sqrt{9+0+16} = 5$, $\lVert b\rVert = \sqrt{0+16+9} = 5$, and $a\cdot b = 3(0) + 0(4) + 4(3) = 12$.

Distances on continuous vectors

Euclidean distance (L2)

Straight-line distance, the one you learned at school extended to any number of dimensions.

For our pair: $\sqrt{3^2 + (-4)^2 + 1^2} = \sqrt{9+16+1} = \sqrt{26} = 5.099$.

L2 is the default in FAISS, and it is the right choice when the magnitude of a vector carries real information — coordinates in space, sensor readings, image feature vectors where intensity matters. It is sensitive to scale: if one dimension of your data is measured in metres and another in millimetres, the millimetre dimension will dominate the distance entirely.

A performance note that shows up everywhere: you almost never need the square root. Since $\sqrt{\cdot}$ is monotonically increasing, ranking by squared Euclidean distance gives the identical ordering, and skipping the square root removes one of the most expensive scalar operations from the inner loop. FAISS's IndexFlatL2 returns squared distances for exactly this reason, and people regularly misread those numbers as if they were distances.

Manhattan distance (L1)

The distance you would walk on a street grid: sum of absolute differences.

For our pair: $|3| + |{-4}| + |1| = 3+4+1 = 8$.

L1 penalises many small differences relatively more than L2 does, and one large difference relatively less — because L2 squares the gaps, a single big gap dominates it. That makes L1 more robust to outliers and a reasonable choice for sparse or heavy-tailed features. It is also cheaper: absolute value instead of multiply. It is rarely the right choice for neural embeddings, which are trained under an L2 or cosine objective.

Chebyshev distance (L∞)

The largest single-dimension difference, ignoring all the others.

For our pair: $\max(3,4,1) = 4$.

This answers "what is the worst discrepancy on any single feature?" — useful for tolerance checking and anomaly detection, almost never useful for semantic retrieval, where meaning is spread across all dimensions and no single one is decisive.

Minkowski distance (Lp) — the general form

All three above are the same formula with a different exponent.

With $p=1$ you get Manhattan, $p=2$ Euclidean, and as $p\to\infty$ the largest term swamps the rest and you get Chebyshev. Our example at $p=3$: $(27 + 64 + 1)^{1/3} = 92^{1/3} = 4.514$.

Line the four up and the pattern is clear:

Distance decreases monotonically as p rises, and increasing p concentrates the measurement onto fewer dimensions. Values of p other than 1 and 2 are curiosities in retrieval; the reason to know the family is that it makes the relationship between L1, L2 and L∞ a single idea instead of three.

Similarity by direction and by magnitude

Cosine similarity

The cosine of the angle between two vectors — how aligned they are, with length divided out.

For our pair: $12 / (5 \times 5) = 0.48$.

It ranges from −1 (opposite) through 0 (orthogonal) to 1 (identical direction). In practice, embeddings from modern text models are almost never negatively correlated; typical similarity scores for unrelated documents cluster around 0.1 to 0.4, and for genuinely related ones around 0.7 to 0.95. This matters when people try to set an absolute cut-off: 0.5 is not a natural boundary, and the right threshold differs per model. Calibrate it against your own data by looking at score distributions for known-relevant and known-irrelevant pairs.

Cosine is the correct default for text retrieval because document length should not determine relevance. A one-sentence answer and a five-paragraph answer that say the same thing get similar vectors, differing mainly in magnitude — and cosine throws that magnitude away.

Inner product (dot product)

For our pair: 12. The unnormalised cousin of cosine: same numerator, no division.

Now the failure from the opening, with numbers. Let the query be a unit vector $q$. Two documents:

• Document A: direction $u$ with $\cos(q,u)=0.6$, but vector length 2.0 — so $q\cdot A = 2.0 \times 0.6 = 1.20$.
• Document B: direction $v$ with $\cos(q,v)=0.9$, and vector length 1.0 — so $q\cdot B = 1.0 \times 0.9 = 0.90$.

Cosine ranks B first, correctly: it is far more about the query. Dot product ranks A first, because A is longer. In a real corpus that length correlates with document length, verbosity, or simply which chunks got padded — none of which is relevance.

Dot product is cosine similarity plus a popularity term you did not ask for. Use it deliberately or normalise it away.

There is one situation where dot product is genuinely the right answer: recommendation systems trained with matrix factorisation or two-tower models, where the length of an item vector is a learned popularity or quality signal and you want it in the score. Also, maximum inner product search on already-normalised vectors is exactly cosine and is the fastest option available, since it is a single fused multiply-add chain with no division.

The identity that ties them together

Expand the squared Euclidean distance:

If both vectors are normalised to unit length, the outer terms are both 1, so

Verify with our example. Normalise: $\hat a = [0.6, 0, 0.8]$ and $\hat b = [0, 0.8, 0.6]$. Their difference is $[0.6, -0.8, 0.2]$, whose squared length is $0.36 + 0.64 + 0.04 = 1.04$. And the formula predicts $2 - 2(0.48) = 1.04$. They agree.

The consequence is important and constantly missed: on normalised vectors, ranking by L2 and ranking by cosine produce exactly the same order. If someone tells you their L2 index gives different results from their cosine index on the same normalised data, one of them is not actually normalised.

The same expansion is also the basis of fast batch distance computation, which we return to below.

Metrics borrowed from adjacent fields

Jaccard similarity

For sets rather than continuous vectors: the size of the intersection over the size of the union.

Two product tag sets:

A = {red, cotton, shirt, medium}B = {red, cotton, shirt, large, sale}intersection = {red, cotton, shirt}                        -> 3union        = {red, cotton, shirt, medium, large, sale}    -> 6J(A,B) = 3 / 6 = 0.5

Jaccard is the metric for categorical attributes, tag overlap, user-item interaction sets, shingled documents for near-duplicate detection, and chemical fingerprints (where it goes by the name Tanimoto). It has no notion of "close" — medium and large are as different as medium and aubergine. That is a limitation and sometimes exactly what you want.

At scale, computing Jaccard against every stored set is as hopeless as brute-force vector scan. The standard trick is MinHash: hash each set with k independent hash functions and keep the minimum value under each. The probability that two sets share a minimum under a random hash function is precisely their Jaccard similarity, so the fraction of matching minima across k hashes estimates J with standard error $\sqrt{J(1-J)/k}$, which is at most $0.5/\sqrt{k}$. With k = 128 that is at most about ±0.044, cheap enough to run over billions of sets.

Hamming distance

For binary vectors: the number of positions where the bits differ.

a = 1 0 1 1 0 1 1 0b = 1 0 0 1 0 1 1 1        ^           ^Hamming distance = 2   similarity = 1 - 2/8 = 0.75

This matters far beyond hash tables now, because binary quantisation of embeddings has become a mainstream technique: take the sign of each dimension, store one bit instead of 32. A 768-dimension vector becomes 96 bytes. Hamming distance is then a hardware XOR followed by a population-count instruction — twelve instruction pairs for 768 dimensions, compared with 768 multiply-adds. Binary search over a hundred million vectors becomes plausible on a single machine, and you recover the lost precision by reranking the top few hundred candidates with the original floats.

Pearson correlation

Cosine similarity computed on mean-centred vectors:

The centring is the whole point, and an example makes it vivid. Two users rate three films:
Text
Copy
User X = [5, 4, 3]     (generous rater)User Y = [3, 2, 1]     (harsh rater, same preference order)cosine  = (15 + 8 + 3) / (sqrt(50) * sqrt(14))        = 26 / (7.0711 * 3.7417) = 26 / 26.458 = 0.983means:  X = 4, Y = 2centred: X' = [1, 0, -1], Y' = [1, 0, -1]Pearson = 2 / (sqrt(2) * sqrt(2)) = 2 / 2 = 1.000
Pearson says these two users agree perfectly, which is true — they rank the films identically and only differ in how generous their scale is. Cosine says 0.983, close but conflating agreement with rating-scale offset. In collaborative filtering that offset is pure noise, which is why Pearson is standard there.
The reverse case is starker:
Text
Copy
X = [1, 2, 3]   Y = [3, 2, 1]      (exactly opposite preferences)cosine  = (3 + 4 + 3) / (sqrt(14) * sqrt(14)) = 10 / 14 = 0.714centred: X' = [-1, 0, 1], Y' = [1, 0, -1]Pearson = -2 / 2 = -1.000
Cosine reports 0.714 — "fairly similar" — for two users with perfectly opposed tastes, because both vectors sit in the positive quadrant and the shared positive offset dominates. Pearson correctly reports −1. For neural embeddings, whose components are already roughly zero-centred by training, the difference between the two is small; for rating data and other non-centred features it can flip the sign of your answer.
Choosing the metric
Data / task
Metric
Why
Trap
Text embeddings, semantic search
Cosine (or dot on normalised vectors)
Meaning is in direction; length is document verbosity
Forgetting to normalise one side
Recommendation, two-tower models
Inner product
Vector length encodes learned popularity
Assuming it behaves like cosine
Image / audio feature vectors
L2
Magnitude carries real signal
Unscaled features letting one dimension dominate
Sparse, outlier-heavy numeric features
L1
Robust to a few extreme dimensions
Using it on embeddings trained under L2
Tolerance / anomaly checks
Chebyshev
Cares only about the worst dimension
Almost never right for retrieval
Tags, categories, interaction sets
Jaccard
Set overlap with no ordering assumed
No graded similarity between categories
Binary-quantised embeddings
Hamming
XOR + popcount, extremely fast
Needs full-precision reranking
User ratings, collaborative filtering
Pearson
Removes per-user scale and offset
Undefined when a vector is constant
The metric is not a free choice. Use the one your embedding model was trained under — anything else is measuring a geometry the model never optimised for.
Concretely: the OpenAI embedding endpoints return already-normalised vectors, so cosine, dot and L2 all agree. The BGE and E5 families are trained with a cosine objective and expect explicit normalisation plus their prefixes (for E5, "query: " for searches and "passage: " for documents; for BGE, an instruction on queries only). Sentence-Transformers models each declare their intended similarity function in the model card, and it is worth reading rather than guessing.
Computing these fast at scale
A metric that is correct but slow is a metric you will not ship. Four techniques account for nearly all of the achievable speed-up.
1. Normalise once, at write time
Cosine similarity requires two norms and a division per comparison. Normalise every vector as it enters the index and cosine collapses to a bare dot product forever after. For a query against 1,000,000 documents that removes 1,000,000 square roots and 1,000,000 divisions from the hot path. Store the vectors normalised; if you also need the original magnitude for some other purpose, save it in a separate metadata field.
2. Turn L2 into one matrix multiplication
Use the expansion from earlier. For a batch of queries $Q$ (shape $m \times d$) and documents $X$ (shape $n \times d$), the full distance matrix is
$$D^2 = \lVert Q\rVert^2_{\text{row}} + \lVert X\rVert^2_{\text{row}} - 2\,Q X^{\!\top}$$
The two norm terms are precomputed once and broadcast. Everything expensive is inside a single GEMM, which is the most heavily optimised routine in computing.
Python
Copy
1import numpy as np23def l2_squared_batch(Q, X, x_sq=None):4    """Distances from every query in Q to every vector in X."""5    q_sq = np.einsum('ij,ij->i', Q, Q)[:, None]      # (m, 1)6    if x_sq is None:7        x_sq = np.einsum('ij,ij->i', X, X)           # (n,) - cache this!8    d2 = q_sq + x_sq[None, :] - 2.0 * (Q @ X.T)9    return np.maximum(d2, 0.0)   # clamp tiny negatives from rounding
That np.maximum(d2, 0.0) is not decoration. Floating-point cancellation makes the formula return values like −1.2e−7 for a vector against itself, and passing a negative into np.sqrt gives you nan that then propagates through your whole ranking. This is one of the most common bugs in hand-rolled vector search.
Size the win: 1,000 queries against 1,000,000 documents at 768 dimensions is $2 \times 1000 \times 10^6 \times 768 = 1.54$ TFLOP. A 16-core server sustaining roughly 500 GFLOP/s on GEMM finishes that in about 3 seconds — 3 ms per query. The same computation in a Python loop over rows takes hours.
3. Let SIMD do the inner loop
Modern CPUs process whole vectors of numbers per instruction. An AVX-512 register holds 16 float32 values, so a 768-dimension dot product is 768 / 16 = 48 fused multiply-add instructions rather than 768 scalar ones. With int8 and the VNNI extension a register holds 64 values, cutting it to 12 instructions. This is why the precision you store in has a speed effect and not only a memory effect.
Precision
Bytes per 768-d vector
Relative memory
Typical speed-up
Recall cost (before reranking)
float32
3,072
1.0×
baseline
—
float16
1,536
0.5×
1.5–2×
< 0.5 points
int8 scalar quantisation
768
0.25×
2–4×
1–2 points
Product quantisation, m = 96
96
0.031×
4–10×
5–15 points
Binary (1 bit per dimension)
96
0.031×
10–40×
10–25 points
The bottom two rows are only sane when paired with reranking: pull 200 candidates from the compressed index, rescore them with float32 vectors, return the best 10. The rescoring cost is fixed and tiny — 200 dot products — and it typically recovers most of the lost recall.
4. Stop computing distances you cannot use
The largest speed-up is structural, not arithmetic. An index that examines 2,000 vectors out of ten million has already beaten any amount of SIMD tuning applied to all ten million. Metric choice interacts with this: engines implement different metrics with different levels of optimisation, and some index types only support a subset. Check what your engine actually accelerates before assuming a metric is available at full speed.
Engine
Cosine
Dot
L2
Manhattan
Hamming / Jaccard
FAISS
via normalised IP
Yes
Yes
Yes (Flat, HNSW, IVF-Flat)
Yes (binary indexes)
Qdrant
Yes
Yes
Yes
Yes
via binary quantisation
Pinecone
Yes
Yes
Yes
No
No
Weaviate
Yes
Yes
Yes
Yes
Hamming
Milvus
Yes
Yes
Yes
No
Yes
pgvector
Yes
Yes
Yes
Yes
Yes (bit type)
Where people get this wrong
"Cosine distance is a distance metric." It is not, and this occasionally breaks index structures that assume the triangle inequality. Take three unit vectors in the plane: $a=(1,0)$, $b=(\tfrac{1}{\sqrt2},\tfrac{1}{\sqrt2})$, $c=(0,1)$. Then $1-\cos(a,b) = 1-0.7071 = 0.2929$ and likewise $1-\cos(b,c)=0.2929$, but $1-\cos(a,c) = 1-0 = 1.0$. The two legs sum to 0.5858, which is less than the direct route of 1.0 — the triangle inequality fails. Angular distance, $\arccos(\cos)/\pi$, does satisfy it: 0.25 + 0.25 = 0.5, exactly meeting the direct distance of 0.5.
Comparing raw scores across metrics or models. A cosine of 0.82 and an inner product of 0.82 mean nothing to each other; a cosine of 0.82 from two different embedding models means nothing to each other either. Any threshold you hardcode — "only show results above 0.75" — is bound to one model, one metric and one corpus, and breaks the day any of the three changes. Prefer relative rules: take the top k, or take results within some ratio of the best score.
Reading FAISS L2 output as a distance. IndexFlatL2 returns squared distances. A displayed "distance" of 0.36 is really 0.6. Rankings are unaffected; any threshold or user-facing number you derive from it is wrong by a square.
Assuming high dimensions break everything. There is a real phenomenon — in high dimensions, distances between random points concentrate, so the nearest and farthest neighbour become nearly equidistant and "nearest" stops being meaningful. Take a thousand uniformly random points and one random query: in 2 dimensions the farthest point is over 100 times as far away as the nearest, in 100 dimensions only about 1.4 times, and in 10,000 dimensions about 1.04 times. Yet 1536-dimension embedding search works well. The resolution: trained embeddings do not fill the space uniformly. They lie on a much lower-dimensional manifold — the intrinsic dimensionality of a text embedding set is usually somewhere in the tens — so distances stay well separated. The lesson is not "high dimensions are fine"; it is that the concentration effect returns the moment your vectors become closer to random, which is precisely what happens with a badly trained or badly matched embedding model.
Mean-pooling then forgetting to normalise. When you build a chunk vector by averaging token or sentence vectors, the result has a smaller magnitude than its parts, and chunks built from more pieces shrink more. Under dot product, short chunks then systematically outrank long ones. Normalise after pooling, always.
What to do on Monday morning
Three checks, each of which takes minutes and each of which catches a class of bug that is otherwise invisible:
Assert the norm. After embedding, assert that abs(np.linalg.norm(v) - 1.0) < 1e-4 for every vector written to the index, and assert the same for query vectors. If your model does not return normalised vectors, normalise in the write path and in the query path, in the same function if possible. Nearly every magnitude bug dies here.
Sanity-check the metric with a known pair. Embed a text and a near-paraphrase of it, and embed something unrelated. Confirm the near-paraphrase scores higher, and that the numbers land where you expect for your model (typically above 0.85 for paraphrases, below 0.4 for unrelated). If a document scores 0.99 against everything, your vectors are collapsing; if paraphrases score 0.55, your prefixes or pooling are wrong.
Record the metric in your schema. Store the embedding model name, version, dimension and intended metric as fields on the collection, and have your write path refuse anything that disagrees. The expensive failures in vector search are never exceptions — they are silently wrong rankings that look fine in a demo and get worse in production as your corpus grows.
And when a ranking looks strange, check the metric before you touch the model. Reranking pipelines, prompt tweaks and chunk-size experiments have all been deployed to fix problems that were, in the end, a missing call to normalize.