☕biscuits in tea
☕ biscuits in teaMathematicsCosine Similarity: When Angle Becomes Meaning↑ Top
Essay

Cosine Similarity: When Angle Becomes Meaning

Normalize the dot product and vector angle becomes a practical measure of similarity.

The dot product mixes length and direction. Cosine similarity deliberately removes the length:

cos_sim⁡(u,v)=u⋅v∥u∥∥v∥=cos⁡θ.\operatorname{cos\_sim}(u,v) = \frac{u\cdot v}{\lVert u\rVert\lVert v\rVert} = \cos\theta.
Several differently sized vectors pointing in nearly the same direction.
Vectors may have very different magnitudes and still have high cosine similarity because their directions agree.

Why normalization matters

Suppose two document vectors encode word counts. A long document may have much larger coordinates than a short one even if they discuss the same subject. Normalizing asks a different question:

Are their patterns similar, regardless of scale?

A query vector with nearby semantic vectors grouped by angle.
In an embedding space, nearest by angle can become a useful proxy for nearest in meaning.

From geometry to language

If an embedding model represents words or documents as vectors, cosine similarity lets us rank neighbours without needing to visualize the high-dimensional space.

But an important warning follows:

Similarity in an embedding is similarity according to the representation learned by that model. It is not an oracle for human meaning.

This is a perfect bridge to Word2Vec: When Words Acquire Geometry.