Which Distance Metric to Use?

📌 Searching the Vector Database

When you type a prompt into a RAG pipeline, the system converts your prompt into a mathematical arrow (Vector) and shoots it into the Vector Database. The database then has to figure out which stored documents are sitting closest to your arrow.

But how does the database measure "closeness"? When you configure a Vector Database (like Pinecone or Milvus), you must choose a specific mathematical formula known as a Distance Metric.

📌 1. Cosine Similarity (The Industry Standard)

Cosine Similarity measures the Angle between two vectors, completely ignoring their length (magnitude).

Why it's the standard for Text: Imagine your search prompt is simply, "I love dogs." Now imagine Document A is a massive 500-page essay where the phrase "I love dogs" is repeated 1,000 times. Because Document A is so long, its mathematical vector (arrow) will be incredibly long. But because it contains the exact same meaning, both arrows will point in the exact same direction.

Cosine Similarity says, "The angle between these arrows is 0, so they are a perfect match." It correctly ignores the fact that one document is much longer than the other.

📌 2. Euclidean Distance (L2)

Euclidean distance measures the physical, straight-line distance between the tips of the two arrows. Imagine taking a physical ruler and measuring the gap.

Because it acts like a ruler, it cares deeply about Magnitude (length). If two arrows point in the exact same direction, but one is much longer than the other, Euclidean distance will say they are very far apart.

When to use it: Engineers rarely use this for text. However, they frequently use it for Image Embeddings, where the length of the vector might represent the physical brightness or color intensity of pixels.

📌 3. Dot Product (The Speed Hack)

Dot Product is a math formula that multiplies the vectors together, taking into account both Angle and Magnitude.

Why is it popular? Because Nvidia GPUs are fundamentally designed to calculate Dot Products incredibly fast. It is much cheaper in compute power than calculating Cosine Similarity.

The Engineering Trick: Because Cosine Similarity is better for text, but Dot Product is faster for GPUs, engineers use a trick. They Normalize their vectors. This means they mathematically stretch or shrink every single arrow in the database until they all have an exact length of 1. Once all the arrows are the exact same length, the math proves that Dot Product and Cosine Similarity yield the exact same ranking results! Thus, engineers get the semantic accuracy of Cosine with the blazing hardware speed of Dot Product.