What is a vector embedding? Definition and examples

A vector embedding is one fixed-length list of numbers that a model produces for a piece of content, placing it near other content with a similar meaning.

How it works

The glossary entry on embeddings covers the general idea. This one looks at a single vector: its size, how two are compared, and what that means in practice.

Dimensions. "An embedding represents each item in n-dimensional space with n floating-point numbers," Google's Machine Learning Crash Course explains, and for word embeddings the number of dimensions "is often 256, 512, or 1024" (Google). Text models for search often use more: OpenAI's text-embedding-3-small returns 1,536 numbers by default and text-embedding-3-large returns 3,072 (OpenAI). The individual numbers mean nothing on their own; only position relative to other vectors matters.

Comparison. Two vectors are compared with a distance or similarity score. OpenAI recommends cosine similarity, notes that its embeddings are normalized to length 1, and points out that for such vectors a plain dot product gives the same result faster, and cosine similarity and Euclidean distance produce identical rankings.

Size. A model can sometimes return shorter vectors. OpenAI's API accepts a dimensions parameter to reduce the length "without losing its concept-representing properties", which saves storage and speeds up search at some cost in accuracy.

Why it matters for your website

You rarely see vectors directly, but they shape how a site assistant or site search behaves:

• One passage, one vector. A vector summarizes whatever text it was made from. Chunking decides what that is, so a page that mixes pricing, security and hiring in one block produces a muddled vector.
• Same model on both sides. The query and the content must be embedded by the same model. Switching models means re-embedding everything.
• Storage adds up. Stored as 32-bit numbers, a 1,536-number vector takes about 6 KB. A site with 100,000 passages needs roughly 600 MB for vectors alone, before any index.
• Close is not correct. The nearest vector is the most similar text, not necessarily the right answer, which is why systems add reranking and a check that the answer is supported.

Example

A toy version with three dimensions (real vectors are far longer):

import math

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.hypot(*a) * math.hypot(*b))

query   = [0.81, 0.12, 0.30]  # "can I pay by bank transfer?"
wire    = [0.78, 0.15, 0.36]  # "wire transfers accepted on annual plans"
offices = [0.04, 0.93, 0.11]  # "our team is based in Lisbon"

print(round(cosine(query, wire), 2))     # 1.0  -> very similar
print(round(cosine(query, offices), 2))  # 0.22 -> unrelated

The wire-transfer passage scores close to 1 and is retrieved; the office passage scores low and is ignored.

Related terms

• Embeddings: the general concept and its uses.
• Vector database: stores vectors and finds the nearest ones fast.
• Semantic search: search that compares vectors instead of words.
• Retrieval: the step that uses vectors to find passages for an AI answer.
• Chunking: decides what each vector represents.