Embeddings: The Math of Meaning

How does a computer know that "Dog" is more similar to "Puppy" than to "Car"? It uses Embeddings. An embedding is an array of numbers (a vector) that represents the "Meaning" of a piece of text.

1. Vector Space

Modern embedding models (like text-embedding-3-small) convert text into 1,536 dimensions. Words with similar meanings are physically "Close" to each other in this 1,536-dimensional space.

2. Cosine Similarity

To find the most relevant documents for a user's question, we:

1. Convert the Question into a vector.
2. Convert our Documents into vectors.
3. Use math (**Cosine Similarity**) to find the documents whose vectors are most 'Aligned' with the question vector.

4. Interview Mastery

Q: "What is an 'Embedding Drift'?"

Architect Answer: "Embedding drift occurs when you change your embedding model (e.g., from OpenAI to local Llama) but don't re-index your database. Since each model has its own unique 'Map' of dimensions, a vector from Model A cannot be compared to a vector from Model B. As an architect, you must plan for a full database re-indexing whenever you upgrade your embedding model."