Embeddings

Embeddings are dense numerical vectors that represent the meaning of text (or other data) in a high-dimensional space. They allow machines to measure semantic similarity between pieces of content -- two sentences with similar meanings will have vectors that are close together, even if they use completely different words.

How It Works

An embedding model takes a piece of text -- a word, sentence, paragraph, or entire document -- and maps it to a fixed-length array of floating-point numbers (typically 768 to 3,072 dimensions). This mapping is learned during training so that semantically related inputs cluster together in vector space.

To compare two pieces of text, you compute their embeddings and measure the distance between them using metrics like cosine similarity. A high similarity score means the texts are semantically related, while a low score indicates they cover different topics or convey different meanings.

Why It Matters

Embeddings unlock powerful capabilities that pure text matching cannot achieve. They power semantic search (finding documents by meaning, not just keywords), recommendation systems, clustering, anomaly detection, and classification. They are also a foundational building block of retrieval-augmented generation (RAG), where relevant documents are fetched based on embedding similarity before being fed to an LLM.

In Practice

In an AI assistant deployment, embeddings are commonly used to build a knowledge base. Documents, code files, or FAQs are pre-processed into embeddings and stored in a vector database. When a user asks a question, the query is embedded and matched against stored vectors to retrieve the most relevant context. This retrieved context is then injected into the LLM's prompt, enabling the assistant to answer questions grounded in your specific data rather than relying solely on its training knowledge.