An embedding is a numerical vector representation produced by a model for an item such as text, an image, audio, a user or a product. The representation places items in a learned space where a chosen distance or similarity measure can support comparison. Text embeddings can represent a word, sentence, query or document chunk and are commonly used for semantic retrieval, clustering, classification and recommendation. An embedding is not a human-readable meaning, a fact store or a unique truth. Its behavior depends on the model, input preparation, task, language, similarity function and data used to evaluate it.
Teams can build a convincing semantic-search demo while retrieving the wrong evidence in production. A long document may be reduced to a vector that hides a critical clause; chunks may split a table from its heading; short queries may use language unlike the corpus. Similarity can return conceptually related content that does not answer the question. Model upgrades can shift every vector and invalidate comparisons with an existing index. Without a task-specific evaluation set, top-k accuracy, latency and storage numbers say little about whether the retrieved material supports the product decision.
Treat embeddings as a learned retrieval component with a measurable contract. Define the unit being represented, preserve the original source and metadata, select a model using representative languages and document types, and evaluate the complete retrieval path. Similarity should nominate candidates, not establish truth or permission. Apply authorization before exposing content, use reranking or deterministic filters where they improve precision, and show source evidence to downstream users or models. Version embeddings together with model, preprocessing, chunking and index.
Representation
Vector proximity is a model-specific signal of similarity
Early neural word-vector work demonstrated that learned continuous representations can capture useful linguistic regularities. Modern embedding models may represent longer passages and multiple modalities, but their dimensions are not named semantic coordinates that teams can interpret independently. The same text embedded by two models normally lives in different spaces. Even with one model, prefixes, truncation, pooling and normalization can affect the resulting comparison.
Cosine similarity, dot product and Euclidean distance are common comparison functions, but the correct choice depends on how the model was trained and served. Higher similarity does not prove equivalence, factual support or business relevance. A good retriever combines the vector signal with source metadata, keyword or structural signals, filters and sometimes a reranker. Product acceptance should be based on retrieved examples and downstream outcomes rather than attractive two-dimensional plots.
| Layer | Design question | Failure to test |
|---|---|---|
| Input | What context enters the model? | Ambiguous representation |
| Model | What languages and tasks fit? | Uneven semantic quality |
| Comparison | Which score is valid? | Misranked candidates |
| Metadata | What must be filtered? | Wrong scope or access |
| Evaluation | What counts as relevant? | Demo quality without evidence |
Retrieval operation
Chunking and evaluation matter as much as the embedding model
Chunk boundaries define what the retriever can return. Fixed character windows are simple but can split obligations and their exceptions. Structural chunking preserves sections but may create units too large for precise ranking. Overlap can recover context while increasing index size and duplicate results. Evaluate several designs on the actual corpus and keep the parent hierarchy so the product can expand context after finding a precise passage.
The original retrieval-augmented generation work combined a generator with retrieved external memory, illustrating why retrieval changes what a language system can use without placing all knowledge in its parameters. In production, preserve the distinction between candidate retrieval and answer evidence. Validate citations against the source text, monitor which relevant items were not retrieved and retain deletion and re-index controls. The vector is disposable derived data; the governed source remains authoritative.
- Design chunks around the task and document structure.
- Keep source identifiers and permission metadata.
- Evaluate hard negatives and multilingual queries.
- Never interpret similarity as factual proof.
- Re-index as a versioned migration.
What good looks like
Useful outcomes from AI embeddings
- Every vector can be traced to its source item, version and access metadata.
- Query and document representations are compatible with the selected model.
- Retrieval quality is measured on real tasks and difficult negative examples.
- Language, document type and source segments reveal uneven performance.
- Model and chunking changes trigger controlled re-embedding and regression.
- Downstream answers retain citations to original content rather than vectors.
Operating model
How to run the work
- 01
Define the retrieval task
Describe the user question, corpus, relevance judgment and downstream consequence. Decide whether the unit should be a clause, paragraph, section, page, image region, product or another object rather than starting from a model.
- 02
Design representation and metadata
Select preprocessing, chunk boundaries, context enrichment, language handling and embedding model. Store stable source identifiers, version, permissions and fields needed for filtering, citation and deletion.
- 03
Build an evaluation set
Collect representative queries with human relevance judgments, hard negatives, ambiguous wording and multilingual cases. Measure candidate recall and ranking quality at the depth the product actually consumes.
- 04
Release and monitor the system
Index a versioned corpus, test authorization and citation paths, and observe live queries, zero-result cases, corrections and latency. Re-evaluate before changing model, distance, chunking, filters or reranker.
Evaluation
Questions that change the decision
- What object and context should one embedding represent?
- Does the model support the languages and content types in production?
- Which similarity measure and normalization match the model guidance?
- What metadata must filter candidates before semantic ranking?
- How many candidates should retrieval expose to reranking or generation?
- What migration plan is required when the embedding model changes?
Failure modes
Where teams lose control
Chunks lose headings, table context or document hierarchy.
Nearest content is related to the topic but does not support the answer.
Queries and documents use incompatible models or preprocessing.
Permission filtering occurs after restricted text reaches model context.
Old and new embedding spaces are mixed in the same comparison.
An aggregate benchmark masks weak retrieval in an important language.
Measurement
Measure the finished job
Measure the completed workflow, including review effort and exceptions. Output volume on its own is not evidence of a better process.
- recall and precision at the candidate depth used by the product
- mean reciprocal rank or normalized discounted cumulative gain where appropriate
- retrieval quality by language, source and document type
- unsupported downstream answers attributable to missing evidence
- index coverage, freshness and orphaned source records
- query latency, cost and re-embedding time by corpus version
Questions
Common questions
What is an embedding in AI?
It is a numerical vector produced by a model to represent an item such as text or an image. Distances or similarities between vectors can support retrieval, grouping, classification and recommendation.
Are embeddings the same as a vector database?
No. The embedding is the representation. A vector database or index stores and searches representations, often with metadata filters. The source content, embedding model and retrieval logic remain separate components.
Do similar embeddings mean two texts say the same thing?
Not necessarily. Similarity is a model-specific ranking signal. Texts can share a topic while disagreeing, and a relevant-looking passage may not support an answer. Inspect and evaluate the original content.
What happens when the embedding model changes?
The vector space generally changes, so old and new vectors should not be assumed comparable. Plan a versioned re-embedding and index migration, then rerun retrieval regression before switching production traffic.
Sources
Primary references
- Efficient Estimation of Word Representations in Vector Space Mikolov et al.
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks Lewis et al., NeurIPS 2020
About the author
Tony Kim
Tony writes about applied AI, dependable product engineering and the systems that turn complex response work into controlled delivery.
Zeke
AI product engineering for moving a software brief into a reliable production product.
Product leaders, founders and engineering teams. Start with the workflow, constraints and evidence you already have.
See Zeke→