Embeddings and Semantic Search Explained

The problem with matching words

Ordinary search compares characters. Search a folder of notes for “battery runs out quickly” and a keyword engine looks for those words. It will miss a note titled “short cell life under load”, even though the two describe the same thing. Semantic search tries to match what you mean rather than the exact words you typed, and embeddings are the tool that makes that possible.

What an embedding is

An embedding is a list of numbers, called a vector, that represents a piece of content such as a word, a sentence, a paragraph or an image. A trained model produces it. The key property is that content with similar meaning tends to get vectors that are close together, while unrelated content ends up far apart.

A helpful picture is a map. On a real map, nearby places are close in distance. In an embedding space, texts about similar topics are close in a space with many dimensions, usually hundreds or more rather than two. You cannot visualise that directly, but the same notion of nearness applies.

The numbers themselves are not individually meaningful. You cannot say that the 37th value means “about batteries”. Meaning lives in the overall pattern and, above all, in the distances between vectors.

How the model learns to place text

Embedding models are trained on large amounts of text so that words and phrases appearing in similar contexts get similar representations. The details vary between model families, but the broad idea follows the general process described in how models are trained: parameters are adjusted over many examples until the output vectors reflect useful relationships.

Because the model was built from its training data, it inherits that data’s strengths and gaps. It may handle everyday language well and technical jargon or less common languages poorly. The influence of data collection choices is discussed under training data, and it applies to embeddings just as it does to chat models.

Measuring similarity

To find related items, you compare vectors. Two common measures:

• Cosine similarity: compares the direction of two vectors, ignoring their length. Values closer to one mean the vectors point the same way.
• Euclidean distance: the straight-line distance between the two points. Smaller means closer.

For many embedding models either works, and the model’s documentation usually says which is intended. The important point is that “similar” is a number you can compute and rank, not a yes-or-no answer.

How a semantic search works

A basic system has two phases.

Indexing (done ahead of time):

1. Split your documents into chunks of a sensible size, such as a paragraph or a section.
2. Run each chunk through an embedding model to get a vector.
3. Store each vector together with the original text and some identifying information.

Querying (done each time someone searches):

1. Embed the query with the same model used for indexing.
2. Compare the query vector against the stored vectors.
3. Return the chunks with the highest similarity.

Using the same model for both steps matters. Vectors from different models live in different spaces and cannot be meaningfully compared.

Storing and searching vectors

With a few hundred or a few thousand chunks, you can simply compare the query against every stored vector. That is fast enough on modest hardware, and it keeps the project simple: vectors can sit in a file or in an ordinary database next to the text. This is a case where database choice matters less than it might seem.

At larger scale, comparing against everything becomes slow, so systems use approximate nearest-neighbour methods that find very close matches quickly without checking every entry. These trade a little accuracy for speed. Specialised vector databases and library features exist for this; as with any tool, check current documentation for details rather than assuming how they behave.

Chunking matters more than people expect

How you split text strongly affects results. Chunks that are too small lose context, so a sentence about “it” becomes meaningless. Chunks that are too large blend several topics into one vector, which blurs the match. A common approach is to use moderate-sized sections with a small overlap between neighbours, then test on questions you know the answers to and adjust.

Adding metadata such as a title or section heading to each chunk before embedding often helps, since it gives the model more context about what the chunk covers.

Where this is useful in maker projects

• Searching your own project notes, build logs or saved datasheets by meaning. Pairing it with good habits for reading datasheets means you can ask a question and land near the right page.
• Finding similar items: related support tickets, similar sensor descriptions, or near-duplicate entries.
• Giving a language model relevant material to work from. A system retrieves the closest chunks, then includes them in a prompt so the model answers from your documents. Done carefully, this can reduce unsupported answers, though it does not eliminate model hallucinations, and the generated answer still needs checking.
• Simple classification, by comparing new text to labelled examples.

Limits and failure modes

Semantic search is not magic, and it has characteristic weaknesses.

• Exact terms can be lost. Part numbers, error codes and names may not embed well, because they carry little general meaning. Many systems combine keyword and semantic search to cover both.
• Similar is not the same as correct. The closest match may be topically related but not answer the question. Always allow users to see the source text.
• Negation and fine distinctions can be weak. “Works with 3.3 volts” and “does not work with 3.3 volts” may look close in vector space.
• Results shift with the model. Changing the embedding model means re-embedding everything.
• Privacy: if you use an online embedding service, your text is sent to it, so consider the points in cloud AI privacy before indexing personal or sensitive material.

Testing your search

Build a small set of realistic queries, with the passages you expect to find, and check whether they appear near the top. Review misses by hand. This kind of simple test, much like automated testing for ordinary code, will tell you more about quality than any headline accuracy claim.