Part 2 — Embeddings: How AI Understands the World Without Knowing a Single Word

Type "apple" into a grocery app → you get fruit. Type "apple" into a stock app → you get a $3 trillion company. Same four letters. Completely different meanings. How does the AI tell them apart — without actually "reading" the word?

The answer is a mathematical trick called Embeddings — and it's the silent engine powering every AI tool you use today.

In our last article on Part 1 — Tokens, we saw how AI breaks words into numbers. But Token #27438 (" Meta") is just a label. It has no meaning. So how does the AI actually understand meaning? Welcome to the concept that makes modern AI possible: Embeddings and Vectors.

To understand how AI encodes meaning, we first need to understand how numbers can represent similarity.

The Coordinate System

To understand how AI thinks, we have to look at how humans classify the world. Imagine describing a person using only three numbers: Height (cm), Weight (kg), and Age (years).

If we represent three people using this exact format:

• Alex: [180, 75, 25]
• Marcus: [178, 74, 26]
• Sarah: [165, 60, 30]

This is just a 2D slice — in reality, AI works in 384+ dimensions, making these clusters far more precise. But even here, Alex and Marcus land right next to each other, while Sarah sits far away.

The computer didn't need to look at their pictures. It didn't need to interview them. By seeing that their numbers are mathematically close, it deduced that they share similar physical characteristics. This flat list of numbers [180, 75, 25] is called a Vector.

Vectors work — but three dimensions aren't nearly enough to capture the complexity of real-world meaning.

The 3D Limit

Vectors are great, but three numbers can't describe the complexity of the real world. Suppose we represent smart devices in a 3D vector space using [Price ($), Weight (g), Features Count]:

The AI looks at the math: the Ray-Bans and the Xreal glasses cluster close together, while the Galaxy Ring is far away. It has successfully grouped the eyewear together — without ever being told what "glasses" are.

But here is the fatal flaw: What if we need to encode battery life? Camera quality? Multilingual translation capability? We can't capture all of that in just three dimensions. To truly capture the meaning and context of a word or an object, we need more numbers. A lot more.

This is where it gets interesting — instead of humans picking the dimensions, we let the AI discover them automatically.

From Vector to Embedding: The Masterpiece

When humans pick the properties (Height, Weight, Price), it's a basic Vector — subjective and limited. When the AI reads the entire internet and mathematically decides on the properties itself? That's an Embedding.

Instead of 3 properties, the model assigns a dense list of 384 to 3,072 numbers to form a digital fingerprint of meaning:

• Small Models: 384 dimensions
• Medium Models: 768 dimensions
• Large Models: 3,072 dimensions

So "Smart Glasses" might become [0.012, -0.443, 0.891, 0.112 ... 380 more dimensions]. Each of those hundreds of numbers represents a hidden layer of meaning the AI learned from reading the internet. We humans don't even know exactly what dimension #247 represents — it might encode "electronic-ness", while #89 might encode "wearable-ness".

We don't actually know these labels — the AI invented these concepts itself. We only see the numbers. But by analyzing patterns across billions of pages, the model mathematically organizes words so that related concepts align.

How AI Learns Meaning: Contrastive Learning

You might wonder: who decides what dimension #89 means? Nobody. The model learns these dimensions automatically through a process called Contrastive Learning. During training, it's shown millions of sentence pairs with labels:

After billions of such iterations, the model develops internal dimensions that capture meaning — not because anyone defined them, but because that geometry is the only way to satisfy all the constraints simultaneously. The coordinates stabilize, creating a highly organized map of human concepts.

Now we can see what that learned geometry actually looks like across languages.

The Cross-Language Magic: Cosine Similarity

To measure the distance between vectors in high-dimensional space, we use a formula called Cosine Similarity. It measures the angle between two vectors:

• 1.0: Pointing in the exact same direction (identical meaning).
• 0.0: Orthogonal/unrelated.
• -1.0: Pointing in opposite directions.

Here are real similarity scores calculated by a multilingual embedding model:

Notice the last pair: they share zero keywords, yet the similarity is 91% because the semantic intent is identical. This is the superpower of semantic search.

Same meaning = same numbers, in any language. The AI isn't translating "café" to "coffee." It just noticed that in its training data, "café" appears in the exact same contexts, next to the same words, as "coffee" does — so their digital fingerprints end up nearly identical. This works because the model was trained on millions of multilingual sentence pairs where the same idea appears side by side, and the contrastive learning we saw in Step 3 forces those equivalents toward the same point in embedding space.

💡 Quick Cosine Similarity Reference

Now that we understand how meaning is encoded in numbers, let's see why this shattered traditional search.

Why This Changed the Internet: Semantic Search

Having a map of meaning changes how we search. This single idea killed traditional keyword search. Think about the old internet — if you searched an online store for "device for my eyes":

This is exactly how TikTok's algorithm knows what video to show you next, and how Netflix recommends movies. Your watch history is converted into a vector, and the system finds the content whose vectors are nearest to yours in the coordinate space.

Code Example: Running Semantic Search in Python

Are you a developer? You can run a full multilingual embedding model and query it locally on your machine for free. We'll use an open-source model called paraphrase-multilingual-MiniLM-L12-v2 — it supports 50 languages and outputs a 384-dimensional vector.

# Install dependencies first:
# %pip install sentence-transformers scikit-learn

from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity

# Load a lightweight, multilingual embedding model
encoder = SentenceTransformer('paraphrase-multilingual-MiniLM-L12-v2')

# Our catalog of devices
devices = [
    "Ray-Ban Meta Ultra - Smart glasses with camera and translation",
    "Sony LinkBuds Open - Lightweight earbuds with translation",
    "Garmin Fenix 9 - Rugged sports watch for running",
]

# Step 1: Convert catalog sentences into vectors (embeddings)
device_embeddings = encoder.encode(devices)

# Step 2: Encode the user search query
query = "I want something very light for translating that isn't too expensive"
query_embedding = encoder.encode(query)

# Step 3: Compute similarity scores between query and catalog
scores = cosine_similarity([query_embedding], device_embeddings)[0]

# Step 4: Output the ranked results
print(f"Query: \"{query}\"\n")
ranked = scores.argsort()[::-1]
for i in ranked:
    print(f"  {scores[i]:.3f}  {devices[i]}")

Output:

Query: "I want something very light for translating that isn't too expensive"

  0.421  Sony LinkBuds Open - Lightweight earbuds with translation
  0.287  Ray-Ban Meta Ultra - Smart glasses with camera and translation
  0.031  Garmin Fenix 9 - Rugged sports watch for running

The AI ranked the Sony LinkBuds first even though the user never typed "Sony", "earbuds", or "LinkBuds". The model aligned the query to the correct product because of the semantic relationship between light/translating and lightweight/translation — and the Garmin sports watch scored almost zero.

The Heart of RAG

If you are building AI applications today, you are likely building something called RAG (Retrieval-Augmented Generation) — the process of giving an LLM access to your private data (PDFs, databases) so it can answer questions grounded in your facts. Embeddings are the silent engine behind it.

You can't do RAG without embeddings. You can't do modern search without embeddings. The AI does not understand words — it understands the distance between numbers in a high-dimensional space. And that turns out to be a far more powerful way to understand the universe.

Try It in Your Head

If we pass two words to an embedding model:

• "Apple" (the fruit)
• "Apple" (the tech company)

Will their vectors be close or far apart in the embedding space? Think about the context.

Pro Tips for Builders

Common Misconceptions

Key Takeaways

Try It Yourself

Run the code above and try these three experiments to build your intuition:

1. Spanish query test: Change the query to "Quiero algo ligero para traducir que no sea muy caro". Does the Sony LinkBuds still rank first? It should — same meaning, different language.
2. The Apple test: Add two new devices, "Apple fruit - fresh red apple from the orchard" and "Apple MacBook - laptop computer". Then query "I want something to eat" — which Apple lands closer?
3. Swap the model: Replace paraphrase-multilingual-MiniLM-L12-v2 with all-MiniLM-L6-v2 (English only, faster). Run the Spanish query and watch the score collapse — proving the multilingual model is doing real cross-language work, not keyword matching.

Up Next in the Series