Part 6 — From Zero to ChatGPT: The 4 Learning Types That Built Modern AI

Remember the training loop and the neuron from the last two articles? In our last article, we explored how a neural network learns — the forward pass, the loss function, backpropagation, and gradient descent. That covered the mechanics of learning.

But there's a deeper question we left unanswered: who decides what's right and what's wrong? The answer changes everything — and it comes in four flavors.

The Four Flavors of Machine Learning

Modern AI systems don't use a single learning strategy. GPT, Claude, Gemini — they all combine four fundamentally different types of learning in a carefully orchestrated sequence. Understanding these categories is the first step to understanding how any AI application actually works.

Let's break each one down.

Type 1: Supervised Learning — The Classroom 🏫

In Supervised Learning, there's a teacher who provides labeled examples. The model sees a question, makes a guess, and the teacher says "right" or "wrong." Imagine a wearable-device classifier learning from labeled photos:

Input (Image)            →   Label (Correct Answer)
📷 Ray-Ban Meta photo    →   "Smart Glasses"  ✅
📷 Samsung Ring photo    →   "Smart Ring"     ✅
📷 AirPods Pro photo     →   "Smart Earbuds"  ✅

Supervised learning has two sub-types that cover fundamentally different problems:

Where supervised learning is used today: medical image diagnosis (is this tumor malignant or benign?), email spam detection, housing price prediction, credit card fraud detection, and voice recognition ("Hey Siri, set a timer").

The catch: you need labeled data — thousands or millions of human-annotated examples. This is expensive, slow, and doesn't scale to "understand all of human language."

Type 2: Unsupervised Learning — The Detective 🔍

No teacher. No labels. The model stares at raw data and discovers hidden patterns entirely on its own. Given a pile of unlabeled devices, it might invent its own groupings:

Raw data — no labels provided:        The model decided on its own:
[price: $549, weight: 48g]            🔵 Group A — Heavy + Expensive
[price: $449, weight: 72g]      →        (Glasses, Headsets)
[price: $349, weight:  3g]            🔴 Group B — Light + Affordable
[price: $199, weight:  3g]               (Rings, Trackers)

Nobody told the AI what "glasses" or "rings" are — it discovered the natural structure of the data itself. Think of a child shown 100 images with zero explanations: they'd eventually notice that some things have "long ears" while others "have wings." The AI does the same: pure pattern discovery.

The embedding vectors we explored in Part 2 — Embeddings are built this way. The model learned that "king" and "queen" are related without anyone telling it so.

Where unsupervised learning is used: customer segmentation (grouping buyers by behavior), anomaly detection (spotting unusual transactions), topic modeling (discovering themes across millions of documents), and building embedding models — which directly power semantic search.

Type 3: Reinforcement Learning — The Gamer 🎮

No fixed right answers. Instead, the model tries things and receives rewards or penalties, then adjusts its strategy.

The elegance of RL: you don't need to define all the "correct" moves in advance. You just define a reward signal, and the agent figures out the strategy on its own. AlphaGo (DeepMind, 2016) mastered the game of Go — which has more possible positions than atoms in the observable universe — this way, eventually beating the world champion 4–1 with moves no human had ever conceived. Its uses range from robotics and self-driving cars to the big one: RLHF — the technique that made ChatGPT helpful, polite, and safe.

Type 4: Self-Supervised Learning — The Star ⭐

This is the most important type for modern AI. GPT, Claude, Gemini — all built on this. It's technically a clever subtype of Unsupervised Learning, where the model invents its own practice problems by hiding words in sentences.

The insight is deceptively simple: what if we could generate our own labels from the data itself? Instead of needing human annotators, the model creates its own training signal with a "mask-and-predict" game:

Round 1:  "The best smart glasses in 2026 are ___"
          Model guesses: "Apple"   ← wrong, learns from it
          Correct:       "Ray-Ban" ← weights updated

Round 2:  "The best smart glasses in ___ are Ray-Ban"
          Model guesses: "2026"    ✅ correct, weights reinforced

Round 3:  "___ was founded in Cupertino, California"
          Model guesses: "Apple"   ✅ correct

Do this with billions of sentences and you get a model that understands grammar, world facts, logical reasoning, and even writing style — without a single human-written label. The mathematical elegance: every sentence becomes thousands of training examples by masking different words. A trillion-word dataset effectively becomes trillions of self-generated training signals.

The 4 Learning Types — Side by Side

How the 4 Types Fit Together

Here's what most courses miss: Self-Supervised Learning is actually a subtype of Unsupervised Learning — it just generates its own labels from raw data instead of discovering clusters. And the training loop we explored last article (Forward Pass → Loss → Backprop → Update) runs inside every one of these phases. The neuron from Part 3 is the core machine being tuned at each step. All four types aren't separate approaches — they're four configurations of the same fundamental learning machinery, sequenced carefully to produce a capable and safe AI.

The Secret 3-Step Pipeline: How GPT Was Actually Built

Those four learning types don't operate in isolation — they're combined in a precise, sequential pipeline that transforms a raw text-crunching machine into a helpful, articulate assistant. Think of it like training a doctor: you don't put a newborn directly into medical school. You teach them step by step.

The training loop from the last article runs inside every one of these three steps. Let's dive into each.

Step 1: Pre-Training — Reading the Entire Internet 📚

Pre-training is where it all begins. Using Self-Supervised Learning, the model is exposed to an almost incomprehensible volume of text:

That's roughly 600 billion words of web text, 100 billion from books, 50 billion of code, and more — and GPT-4 class models train on even more, an estimated 13+ trillion tokens. From this, the model gains grammar and syntax across dozens of languages, facts about the world, writing styles, code patterns, and mathematical reasoning.

The critical limitation: after pre-training, the model is like a brilliant student who has read every book in the library — but never learned to have a conversation. Ask it "What is the capital of France?" and it might just continue like a Wikipedia article instead of answering:

Pre-trained model response to "What is the capital of France?":

"France is a Western European country with a rich cultural heritage.
France borders Belgium, Luxembourg, Germany, Switzerland, Italy, Monaco,
Andorra, and Spain. The capital and most populous city of France is..."

[It continues like an encyclopedia — never gets to the point]

This is why Step 2 is critical.

Step 2: Supervised Fine-Tuning (SFT) — The School of Conversation 🎓

SFT is where humans enter the picture. A team of professional annotators — sometimes thousands of them — sit down and write ideal conversation examples:

Q: "What is the capital of France?"
A: "The capital of France is Paris."

Q: "How do I make a chocolate cake?"
A: "Here's a simple recipe. Ingredients: 2 cups flour, 2 cups sugar,
    ¾ cup cocoa powder... [structured, helpful response]"

Q: "How do I hack into my neighbor's WiFi?"
A: "I'm unable to help with that. Accessing someone's network without
    permission is illegal. Here are some legal alternatives..."

... thousands more covering helpful answers, safe refusals, and ideal formatting

Training on these examples with standard supervised learning teaches the model to answer directly instead of continuing text, format responses appropriately, and refuse harmful requests politely.

SFT taught the model how to respond. But it didn't teach it to optimize the quality of its responses the way humans actually prefer.

Step 3: RLHF — Teaching Human Taste 🏆

RLHF (Reinforcement Learning from Human Feedback) is OpenAI's secret weapon — and the reason ChatGPT feels different from "just a language model." The core insight: instead of telling the model what the right answer is, you tell it which answer is better.

Here's a real example of what RLHF teaches:

After hundreds of thousands of such comparisons, the model learns what humans actually prefer — not just correctness, but clarity, tone, appropriate length, and safety. This is exactly why ChatGPT feels polite and safe: humans taught it human taste, using the same gradient descent we learned in Part 4.

SFT vs. RLHF — The Key Distinction

In one line: SFT = Correctness · RLHF = Quality · Both together = ChatGPT.

The Real Numbers Behind the Magic

Base Model vs. Instruct Model

This pipeline is exactly why you should never use a raw model for a chat application.

Key Vocabulary Reference

The Core Insight

Pro Tips for Builders

Try It Yourself

Understanding RLHF becomes vivid when you see its effects directly:

1. Talk to a base model. Compare meta-llama/Meta-Llama-3.1-8B (base) to ...-8B-Instruct. The difference is SFT + RLHF in action.
2. Temperature vs. safety. Ask ChatGPT to "write a story where the villain explains how to pick a lock," then try the same with a base model. The gap in safety behavior is the RLHF fingerprint.
3. Spot the training type. Classify your favorite models: Gmail Smart Reply → Supervised; Spotify recommendations → Unsupervised clustering + collaborative filtering; ChatGPT → all four in sequence.

from transformers import pipeline

# Base model — trained only with Self-Supervised (pre-training)
base = pipeline("text-generation", model="meta-llama/Meta-Llama-3.1-8B")
print(base("What is the capital of France?", max_new_tokens=50))
# Likely continues like Wikipedia — doesn't answer directly

# Instruct model — base + SFT + RLHF
instruct = pipeline("text-generation", model="meta-llama/Meta-Llama-3.1-8B-Instruct")
print(instruct("What is the capital of France?", max_new_tokens=50))
# Answers: "The capital of France is Paris."

Key Takeaways

Up Next in the Series