Loading learning content...
Machine learning approaches are categorized by the type of feedback available during training. This fundamental distinction determines which algorithms apply, what problems can be solved, and how models learn. The three primary paradigms are Supervised Learning, Unsupervised Learning, and Reinforcement Learning.
| Aspect | Supervised | Unsupervised | Reinforcement |
|---|---|---|---|
| Training Signal | Labeled examples (X→Y) | Unlabeled data (X only) | Rewards/penalties |
| Goal | Predict labels for new data | Discover hidden structure | Maximize cumulative reward |
| Feedback | Explicit (correct answer) | None (find patterns) | Delayed (action outcomes) |
| Example | Spam detection | Customer segmentation | Game playing AI |
Supervised learning is the most common and well-understood ML paradigm. The algorithm learns from labeled examples—input-output pairs where both the input features (X) and the correct output (Y) are provided. The goal is to learn a mapping function f(X) → Y that can predict outputs for new inputs.
The name comes from the idea of a 'supervisor' providing the correct answers during training. Like a teacher grading homework, each training example shows the algorithm what the right answer should be. The algorithm adjusts its internal parameters to minimize the difference between its predictions and the correct labels.
Classification predicts which category an input belongs to.
Types:
Common Algorithms:
Metrics:
Real-World Applications:
Training Data:
• House A: 2000 sqft, 3 bed, 2 bath → $450,000
• House B: 1500 sqft, 2 bed, 1 bath → $320,000
• House C: 3000 sqft, 4 bed, 3 bath → $680,000
• ... (10,000 more examples)Learned Model: price ≈ 150 × sqft + 20,000 × bedrooms + 15,000 × bathrooms + ...
New Prediction: House D (2500 sqft, 3 bed, 2 bath) → $525,000The algorithm found the relationship between features and prices by analyzing thousands of examples. It can now estimate prices for houses it's never seen, generalizing from the training data.
Unsupervised learning discovers patterns in data without labeled examples. The algorithm receives only input data (X) with no corresponding outputs. It must find meaningful structure on its own—grouping similar items, reducing dimensionality, or uncovering hidden patterns.
Without labels, there's no 'teacher' telling the algorithm what's right or wrong. It's like giving someone a pile of photos and asking them to organize it without any instructions—they might group by color, subject, time period, or some other structure they discover.
Clustering groups similar data points together.
Common Algorithms:
Applications:
Key Challenge: How many clusters? This is often domain-dependent and requires techniques like the elbow method or silhouette analysis.
Reinforcement Learning (RL) is fundamentally different from supervised and unsupervised learning. An agent learns by interacting with an environment, receiving rewards or penalties based on its actions. The goal is to learn a policy—a mapping from states to actions—that maximizes cumulative reward over time.
| Component | Description | Example (Chess AI) |
|---|---|---|
| Agent | The learner/decision-maker | The chess-playing program |
| Environment | What the agent interacts with | The chess board and opponent |
| State | Current situation | Current board position |
| Action | What the agent can do | Move a piece |
| Reward | Feedback signal | +1 for win, -1 for loss, 0 otherwise |
| Policy | Action selection strategy | Which move to make in each position |
• AlphaGo: Defeated world champion Go player using RL + deep learning • OpenAI Five: Beat professional Dota 2 teams • DeepMind's AlphaFold: Solved protein folding (partial RL) • Robotics: Teaching robots to walk, grasp objects • Game AI: Atari games from raw pixels • Autonomous driving: Decision making in complex traffic
Modern machine learning has evolved beyond the classic three-way division. Several important paradigms blend or extend these approaches:
| Paradigm | Description | When to Use |
|---|---|---|
| Semi-Supervised Learning | Small amount of labeled data + large amount of unlabeled data | Labels are expensive to obtain; unlabeled data is abundant |
| Self-Supervised Learning | Create labels from data structure itself | Massive unlabeled datasets (e.g., predicting next word in text) |
| Transfer Learning | Apply knowledge from one task to another | Limited data for target task; related source task has more data |
| Active Learning | Algorithm requests labels for specific examples | Labeling is costly; want to minimize labeling effort |
| Meta-Learning | Learning to learn; adapting quickly to new tasks | Many related tasks; need rapid adaptation |
| Multi-Task Learning | Learn multiple related tasks simultaneously | Tasks share common features; can benefit from shared representations |
| Federated Learning | Train on decentralized data without sharing it | Privacy-sensitive data; data cannot leave devices |
Large language models like GPT and BERT use self-supervised learning. They predict masked words or next words in text—a task that requires no human labeling. This enables training on trillions of words from the internet, leading to remarkable language understanding. Similarly, contrastive learning in vision (SimCLR, CLIP) learns by comparing augmented views of images.
You now understand the three core learning paradigms—supervised, unsupervised, and reinforcement learning—plus the emerging approaches that extend them. Next, we'll explore the historical perspective: how ML evolved from theoretical ideas to today's transformative technology.