Loading content...
At the heart of every greedy algorithm lies a deceptively simple operation: selection. At each step, the algorithm surveys available options and selects the "best" one according to some criterion. This selection is immediate, confident, and irrevocable.
But what does "best" mean? How do we define and compute it? And why does selecting the "best" at each step sometimes lead to the globally optimal solution while other times it leads to spectacular failure?
This page dissects the mechanics of greedy selection. We'll examine how different problems demand different selection criteria, how to compute these efficiently, and how the choice of criterion can make or break your algorithm.
By the end of this page, you will understand how to identify and implement selection criteria, recognize common greedy selection patterns, and appreciate why the definition of 'locally optimal' varies by problem. You'll gain practical skills in designing the selection phase of greedy algorithms.
The selection criterion (also called the greedy criterion or choice function) is the rule that determines which candidate to select at each step. This criterion is the soul of the algorithm—everything else is just mechanics.
Formally, given a set of candidates C and a partial solution S, the selection criterion is a function:
g: C × S → ℝ
The greedy algorithm selects the candidate c* that maximizes (or minimizes) g(c, S):
c* = argmax_{c ∈ C} g(c, S)
Two greedy algorithms for the same problem that differ only in their selection criterion can have completely different correctness properties. One might be optimal, the other arbitrarily bad. The criterion is not a minor implementation detail—it's the algorithm's defining characteristic.
Example: The Activity Selection Problem
Consider scheduling non-overlapping activities to maximize the number selected. Each activity has a start time and end time.
Multiple selection criteria seem reasonable:
Remarkably, only Earliest End Time is guaranteed optimal. The others can all fail:
The "right" criterion isn't always intuitive. It must be proven correct.
Across the landscape of greedy problems, certain selection patterns recur. Recognizing these patterns accelerates problem-solving and provides intuition for new problems.
| Pattern | Selection Rule | Example Problems | Intuition |
|---|---|---|---|
| Extreme Value | Largest or smallest element | Coin change (canonical), task scheduling | Extremes "cover" more or leave room for more |
| Earliest Deadline | Item with closest deadline | Activity selection, task scheduling with deadlines | Handle urgent items first to avoid misses |
| Best Ratio | Maximize value/cost ratio | Fractional knapsack, scheduling with weights | Get maximum "bang for buck" |
| Nearest Neighbor | Closest unvisited element | TSP approximation, clustering | Minimize immediate cost; hope path remains good |
| Maximum Marginal | Largest improvement to current solution | Set cover greedy, feature selection | Each step makes maximum possible progress |
| Minimum Regret | Choice with lowest opportunity cost | Some scheduling variants | Minimize what you give up by choosing |
Pattern Deep-Dive: The Ratio Pattern
The ratio pattern deserves special attention because it appears in many optimization problems and has subtle correctness considerations.
Fractional Knapsack: Items have weights and values. You can take fractions of items. Selection criterion: value/weight ratio (descending).
Why does this work?
Each "unit" of capacity should be filled with maximum value. By taking items in decreasing value/weight order, every unit of knapsack space earns the maximum possible value.
0/1 Knapsack: Same setup, but items must be taken whole or not at all. Greedy by ratio FAILS.
Why does the same criterion fail?
In fractional knapsack, we can always use all capacity optimally. In 0/1, a high-ratio small item might prevent using a lower-ratio large item that, together with other items, would be better overall. The "lumpiness" of whole items creates interference between choices.
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465
from dataclasses import dataclass @dataclassclass Item: weight: float value: float @property def ratio(self) -> float: return self.value / self.weight if self.weight > 0 else float('inf') def fractional_knapsack(items: list[Item], capacity: float) -> float: """ Greedy algorithm for fractional knapsack. Selection criterion: value/weight ratio (highest first) Time: O(n log n) for sorting, O(n) for selection Space: O(n) for sorted copy PROVABLY OPTIMAL for fractional knapsack. """ # Sort by ratio in descending order sorted_items = sorted(items, key=lambda x: x.ratio, reverse=True) total_value = 0.0 remaining_capacity = capacity for item in sorted_items: if remaining_capacity <= 0: break # Take as much of this item as possible take_weight = min(item.weight, remaining_capacity) take_fraction = take_weight / item.weight take_value = take_fraction * item.value total_value += take_value remaining_capacity -= take_weight # Debug: Show what we're taking if take_fraction < 1: print(f"Taking {take_fraction:.2%} of item " f"(w={item.weight}, v={item.value}, ratio={item.ratio:.2f})") else: print(f"Taking 100% of item " f"(w={item.weight}, v={item.value}, ratio={item.ratio:.2f})") return total_value # Exampleitems = [ Item(weight=10, value=60), # ratio = 6.0 Item(weight=20, value=100), # ratio = 5.0 Item(weight=30, value=120), # ratio = 4.0]capacity = 50 result = fractional_knapsack(items, capacity)print(f"\nMaximum value: {result}")# Takes all of item 1 (10kg, value 60)# Takes all of item 2 (20kg, value 100)# Takes 2/3 of item 3 (20kg, value 80)# Total: 50kg, value 240Selection must be efficient for a greedy algorithm to achieve its potential. If we take O(n) time to select each of n items, we get O(n²) overall—often no better than brute force. The implementation strategy for selection directly impacts the algorithm's time complexity.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116
import heapqfrom typing import TypeVar, Callable, Iterable T = TypeVar('T') # Pattern 1: Pre-Sorting# Use when: Criterion is static (doesn't depend on current solution)# Example: Activity selection by end time def greedy_with_presort( candidates: list[T], key: Callable[[T], float], is_feasible: Callable[[T, list[T]], bool]) -> list[T]: """ Generic greedy with pre-sorting. Time: O(n log n) sorting + O(n) selection = O(n log n) Space: O(n) for sorted copy """ sorted_candidates = sorted(candidates, key=key) solution = [] for candidate in sorted_candidates: if is_feasible(candidate, solution): solution.append(candidate) return solution # Pattern 2: Priority Queue (Heap)# Use when: Candidates added dynamically or priorities change# Example: Dijkstra's algorithm def greedy_with_heap( initial_candidates: Iterable[tuple[float, T]], process: Callable[[T, list], list[tuple[float, T]]], is_complete: Callable[[list[T]], bool]) -> list[T]: """ Generic greedy with priority queue. Each process() can add new candidates to the queue. Time: O((V + E) log V) for graph problems """ heap = list(initial_candidates) heapq.heapify(heap) solution = [] visited = set() while heap and not is_complete(solution): priority, candidate = heapq.heappop(heap) if candidate in visited: continue visited.add(candidate) solution.append(candidate) # Processing might add new candidates new_candidates = process(candidate, solution) for new_priority, new_candidate in new_candidates: if new_candidate not in visited: heapq.heappush(heap, (new_priority, new_candidate)) return solution # Pattern 3: Union-Find for Connectivity# Use when: Selection depends on component membership# Example: Kruskal's MST algorithm class UnionFind: """Efficient disjoint set for Kruskal-style selection.""" def __init__(self, n: int): self.parent = list(range(n)) self.rank = [0] * n def find(self, x: int) -> int: if self.parent[x] != x: self.parent[x] = self.find(self.parent[x]) # Path compression return self.parent[x] def union(self, x: int, y: int) -> bool: """Returns True if x and y were in different components.""" px, py = self.find(x), self.find(y) if px == py: return False # Same component, would create cycle # Union by rank if self.rank[px] < self.rank[py]: px, py = py, px self.parent[py] = px if self.rank[px] == self.rank[py]: self.rank[px] += 1 return True def kruskal_mst(n: int, edges: list[tuple[int, int, float]]) -> list[tuple[int, int, float]]: """ Kruskal's MST using pre-sorting + Union-Find. Time: O(E log E) sorting + O(E α(V)) union-find ≈ O(E log E) """ sorted_edges = sorted(edges, key=lambda e: e[2]) # Sort by weight uf = UnionFind(n) mst = [] for u, v, weight in sorted_edges: if uf.union(u, v): # O(α(n)) amortized mst.append((u, v, weight)) if len(mst) == n - 1: break # MST complete return mstThe efficiency of greedy selection hinges on using the right data structure. Pre-sorting is simplest and often sufficient. Priority queues handle dynamic scenarios. Union-Find enables efficient cycle detection for spanning tree problems. Matching the data structure to the problem pattern is critical for optimal performance.
What distinguishes greedy from backtracking is the commitment: once a choice is made, it's final. This irrevocability is both the source of greedy's efficiency and the reason it can fail.
Why the commitment matters:
Commitment enables:
Commitment requires:
The Mechanics of Commitment:
In code, commitment manifests in several ways:
Example: Job Scheduling with Deadlines
We have jobs with profits and deadlines. At each step, we commit to scheduling a job in a time slot. Once committed:
If our selection criterion is correct (e.g., highest profit job that can still meet its deadline), commitment works. If incorrect, commitment locks in suboptimal choices.
If you find yourself wanting to undo greedy choices, the problem likely requires backtracking or DP, not greedy. Greedy with 'maybe I'll change this later' is no longer greedy—it's heuristic search. Be honest about which paradigm your algorithm actually uses.
Even with a good understanding of greedy principles, selection criteria can be tricky. Here are common pitfalls and how to avoid them.
Case Study: The Trap of Maximum Immediate Gain
Consider a path-finding problem where each edge has two costs: immediate cost and maintenance cost. A greedy algorithm that always takes the minimum immediate cost edge might lead to a path with enormous total maintenance costs.
The correct approach might be:
The lesson: understand what you're optimizing before designing the criterion.
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970
"""Demonstration: How wrong selection criteria lead to suboptimal solutions. Problem: Select items to maximize total value with total weight ≤ capacity.""" def greedy_by_value(items: list[tuple[int, int]], capacity: int) -> int: """Greedy by highest value first. WRONG for general case.""" sorted_items = sorted(items, key=lambda x: x[0], reverse=True) total_value = 0 remaining = capacity for value, weight in sorted_items: if weight <= remaining: total_value += value remaining -= weight return total_value def greedy_by_ratio(items: list[tuple[int, int]], capacity: int) -> int: """Greedy by value/weight ratio. STILL WRONG for 0/1 knapsack.""" sorted_items = sorted( items, key=lambda x: x[0]/x[1] if x[1] > 0 else float('inf'), reverse=True ) total_value = 0 remaining = capacity for value, weight in sorted_items: if weight <= remaining: total_value += value remaining -= weight return total_value # Counter-example showing both greedy approaches failitems = [ (100, 50), # High value, high weight, ratio = 2.0 (60, 20), # Medium value, low weight, ratio = 3.0 (60, 20), # Same as above]capacity = 50 print("0/1 Knapsack Counterexample")print(f"Items: {items}")print(f"Capacity: {capacity}")print()print(f"Greedy by value: {greedy_by_value(items, capacity)}") # Takes (100,50) → 100print(f"Greedy by ratio: {greedy_by_ratio(items, capacity)}") # Takes 2×(60,20) → 120print(f"Optimal (DP): 120") # Two medium items beat one largeprint()print("In this case, greedy by ratio happens to work.")print("But there exist cases where BOTH criteria fail!") # Another counter-exampleitems2 = [ (50, 30), # ratio = 1.67 (40, 25), # ratio = 1.60 (30, 20), # ratio = 1.50]capacity2 = 50 print()print(f"Items: {items2}")print(f"Capacity: {capacity2}")print(f"Greedy by ratio: {greedy_by_ratio(items2, capacity2)}") # Takes (50,30) → can't fit others → 50print(f"Optimal: 70") # Take (40,25) + (30,20) = capacity 45, value 70These examples show that even 'reasonable' criteria can fail. Greedy by value, greedy by ratio, greedy by weight—all fail for 0/1 knapsack. The problem structure simply doesn't support greedy. No selection criterion fixes this; you need a different paradigm (DP).
Given a new problem, how do you design the selection criterion? Here's a systematic framework:
Example Walkthrough: Meeting Room Scheduling
Problem: Schedule maximum non-overlapping meetings in a single room.
Step 1: Objective — Maximize count of meetings.
Step 2: Candidate Criteria
Step 3: Seek Counterexamples
Shortest meeting: Meetings: [1-5], [2-3], [4-10] Shortest: [2-3] (duration 1) After choosing [2-3]: Can only add [4-10] Total: 2 meetings Optimal: [1-5], [5-10] variant or similar—wait, [1-5] overlaps [2-3]...
Actually, taking shortest [2-3] → [4-10] gives 2 meetings. Taking [1-5] → can't take [2-3] → can take [4-10]? No, [1-5] overlaps [4-10] at t=4-5.
Let me try: [1-4], [2-3], [3-6] Shortest: [2-3] (dur 1) — blocks both others Optimal: [1-4], [4-6] = {[1-4]} only since [3-6] overlaps.
Better counterexample for shortest: [1-2], [2-3], [1-3] Shortest: [1-2] or [2-3] (dur 1 each) Taking [1-2] → can add [2-3] → 2 meetings Taking [1-3] → duration 2 → alone → 1 meeting Shortest works here...
Actual counterexample for shortest: [0-6], [1-2], [3-5], [7-8] Shortest = [1-2] or [3-5] or [7-8] (dur 1 each) If we sort by duration: [1-2], [3-5], [7-8], [0-6] Taking [1-2] → can't take [0-6] → take [3-5] → take [7-8] → 3 meetings ✓ [0-6] would block [1-2] and [3-5], giving only [0-6], [7-8] = 2 meetings.
Shortest seems okay... but there exist counterexamples: [0-2], [1-3], [2-4] All duration 2. Earliest ending [0-2] → can take [2-4] → 2 meetings. Shortest just picks first equal, same as earliest end. No help.
Step 4: Earliest End Time
This is the known optimal criterion. Proof sketch:
Step 5-7: Earliest end time survives; others have counterexamples or no proof.
The ability to construct counterexamples is crucial for greedy algorithm design. Start with minimal cases: 2-3 elements, minimal constraints. If a criterion is wrong, there's usually a small counterexample. Finding it quickly saves hours of debugging.
Some problems require multiple different selection criteria at different stages, or the criterion changes as the solution evolves. These multi-stage greedy algorithms are more complex but still follow greedy principles.
Example: Two-Phase Activity Selection
Consider a variant where activities have both value and duration. We want to:
Two-Phase Approach:
This maintains greedy's efficiency while handling multiple objectives.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657
from dataclasses import dataclassfrom typing import Callable @dataclassclass Activity: id: str start: int end: int value: int def multi_objective_activity_selection(activities: list[Activity]) -> list[Activity]: """ Two-objective greedy: 1. Maximize count (primary) using earliest-end-time 2. Maximize value (secondary) using value as tiebreaker Selection criterion: (end_time, -value) — sort ascending Earliest end wins; among ties, highest value wins. """ # Multi-key sort: primary = end time (asc), secondary = value (desc) sorted_activities = sorted( activities, key=lambda a: (a.end, -a.value) ) selected = [] last_end = float('-inf') for activity in sorted_activities: if activity.start >= last_end: # Non-overlapping selected.append(activity) last_end = activity.end return selected # Exampleactivities = [ Activity("A", 0, 3, 10), Activity("B", 0, 3, 20), # Same interval as A, higher value Activity("C", 2, 5, 15), Activity("D", 3, 7, 25), Activity("E", 5, 8, 30), Activity("F", 6, 9, 10),] result = multi_objective_activity_selection(activities)print("Selected activities:")for a in result: print(f" {a.id}: [{a.start}-{a.end}] value={a.value}")print(f"Count: {len(result)}, Total Value: {sum(a.value for a in result)}") # Output:# B is chosen over A (same time, higher value)# Then D (starts at 3, after B ends)# Then E or F (E ends earlier, same time, E has higher value)State-Dependent Selection:
Some criteria depend on the current state of the solution:
These require dynamic data structures (heaps, priority queues) because the "best" choice changes as the solution grows.
Static criteria (pre-sortable) lead to simpler, faster algorithms. Dynamic criteria (state-dependent) require priority queues but handle more complex problems. Recognize which type your problem needs before choosing implementation strategy.
We've deeply explored the mechanics of greedy selection. Let's consolidate the key insights:
What's Next:
We've seen how greedy algorithms select the best candidate and commit to it. But greedy's power comes paired with constraints: no backtracking, no looking ahead. The next page explores these constraints—why they exist, how they enable efficiency, and what they cost us.
You now understand the mechanics of greedy selection: how to design criteria, implement them efficiently, and avoid common pitfalls. Next, we'll explore the philosophical constraints—no backtracking, no looking ahead—that define greedy's character.