Fallback Patterns
Feature Degradation
Read aloud is unavailable in this browser.
The Art of Knowing What to Sacrifice
On Black Friday 2021, a major e-commerce platform faced unprecedented traffic—3x their previous peak. Their monitoring showed system resources approaching critical thresholds. Rather than wait for cascading failures, the engineering team made a deliberate decision: disable personalized recommendations, switch from real-time to polling-based inventory updates, reduce image quality, and disable the product comparison feature.
The result? The core shopping experience—browse, add to cart, checkout—remained fully functional throughout the traffic surge. Revenue was protected because engineers knew exactly which features could be sacrificed and had the tools to do so instantly.
This is feature degradation: the intentional, controlled reduction of system functionality to protect core operations during stress or failure conditions.
This page provides comprehensive coverage of feature degradation strategies—from identifying which features can be degraded to implementing the technical mechanisms that enable rapid, safe degradation. You'll learn how to classify features by criticality, implement feature flags for degradation, design load shedding strategies, and build operational playbooks for degradation decisions.
Understanding Feature Degradation
Feature degradation is the deliberate simplification or disabling of system features to reduce resource consumption and protect core functionality. Unlike cache fallbacks (which provide substitute data) or default responses (which provide predetermined values), feature degradation removes or simplifies functionality entirely.
The core philosophy:
Modern systems have features of varying importance. During stress, resources should flow to the most important features. Feature degradation explicitly prioritizes—deciding which features matter most and which can be temporarily sacrificed.
Types of feature degradation:
| Type | Description | Example |
|---|---|---|
| Full Disabling | Feature completely turned off | Disable product recommendations entirely |
| Simplification | Feature works but with reduced complexity | Show top 3 recommendations instead of 12 |
| Async Conversion | Real-time feature becomes eventual | Switch from live chat to 'leave a message' |
| Static Substitution | Dynamic feature replaced with static content | Replace personalized banner with global promotion |
| Rate Limiting | Feature available to fewer requests | Expensive search available to 10% of users |
| Quality Reduction | Feature works but at lower quality | Serve smaller images, disable animations |
When to apply feature degradation:
-
Proactive degradation: Applied before problems occur, based on anticipated load or known issues. Example: Disabling heavy features before a marketing campaign.
-
Reactive degradation: Applied in response to observed system stress. Example: Degrading features when CPU or latency crosses thresholds.
-
Cascade prevention: Applied when downstream dependencies fail. Example: Disabling features that depend on a failing service.
-
Incident response: Applied during active incidents to reduce impact scope. Example: Disabling a feature suspected of causing issues.
Feature degradation is proactive—you decide to reduce functionality before failure occurs. Graceful failure is reactive—the system automatically handles failures when they occur. Both are needed: degradation prevents failures; graceful failure handles failures that occur despite prevention.
Feature Criticality Classification
Effective feature degradation requires knowing which features can be degraded and in what order. This requires explicit feature criticality classification—a systematic ranking of features by their importance to business objectives.
Classification framework:
- •Tier 0: Life-Safety Critical — Features where failure could cause physical harm. Healthcare systems, industrial controls, emergency services. Never degraded under any circumstances.
- •Tier 1: Business Critical — Features that directly generate revenue or are legally required. Purchase flows, payment processing, authentication. Degraded only as last resort.
- •Tier 2: User Experience Important — Features that significantly impact user satisfaction and retention. Search, navigation, account management. Can be simplified but should remain functional.
- •Tier 3: Enhancement Features — Features that improve experience but aren't essential. Personalization, recommendations, advanced filters. Can be disabled during stress.
- •Tier 4: Nice-to-Have Features — Features that add polish. Animations, tooltips, social sharing, analytics tracking. First to be disabled, often proactively.
Classification process:
To classify features, ask these questions for each feature:
- Revenue impact: What's the revenue loss if this feature is disabled for 1 hour? 1 day?
- User task completion: Can users complete their primary goals without this feature?
- Regulatory requirements: Are there legal or compliance requirements for this feature?
- Competitive differentiation: Is this feature a key reason users choose your product?
- Dependency footprint: How many other features depend on this one?
Score features on these dimensions and stack-rank them. The resulting order is your degradation priority—Tier 4 features are disabled first, Tier 1 features are protected longest.
A Tier 4 feature may become Tier 1 if its failure crashes a Tier 1 page. Map feature dependencies carefully. The 'show social share count' badge might be Tier 4, but if its API failure causes the product page to error, it's effectively Tier 1.
| Feature | Tier | Degradation Strategy | Order |
|---|---|---|---|
| Checkout/Payment | Tier 1 | Never degrade | Protected |
| Product Search | Tier 2 | Simplify (basic search only) | Last to degrade |
| Personalized Recommendations | Tier 3 | Switch to trending items | Early degradation |
| Product Comparison | Tier 4 | Disable completely | First to disable |
| Social Share Buttons | Tier 4 | Disable completely | First to disable |
Feature Flags for Degradation
Feature flags are the technical foundation for feature degradation. They enable instant, deployment-free control over feature availability. While feature flags have many uses (A/B testing, gradual rollout), their use for degradation requires specific design considerations.
Degradation-specific feature flag types:
- •Kill switches — Binary on/off flags for complete feature disabling. Simple but coarse. Example:
recommendations.enabled = false - •Degradation levels — Multi-value flags representing degradation states. Example:
search.mode = 'full' | 'simple' | 'disabled' - •Percentage rollout — Feature available to percentage of traffic. Useful for load shedding. Example:
advanced_filters.percentage = 25 - •Conditional flags — Flag value depends on runtime conditions. Example:
real_time_updates.enabled_if = cpu_usage < 80 - •User segment flags — Different behavior for different user groups. Example:
heavy_features.enabled_for = 'premium_users'
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556
// Feature flag schema optimized for degradation controlinterface DegradationFlag { // Identifier name: string; feature: string; // Current state enabled: boolean; // Degradation level (for multi-level flags) level?: 'full' | 'reduced' | 'minimal' | 'disabled'; // Percentage of traffic to receive degraded experience degradationPercentage?: number; // Conditions that auto-trigger degradation autoTriggers?: { cpuThreshold?: number; // Degrade when CPU exceeds latencyThreshold?: number; // Degrade when P99 latency exceeds errorRateThreshold?: number; // Degrade when error rate exceeds dependencyHealth?: string[]; // Degrade when these deps are unhealthy }; // Who/what can control this flag permissions: { autoModify: boolean; // Can system auto-modify manualModify: string[]; // Roles that can manually modify requiresApproval: boolean; // Requires second person approval }; // Audit trail lastModified: Date; lastModifiedBy: string; modificationReason: string;} // Example usageconst recommendationFlag: DegradationFlag = { name: 'product_recommendations', feature: 'recommendations', enabled: true, level: 'full', autoTriggers: { cpuThreshold: 85, latencyThreshold: 500, dependencyHealth: ['ml-model-service', 'user-preference-service'] }, permissions: { autoModify: true, manualModify: ['oncall', 'platform-eng'], requiresApproval: false }, lastModified: new Date(), lastModifiedBy: 'system', modificationReason: 'Initial configuration'};Feature flag infrastructure requirements:
For degradation use cases, feature flag infrastructure must be:
-
Highly available — If the flag system is down, you can't degrade. Flag systems must be more reliable than the features they control.
-
Low latency — Flag evaluation happens on every request. Slow flag checks add latency. Target < 1ms evaluation time.
-
Locally cached — Services should cache flags locally with fast refresh. Don't call the flag service on every request.
-
Fail-safe — If flag service is unreachable, use cached values or safe defaults (often 'degraded').
-
Observable — Every flag change must be logged. You need to correlate flag changes with system behavior.
-
Fast to propagate — Flag changes should propagate to all instances within seconds, not minutes.
Feature flags for degradation should be long-lived and maintained, unlike release flags that should be removed after rollout. Document each degradation flag: what it controls, what degraded behavior looks like, when to use it, and how to verify it worked.
Load Shedding Strategies
Load shedding is a specific form of feature degradation focused on reducing the volume of work the system processes. When load exceeds capacity, load shedding deliberately drops or deprioritizes work to keep the system functional.
Load shedding principles:
- Shed low-priority work first — Use request/feature classification to determine what to shed
- Shed early, shed gracefully — It's better to reject 10% of traffic cleanly than let 100% degrade uncontrollably
- Communicate clearly — Rejected requests should receive helpful error messages, not cryptic failures
- Recover automatically — When load decreases, stop shedding without manual intervention
- •Request priority shedding — Classify requests by priority; shed lowest priority first. API requests might be higher priority than background jobs. Authenticated users higher than anonymous.
- •Feature-based shedding — Disable specific features to reduce per-request resource consumption. Instead of serving fewer requests, serve all requests but with less expensive features.
- •Random shedding — Reject random percentage of requests. Fair but doesn't prioritize. Useful when no other classification is available.
- •Client-based shedding — Different treatment for different clients. Internal services might be prioritized over third-party integrations. Premium customers over free tier.
- •Queue-based shedding — Bound queue sizes; reject new work when queues are full. Prevents unbounded latency growth during overload.
- •Time-based shedding — Reject requests that have already waited too long. If a request has been queued for 5 seconds, the client likely already timed out.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566
class LoadShedder { constructor( private metrics: MetricsClient, private config: LoadShedderConfig ) {} shouldAccept(request: IncomingRequest): SheddingDecision { const systemLoad = this.metrics.getCurrentLoad(); const requestPriority = this.classifyRequest(request); // Normal operation - accept everything if (systemLoad < this.config.normalThreshold) { return { accept: true, reason: 'normal_operation' }; } // Elevated load - shed lowest priority if (systemLoad < this.config.elevatedThreshold) { if (requestPriority === 'low') { return { accept: false, reason: 'shedding_low_priority', retryAfter: 30 }; } return { accept: true, reason: 'elevated_priority_pass' }; } // High load - shed low and medium priority if (systemLoad < this.config.criticalThreshold) { if (requestPriority === 'low' || requestPriority === 'medium') { return { accept: false, reason: 'shedding_medium_priority', retryAfter: 60 }; } return { accept: true, reason: 'high_load_priority_pass' }; } // Critical load - only accept highest priority if (requestPriority !== 'critical') { return { accept: false, reason: 'critical_load_shedding', retryAfter: 120 }; } return { accept: true, reason: 'critical_priority_pass' }; } private classifyRequest(request: IncomingRequest): 'critical' | 'high' | 'medium' | 'low' { // Critical: Health checks, payment completions if (request.path.startsWith('/health') || request.path.includes('/payment/complete')) { return 'critical'; } // High: Authenticated user core actions if (request.authenticated && (request.path.includes('/cart') || request.path.includes('/checkout'))) { return 'high'; } // Medium: Authenticated user browsing if (request.authenticated) { return 'medium'; } // Low: Anonymous browsing, bots return 'low'; }} interface SheddingDecision { accept: boolean; reason: string; retryAfter?: number; // Seconds before client should retry}When shedding load, return appropriate HTTP headers (429 with Retry-After) so clients back off appropriately. Clients that retry immediately after rejection cause thundering herd effects, making the overload worse. Coordinate with client teams on retry behavior.
Automatic vs. Manual Degradation
Feature degradation can be triggered automatically (based on system conditions) or manually (by operator decision). Both approaches have trade-offs.
Automatic degradation:
System monitors trigger degradation when conditions cross thresholds. Example: When CPU exceeds 85%, automatically disable Tier 4 features.
Automatic vs. Manual Degradation
| Automatic Degradation | Manual Degradation |
|---|---|
| Responds in seconds to milliseconds | Responds in minutes to hours |
| No human lag time | Requires human availability and response |
| Works 24/7 without on-call fatigue | Depends on on-call engineer judgment |
| May trigger unnecessarily (false positives) | Can apply judgment to avoid false positives |
| Limited to predefined conditions | Can handle novel, unexpected situations |
| Can oscillate if thresholds poorly tuned | Single decision, less oscillation risk |
When to use each:
Use automatic degradation when:
- The degradation impact is low risk
- Speed of response is critical
- The trigger conditions are well-understood and stable
- False positives cause minor inconvenience
Use manual degradation when:
- The degradation has significant user impact
- The situation requires judgment (novel failures)
- Automatic thresholds are unreliable or untested
- Regulatory or compliance review is required
Hybrid approach:
Most production systems use a hybrid:
- Automatic degradation for low-risk features (Tier 4)
- Automatic + notification for medium risk (Tier 3)
- Manual approval required for high risk (Tier 1-2)
- Manual override available for all levels
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768
class DegradationController { async evaluateDegradation(systemState: SystemState): Promise<DegradationAction[]> { const actions: DegradationAction[] = []; for (const feature of this.features) { const shouldDegrade = this.evaluateConditions(feature, systemState); if (!shouldDegrade) continue; switch (feature.degradationType) { case 'automatic_silent': // Tier 4: Just do it, log for audit await this.applyDegradation(feature); actions.push({ feature: feature.name, action: 'degraded', approval: 'auto' }); break; case 'automatic_notify': // Tier 3: Do it, but notify operators await this.applyDegradation(feature); await this.notifyOperators(feature, systemState); actions.push({ feature: feature.name, action: 'degraded', approval: 'auto_notified' }); break; case 'recommend': // Tier 2: Don't do it, but recommend to operators await this.createRecommendation(feature, systemState); actions.push({ feature: feature.name, action: 'recommended', approval: 'pending' }); break; case 'manual_only': // Tier 1: Only operators can degrade // Log that conditions were met, no action taken await this.logConditionMet(feature, systemState); break; } } return actions; } // Manual override - can be used to force degradation or prevent it async manualOverride(featureName: string, action: 'degrade' | 'restore' | 'lock', operator: string, reason: string): Promise<void> { const feature = this.getFeature(featureName); // Audit trail await this.auditLog.record({ feature: featureName, action, operator, reason, timestamp: new Date(), previousState: feature.currentState }); switch (action) { case 'degrade': await this.applyDegradation(feature); break; case 'restore': await this.restoreFeature(feature); break; case 'lock': await this.lockFeature(feature); // Prevent auto-degradation break; } }}Graceful Degradation UX
Feature degradation creates changed user experiences. How those changes are presented to users significantly impacts perceived quality and user trust. Poor degradation UX can be more damaging than the degradation itself.
Degradation UX principles:
- •Smooth removal — Features should disappear gracefully, not leave broken UI elements. If recommendations are disabled, the space should collapse or show alternative content, not an empty box.
- •Consistent styling — Degraded states should use the same design system as normal states. A degraded page should still look like your product, not a beta or error page.
- •Minimal user confusion — Users shouldn't be confused about whether the site is broken. Be intentional about what signals 'degraded but intentional' vs 'broken and needs fixing'.
- •Preserve core user journey — Degradation should not block users from completing their primary task. If they came to buy something, ensure they can still buy.
- •Appropriate communication — Sometimes transparency helps ('Some features temporarily unavailable'); sometimes silent degradation is better (users don't need to know recommendations are generic).
- •Recovery is invisible — When features return, the transition should be smooth. Users shouldn't experience jarring page reloads or layout shifts.
| Degradation Type | Bad UX | Good UX |
|---|---|---|
| Recommendations disabled | Empty 'Recommended for you' section | Replace with 'Popular products' or remove section entirely |
| Search simplified | Advanced filters grayed out with no explanation | Show simplified search; advanced filters hidden or with 'coming back soon' tooltip |
| Real-time updates disabled | Data appears frozen with no indication | Show 'Last updated: X minutes ago' timestamp |
| Images low quality | Blurry images with no context | Slightly lower quality (often unnoticeable) or 'Loading high quality...' placeholder |
| Feature completely removed | 404 errors or broken links | All navigation to feature redirects with explanation, or links removed |
Include degradation states in your design system. When designers create new features, they should also design the degraded version. This ensures consistent, intentional degradation UX rather than ad-hoc decisions during incidents.
Dependency-Triggered Degradation
Many features depend on downstream services. When a dependency becomes unhealthy, features that rely on it should degrade rather than fail. This requires mapping dependencies and automatically triggering degradation based on dependency health.
Dependency mapping:
For each feature, document:
- Which services it depends on
- Whether each dependency is required or optional
- What happens when each dependency fails
- What degraded mode looks like for each failure
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273
interface FeatureDependencyConfig { featureName: string; dependencies: { service: string; required: boolean; // Feature fails if this does, or degrades? healthCheck: string; // Endpoint or metric to check health // What to do when this dependency fails onFailure: { action: 'disable_feature' | 'degrade_feature' | 'use_fallback'; degradedBehavior?: string; // Description of degraded mode fallbackSource?: string; // Where to get fallback data }; }[];} // Example: Product recommendations featureconst recommendationsConfig: FeatureDependencyConfig = { featureName: 'personalized_recommendations', dependencies: [ { service: 'ml-model-serving', required: false, // Can degrade healthCheck: '/health', onFailure: { action: 'degrade_feature', degradedBehavior: 'Show trending items instead of personalized', fallbackSource: 'trending-service' } }, { service: 'user-preferences', required: false, healthCheck: '/health', onFailure: { action: 'use_fallback', fallbackSource: 'cached-preferences', degradedBehavior: 'Use cached preferences up to 24h old' } }, { service: 'product-catalog', required: true, // Can't show anything without products healthCheck: '/health', onFailure: { action: 'disable_feature', degradedBehavior: 'Hide recommendations section entirely' } } ]}; // Degradation controller uses this configclass DependencyAwareDegradation { async evaluateFeature(featureConfig: FeatureDependencyConfig): Promise<FeatureState> { const unhealthyDeps = await this.checkDependencies(featureConfig.dependencies); if (unhealthyDeps.length === 0) { return { status: 'healthy', mode: 'full' }; } // Check if any required deps are unhealthy const requiredUnhealthy = unhealthyDeps.filter(d => d.required); if (requiredUnhealthy.length > 0) { return { status: 'disabled', mode: 'off', reason: `Required dependency ${requiredUnhealthy[0].service} unavailable` }; } // Degrade based on optional dependency failures const degradations = unhealthyDeps.map(d => d.onFailure.degradedBehavior); return { status: 'degraded', mode: 'degraded', degradations }; }}Circuit breakers protect against calling failing dependencies. Combine circuit breaker state with feature degradation: when a circuit opens, automatically trigger the appropriate feature degradation mode. When the circuit closes, restore the feature. This creates a unified resilience response.
KEEP LEARNING
There’s more to understand.
You’re reading the introduction. Create a free account to read the complete lesson, save notes and keep track of your progress.