ML System Design

Every failed ML project shares a common origin: insufficient understanding of requirements. Not bad algorithms. Not inadequate data. Not poor engineering. The root cause is almost always a gap between what was built and what was actually needed.

This isn't hyperbole. Industry surveys consistently show that 85-90% of ML projects fail to make it into production, and the primary reason isn't technical—it's organizational. The model works on test data but doesn't solve the actual business problem. The system achieves impressive metrics but doesn't integrate with existing workflows. The solution is technically elegant but answers the wrong question.

Requirements gathering for ML systems is fundamentally different from traditional software development. In conventional software, you can often specify exact inputs, outputs, and behaviors. In ML, you're building systems that operate under uncertainty, whose behavior emerges from data, and whose success depends on probabilistic guarantees that stakeholders may not intuitively understand.

Effective ML requirements gathering operates across multiple dimensions simultaneously. Unlike traditional software where functional requirements dominate, ML systems require careful consideration of data, model behavior, operational constraints, and business objectives—all intertwined and mutually constraining.

The ML Requirements Framework organizes these concerns into a structured approach that ensures nothing critical is overlooked. Each dimension informs the others, creating a coherent picture of what success looks like and how to achieve it.

The most critical and most frequently botched aspect of ML requirements is problem definition. Stakeholders often come with solutions in mind ("we need a recommendation engine" or "let's use deep learning for this") rather than clearly articulated problems. Your first task is to work backward from the proposed solution to the underlying problem, then forward again to potentially different—and often simpler—solutions.

The Problem Definition Canvas captures the essential questions that must be answered before any ML work begins:

The Translation Challenge

Business stakeholders speak in terms of outcomes: "We want to reduce customer churn" or "We need to optimize pricing." These high-level objectives must be translated into precise ML problem formulations:

• Reduce churn → Predict probability of churn per customer → Binary classification with temporal dynamics
• Optimize pricing → Predict demand elasticity curves → Regression with causal considerations
• Improve recommendations → Rank items by relevance for each user → Learning-to-rank or matrix factorization
• Detect fraud → Identify anomalous transactions → Anomaly detection with extreme class imbalance

Each translation involves choices that significantly impact the solution approach. Framing churn as binary classification is simpler than predicting days-until-churn (regression) or optimal intervention timing (reinforcement learning). The right framing depends on what decisions will actually be made with the predictions.

Problem Decomposition

Complex business problems often require decomposition into multiple ML tasks. Consider a content moderation system:

High-level need: Remove harmful content from the platform.

Decomposed ML tasks:

1. Classification: Is this content potentially harmful? (High recall, moderate precision)
2. Categorization: What type of harm? (Violence, hate speech, misinformation, etc.)
3. Severity scoring: How severe is the violation? (Prioritizes human review queue)
4. User risk modeling: Is this user a repeat offender? (Informs action severity)
5. False positive detection: Is this likely a false positive? (Prevents over-enforcement)

Each sub-problem has different requirements, evaluation criteria, and acceptable error rates. Treating this as a single classification task would miss the nuanced decision-making that effective moderation requires.

Defining success metrics for ML systems is deceptively complex. The challenge is bridging three distinct worlds: the business metrics that stakeholders care about, the model metrics that ML engineers optimize, and the system metrics that ensure reliable operation. These three types of metrics are interconnected but distinct, and confusion between them is a leading cause of ML project failure.

The Metrics Alignment Problem

The fundamental challenge is that improving model metrics doesn't guarantee improving business metrics. This happens for several reasons:

1. Offline-Online Gap: A model trained to minimize log loss on historical data might not perform well on live traffic where data distributions shift.

2. Proxy Metric Divergence: Click-through rate is easy to measure but may not correlate with user satisfaction or purchase intent.

3. Goodhart's Law: Once a metric becomes a target, it ceases to be a good metric. Optimizing for engagement can lead to addictive design patterns that hurt long-term retention.

4. Simpson's Paradox: Model improvements within segments can disappear or reverse when aggregated, especially if the segment distribution shifts.

5. Confounding Variables: Business metrics are affected by seasonality, marketing campaigns, product changes, and competitor actions—making causal attribution difficult.

Establishing Metric Baselines

Before any ML work begins, establish baselines for all relevant metrics:

Simple Baselines:

• Random predictions (establishes floor)
• Most-popular/majority class (surprisingly hard to beat)
• Last-value persistence (for time series)
• Rule-based heuristics (often captures 80% of value)

Current State Baseline:

• Existing system performance (if replacing something)
• Human expert performance (for comparison)
• Business metrics before intervention

Theoretical Ceiling:

• Inter-rater agreement (human upper bound)
• Irreducible noise in the data
• Best published results on similar problems

The Improvement Target: With baselines established, define the minimum viable improvement. A 2% lift in AUC might be meaningless, or it might translate to $10M in annual revenue. The business translation determines whether the project is worth pursuing.

ML projects typically involve stakeholders with fundamentally different perspectives, incentives, and risk tolerances. Aligning these stakeholders is not a soft skill afterthought—it's a core technical requirement. Misalignment results in projects that are technically successful but organizationally rejected.

Understanding the different stakeholder archetypes and their concerns enables proactive alignment:

Managing Expectations with the ML Uncertainty Principle

Unlike traditional software where you can often guarantee specific behaviors, ML systems have inherent uncertainty. Stakeholders accustomed to deterministic software need to understand this fundamental difference:

What ML can provide:

• Probabilistic predictions with quantified uncertainty
• Aggregate performance guarantees (on average, over time)
• Continuous improvement as more data is collected
• Patterns and insights invisible to human review at scale

What ML cannot provide:

• Perfect predictions for every individual case
• Guarantees against any specific error type
• Static behavior as data distributions shift
• Causal explanations (only correlations)

This isn't a limitation to hide—it's a fundamental characteristic to communicate early and often. Stakeholders who expect 100% accuracy will be disappointed; stakeholders who understand they're getting probabilistic decision support will be satisfied.

The Requirements Sign-Off Process

Formalize stakeholder alignment with a requirements sign-off that includes:

1. Problem Statement: Clear, agreed-upon definition of what we're solving
2. Success Criteria: Specific metrics with target values and measurement methodology
3. Failure Criteria: What would cause us to abandon this approach
4. Timeline: Realistic milestones for data, MVP, production deployment
5. Resource Commitment: Data access, engineering support, review bandwidth
6. Risk Register: Known risks, mitigations, and acceptable risk levels
7. Decision Rights: Who approves changes to scope, metrics, or approach

This document serves as a reference point throughout the project, preventing scope creep and misunderstandings.

Data is the fuel for ML systems, and data requirements gathering is often where projects discover they can't proceed. Better to discover data gaps early—before investing in model development—than to build a sophisticated model that can't be trained or deployed due to data limitations.

The Data Requirements Assessment covers seven critical dimensions:

The Data Audit Checklist

For each data source identified as relevant, conduct a systematic audit:

Feature Availability Analysis

Beyond raw data, assess whether the features needed for prediction will be available at inference time:

Training-Serving Skew Risks:

• Features computed from future data (look-ahead bias)
• Features requiring expensive computation (feasible offline, not in real-time)
• Features from third-party sources with availability gaps
• Features that aggregate over time windows differently in batch vs. streaming

Example: A churn prediction model uses 'last 30 days of activity' as a feature. During training, this is computed accurately from historical data. In production, for real-time serving, you need streaming infrastructure to maintain rolling 30-day aggregations—fundamentally different architecture.

The Feature Feasibility Matrix: For each candidate feature, assess:

1. Can it be computed in training? (data exists historically)
2. Can it be computed at serving time? (data available, latency acceptable)
3. Does it change the system architecture? (new pipelines needed)
4. What's the marginal predictive value? (worth the complexity?)

Every ML system operates within constraints—computational resources, latency requirements, budget limitations, regulatory mandates, organizational capabilities. Identifying these constraints early prevents building systems that can't be deployed or sustained.

The Constraint Categories:

The Iron Triangle of ML

ML systems face fundamental trade-offs that cannot be escaped—only navigated. Understanding these trade-offs enables explicit prioritization:

Accuracy vs. Latency: More complex models are often more accurate but slower. A 100-layer transformer beats a logistic regression on most tasks—but not if you need sub-millisecond response times.

Precision vs. Recall: You cannot maximize both. A spam filter with high precision rarely catches legitimate email—but also catches less spam. High recall catches all spam—but also many legitimate messages.

Freshness vs. Cost: Real-time model updates enable rapid adaptation but require expensive streaming infrastructure. Daily retraining is cheaper but responds slowly to distribution shifts.

Generalization vs. Personalization: Global models are simpler to train and deploy. Per-user models capture individual preferences but create cold-start problems and scalability challenges.

Interpretability vs. Performance: Linear models are easy to explain but limited in what they can learn. Deep networks capture complex patterns but act as black boxes.

A comprehensive ML requirements document synthesizes all gathered information into a single reference that guides development and enables accountability. This document should be living—updated as understanding evolves—but versioned to track how requirements changed over time.

Standard ML Requirements Document Structure:

Experience reveals recurring patterns of requirements failure. Recognizing these anti-patterns helps you avoid them in your own projects:

Requirements gathering for ML systems is a discipline unto itself—distinct from traditional software requirements and deserving of serious investment. The time spent here pays dividends throughout the project lifecycle.

What's next:

With requirements gathered and documented, the next step is designing the data infrastructure that will feed your ML system. The next page explores Data Pipeline Design—how to build the data infrastructure that transforms raw data into training sets and features, enabling reliable model development and production serving.