Problem Formulation

Every successful machine learning project begins not with data, not with algorithms, and not with code—but with a precisely formulated problem. Problem formulation is the art and science of translating vague business objectives into concrete, measurable machine learning tasks. It is, without exaggeration, the single most consequential decision in any ML endeavor.

Yet this critical step is routinely overlooked. Teams dive into data exploration, debate model architectures, and tune hyperparameters—only to discover months later that they've been solving the wrong problem entirely. The algorithm performs brilliantly on the metrics they chose, but the business sees no value. The model achieves state-of-the-art accuracy, but users don't adopt the feature.

Problem formulation failure is the leading cause of ML project failure. Not bad data. Not algorithmic complexity. Not infrastructure challenges. Simply: solving the wrong problem.

The journey from a business problem to an ML problem involves several translation layers. Consider a simple-sounding request:

"We want to reduce customer churn."

This is a business objective—a desired outcome expressed in business terms. It cannot be directly optimized by a machine learning algorithm. To make it actionable, we must perform a series of translations:

Step 1: Define what 'churn' means precisely

Churn seems obvious, but is a customer churned when they:

• Cancel their subscription?
• Haven't logged in for 30 days? 60 days? 90 days?
• Downgrade from paid to free tier?
• Reduce usage below a threshold?

Each definition leads to different datasets, different models, and different interventions. A customer who hasn't logged in for 30 days but renews their annual subscription isn't churned—but a 30-day inactivity definition would label them as such.

Step 2: Determine what can be predicted

ML models predict outcomes. For churn, we must decide:

• Are we predicting whether a customer will churn (binary classification)?
• Are we predicting when they'll churn (survival analysis)?
• Are we predicting probability of churn (probabilistic classification)?
• Are we predicting churn risk score (regression/ranking)?

Step 3: Define the prediction window

When do we make predictions, and when must the outcome occur to count?

• Predict churn within 30 days of the prediction date?
• Predict churn before the next billing cycle?
• Predict lifetime value decrease over 12 months?

Step 4: Determine actionability

A prediction is only valuable if it enables action:

• If we predict churn 1 day before cancellation, can the business intervene in time?
• Do we need predictions far enough in advance to run a retention campaign?
• Can the predicted customers actually be influenced, or is churn inevitable for some segments?

The target variable (also called the label, response, or dependent variable) is what your model learns to predict. Defining it correctly is perhaps the most consequential technical decision in an ML project.

The target variable must satisfy several properties:

1. Observable at Training Time

You can only train a model to predict something you can measure in your historical data. If you want to predict 'customer satisfaction,' you need historical satisfaction measurements (surveys, NPS scores, support tickets). If those don't exist, you must use a proxy—which introduces potential mismatch between what you optimize and what you actually care about.

2. Observable at Prediction Time (Eventually)

For supervised learning, you need ground truth labels to evaluate your model. If your target takes 2 years to observe (e.g., 'will this customer remain active for 24 months?'), you may need to redefine the problem or accept that model evaluation will be delayed.

3. Aligned with Business Value

Optimizing for click-through rate when you care about conversions leads to models that generate clicks but not revenue. Optimizing for accuracy when false positives and false negatives have vastly different costs leads to models that are mathematically impressive but operationally useless.

4. Stable and Consistent

If your definition of the target changes over time (e.g., the company redefines 'active user'), historical labels become inconsistent with future labels, degrading model performance.

Once you've defined your target variable, you must identify which type of machine learning problem you're solving. This classification determines which algorithms to consider, how to evaluate performance, and what training procedures to apply.

The ML Task Taxonomy:

Machine learning problems generally fall into one of several categories, each with distinct characteristics:

Choosing the Right Task Type:

The same business problem can often be formulated as different ML tasks. Consider predicting house prices:

• As regression: Predict the exact selling price in dollars
• As classification: Predict if price is 'low', 'medium', or 'high' (discretized buckets)
• As ranking: For a given buyer, rank houses by likelihood of purchase

Each formulation has tradeoffs:

The right choice depends on how predictions will be used. If users need exact prices for budgeting, regression is essential. If the goal is to surface relevant properties, ranking may suffice.

A metric is how you measure success. In ML, we distinguish between optimization metrics (what the algorithm directly optimizes) and evaluation metrics (what we use to assess model quality).

The Metric Hierarchy:

1. Business Metrics — Revenue, conversion rate, customer lifetime value, NPS. These are what the business ultimately cares about but often can't be directly optimized (too slow to observe, too noisy, confounded by other factors).

2. Evaluation Metrics — Metrics used to assess model quality and select between models. Should be as close to business metrics as possible. Examples: precision, recall, RMSE, AUC-ROC.

3. Optimization Metrics — The loss function that the model directly minimizes during training. Examples: cross-entropy, mean squared error, hinge loss.

The goal is to choose optimization and evaluation metrics that are aligned with business metrics—when the model improves on evaluation metrics, business outcomes improve too.

Choosing Evaluation Metrics:

Cost-Sensitive Metrics:

For many problems, not all errors are equal. In fraud detection:

• A false positive (blocking a legitimate transaction) annoys a customer, but costs relatively little
• A false negative (allowing a fraudulent transaction) causes direct financial loss

For medical diagnosis:

• A false positive (telling a healthy patient they're sick) causes anxiety and unnecessary tests
• A false negative (missing a disease) can be life-threatening

When errors have asymmetric costs, standard metrics like accuracy are misleading. You need cost-sensitive evaluation:

Expected Cost = Σ (Cost of Error Type × Probability of Error Type)

This requires estimating the actual business costs of each error type, which often involves conversations with business stakeholders who may have never quantified these costs before.

Problem formulation isn't complete until you've documented the constraints under which the solution must operate. These constraints often eliminate entire classes of solutions and fundamentally shape which approaches are viable.

Categories of Constraints:

Edge Cases and Failure Modes:

Problem formulation should also anticipate edge cases and failure modes:

• What happens when the model is highly uncertain?
• What's the fallback if the model fails or is unavailable?
• How should the system handle inputs outside the training distribution?
• What's the process when users dispute predictions?

Documenting these upfront ensures the system is designed for robustness, not just average-case performance.

To systematize problem formulation, consider using a Problem Formulation Canvas—a structured template that captures all critical decisions before any modeling begins.

The Canvas Elements:

Even experienced ML practitioners fall into formulation traps. Here are the most common—and how to avoid them: