Loading content...
Consider two questions a loan officer might ask about an ML-based credit decision system:
"Why was this specific applicant denied?" — The officer needs to explain to John Smith why his application was rejected. What factors specific to his application led to this outcome?
"How does the model generally work?" — The compliance team needs to audit the system. What factors does the model consider important overall? What patterns drive its decisions across all applicants?
These are fundamentally different questions requiring fundamentally different answers. The first asks for a local explanation—understanding a single prediction. The second asks for a global explanation—understanding the model's overall behavior.
This local vs. global distinction is the second major dimension of interpretability, orthogonal to the intrinsic/post-hoc distinction we explored earlier.
By the end of this page, you will understand: the precise distinction between local and global interpretability, the key methods for each scope, how local explanations can be aggregated to create global understanding, the different stakeholder needs each serves, and how to combine both for complete model understanding.
Local interpretability explains why a model made a specific prediction for a specific input. It answers the question: "Given this particular input, why did the model produce this particular output?"
Local explanations are instance-specific. The explanation for one input may be completely different from the explanation for another, because different features may be relevant in different contexts.
Example: Medical diagnosis
Consider a model that predicts diabetes risk. For two patients:
Patient A (predicted high risk):
Patient B (also predicted high risk):
Both patients have high risk predictions, but for completely different reasons. A global explanation saying 'fasting glucose is the most important feature' would be misleading for Patient B. Local interpretability captures this instance-specific nuance.
Key insight: Local explanations reveal that a model may behave very differently in different regions of the input space. What matters for one prediction may be irrelevant for another.
Local interpretability can be thought of as describing the model's decision boundary in the vicinity of the specific input. Even if the global decision boundary is complex and non-linear, it can often be approximated locally by a simple linear function—this is the core insight behind methods like LIME.
Several methods have been developed specifically for local interpretability. Each takes a different approach to explaining individual predictions.
LIME (Local Interpretable Model-agnostic Explanations)
Core idea: Approximate the complex model locally with a simple, interpretable linear model. Even if the global decision boundary is complex, it can often be approximated by a linear function in the neighborhood of any specific point.
Algorithm:
Strengths:
Limitations:
Best for: Quick, model-agnostic local explanations when perfect faithfulness isn't critical.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105
import shapimport limeimport lime.lime_tabularimport numpy as npfrom sklearn.ensemble import GradientBoostingClassifier # Train modelmodel = GradientBoostingClassifier(n_estimators=100, random_state=42)model.fit(X_train, y_train) # Select instance to explaininstance_idx = 42instance = X_test[instance_idx:instance_idx+1]prediction = model.predict(instance)[0]probability = model.predict_proba(instance)[0, 1] print(f"Instance {instance_idx}: Prediction = {prediction}, P(positive) = {probability:.3f}")print("="*60) # Method 1: SHAP Local Explanationexplainer_shap = shap.TreeExplainer(model)shap_values = explainer_shap.shap_values(instance) print("📊 SHAP Local Explanation:")print(f"Base value (expected): {explainer_shap.expected_value[1]:.4f}")print(f"Sum of SHAP values: {shap_values[1][0].sum():.4f}")print(f"Prediction log-odds: {explainer_shap.expected_value[1] + shap_values[1][0].sum():.4f}")print("Top 5 feature contributions:")contributions = list(zip(feature_names, instance[0], shap_values[1][0]))for name, val, shap_val in sorted(contributions, key=lambda x: -abs(x[2]))[:5]: direction = "↑" if shap_val > 0 else "↓" print(f" {name}={val:.2f}: {shap_val:+.4f} {direction}") # Method 2: LIME Local Explanationexplainer_lime = lime.lime_tabular.LimeTabularExplainer( X_train, feature_names=feature_names, class_names=['Negative', 'Positive'], mode='classification') lime_exp = explainer_lime.explain_instance( instance[0], model.predict_proba, num_features=5) print("📊 LIME Local Explanation:")print(f"Local model intercept: {lime_exp.intercept[1]:.4f}")print(f"Local prediction: {lime_exp.local_pred[0]:.4f}")print("Top 5 local linear coefficients:")for feature_rule, weight in lime_exp.as_list()[:5]: direction = "↑" if weight > 0 else "↓" print(f" {feature_rule}: {weight:+.4f} {direction}") # Method 3: Counterfactual Explanation (using DiCE)try: import dice_ml # Setup DiCE data = dice_ml.Data( dataframe=train_df, continuous_features=continuous_features, outcome_name='target' ) dice_model = dice_ml.Model(model=model, backend='sklearn') exp = dice_ml.Dice(data, dice_model, method='random') # Generate counterfactuals query_instance = pd.DataFrame([instance[0]], columns=feature_names) cf = exp.generate_counterfactuals( query_instance, total_CFs=3, desired_class='opposite' ) print("📊 Counterfactual Explanations:") print("To flip the prediction, consider these changes:") cf.visualize_as_dataframe()except ImportError: print("(DiCE not installed - skipping counterfactual example)") # Comparison: Do methods agree?print("" + "="*60)print("📊 Method Agreement Analysis")print("="*60) # Get top 3 features from each methodshap_top3 = set([name for name, _, _ in sorted(contributions, key=lambda x: -abs(x[2]))[:3]])lime_top3 = set([feat.split()[0] for feat, _ in lime_exp.as_list()[:3]]) overlap = shap_top3.intersection(lime_top3)print(f"SHAP top 3: {shap_top3}")print(f"LIME top 3: {lime_top3}")print(f"Agreement: {len(overlap)}/3 features overlap") # Note: Some disagreement is expected due to different methodologies# High disagreement may indicate instability or complex local behaviorGlobal interpretability explains the model's overall behavior across all inputs. It answers questions like: "What features does this model generally consider important? What patterns has it learned? How does it behave across the input space?"
Global explanations provide a bird's-eye view of the model, abstracting away instance-specific details to reveal general principles.
The abstraction challenge:
Global explanations necessarily involve abstraction. A complex model may behave differently in different regions of the input space. A global explanation must somehow summarize this heterogeneous behavior.
Consider a credit model:
A global explanation might say 'income is the most important feature on average,' but this average obscures the heterogeneous local behavior. This is both the power and the limitation of global interpretability—it simplifies, which aids comprehension but can mislead.
Types of global explanation:
Global explanations can be dangerously misleading when local behavior varies significantly. Simpson's paradox applies: a feature might have positive effect globally but negative effect for specific subgroups. Always combine global explanations with local analysis for high-stakes applications.
Global interpretability methods provide various lenses for understanding overall model behavior.
Global Feature Importance Methods
Feature importance methods rank features by their overall contribution to model predictions.
Permutation Importance:
Impurity-based Importance (Trees):
SHAP Global Importance:
Comparison:
| Method | Speed | Bias Issues | Handles Interactions | Theory |
|---|---|---|---|---|
| Permutation | Medium | Low | Yes | Sound |
| Impurity | Fast | High-cardinality bias | Partial | Heuristic |
| SHAP | Slow | Low | Yes | Axiomatic |
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128
import shapimport numpy as npimport matplotlib.pyplot as pltfrom sklearn.inspection import permutation_importance, PartialDependenceDisplayfrom sklearn.ensemble import GradientBoostingClassifierfrom sklearn.tree import DecisionTreeClassifier # Train complex modelmodel = GradientBoostingClassifier(n_estimators=100, random_state=42)model.fit(X_train, y_train) # ============================================# Method 1: Permutation Importance# ============================================print("📊 Permutation Feature Importance")print("="*50) perm_imp = permutation_importance( model, X_test, y_test, n_repeats=30, random_state=42) for name, imp, std in sorted( zip(feature_names, perm_imp.importances_mean, perm_imp.importances_std), key=lambda x: -x[1]): print(f" {name}: {imp:.4f} ± {std:.4f}") # ============================================# Method 2: SHAP Global Importance# ============================================print("📊 SHAP Global Feature Importance")print("="*50) explainer = shap.TreeExplainer(model)shap_values = explainer.shap_values(X_test) # Mean absolute SHAP value per featureshap_importance = np.abs(shap_values[1]).mean(axis=0)for name, imp in sorted(zip(feature_names, shap_importance), key=lambda x: -x[1]): print(f" {name}: {imp:.4f}") # SHAP summary plot (beeswarm) - shows distribution of effectsshap.summary_plot(shap_values[1], X_test, feature_names=feature_names, show=False)plt.title("SHAP Summary Plot")plt.tight_layout()plt.savefig("shap_summary.png", dpi=150)plt.close() # ============================================# Method 3: Partial Dependence Plots# ============================================print("📊 Generating Partial Dependence Plots...") fig, axes = plt.subplots(2, 3, figsize=(15, 10))top_features = [name for name, _ in sorted( zip(feature_names, shap_importance), key=lambda x: -x[1])[:6]] for ax, feature in zip(axes.flatten(), top_features): feature_idx = list(feature_names).index(feature) PartialDependenceDisplay.from_estimator( model, X_test, [feature_idx], feature_names=feature_names, ax=ax ) ax.set_title(f"PDP: {feature}") plt.tight_layout()plt.savefig("pdp_plots.png", dpi=150)plt.close()print(" Saved to pdp_plots.png") # ============================================# Method 4: Global Surrogate Model# ============================================print("📊 Global Surrogate Model Analysis")print("="*50) # Use black-box predictions as targetbb_predictions = model.predict(X_train) # Train interpretable surrogatesurrogate = DecisionTreeClassifier(max_depth=5, random_state=42)surrogate.fit(X_train, bb_predictions) # Evaluate fidelitytrain_fidelity = (surrogate.predict(X_train) == bb_predictions).mean()test_predictions = model.predict(X_test)test_fidelity = (surrogate.predict(X_test) == test_predictions).mean() print(f" Surrogate fidelity (train): {train_fidelity:.3f}")print(f" Surrogate fidelity (test): {test_fidelity:.3f}") if test_fidelity > 0.85: print(" ✓ High fidelity - surrogate explanations are reasonably faithful") # Export tree rules from sklearn.tree import export_text print(" Decision tree rules (simplified global explanation):") print(export_text(surrogate, feature_names=list(feature_names), max_depth=3))else: print(" ⚠ Low fidelity - interpret surrogate with caution") # ============================================# Comparing Global Methods# ============================================print("📊 Cross-Method Agreement")print("="*50) # Get top 5 from each methodperm_top5 = set([name for name, _, _ in sorted( zip(feature_names, perm_imp.importances_mean, perm_imp.importances_std), key=lambda x: -x[1])[:5]]) shap_top5 = set([name for name, _ in sorted( zip(feature_names, shap_importance), key=lambda x: -x[1])[:5]]) print(f" Permutation top 5: {perm_top5}")print(f" SHAP top 5: {shap_top5}")print(f" Agreement: {len(perm_top5.intersection(shap_top5))}/5 features")A powerful approach to global interpretability is aggregating local explanations. This preserves the faithfulness of local methods while providing global insight.
The key insight: If we have a local explanation for every prediction, we can analyze patterns in these explanations to understand global behavior.
Why aggregation beats direct global methods:
Preserves heterogeneity: By keeping individual explanations before aggregating, we can analyze variation, not just averages.
Consistent interpretation: Local and global explanations use the same units and logic. SHAP values mean the same thing locally and globally.
Faithful to complex models: Direct global methods sometimes oversimplify. Aggregating faithful local explanations maintains accuracy.
Enables stratified analysis: We can aggregate differently for different subgroups, revealing heterogeneous treatment.
Supports auditing: Regulators can examine both individual explanations and aggregate patterns.
SHAP's summary plot (beeswarm plot) is the quintessential local-to-global visualization. Each dot is a local SHAP value, positioned by feature importance and magnitude. The distribution of dots reveals both feature rankings AND how feature values affect predictions. This single visualization bridges local and global interpretability beautifully.
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485
import shapimport numpy as npimport pandas as pdfrom sklearn.cluster import KMeans # Compute SHAP values for all test instancesexplainer = shap.TreeExplainer(model)shap_values = explainer.shap_values(X_test)[1] # For positive class # ============================================# Aggregation 1: Mean Absolute SHAP (Global Importance)# ============================================print("📊 Global Feature Importance (Mean |SHAP|)")global_importance = np.abs(shap_values).mean(axis=0)for name, imp in sorted(zip(feature_names, global_importance), key=lambda x: -x[1]): print(f" {name}: {imp:.4f}") # ============================================# Aggregation 2: SHAP Distribution Analysis# ============================================print("📊 SHAP Value Distribution Analysis")print("(Heterogeneity indicator: std/mean)") for i, name in enumerate(feature_names): mean_abs = np.abs(shap_values[:, i]).mean() std = np.abs(shap_values[:, i]).std() heterogeneity = std / mean_abs if mean_abs > 0 else 0 print(f" {name}: mean={mean_abs:.4f}, std={std:.4f}, CV={heterogeneity:.2f}") # High CV = heterogeneous effect (feature matters more for some instances)# Low CV = homogeneous effect (feature matters similarly across instances) # ============================================# Aggregation 3: Subgroup Analysis# ============================================print("📊 Subgroup SHAP Analysis (by demographic)") # Assume we have a demographic variable# This reveals if model treats groups differentlyfor group in ['A', 'B']: mask = X_test_df['demographic'] == group group_shap = shap_values[mask] group_importance = np.abs(group_shap).mean(axis=0) print(f" Group {group} (n={mask.sum()}):") for name, imp in sorted(zip(feature_names, group_importance), key=lambda x: -x[1])[:3]: print(f" {name}: {imp:.4f}") # If top features differ between groups, model uses different decision logic # ============================================# Aggregation 4: Explanation Clustering# ============================================print("📊 Explanation Clustering (finding decision modes)") # Cluster instances by their SHAP profilesn_clusters = 3kmeans = KMeans(n_clusters=n_clusters, random_state=42)cluster_labels = kmeans.fit_predict(shap_values) for c in range(n_clusters): mask = cluster_labels == c cluster_shap = shap_values[mask] mean_shap = cluster_shap.mean(axis=0) print(f" Cluster {c} (n={mask.sum()}, {100*mask.mean():.1f}% of data):") # Top features for this cluster top_idx = np.argsort(-np.abs(mean_shap))[:3] for idx in top_idx: direction = "↑" if mean_shap[idx] > 0 else "↓" print(f" {feature_names[idx]}: {mean_shap[idx]:+.4f} {direction}") # Characterize cluster cluster_X = X_test[mask] print(f" Avg prediction: {model.predict_proba(cluster_X)[:, 1].mean():.3f}") # Each cluster represents a distinct "explanation mode"# Cluster 0 might be "approved due to high income"# Cluster 1 might be "denied due to poor credit history"# etc.Neither local nor global interpretability alone tells the complete story. A comprehensive interpretability strategy combines both perspectives, using each where it's most valuable.
| Scenario | Best Approach | Why |
|---|---|---|
| Explaining individual decisions to users | Local | Users need personalized, specific explanations |
| Regulatory compliance (ECOA reason codes) | Local | Regulations require per-decision explanations |
| Debugging specific misclassifications | Local | Need to understand what went wrong for that instance |
| Model documentation and model cards | Global | Stakeholders need overall behavior summary |
| Bias detection across populations | Global + Subgroup | Need to compare treatment across groups globally |
| Scientific discovery of patterns | Global | Looking for general relationships, not individual cases |
| Human-in-the-loop decision support | Both | Global for mental model, local for specific decision |
| Model comparison and selection | Global | Need to compare overall behavior, not individual predictions |
| Edge case analysis | Local with sampling | Aggregate local explanations for outlier instances |
| Stakeholder presentation | Both | Global overview + specific illustrative examples |
A comprehensive interpretability workflow:
Start with global analysis:
Stratify by subgroups:
Deep dive with local analysis:
Examine edge cases:
Synthesize and communicate:
Think of global and local interpretability like a map. Global interpretability is like viewing a city from above—you see neighborhoods, major roads, and overall layout. Local interpretability is like street view—you see specific buildings, signs, and details. Both perspectives are needed to truly understand the territory.
Choosing between local and global methods—and how to combine them—depends on practical constraints and stakeholder needs.
| Constraint | Recommended Approach |
|---|---|
| Real-time explanations needed | LIME or cached SHAP; avoid expensive exact computation |
| Regulatory audit required | Global analysis + sampled local explanations; document methodology |
| User-facing explanation API | Local SHAP or counterfactuals with caching |
| Model comparison during development | Global importance + PDP; consistent across models |
| Bias detection | Subgroup global analysis + local examples of disparate treatment |
| Scientific publication | Both global and local with statistical significance tests |
We've explored the scope dimension of interpretability. Let's consolidate the key insights:
What's next:
We've now covered two major taxonomic dimensions: intrinsic vs. post-hoc (when interpretability is built) and local vs. global (what scope of behavior is explained). Next, we'll explore another important distinction: model-specific vs. model-agnostic methods. Should we use methods tailored to specific model types, or general methods that work across all models?
You now understand the local vs. global interpretability dimension. Local explanations answer 'why this prediction?' while global explanations answer 'how does the model work?' Mastering both perspectives—and knowing when to use each—is essential for comprehensive model understanding.