Practical Interpretability

When you purchase a food product, you expect a nutrition label. When you buy electronics, you expect safety certifications and specifications. But when organizations deploy machine learning models—systems that may affect millions of lives—no standardized documentation format has historically existed. Users, auditors, and even the deploying organizations themselves often lack crucial information about what models do, how they perform across different populations, and what their limitations are.

Model Cards address this gap. Introduced by Mitchell et al. at Google in 2019, Model Cards provide a standardized framework for documenting machine learning models. They serve as the "nutrition labels" of ML—concise, accessible summaries that communicate essential information about trained models to diverse stakeholders.

This page examines Model Cards comprehensively: their purpose and structure, how to create effective model cards, real-world examples, tooling for automation, and emerging extensions of the concept.

Before diving into structure and implementation, let's understand why Model Cards emerged and what problems they solve.

The Documentation Crisis:

Machine learning models have historically been documented inconsistently, if at all. Common patterns include:

• No documentation: Model exists as a serialized file with no accompanying information
• Code-only documentation: README files that explain how to run the model but not its behavior
• Research paper documentation: Academic papers describing methodology but not deployment considerations
• Scattered documentation: Information spread across wikis, tickets, and tribal knowledge

This creates serious problems across the ML lifecycle:

What Model Cards Provide:

Model Cards are not comprehensive technical documentation—they're accessible summaries designed for multiple audiences:

The Standardization Benefit:

By using a common format, Model Cards enable:

1. Comparison: Evaluate alternative models using consistent criteria
2. Efficiency: Reviewers know where to find information without searching
3. Completeness: Standardized sections ensure critical information isn't forgotten
4. Interoperability: Tools can parse and aggregate information across models
5. Accountability: Clear expectations for what documentation should include

The original Model Card framework proposed by Mitchell et al. includes several core sections. While organizations adapt this structure to their needs, the fundamental components remain consistent.

Core Model Card Sections:

Section Deep Dives:

1. Model Details

This section provides the "metadata" of the model—who built it, what it is, and how to learn more:

• Person or Organization Developing the Model: Clear accountability
• Model Date: When the model was created and last updated
• Model Version: Unique identifier for this version
• Model Type: Architecture (e.g., "XGBoost classifier," "BERT-base transformer")
• Paper or Resource: Link to research paper, blog post, or detailed documentation
• Citation: How to cite this model in publications
• License: Terms for using the model
• Contact: Where to send questions or report issues

2. Intended Use

Critically important—this section prevents misuse by explicitly defining appropriate use:

• Primary Intended Uses: What the model was designed and validated for
• Primary Intended Users: Who should be using this model
• Out-of-Scope Uses: Explicit examples of what the model should NOT be used for
• Additional Use Cases: Secondary uses that may be appropriate with care

Example:

Primary Intended Use: Toxicity classification for content moderation in English-language social media comments

Out-of-Scope Uses: Legal evidence for defamation cases, classification of non-English content, classification of long-form articles (trained on short-form only)

3. Factors

This section specifies what factors are relevant to model performance:

• Relevant Factors: Demographic groups, instruments, environments where performance may vary
• Evaluation Factors: Which factors performance was actually disaggregated across

The distinction matters: some relevant factors may not have been evaluated due to data limitations, and the card should acknowledge this gap.

4. Metrics

Specify how performance is measured:

• Performance Measures: What metrics quantify performance (accuracy, AUC, F1, MAE, etc.)
• Decision Thresholds: If the model outputs probabilities, what thresholds define decisions
• Variation Approaches: How uncertainty is estimated (confidence intervals, cross-validation, etc.)

Example:

Primary Metrics: AUC-ROC (overall discrimination), Precision@90%Recall (operational threshold)

Decision Threshold: 0.72 probability triggers positive classification, optimized for equal error rate

Uncertainty: 95% confidence intervals via bootstrap (n=1000)

5. Evaluation Data & Training Data

Describe what data the model was tested on and trained on:

• Dataset Description: What data was used, how much, when collected
• Motivation: Why this data was chosen
• Preprocessing: What transformations were applied
• Limitations: Known gaps or biases in the data

Note: Training data may be less detailed for proprietary models, but evaluation data should be transparent.

6. Quantitative Analyses

The heart of disaggregated evaluation:

• Unitary Results: Overall performance metrics
• Intersectional Results: Performance broken down by relevant factors and their intersections

This section enables detection of performance disparities that aggregate metrics hide.

7. Ethical Considerations

Explicit discussion of ethical dimensions:

• Potential Risks: What could go wrong if the model is misused or performs poorly
• Sensitive Use Cases: Applications that require additional scrutiny
• Mitigation Strategies: What has been done to address identified ethical concerns
• Human Oversight: Recommended human involvement in model use

8. Caveats and Recommendations

Practical advice for users:

• Known Limitations: Where the model is known to underperform
• Recommendations for Use: Best practices for deployment
• Recommendations Against Use: When NOT to use this model

Let's examine a complete Model Card example to see how the sections come together. This example is for a hypothetical customer churn prediction model.

Creating a useful Model Card requires more than filling in a template—it requires understanding your audience, being honest about limitations, and providing actionable information.

The Model Card Creation Process:

Best Practices:

1. Write for Multiple Audiences Simultaneously

The same card will be read by technical and non-technical stakeholders. Use layering:

• Headlines and summaries for quick scanning
• Tables for structured comparison
• Detailed text for those who need depth
• Links to full documentation for technical details

2. Be Specific, Not Vague

❌ "The model may have some biases" ✓ "The model shows 8% lower precision for customers in the Healthcare industry (AUC 0.79 vs 0.85 overall)"

Vague warnings provide no actionable information. Specific findings enable informed decisions.

3. Explain What's Missing

A Model Card's value includes what it reveals about gaps:

• "Performance not evaluated by disability status due to data unavailability"
• "Geographic analysis limited to country level; regional variation not assessed"

4. Connect Metrics to Impact

Raw metrics are insufficient. Explain implications:

• "At the default threshold, expect approximately 1 in 3 'high-risk' predictions to actually churn"
• "False positive rate of 12% means ~1,200 customers per month contacted unnecessarily"

5. Include Visual Summaries Where Helpful

Confusion matrices, reliability diagrams, and subgroup comparison charts can communicate patterns faster than tables of numbers.

6. Link Limitations to Recommendations

Every documented limitation should connect to guidance:

• Limitation: "Lower performance for new customers"
• Recommendation: "Supplement model scores with product usage signals for customers under 6 months"

Several tools exist to streamline Model Card creation, from templates to automated generators that extract information from training pipelines.

Automation Strategies:

1. Pipeline Integration

Integrate Model Card generation into your ML pipeline:

• Training pipeline captures training data statistics, hyperparameters
• Evaluation pipeline computes and stores disaggregated metrics
• Card generation extracts information automatically
• Human review adds context like intended use and ethical considerations

2. Version Control

Store Model Cards alongside models in version control:

• Model artifacts and Model Cards versioned together
• Changes to models trigger Card updates
• History of documentation available for audit

3. Validation Checks

Automate Model Card quality checks:

• All required sections populated?
• Subgroup analyses for mandated factors complete?
• Performance thresholds met?
• Recent update date?
• Links valid?

4. Registry Integration

Model registries should require Model Cards:

• No model deployment without compliant Card
• Cards discoverable from registry
• Cards attached to versions, not just model names

The original Model Card concept has spawned numerous extensions addressing specific needs and domains.

Model Cards directly support regulatory compliance by providing standardized documentation that addresses many legal requirements.

Model Cards as Compliance Evidence:

Model Cards can serve as evidence of compliance efforts:

1. Documentation of Due Diligence: Shows that relevant factors were considered
2. Contemporaneous Records: Created alongside development, not reconstructed later
3. Transparency Commitment: Public or regulator-accessible cards demonstrate openness
4. Standardized Format: Regulators can compare across organizations
5. Update History: Demonstrates ongoing monitoring and improvement

Limitations as Compliance Documentation:

However, Model Cards alone may not satisfy all requirements:

• They're summaries, not complete technical specifications
• They don't replace formal risk management documentation
• Regulatory requirements may mandate specific formats
• Some requirements need supporting evidence beyond what Cards contain
• Model Cards don't automatically prove the claims they make

Best Practice:

Use Model Cards as the accessible layer of a documentation hierarchy:

Model Card (Public/Accessible Summary)
    ├── Technical Specification (Detailed architecture, parameters)
    ├── Validation Report (Complete evaluation methodology and results)
    ├── Risk Assessment (Formal risk management documentation)
    ├── Data Documentation (Data Cards, data lineage)
    └── Operational Runbook (Deployment, monitoring, incident response)

Model Cards provide navigation and summary; supporting documents provide depth.

Successfully implementing Model Cards across an organization requires more than choosing a template—it requires process integration, cultural change, and sustained effort.

Common Implementation Challenges:

Measuring Model Card Program Success:

• Coverage: % of production models with Cards
• Completeness: % of Card sections populated
• Currency: % of Cards updated within policy timeframe
• Quality: % passing review criteria
• Usage: Card views, references in decisions/incidents
• Stakeholder Satisfaction: Survey of Card users

Model Cards provide standardized, accessible documentation for machine learning models—serving as nutrition labels that inform diverse stakeholders about model purpose, performance, limitations, and appropriate use. Let's consolidate the key insights:

What's Next:

Model Cards focus on individual models, but comprehensive ML documentation requires broader practices. The next page examines Documentation and Governance for interpretability—covering documentation strategies, governance structures, and organizational practices that ensure interpretability is sustained throughout the ML lifecycle.