Batch Inference Health Monitor

In production machine learning systems, maintaining visibility into the health and reliability of batch prediction pipelines is critical for operational excellence. MLOps teams rely on real-time dashboards and automated alerting systems to detect anomalies, performance degradation, and failures before they impact downstream consumers.

When running large-scale batch inference jobs—such as overnight scoring of millions of customer records, daily fraud detection sweeps, or hourly recommendation updates—the system generates prediction results that need continuous monitoring. Each prediction result carries metadata about its execution status and, for successful predictions, a confidence score indicating the model's certainty in its output.

Your Task:

Given a list of prediction results from a batch inference job, implement a function that computes essential health metrics commonly tracked in MLOps observability platforms. These metrics help engineering teams assess:

1. Overall Reliability: What percentage of predictions completed without errors?
2. Model Confidence Quality: How confident is the model in its successful predictions on average?
3. Threshold Violation Rate: What proportion of predictions fall below an acceptable confidence level?

Input Format:

You will receive:

• predictions: A list of dictionaries, where each dictionary represents a single prediction result with:
  • status: A string that is either 'success' (prediction completed) or 'error' (prediction failed)
  • confidence: A float between 0 and 1 representing model certainty (only present when status is 'success')
• confidence_threshold: A float between 0 and 1 representing the minimum acceptable confidence level

Output Format:

Return a dictionary containing three metrics, all rounded to 2 decimal places:

• success_rate: The percentage of predictions with status = 'success' out of all predictions
• avg_confidence: The mean confidence score across all successful predictions, expressed as a percentage (0-100)
• low_confidence_rate: The percentage of successful predictions whose confidence falls strictly below the threshold

Edge Cases:

• If the input list is empty, return an empty dictionary {}
• If there are no successful predictions, return success_rate as calculated, and both avg_confidence and low_confidence_rate as 0.0