Robust Hybrid Regression Loss (Medium) — Practice with Code Visualizer

In regression tasks, selecting an appropriate loss function is critical for model performance, especially when dealing with datasets containing outliers. Traditional loss functions each have their limitations:

Squared Error Loss (L2): Highly sensitive to outliers because it squares the error, causing large deviations to dominate gradient updates and potentially destabilize training.
Absolute Error Loss (L1): More robust to outliers but has non-differentiable points at zero error, which can cause optimization challenges and slower convergence near the optimum.

The Robust Hybrid Regression Loss elegantly bridges these two approaches by behaving like squared error for small residuals and transitioning to linear error for larger residuals. This adaptive behavior is governed by a threshold parameter δ (delta), which defines the boundary between the two regimes.

Mathematical Formulation:

For a single prediction with residual (r = y_{\text{true}} - y_{\text{pred}}):

$$L(r) = \begin{cases} \frac{1}{2}r^2 & \text{if } |r| \leq \delta \ \delta \cdot (|r| - \frac{1}{2}\delta) & \text{if } |r| > \delta \end{cases}$$

This piecewise definition ensures:

Smooth gradient near zero: Small errors are penalized quadratically, providing strong gradient signals for fine-tuning predictions
Bounded gradient for outliers: Large errors grow only linearly, preventing any single outlier from dominating the loss
Continuous and differentiable everywhere: Unlike L1 loss, this function has continuous first derivatives

Key Properties:

When (|r| = \delta), both formulas yield the same value: (\frac{1}{2}\delta^2)
The gradient is continuous at the transition point, ensuring smooth optimization
As (\delta \to 0), the loss approaches L1 behavior
As (\delta \to \infty), the loss approaches L2 behavior

Your Task: Implement a function that computes the Robust Hybrid Regression Loss given true values, predicted values, and a delta threshold. The function should:

Handle both scalar and list inputs for true and predicted values
Compute the loss for each element using the piecewise formula above
Return the average loss across all elements, rounded to 4 decimal places

Computing the loss for each element:

• Element 1: |2.0 - 2.5| = 0.5 ≤ δ (1.0), so use quadratic formula: 0.5 × (0.5)² = 0.125 • Element 2: |3.0 - 2.0| = 1.0 ≤ δ (1.0), so use quadratic formula: 0.5 × (1.0)² = 0.5 • Element 3: |4.0 - 4.0| = 0.0 ≤ δ (1.0), so use quadratic formula: 0.5 × (0.0)² = 0.0

Average loss = (0.125 + 0.5 + 0.0) / 3 = 0.208333...

Rounded to 4 decimal places: 0.2083

When the predicted value exactly matches the true value:

• Residual: |5 - 5| = 0.0 ≤ δ (1.0) • Loss: 0.5 × (0.0)² = 0.0

Perfect predictions incur zero loss, as expected from any well-designed loss function.

All residuals exceed the delta threshold, triggering linear loss behavior:

• Element 1: |1.0 - 4.0| = 3.0 > δ (1.0), so use linear formula: δ × (|r| - 0.5δ) = 1.0 × (3.0 - 0.5) = 2.5 • Element 2: |2.0 - 5.0| = 3.0 > δ (1.0), so use linear formula: 1.0 × (3.0 - 0.5) = 2.5 • Element 3: |3.0 - 6.0| = 3.0 > δ (1.0), so use linear formula: 1.0 × (3.0 - 0.5) = 2.5

Average loss = (2.5 + 2.5 + 2.5) / 3 = 2.5

Note: The linear penalty (2.5) is much smaller than what squared error would give (4.5), demonstrating outlier robustness.

Squared Error Loss (L2): Highly sensitive to outliers because it squares the error, causing large deviations to dominate gradient updates and potentially destabilize training.
Absolute Error Loss (L1): More robust to outliers but has non-differentiable points at zero error, which can cause optimization challenges and slower convergence near the optimum.