Exponential Decay Learning Rate Scheduler

In deep learning and optimization, the learning rate is one of the most critical hyperparameters that determines how quickly or slowly a model learns. A constant learning rate throughout training is often suboptimal—starting with a high learning rate enables rapid initial progress, but as the model approaches convergence, smaller steps are needed for fine-tuning.

Exponential decay is a widely-used technique for dynamically adjusting the learning rate during training. With this strategy, the learning rate decreases by a constant multiplicative factor after each epoch, creating a smooth exponential curve that naturally balances exploration and exploitation throughout the training process.

The learning rate at any epoch t is computed using the formula:

$$lr_t = lr_0 \times \gamma^t$$

Where:

• lr₀ is the initial learning rate at epoch 0
• γ (gamma) is the decay factor (typically between 0 and 1)
• t is the current epoch number

Your Task: Implement a Python class that serves as an exponential decay learning rate scheduler. The class should:

• Initialize with an initial learning rate and a decay factor
• Provide a method to compute the learning rate for any given epoch
• Return the learning rate rounded to 4 decimal places

Key Insight: When γ = 0.9 and the initial learning rate is 0.1:

• At epoch 0: lr = 0.1 × 0.9⁰ = 0.1
• At epoch 1: lr = 0.1 × 0.9¹ = 0.09
• At epoch 5: lr = 0.1 × 0.9⁵ ≈ 0.059
• At epoch 10: lr = 0.1 × 0.9¹⁰ ≈ 0.035

This gradual decay helps the optimizer take large steps initially to escape local minima and smaller steps later for precise convergence.