Cumulative Reward with Temporal Decay

In reinforcement learning and sequential decision-making, agents collect rewards over time as they interact with an environment. However, immediate rewards are typically valued more than future rewards—a fundamental concept known as temporal discounting. This principle reflects the uncertainty of future outcomes and the preference for immediate gratification.

The cumulative reward with temporal decay (also known as the discounted return or discounted cumulative reward) quantifies the total value of a sequence of rewards, where each future reward is progressively diminished by a decay factor.

Mathematical Formulation:

Given a sequence of rewards ( R = [r_0, r_1, r_2, ..., r_{T-1}] ) and a decay factor ( \gamma ) (gamma), where ( 0 < \gamma \leq 1 ), the cumulative decayed reward ( G ) is computed as:

$$G = \sum_{t=0}^{T-1} \gamma^t \cdot r_t = r_0 + \gamma \cdot r_1 + \gamma^2 \cdot r_2 + ... + \gamma^{T-1} \cdot r_{T-1}$$

Understanding the Decay Factor (γ):

• When γ = 1: No decay is applied; all future rewards are valued equally as immediate rewards. This represents a myopic agent that doesn't discount the future.
• When γ → 0: Future rewards become negligible; only the immediate reward matters. The agent is extremely short-sighted.
• When 0 < γ < 1: A balanced approach where future rewards matter but are progressively discounted. Typical values in practice range from 0.9 to 0.99.

Your Task: Write a Python function that computes the cumulative reward with temporal decay given a list of rewards and a decay factor gamma. The function should use NumPy for efficient computation and return the scalar value representing the total decayed cumulative reward.