Convolutional Network Backpropagation Trainer

Convolutional Neural Networks (CNNs) have revolutionized the field of computer vision and pattern recognition. At the heart of training these networks lies backpropagation—an elegant algorithm that computes gradients by propagating error signals backward through the network's layers. This enables the model to learn hierarchical feature representations directly from raw pixel data.

In this challenge, you will implement a complete training pipeline for a simplified CNN architecture that includes:

Network Architecture

1. Convolutional Layer: Applies learnable filters to detect local patterns in the input

   • Filters of size kernel_size × kernel_size
   • Number of filters controlled by num_filters
   • Uses "valid" convolution (no padding)

2. ReLU Activation: Introduces non-linearity by applying the rectified linear unit function $$\text{ReLU}(x) = \max(0, x)$$

3. Flattening: Reshapes the 2D feature maps into a 1D vector for the dense layer

4. Dense Layer: Fully-connected layer that maps features to class scores $$z = W_{dense} \cdot flatten(activation) + b_{dense}$$

5. Softmax Output: Converts raw scores to probability distribution $$\text{softmax}(z_i) = \frac{e^{z_i}}{\sum_{j} e^{z_j}}$$

6. Cross-Entropy Loss: Measures the discrepancy between predictions and true labels $$L = -\sum_{i} y_i \log(\hat{y}_i)$$

Training Process

Your implementation must handle the complete training cycle:

Forward Pass:

• Convolve input images with learned filters
• Apply ReLU activation
• Flatten spatial dimensions
• Compute dense layer output
• Apply softmax to get predictions
• Calculate cross-entropy loss

Backward Pass (Backpropagation):

• Compute gradients of loss with respect to softmax output
• Propagate gradients through the dense layer
• Propagate gradients through the flattening operation
• Handle ReLU derivative (gradient is zero where input was negative)
• Compute gradients for convolutional weights via correlation

Weight Update (Stochastic Gradient Descent): $$W \leftarrow W - \eta \cdot \frac{\partial L}{\partial W}$$

where (\eta) is the learning rate.

Technical Requirements

• Initialize all weights using NumPy's random number generator with seed 42
• Weights should be initialized from a standard normal distribution scaled by 0.01
• Biases should be initialized to zeros
• Process samples one at a time (stochastic gradient descent)
• Repeat the entire dataset for the specified number of epochs

Your Task

Implement the train_cnn_with_gradient_descent function that trains the described network and returns the final trained parameters after completing all epochs.