Loading problem...
In biological neural systems, lateral inhibition is a fundamental mechanism where activated neurons suppress the activity of their neighbors, enhancing contrast and enabling sharper, more discriminative feature detection. This principle has been successfully adapted in deep learning architectures to create a normalization technique that promotes competition among feature maps.
Cross-Channel Lateral Inhibition Normalization applies a normalization operation across neighboring channels (feature maps) in a convolutional neural network. For each activation value at position (i, x, y) in the feature tensor, the normalization considers the squared activations of channels within a local window centered around channel i and divides by a function of their sum.
Given a 4D input tensor X of shape (N, C, H, W) where:
For each activation x_{n,i,x,y}, the normalized output y_{n,i,x,y} is computed as:
$$y_{n,i,x,y} = \frac{x_{n,i,x,y}}{\left( k + \alpha \sum_{j=\max(0, i-\lfloor n/2 \rfloor)}^{\min(C-1, i+\lfloor n/2 \rfloor)} x_{n,j,x,y}^2 \right)^\beta}$$
Where:
The window spans n channels centered at channel i, bounded by the valid channel indices [0, C-1].
Your Task: Implement a function that applies cross-channel lateral inhibition normalization to a 4D tensor. The function should normalize each activation based on the sum of squared values in its local channel neighborhood and return the result rounded to 4 decimal places.
x = [[[[0.496714, -0.138264], [0.647689, 1.52303]], [[-0.234153, -0.234137], [1.579213, 0.767435]], [[-0.469474, 0.54256], [-0.463418, -0.46573]]]]
n = 3, k = 2, alpha = 0.0001, beta = 0.75[[[[0.2953, -0.0822], [0.3851, 0.9055]], [[-0.1392, -0.1392], [0.9389, 0.4563]], [[-0.2791, 0.3226], [-0.2755, -0.2769]]]]The input is a 4D tensor with shape (1, 3, 2, 2) - 1 sample, 3 channels, 2×2 spatial dimensions.
For each element, we compute the normalization using a window of n=3 adjacent channels:
• For channel 0 at position (0,0): value = 0.496714
• For channel 1 at position (0,0): value = -0.234153
This process continues for all positions, producing the normalized output tensor.
x = [[[[1.0, 2.0], [3.0, 4.0]], [[0.5, 1.5], [2.5, 3.5]], [[0.1, 0.2], [0.3, 0.4]], [[1.2, 2.2], [3.2, 4.2]]]]
n = 3, k = 2, alpha = 0.0001, beta = 0.75[[[[0.5946, 1.1889], [1.7828, 2.3759]], [[0.2973, 0.8917], [1.4857, 2.0789]], [[0.0595, 0.1189], [0.1783, 0.2376]], [[0.7135, 1.3079], [1.902, 2.4957]]]]The input tensor has shape (1, 4, 2, 2) - 1 sample, 4 channels, 2×2 spatial dimensions.
With n=3, the normalization window spans 3 adjacent channels centered at each position:
• Channel 0: considers channels 0 and 1 (bounded at start) • Channel 1: considers channels 0, 1, and 2 (full window) • Channel 2: considers channels 1, 2, and 3 (full window) • Channel 3: considers channels 2 and 3 (bounded at end)
The normalization divides each activation by a power of the sum of squared neighboring values plus the bias constant, creating inter-channel competition that emphasizes relatively stronger activations.
x = [[[[0.120981, -0.95664, -0.862459], [-0.281144, -0.506416, 0.157124], [-0.454012, -0.706152, 0.732824]], [[-0.112888, 0.033764, -0.712374], [-0.272191, 0.055461, -0.575497], [0.187849, -0.300319, -0.145847]], [[-0.300853, 0.926139, -0.006749], [-0.528855, 0.411272, -0.610422], [0.104432, -0.979835, -0.664093]], [[0.098431, 0.369233, 0.085684], [-0.057824, -0.150552, -0.739261], [-0.359922, -0.230319, 0.528561]], [[0.171809, -0.88152, 0.162042], [-0.192541, -0.338461, 0.305838], [0.5155, 0.46564, -0.419609]]]]
n = 5, k = 1, alpha = 0.001, beta = 0.5[[[[0.121, -0.9558, -0.8619], [-0.2811, -0.5063, 0.1571], [-0.454, -0.7056, 0.7325]], [[-0.1129, 0.0337, -0.7119], [-0.2721, 0.0554, -0.5751], [0.1878, -0.3001, -0.1458]], [[-0.3008, 0.9249, -0.0067], [-0.5287, 0.4112, -0.61], [0.1044, -0.9789, -0.6636]], [[0.0984, 0.3689, 0.0857], [-0.0578, -0.1505, -0.7388], [-0.3598, -0.2302, 0.5283]], [[0.1718, -0.8807, 0.162], [-0.1925, -0.3384, 0.3057], [0.5154, 0.4654, -0.4194]]]]This example uses a larger window size (n=5) with different hyperparameters: k=1 (smaller bias), alpha=0.001 (larger scaling), and beta=0.5 (square root normalization).
The input tensor has shape (1, 5, 3, 3). With n=5, the window can potentially span all 5 channels, but is bounded by valid indices:
• With a smaller k value, the normalization is more sensitive to the sum of squares • The larger alpha increases the contribution of neighboring activations • The smaller beta (0.5 vs 0.75) results in a gentler normalization effect
Notice how the output values are very close to the input values, since with appropriate hyperparameters, the normalization can be configured to have minimal impact while still providing slight regularization.
Constraints