OneNoughtOne
0/106
Loading content...
In the world of neural networks, the single neuron (also called a perceptron with a non-linear activation) represents the most fundamental computational unit. This building block forms the basis of all modern deep learning architectures, from simple classifiers to complex transformer models.
A single neuron performs a two-step computation:
Step 1: Weighted Sum (Linear Combination) For an input feature vector x = [x₁, x₂, ..., xₘ] with corresponding weights w = [w₁, w₂, ..., wₘ] and a bias term b, the neuron first computes the weighted sum (also called pre-activation or logit):
$$z = \sum_{j=1}^{m} w_j \cdot x_j + b = w_1x_1 + w_2x_2 + ... + w_mx_m + b$$
Step 2: Sigmoid Activation The weighted sum is then passed through the sigmoid activation function to produce a probability output between 0 and 1:
$$\sigma(z) = \frac{1}{1 + e^{-z}}$$
The sigmoid function squashes any real-valued input into the range (0, 1), making it ideal for binary classification tasks where we interpret the output as the probability of belonging to class 1.
Mean Squared Error (MSE) To evaluate how well the neuron's predictions match the true labels, we compute the Mean Squared Error:
$$MSE = \frac{1}{n} \sum_{i=1}^{n} (\hat{y}_i - y_i)^2$$
where ŷᵢ is the predicted probability and yᵢ is the true label (0 or 1) for each example.
Your Task: Write a Python function that simulates a single neuron with sigmoid activation for binary classification. Given a collection of feature vectors, their corresponding binary labels, the neuron's weights, and its bias term, compute:
Both values should be rounded to 4 decimal places.
features = [[0.5, 1.0], [-1.5, -2.0], [2.0, 1.5]]
labels = [0, 1, 0]
weights = [0.7, -0.4]
bias = -0.1([0.4626, 0.4134, 0.6682], 0.3349)Explanation
For each input vector, we compute the weighted sum and apply the sigmoid function: **Example 1:** x = [0.5, 1.0] • Weighted sum: z = (0.7 × 0.5) + (-0.4 × 1.0) + (-0.1) = 0.35 - 0.4 - 0.1 = -0.15 • Sigmoid: σ(-0.15) = 1/(1 + e^0.15) ≈ 0.4626 **Example 2:** x = [-1.5, -2.0] • Weighted sum: z = (0.7 × -1.5) + (-0.4 × -2.0) + (-0.1) = -1.05 + 0.8 - 0.1 = -0.35 • Sigmoid: σ(-0.35) = 1/(1 + e^0.35) ≈ 0.4134 **Example 3:** x = [2.0, 1.5] • Weighted sum: z = (0.7 × 2.0) + (-0.4 × 1.5) + (-0.1) = 1.4 - 0.6 - 0.1 = 0.7 • Sigmoid: σ(0.7) = 1/(1 + e^-0.7) ≈ 0.6682 **MSE Calculation:** • Errors: (0.4626-0)² + (0.4134-1)² + (0.6682-0)² = 0.214 + 0.344 + 0.446 = 1.004 • MSE = 1.004 / 3 ≈ 0.3349
features = [[1.0], [0.0], [-1.0]]
labels = [1, 0, 0]
weights = [0.5]
bias = 0.0([0.6225, 0.5, 0.3775], 0.1783)Explanation
With a single feature and zero bias: **Example 1:** x = [1.0] • Weighted sum: z = 0.5 × 1.0 + 0.0 = 0.5 • Sigmoid: σ(0.5) = 1/(1 + e^-0.5) ≈ 0.6225 • True label: 1, Error: (0.6225 - 1)² = 0.1425 **Example 2:** x = [0.0] • Weighted sum: z = 0.5 × 0.0 + 0.0 = 0.0 • Sigmoid: σ(0) = 1/(1 + e^0) = 0.5 (sigmoid of zero is always 0.5) • True label: 0, Error: (0.5 - 0)² = 0.25 **Example 3:** x = [-1.0] • Weighted sum: z = 0.5 × (-1.0) + 0.0 = -0.5 • Sigmoid: σ(-0.5) = 1/(1 + e^0.5) ≈ 0.3775 • True label: 0, Error: (0.3775 - 0)² = 0.1425 **MSE:** (0.1425 + 0.25 + 0.1425) / 3 ≈ 0.1783
features = [[10.0], [-10.0]]
labels = [1, 0]
weights = [1.0]
bias = 0.0([1.0, 0.0], 0.0)Explanation
This demonstrates the sigmoid function's saturation behavior with extreme inputs: **Example 1:** x = [10.0] • Weighted sum: z = 1.0 × 10.0 + 0.0 = 10.0 • Sigmoid: σ(10) = 1/(1 + e^-10) ≈ 0.9999546 → rounds to 1.0 • True label: 1, Error: (1.0 - 1)² = 0.0 **Example 2:** x = [-10.0] • Weighted sum: z = 1.0 × (-10.0) + 0.0 = -10.0 • Sigmoid: σ(-10) = 1/(1 + e^10) ≈ 0.0000454 → rounds to 0.0 • True label: 0, Error: (0.0 - 0)² = 0.0 **MSE:** (0.0 + 0.0) / 2 = 0.0 This shows that the neuron achieves perfect predictions on these examples, demonstrating how strongly positive/negative weighted sums produce confident probability predictions.
Constraints