Loading...
Loading...
How neural networks learn — neurons, layers, backpropagation, and why depth matters for modern AI
Deep Learning uses artificial neural networks with many layers to learn hierarchical representations. Each layer transforms the input into increasingly abstract features.
An artificial neuron:
Common activations: ReLU (default for hidden layers), Sigmoid (binary classification), Tanh (-1 to 1 range), Softmax (multi-class)
Training happens in three phases:
Shallow networks learn simple patterns. Deep networks learn hierarchical features — first layers detect edges, middle layers detect shapes, final layers detect concepts.
| Wrong | Why | Right |
|---|---|---|
| Neurons = brain cells | Mathematical abstraction | Simplified models |
| More layers always helps | Vanishing gradients | Residual connections help |
| Deep learning solves everything | Needs massive data | Simple ML sometimes better |
Implement a single neuron:
import math
def neuron(inputs, weights, bias, activation='relu'):
z = sum(w * x for w, x in zip(weights, inputs)) + bias
if activation == 'relu':
return max(0, z)
elif activation == 'sigmoid':
return 1 / (1 + math.exp(-z))
# Test AND gate
inputs = [(0, 0), (0, 1), (1, 0), (1, 1)]
for x1, x2 in inputs:
output = neuron([x1, x2], [0.5, 0.5], bias=-0.7, activation='sigmoid')
print(f"AND({x1}, {x2}) = {output:.3f} (expected {x1 and x2})")