Deep Learning & Neural Networks

What Is Deep Learning?

Deep Learning uses artificial neural networks with many layers to learn hierarchical representations. Each layer transforms the input into increasingly abstract features.

The Neuron

An artificial neuron:

Common activations: ReLU (default for hidden layers), Sigmoid (binary classification), Tanh (-1 to 1 range), Softmax (multi-class)

Backpropagation

Training happens in three phases:

Why Depth Matters

Shallow networks learn simple patterns. Deep networks learn hierarchical features — first layers detect edges, middle layers detect shapes, final layers detect concepts.

Common Mistakes

Your Turn!

Implement a single neuron:

import math

def neuron(inputs, weights, bias, activation='relu'):
    z = sum(w * x for w, x in zip(weights, inputs)) + bias
    if activation == 'relu':
        return max(0, z)
    elif activation == 'sigmoid':
        return 1 / (1 + math.exp(-z))

# Test AND gate
inputs = [(0, 0), (0, 1), (1, 0), (1, 1)]
for x1, x2 in inputs:
    output = neuron([x1, x2], [0.5, 0.5], bias=-0.7, activation='sigmoid')
    print(f"AND({x1}, {x2}) = {output:.3f} (expected {x1 and x2})")