Understanding Neural Networks

Neural networks are the foundation of deep learning, inspired by biological neurons.

Architecture

A neural network consists of:

1. Input Layer - Receives raw data
2. Hidden Layers - Process and transform data
3. Output Layer - Produces predictions

Activation Functions

• ReLU: f(x) = max(0, x)
• Sigmoid: f(x) = 1 / (1 + e^(-x))
• Softmax: Used for multi-class classification

Backpropagation

The algorithm that computes gradients and updates weights using the chain rule of calculus.

import torch
import torch.nn as nn

class SimpleNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.layers = nn.Sequential(
            nn.Linear(784, 128),
            nn.ReLU(),
            nn.Linear(128, 10)
        )
    
    def forward(self, x):
        return self.layers(x)