This repository is a feed-forward neural network in NumPy, with no autograd and no deep-learning framework. The models train on Fashion-MNIST and CIFAR-10. The project uses Weights & Biases (WandB) heavily for experiment tracking.
Engineering without abstractions
Dropping autograd means numerical stability is handled directly. The code does manual forward and backward passes, including:
- Stable softmax and cross-entropy so gradients do not explode.
- Weight-initialization scaling so variance stays usable across depth.
- Manual optimizers (SGD with momentum, Adam) and how they change the loss landscape.
- L2 regularization and gradient clipping.
The repo also has the operational pieces a real pipeline needs: minibatching, early stopping, checkpointing, and reproducible runs.
Experiment tracking over cherry-picking
The write-up does not lead with a single vanity metric. It commits to concrete baselines (~89% on Fashion-MNIST, ~45% on CIFAR-10 for an FFNN) and documents the hyperparameter search. WandB keeps the sweep history, failures, and tuning decisions visible. Measurement over cherry-picked peak accuracy.
Note: This was a DTU group project. The repository lives under the group’s GitHub organization.