Multi-Token Prediction LM
A small language model that predicts several tokens ahead, reimplemented from the paper (arXiv:2404.19737).
- Generates with 1.28× fewer forward passes by checking its own drafts.
- Peak training memory cut from 1,778 MB to 790 MB.
- Compared against a baseline with exactly the same 5,836,032 parameters.
GitHub ↗
Transformer from Scratch
A complete transformer written in NumPy, with no deep-learning framework.
- Attention, multi-head attention, positional encoding and layer norm, by hand.
GitHub ↗
Neural Network from Scratch
A neural network in NumPy alone that reaches 97.58% on MNIST.
- Backpropagation, ReLU, softmax and He initialization, by hand.
GitHub ↗
Paper Explainer
Explains arXiv papers in plain language with a fine-tuned Llama 3.2 1B.
- A LoRA fine-tune that trains only 0.07% of the parameters.
GitHub ↗