Language models · September 2026
Modern Model Architectures
Block by Block
Learn the basics. Compute the blocks. Compare modern model architectures. A practical, illustrated path from vectors and probability to attention, memory, expert routing, and the code that makes them work.
89 pages · 7 model families · 26 illustrations
Working manuscript. Corrections and reader feedback are welcome.
Inside the book
Build the foundations
Vectors, matrices, probabilities, loss, and a tiny decoder. Start with basic Python and arithmetic, then work through concrete examples.
Understand the architecture
Explore 44 concepts across attention, recurrent memory, normalization, position, expert routing, residuals, and depth.
Connect it to code
Compare GPT, Qwen, DeepSeek, Qwen-Next, Looped, Gemma, and Mamba through small PyTorch implementations and exercises.
Read online
Open PDF in a new tab ↗Use your browser’s PDF controls to search, zoom, or browse the contents. If the reader isn’t available on your device, open the PDF directly or download a copy.
Help shape the next draft
What could be clearer?
Found an error, got stuck on an explanation, or tried an exercise? I’d love to hear from you. Include the page or chapter and what you think could improve.
Email feedbackOpens your email app. You can also write to ayoubkirouane3@gmail.com.