AI Spaceship

LLM Engineering From First Principles

8 modules

Build Your Own LLM

Learn the full LLM pipeline through compact lessons and executable labs.

Build, train, and adapt GPT-style language models through structured hands-on lessons.

Course Snapshot

Modules
8
Lessons
39
Completion0%

Sign in to track progress

Module 1: Introduction

3 lessons

The LLM PipelineOpen

Lesson in Module 1: Introduction

Open lesson
NumPy for Neural Networks Locked

Lesson in Module 1: Introduction

Locked
Vectors and Similarity Locked

Lesson in Module 1: Introduction

Locked

Module 2: Data Preparation

5 lessons

Reading Input Data Locked

Lesson in Module 2: Data Preparation

Locked
BPE Tokenization with tiktoken Locked

Lesson in Module 2: Data Preparation

Locked
Data Sampling with Sliding Window Locked

Lesson in Module 2: Data Preparation

Locked
Token and Positional Embeddings Locked

Lesson in Module 2: Data Preparation

Locked
Batching with a DataLoader Locked

Lesson in Module 2: Data Preparation

Locked

Module 3: Attention Mechanisms

5 lessons

Dot-Product Similarity Locked

Lesson in Module 3: Attention Mechanisms

Locked
Self-Attention from Scratch Locked

Lesson in Module 3: Attention Mechanisms

Locked
Scaled Dot-Product Attention Locked

Lesson in Module 3: Attention Mechanisms

Locked
Causal (Masked) Attention Locked

Lesson in Module 3: Attention Mechanisms

Locked
Multi-Head Attention Locked

Lesson in Module 3: Attention Mechanisms

Locked

Module 4: The Transformer Block

5 lessons

GELU Activation Function Locked

Lesson in Module 4: The Transformer Block

Locked
Position-Wise Feed-Forward Network Locked

Lesson in Module 4: The Transformer Block

Locked
Layer Normalization Locked

Lesson in Module 4: The Transformer Block

Locked
The Transformer Block Locked

Lesson in Module 4: The Transformer Block

Locked
Stacking Blocks into a GPT Locked

Lesson in Module 4: The Transformer Block

Locked

Module 5: Training the GPT

6 lessons

Cross-Entropy Loss Locked

Lesson in Module 5: Training the GPT

Locked
Loss Gradients Locked

Lesson in Module 5: Training the GPT

Locked
Backpropagation Through a Linear Layer Locked

Lesson in Module 5: Training the GPT

Locked
Gradient Descent Locked

Lesson in Module 5: Training the GPT

Locked
The Training Loop Locked

Lesson in Module 5: Training the GPT

Locked
Autoregressive Text Generation Locked

Lesson in Module 5: Training the GPT

Locked

Module 6: Inference and Evaluation

5 lessons

Saving and Loading Weights Locked

Lesson in Module 6: Inference and Evaluation

Locked
Perplexity Locked

Lesson in Module 6: Inference and Evaluation

Locked
End-to-End Inference Pipeline Locked

Lesson in Module 6: Inference and Evaluation

Locked
KV Cache for Fast Inference Locked

Lesson in Module 6: Inference and Evaluation

Locked
Beam Search Locked

Lesson in Module 6: Inference and Evaluation

Locked

Module 7: Fine-Tuning for Classification

5 lessons

Transfer Learning Locked

Lesson in Module 7: Fine-Tuning for Classification

Locked
Replacing the Output Head Locked

Lesson in Module 7: Fine-Tuning for Classification

Locked
Freezing Layers Locked

Lesson in Module 7: Fine-Tuning for Classification

Locked
Classification Training Loop Locked

Lesson in Module 7: Fine-Tuning for Classification

Locked
Evaluation Metrics Locked

Lesson in Module 7: Fine-Tuning for Classification

Locked

Module 8: Instruction Fine-Tuning

5 lessons

Instruction Data Format Locked

Lesson in Module 8: Instruction Fine-Tuning

Locked
Chat Templates Locked

Lesson in Module 8: Instruction Fine-Tuning

Locked
Supervised Fine-Tuning Locked

Lesson in Module 8: Instruction Fine-Tuning

Locked
Response Extraction Locked

Lesson in Module 8: Instruction Fine-Tuning

Locked
Instruction-Following Evaluation Locked

Lesson in Module 8: Instruction Fine-Tuning

Locked