Illustrate GPT

How does ChatGPT actually write? Illustrate GPT answers that question with a real language model running entirely on your iPhone. You don't just read about it: you open it up and watch it think. Illustrate GPT runs Baby Llama 2, a tiny GPT trained on children's stories. It has the same architecture as the big models, small enough to run offline in real time. Eight short interactive chapters take you from raw text to the next word, and every chart comes from the live model. THE COURSE 1. Meet Baby GPT. Watch a story being written one token at a time. Each word is colored by how confident the model was. Tap any word to see the alternatives it considered, and pick a different one to rewrite the story from that point. 2. Tokens. Type anything and see how it gets chopped into pieces and turned into numbers. Find out why " cat", "Cat" and " CAT" are different things to a model. 3. Embeddings. Every token is a list of 288 numbers. See a word's vector, find its nearest neighbors (king → queen, prince, crown), and measure how similar two words are. 4. Attention. Tap a word and see which earlier words it looks at. Explore all 36 attention heads. Each one's role (previous-token, duplicate-token, attention sink, and more) is detected automatically. 5. Layers. Use the "logit lens" to watch the prediction take shape block by block, from a vague guess at the bottom to a confident answer at the top. Then browse the full model blueprint with real weight shapes. 6. Sampling. Turn the temperature, top-k and top-p knobs and watch the probabilities change in real time. Roll the dice, and write three different futures from the same sentence. 7. The Loop. Step through the whole pipeline for a single token, from tokenize and embed through 6 transformer blocks to probabilities, sample and append, then do it again. 8. Your Turn. Now you are the model. Guess the most likely next word and see how your intuition compares with a neural network. BUILT FOR LEARNING • Every chapter starts with a "predict first" question, so you form a guess before you see the answer. • Key ideas, common myths vs. reality, and an optional "Go deeper" section in every chapter. • Your progress is saved, so you can learn at your own pace. PRIVATE AND OFFLINE The model runs on your device with Apple's MLX framework. No account, no server, no data leaves your phone. FREE Every feature is free. An optional tip supports development. Perfect for students, teachers, developers, and anyone curious about what's really going on inside AI.