Build your own small language model from any text – and see exactly how it chooses the next word, one word at a time.
The order is a suggested workflow – feel free to jump freely between steps.
🗂️ Training Data
✓🌦️The Weather ReportSimple, repetitive Swedish text
✓⚽The Sports ReportMore varied Swedish text
✓📖HemsöbornaSwedish novel (Strindberg)
✓🎭ShakespeareLarge English collection
Type at least one word.
Train the model and it will start writing.
The model has not been trained yet.
context (memory) chosen worddead end
✓🎲Weighted randomRandomizes by probability
✓👑Greedy choiceAlways picks the most common word
1.0
🌡️ Drag the slider and see what happens to the text.
Language Model Lab
Discover how a language model works by building, analyzing, and evaluating your own model step by step.
1
How does the model choose the next word? Explore how the model continues a text and discover how it chooses between different possible words.
2
What does the model remember? Examine how many previous words the model looks back at when choosing the next word – and what that does to the text.
3
How is the model's choice affected? Experiment with temperature and different ways of choosing the next word.
4
How does the training data shape the model? Switch between different texts and see how the language changes – then combine everything freely and finish by comparing the model with ChatGPT.