Inside an LLM: How a Language Model Picks the Next Word

A large language model writes by repeating one step: score every possible next word, turn the scores into probabilities, pick one, and add it to the text. This toy tea-shop model uses a small hand-written table of odds (illustrative numbers, not a real model) so you can watch that loop at human speed.

Choose a starting sentence, then press Next word to add one word at a time, or Finish sentence to run it to the end. The bars show the model’s odds for the next word. Drag Temperature to flatten or sharpen those odds, and press Start over to try again.

What this teaches: Watch a language model choose each word from a list of probabilities, and see how temperature changes the odds. It teaches next-token prediction, probability and why a fluent answer is not the same as a true one.

Inside an LLM shows the next-word prediction at the heart of the generative AI covered in DAVHILL’s AI Fundamentals for Professionals (opens in a new tab) course.

Where large language models fit among the approaches to data science is part of DAVHILL’s Intro to Data Science (opens in a new tab) course.