Artificial intelligence · Research Worlds
A simple change, a huge jump
Instead of reading a sentence one word after another, the model looks at all the words at once and decides what to pay attention to.
Free · No account required · Runs in your browser
Watch the 90-second visual story ↗What will you explore?
Vaswani et al., 2017
Attention connects tokens directly; the Transformer can train on many positions in parallel.
How does it work?
Watch a 90-second cinematic explanation, or explore a story that pauses so you can change its central mechanism. Inspect the source evidence, compare outcomes and resume the narration. This free Lesson Worlds adaptation needs no account or download; younger learners may want help from a parent or teacher.
Inside the cinematic story
A sentence is a network.
Context travels between positions.
Ask. Compare. Combine.
Queries · keys · weighted values
Change the point of view.
Different positions gather different context.
Take control: Which position should gather information? First word · Middle word · Last word
Training can work in parallel.
Parallel training across known positions
The future must stay hidden.
A causal mask blocks future positions.
Take control: Should future positions be visible? Decoder mask · Unmasked comparison
Information takes a new route.
Explore connections and their limits.
Discuss the evidence
Before starting, explain what you think this claim means: Attention connects tokens directly; the Transformer can train on many positions in parallel. After exploring, choose one stage above and describe what its model helps you see. Separate what you observed in the adaptation from what the original researchers measured.
Read the original publication for its methods, evidence and limitations. Ask which part of the explanation the interactive model leaves out, and what additional evidence you would need to test the claim yourself.
Original token network with invented normalized weights, not values from a trained Transformer or an explanation of a model's reasoning. Parallel training and autoregressive generation are distinguished.
Original research
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser & Illia Polosukhin · NeurIPS 2017 (arXiv:1706.03762) · 2017
Educational adaptation by Lesson Worlds. Not affiliated with the authors, Google or NeurIPS. The original publication may have its own access terms.