Vaswani et al., 2017

A simple change, a huge jump

Instead of reading a sentence one word after another, the model looks at all the words at once and decides what to pay attention to.