attention mechanism

The part of a transformer that decides which earlier tokens matter for the token being produced now. Its cost grows with the square of the context, which is why long contexts are expensive.

ai 1 video 1 mention also: self-attention

Where it comes up

Training Composer 2

May 14, 2026first said at 3:511 mention

“It also uses uh multi-head latent attention for its attention mechanism, which makes it efficient to serve. When deciding on which base model to use, we tested several different open models on several different benchmarks. Some of these loo…”