KV cache
The intermediate state a model keeps for tokens it has already read, so generating token 5,001 does not mean re-reading the first 5,000.
ai
Nobody says this one in the transcripts we have yet.
The intermediate state a model keeps for tokens it has already read, so generating token 5,001 does not mean re-reading the first 5,000.