prompt caching

Reusing the model's processing of a prompt prefix that has not changed, so a long unchanging system prompt is only paid for properly once.

ai 1 video 1 mention

Where it comes up

Refactoring Legacy Codebases

Jan 8, 2026first said at 20:421 mention

“long time. So, you have to be mindful of that. Um, I think that in the ideal state, the agent harnesses will build in the tools that get very very good at doing summarization over long periods of time because then you can take advantage of…”