eval
A repeatable test set for a prompt, model, or agent. Without one, 'the new prompt seems better' is a feeling rather than a fact.
ai
2 videos
3 mentions
also: evaluation suite, evals
Where it comes up
Training Composer 2
“And I can give you a brief preview of Composer 2.5. Um so, this model coming out soon will have even stronger terminal band scores, as well as other similar benchmarks. Uh and that's really from refining this reinforcement learning process,…”
Cursor 201
“models then you know, that that'd be your that'd be your answer. If you if you I would do a formal eval on it though. You know, don't just don't totally distrust it, but I think we may be seeing that answer kind of shift over time. >> Now a…”