Benchmark
A benchmark is a fixed set of tasks and scoring rules used to compare tools on the same test.
What it means
A benchmark gives tools the same tasks, scores the outputs with a shared method, and reports the results. For research AIs, benchmarks such as DRACO and BrowseComp try to measure report quality or search accuracy. Because all tools face the same questions, the scores are more comparable than marketing claims.
Benchmarks have limits. Scores depend on the task set, the judge, and the rubric. A small task count makes results noisy, and a vendor that runs its own test may choose tasks that suit it. Read the method before the number, and run a benchmark on your own questions before you commit to a tool. A useful habit is to note which step of the process produced each claim you rely on. Treat it as one input among several, not as proof on its own.
Continue this reading path
Cite this entry
https://bestresearchai.com/glossary/benchmark/