Harvey Legal Agent Benchmark
What it measures and how
Fix Harvey’s private test set, six file tools and docx, xlsx and pptx skills, with internet access disabled. Use the mean whole-task pass rate from two judges, rather than individual-condition pass rates. Fable 5’s 11.25% with Opus 4.8 fallback cannot represent one model. The official 10.42% without fallback requires review under the common configuration policy.
How this evidence is used
Under observation: complete legal-work evidence is a priority. Stable score exports, configurations and display terms remain unresolved. Only the source is listed; no automated collection or ranking yet.
Limitations and attribution
Model fallback and judging conditions affect comparability. Roughly 90% on individual conditions does not mean whole-task completion. Vals specialties are not independent organizations, and more boards do not increase total weight.
Data license: Official public scores are available for review. Harvey’s open-source tasks and code licenses do not automatically cover Vals private test results. Leaderboard display terms remain unconfirmed.
Results are published by Vals AI / Harvey. Raw scores and the e/acc consensus index use different scales and cannot be added directly.