Harvey AI case study: 0.853 → 0.913

What they built

GLM-5.1 post-trained with full-parameter asynchronous RL inside Harvey's Legal Agent Benchmark harness, with a cheap batched grader replacing frontier graders.

The result

Rubric pass rate 0.853 to 0.913, above Opus 4.8 Max and GPT-5.5 xhigh; all-pass rate 0.059 to 0.126; grading cost cut 40x by a smaller model with batched criteria.

The number covers 180-item held-out set of Harvey's Legal Agent Benchmark.

“…it outperforms Opus 4.8 Max and GPT-5.5 xhigh, a threshold that previous trained models across the industry had not yet reached,” Harvey said.

The layer: The learning loop

Reviewed corrections are kept as test cases and later used to train models.

A log of every correction and who reviewed it
Eval sets that grow from reviewed corrections
New versions, then post-trained models, from that signal

Read the guide to the learning loop.

More cases on the learning loop

Shopify

96%

lower estimated serving cost for one agent

A daily continual-learning loop for Sidekick's GraphQL agent: an expert-calibrated LLM judge scores production conversations, a self-healing pipeline repairs failures, and the repaired trajectories are trained into a smaller model (SFT, then GRPO against the judge) that replaces the frontier model (source).

Bridgewater Associates

13.8x

lower cost per task, with higher accuracy than the best frontier model

A Qwen3-235B model fine-tuned on Tinker with investor-labelled data to do six daily information-filtering tasks (e.g. which news matters to a macro investor) (source).

Atos

1% → 99%

valid structured output after fine-tuning a small model

A proof of value on Red Hat OpenShift AI: GPT-4.1 generated synthetic training data, then small models (Phi-4 Mini 3.8B, Qwen 2.5 1.5B/7B) were fine-tuned for multilingual structured extraction from social media, deployable locally (source).

Harvey: 0.853 → 0.913 rubric pass rate after RL post-training