13.8x
lower cost per task, with higher accuracy than the best frontier model
A Qwen3-235B model fine-tuned on Tinker with investor-labelled data to do six daily information-filtering tasks (e.g. which news matters to a macro investor) (source).
Shopify: ~7x faster responses from a post-trained open model
Shopify (commerce software, ~8,000 employees) reported this on 21 Oct 2026 in company self-report (source). It belongs to the learning loop layer.
What they built
A post-trained NVIDIA Nemotron 3.5 Lightning (30B MoE, 3B active) 'task expert' that took over Sidekick's GraphQL sub-agent from a frontier model, served on Shopify's own GPUs.
The result
Judge score rose from 37.7 (base) to 74.5 after six loops and ~61,000 repaired examples, vs 73 for the frontier model; median full response 0.87 s vs 6.50 s (~7x faster); ~90% lower serving cost.
The number covers graphQL sub-agent on Shopify's internal judge and replayed production traffic.
“…because Shopify controls the post-trained weights, we can serve the model on our own GPUs and decide when it changes,” Shopify said.
The layer: The learning loop
Reviewed corrections are kept as test cases and later used to train models.
- A log of every correction and who reviewed it
- Eval sets that grow from reviewed corrections
- New versions, then post-trained models, from that signal
Read the guide to the learning loop.
Playbooks that use this case
More cases on the learning loop
1% → 99%
valid structured output after fine-tuning a small model
A proof of value on Red Hat OpenShift AI: GPT-4.1 generated synthetic training data, then small models (Phi-4 Mini 3.8B, Qwen 2.5 1.5B/7B) were fine-tuned for multilingual structured extraction from social media, deployable locally (source).
Own LLM
beat commercial models in internal tests, no figures published
TravelersLLM, a proprietary insurance LLM built by in-house engineers on millions of company documents with underwriting, claims and service experts; applications route each query to it or to a frontier model (source).