Ito: 55% lower median cost per pipeline run after moving to an open model
Ito (software qa) reported this on 13 Jul 2026 in company self-report (source). It belongs to the models and routing layer.
What they built
Moved its largest workload (PR QA agents, ~100B tokens/week) from GPT-5.3-codex to open-weight MiniMax M3 over 11 days, repo by repo, with a same-day revert path.
The result
Median cost per pipeline run -55%; weekly spend -28% while volume +8%. Trade-offs: median step 23 to 29 min; failed terminal states 2.4% to 4.5%; open model used 2.18x output tokens.
The number covers one production pipeline (PR QA test-runner agents); within-repository before/after comparison from Ito's billing data.
“Cheap tokens are not efficient tokens,” Ito said.
The layer: models and routing
One gateway picks the AI model for each task and records what it costs.
An internal AI Gateway that, turn by turn, weighs speed, cost and expected quality and routes each prompt to the most cost-effective model, switching models mid-session, plus a library of post-trained open telco models (OTel 2.0) that run on-prem or in cloud (source).
A blended model portfolio (in-house compact models, open-source models post-trained on Pinterest data in its own secure cloud, closed models only in 'very limited cases') behind a centralized routing layer that sends routine work to lighter models; being extended to internal engineering and sales tools in H2 2026 (source).
A cost equation and weekly metric stack for its agentic 'software factory' (70%+ of PRs agent-attributed), with benchmark-driven Pareto model selection (incl. open-weight models), cheaper default subagent models, Medium reasoning effort default, 400K-token compaction cap and prompt-cache strategy (source).
We map how your company runs, pick the first processes with your leadership team, and put them on agents.