New playbook: when your own AI model pays. Read

DoorDash: 2,000 sessions auto-graded a day before launch

DoorDash (delivery marketplace, ~20,000+ employees) reported this on 30 Jun 2026 in company self-report (source). It belongs to the agents, traceability and evals layer.

What they built

An evaluation harness for the multi-agent Ask DoorDash ordering assistant: rubrics per agent, a transcript builder, a conversation simulator and auto-grading of live sessions, used as the launch bar.

The result

Quality signal grew from ~1 employee feedback report to 2,000 auto-graded sessions a day; agent quality score up 8 points and error rate nearly halved before nationwide launch; a manual regression test that took more than six hours now runs in ~20 minutes.

The number covers ask DoorDash agent, pre-launch and production evaluation.

“…we drove an 8-point improvement in agent quality scores ahead of nationwide launch,” DoorDash said.

Guardrails

Separate evals for routing, guardrails (trust and integrity) and per-agent capability; offline replay before any change; the harness validated a base-model swap that cut latency 35% without quality loss.

The layer: agents, traceability and evals

Each agent has one narrow task, a named owner and an action log.

  • Agents with one narrow job and an identity
  • A trace of every tool call and decision
  • Eval sets that gate every change

Read the guide to agents, traceability and evals.

Playbooks that use this case

More cases on agents, traceability and evals

  • 12,500

    banker hours saved in the first month of operation, July 2026

    Five specialist agents on Amazon Bedrock AgentCore that classify and read payslips and bank statements, run serviceability calculations and verify them against lending policy, then hand the banker a result to review for mortgage and credit-card applications (source).

  • $10M

    EBITDA uplift from two agent systems, Partners Group says

    Two agentic systems built on a cleaned proprietary policy-level dataset: one runs new-client policy processing, the other automates human-intensive policy checking; delivered by a seven-person team with Version 1 (source).

  • 2–3 minutes

    per ocean or air freight quote, down from one to three days

    Hundreds of in-house AI agents embedded across the shipment lifecycle (quoting, orders, appointments, missed-pickup checks) and now ocean and air quoting, ocean booking and customs brokerage, built and run under a lean waste-removal discipline by 450+ in-house engineers and data scientists (source).

We map how your company runs, pick the first processes with your leadership team, and put them on agents.