Inside DoorDash's software factory
Flux, DoorDash's in-house cloud-agent platform, started with one narrow use case, code review, before expanding to 130,000 automated engineering tasks a month; the reviewer it grew from now covers 10,000+ pull requests a week across 56 repositories and can push its own suggested fixes straight back to the PR.
DoorDash at a glance
The company
| Sector | "Our mission is to grow and empower local economies" 6 |
|---|---|
| Employees | 31,400+ worldwide as of December 31, 2025 6 |
| Customers | 56m+ monthly active users; 35m+ DashPass, Wolt+, and Deliveroo Plus members 7 |
| Revenue | $13.72bn full-year 2025 revenue; Marketplace GOV grew 27% YoY for the full year 7 |
No source discloses an engineering-specific headcount separate from the company-wide employee count.
The engineering organisation
| Engineering size | Not disclosed as a headcount, though individual contributors are named across three separate initiatives: Flux (Santosh Banda, Jeffrey Hwang, Siddarth Kodwani, all Tech Lead Engineers, plus 16 acknowledged contributors including Adam Rogal, Adam Yarger, and Andy Fang), the code reviewer (Adam Yarger, Infrastructure Engineering; Adam Rogal, Director of Engineering), and DashBench (Saketh Dhulipalla, Vasily Vlasov, Marcus Yearwood, and Adam Yarger again) 1,2,3 |
|---|---|
| AI tooling since | A third-party AI review tool (46% acceptance) was already in use before DoorDash replaced it with an in-house reviewer that itself went through three internal architecture versions; Flux, the platform that reviewer now runs on, debuted in Q1 2026 with code review as its first production use case 1,2 |
| Tool policy | Built in-house — Flux, the code-review agent, DashBench, and the Agent Gateway are all custom-built, replacing an earlier third-party review tool, though DoorDash still uses third-party models and tools inside its own primitives 1,2 |
| Agents in the SDLC | A cloud background-agent platform (Flux) running CI triage, on-call tasks, maintenance workflows, and ticket-driven development; a three-stage automated code-review agent (scout plus two deep reviewers) that runs as a Flux playbook; a "Fixer" agent that applies suggested changes when tagged in a PR thread; an in-house MCP-based Agent Gateway scoping tool access per playbook 1,2 |
| Reported adoption | "Flux: 130,000 automated engineering tasks in one month, 300+ playbooks, 10,000+ playbook invocations per week (2026) 1. Code reviewer: 10,000+ pull requests per week across 56 onboarded repositories, a 60.2% acceptance rate on settled high/critical findings (n=2,256), up from 46% with the prior third-party tool 2,3" |
| Published metrics | Task and invocation counts, playbook counts, PR-review volume and repo count, review acceptance rate (with an n), review cost and latency, a 105-case internal benchmark with weighted precision/recall/F1 by model configuration and severity tier 1,2,3 |
Acceptance rate can't carry the full weight of this claim on its own: in confusion-matrix terms, it only ever populates the true-positive and false-positive cells, never the false negatives (missed bugs) or true negatives (justified silence). The 60.2% figure is a production-usage number under that limitation; DashBench's own 105-case benchmark puts the same production configuration at 53.6% weighted recall and 87.0% weighted precision 3.
People
What they set out to solve
By 2026, DoorDash had three separate AI efforts running inside its own engineering organization: a cloud platform for background engineering tasks, an automated pull-request reviewer, and a benchmark built specifically to decide whether the reviewer's own numbers could be trusted. The reviewer's story starts with a replacement, not an invention. An earlier third-party AI reviewer was already running against pull requests before DoorDash built its own, converting only 46% of its settled high- and critical-severity findings into an actual code change. DoorDash's in-house reviewer pushed that to 60.2%. Bolted-on AI review has a trust problem generally, not just the version DoorDash was running: the comments get noisy, the suggestions get generic, and engineers learn to scroll past them. The fix treats this as a problem of attention, not raw model quality, getting the agent to focus on the parts of a change that deserve scrutiny and stay quiet on the rest 2.
Running agentic workloads on laptops hit three separate limits at once: fixed CPU, memory, and uptime shared with everything else on the machine; broad credential and VPN access an autonomous agent didn't need; and fragmented, hard-to-audit execution once work scattered across individual machines. Flux delegates that work to secure, autonomous agents instead, freeing engineers for "innovation, critical thinking, and solving complex problems." Hosted coding agents also forced an uncomfortable choice, sending sensitive code to a third party or opening a path from that third party back into internal systems, so DoorDash built the surrounding primitives, sandboxes, gateway, playbooks, in-house, while still using third-party models and tools inside them 1.
Leaning on acceptance rate to judge a reviewer is optimizing against an incomplete signal: it can only ever confirm that a comment was accepted or rejected, never that a missed bug went uncaught or that silence on clean code was the right call 3. DashBench exists to close that gap.
Maturity level
Assigned by testing published facts against fixed criteria, and capped by what the sources state.
How DoorDash's software factory works
Flux didn't start broad. It debuted in Q1 2026 with one narrow use case, automated code review, chosen because it was "frequent, measurable, and easy for engineers to evaluate," and that one production workflow is what DoorDash used to tune quality, latency, cost, and behavior before expanding into CI triage, on-call tasks, maintenance playbooks, and ticket-driven development. The two efforts share infrastructure: "we run both the evals for our code review on Flux infrastructure" 1.
Flux runs on four primitives. Firecracker microVM sandboxes give each agent run an isolated, pre-loaded environment, repository, developer tools, secrets, and runtime dependencies already in place, targeting a 95th-percentile full end-to-end setup, starting the microVM through cloning repositories, installing build tools, and configuring whichever coding-agent harness is in use, under five seconds; the platform runs multiple such harnesses rather than one proprietary agent . Work is packaged into "playbooks," YAML specifications DoorDash calls "the equivalent of a Docker container for skills and agent-driven tasks," mixing agentic steps (flexible, judgment-driven) with deterministic ones (predictable, cheap, easy to validate) so logic moves between the two as a workflow matures. Playbooks trigger from wherever the work already starts, Slack, GitHub, a cron schedule, a CLI, or a conversational skill, all converging on the same execution path. An in-house MCP-based Agent Gateway sits underneath, scoping exactly which tools a given playbook can reach and logging every action. In a single month, Flux ran 130,000 automated engineering tasks this way, across more than 300 playbooks and over 10,000 invocations a week 1.
The code reviewer went through three internal versions before shipping. The first fanned review out across specialist agents, security, tests, performance, code quality, which caught mechanical bugs well but consistently missed problems that crossed domain boundaries, because no specialist was looking at the whole change. The second gave two general-purpose reviewers the full context, which improved architectural coverage but spread attention too thin across everything a reviewer had to check, so real findings still got lost despite the reviewers having the information needed to catch them. The version that shipped separates noticing from verifying: a "lead scout" reads the full diff and flags places that "feel off" without trying to confirm anything, and two parallel "deep reviewer" agents then investigate the scout's strongest leads, confirming what holds up and dropping what doesn't. Code review agents don't fail only because models make mistakes; they fail because they spend attention poorly and spend trust too quickly 2.
Every finding survives a deliberate filter before it reaches an author. The design principle is precision over recall: rather than surface every possible concern and let a human filter the list, each finding passes a "disprove-it" pass, an explicit self-falsification step, before it's posted at all. Review knowledge is scoped per domain rather than universal, mined from four sources, the review-relevant subset of AGENTS.md files (architectural boundaries and contract rules, not setup instructions), historical PR review comments, Slack decisions and incident postmortems, and incident history, then filtered again: drop it if CI would already catch it, drop it if the model already knows it from general training, drop it if it can't be pointed at with a concrete file-and-line citation. A change to the payment-service-provider gateway loads a different rule set than a change to the consumer feed, which is how acceptance rates hold up across 56 structurally different repositories rather than one universal standard being stretched thin. The agent runs on remote VMs with a full repository clone, not just the diff, tracing callers and comparing sibling implementations the way a human reviewer would, on infrastructure that supports "modern coding-agent harnesses with full filesystem and tooling access" 2.
In a typical week, this reviews more than 10,000 pull requests across 56 onboarded repositories, Go backend services, iOS and Android apps, web frontends, infrastructure, and data pipelines, many fired automatically on PR open, often before a human has looked at the diff. At roughly $3 and seven minutes per review, 60.2% of settled high- and critical-severity findings (n=2,256) get an author to change code before merge, up from 46% under the third-party tool it replaced; webhook- triggered findings, not manually invoked ones, land at 59.0% 2.
Closing the loop: the fixer
Finding an issue and fixing it used to be two separate handoffs. Now anyone can tag the review agent directly in a GitHub PR thread, "Can you handle the nil check here?", and a "Fixer" agent picks it up: it runs in a remote VM with a full checkout of the repository and the original review context, the diff, the finding, the surrounding code, and the suggested direction, carried forward as its prompt, then pushes the change back to the same PR. Multiple comments on one PR get addressed concurrently, each in its own isolated worktree, merged back to the head branch in order. The result is deliberately unremarkable: a normal PR commit or diff, subject to the same CI and human review as any other change 2. Nothing skips that review step.
DashBench: measuring a reviewer without trusting any single signal
Trusting the 60.2% figure took a separate measurement effort. Acceptance rate is structurally incomplete: in confusion-matrix terms, an accepted comment books a true positive and a rejected one a false positive, but acceptance can never register a false negative (a bug the reviewer missed) or a true negative (clean code where staying silent was correct) 3. DashBench closes that gap. Disagreement audits, comparing author annotations, the original candidate findings, and an independent agentic judge, turned up mistakes on both sides, human review and the agent's own verification step alike; the point wasn't to declare either side right, but to separate "was accepted" from "was real" as genuinely distinct questions 3.
The benchmark started from roughly 1,000 raw PR candidates, narrowed to a 105-case valid report spanning historical PRs with real findings, benign PRs used to measure restraint (a reviewer loud on clean code is its own failure mode), and PRs later reverted or hotfixed, testing whether a system would have caught the regression before merge. On that set, the production configuration (Claude Sonnet 4.6 high scout plus Claude Opus 4.8 high reviewer) found 504 real findings at 53.6% weighted recall and 87.0% weighted precision, at $3.91 and 725 seconds per PR, against 164 findings at 30.7% recall and 84.1% precision, at $0.75 and 170 seconds, for a no-scout single-pass baseline. No single model pairing won on every axis: a Kimi K2.6-scout, Claude-Fable-5-reviewer configuration led weighted recall and F1 (65.2% and 75.3%, 537 real findings, $3.81 per PR), while a Composer-2.5-scout, GPT-5.5-medium-reviewer pairing led weighted precision (92.2%) at substantially lower recall. Four lessons came out of it: no single signal, human, production, or agentic, is reliable enough alone as ground truth; human labeling doesn't scale at this volume, so AI judging is what makes evaluation tractable, under calibration; model and judge variance is a feature to design around, not a bug to average away; and a single score is inherently misleading, which is why the report tracks weighted precision, recall, F1, cost, and latency together instead of naming one winner 3.
Operating the software factory: teams, roles, and skills
Three separate DoorDash teams show up in the SDLC-relevant record, tied together only by byline, not by any disclosed org chart. Santosh Banda, Jeffrey Hwang, and Siddarth Kodwani, all Tech Lead Engineers, Banda and Hwang in agentic software development, Kodwani on the Generative AI platform, built Flux, with thirteen more people credited as contributors 1. Adam Yarger, a software engineer on the Infrastructure Engineering team, and Adam Rogal, a Director of Engineering, built the reviewer's scout-plus-deep-reviewer architecture 2. Saketh Dhulipalla (Member of Technical Staff), Vasily Vlasov (Principal Engineer), Marcus Yearwood (Member of Technical Staff, Lead), and Adam Yarger again, the one name spanning both code-reviewer posts, built DashBench 3. None of these efforts has a disclosed headcount. Co-founder Andy Fang is credited as a Flux contributor 1.
A separate, unrelated team, DoorDash's Data Platform organization, Aydar Akhmetzyanov, Harsha Venkat Annapa Reddy, Jash Radia, Gurudev Jagdale, Sahal Sadique, and Lokesh Sharma, built a business-intelligence agent platform for internal knowledge access, entirely apart from Flux, the reviewer, and DashBench 4.
Implementation journey
- November 11, 2025DoorDash's Data Platform team publishes the earliest dated agent-related account found for the company, an internal business-intelligence agent platform, unrelated to the SDLC 4.
- Q1 2026 (exact date undisclosed): Flux debuts, with automated code review, replacing an earlier third-party review tool, as its first production use case 1.
- May 11, 2026DoorDash publishes the reviewer's own account of reaching a three-version scout-plus-deep-reviewer architecture; a typical week already covers 10,000+ pull requests across 56 repositories, and the "Fixer" agent for tagging in a fix is described 2.
- July 6, 2026DashBench publishes, with the confusion-matrix argument against trusting acceptance rate alone and a 105-case benchmark of scout/reviewer model configurations 3.
- August 11, 2026
Measuring the software factory
The reviewer's before-and-after is the sharpest comparison in the record: 46% of settled high/critical findings converted to a code change under the prior third-party tool, versus 60.2% (n=2,256) under the in-house version, with webhook-triggered findings specifically at 59.0% 2. DashBench's own benchmark gives a cleaner, independently adjudicated number for the same production configuration: 53.6% weighted recall and 87.0% weighted precision on a 105-case set, against 30.7% recall for a no-scout baseline. None of these numbers hold up alone: acceptance rate can't see missed bugs or justified silence, and a single benchmark score hides tradeoffs between coverage, precision, cost, and latency that move independently of each other 3.
Flux's own figures are volume, not quality or outcome: 130,000 tasks in a month, 300-plus playbooks, 10,000-plus weekly invocations. There's no failure rate, rollback rate, or rejection rate for what Flux produces outside the code-review playbook specifically, and "ticket-driven development" gets named once with no metric attached 1.
Lessons learned
Flux's own lessons
Three stand out. Start narrow to earn trust: automated code review became the first use case specifically because it was frequent and measurable, giving a production workflow to tune before expanding to CI triage, on-call, maintenance, and ticket-driven development. Make the work visible: an early Slack integration created private per-run channels, which made Flux useful to individuals but didn't build team habits; moving delegation into public threads is what actually built adoption, because engineers could watch each other's runs and build trust together. Playbooks need enablement: reusable workflows didn't appear just because the primitives existed, workshops and hackathons are what helped teams recognize which of their own repeated operational work was worth encoding 1.
The reviewer's engineering lessons
Several only showed up running against real PRs. A stuck agent once looped without advancing a turn counter, burning twenty minutes of tokens before a single overall timeout caught it; that became per-agent soft timeouts (stop investigating, return only verified findings) and hard timeouts (a kill switch). The cheapest model isn't always the cheapest review, a weaker model retrying invalid structured output can cost more per successful review than a stronger model that gets it right the first time. A correct finding can still be a bad comment: broad, unanchored "consider checking" language erodes trust even when the underlying concern is real, so the system withholds anything it can't anchor to a specific file, line, and action point. And the final report needed its own guardrails, against posting a false-clean review when issues exist, against stale findings on a PR that changed mid-review, against duplicate comments piling up across re-reviews 2.
DashBench's measurement lessons
No single signal, human acceptance, production telemetry, agentic judgment, is reliable enough alone as ground truth; each was wrong often enough in disagreement audits that trusting any one of them exclusively would have quietly poisoned the scores. Human labeling doesn't scale to this volume, so AI judging is necessary, not optional, but only under calibration, audit sets, and stable rubrics. Model and judge variance is something to design around, run repeatedly and aggregate, rather than a noise source to average away in a single pass. And a single number is misleading by construction: weighted precision, recall, F1, cost, and latency move independently enough that "better" has no meaning until the axis is named 3.
What's next for DoorDash's software factory
None of the three SDLC-relevant efforts converge on one roadmap. Flux has more posts coming on its own platform primitives and developer experience, and an upcoming application, "Flux Responder," an internal Slack agent, with no further detail yet 1. The reviewer's team is building a continuous evaluation harness that tests every change automatically against a growing corpus of real incidents 2. DashBench is moving to continuous benchmarking as new models and agent harnesses ship, shifting from a single LLM judge to an "agentic jury" to cut judge bias, adding benchmarks against external code-review products, and starting to benchmark coding agents themselves, not just reviewers, "on a real enterprise codebase with real tasks and features" 3. DoorDash's Data Platform team has its own separate roadmap for business-intelligence agents, a marketplace and first deep-agent systems in preview, A2A-protocol swarm work still exploratory, but nothing ties that roadmap to Flux or the reviewer 4.
