Inside Shopify's software factory
The famous CEO memo wasn't the whole record. Underneath it sits a Slack-native agent coauthoring one in eight merged PRs, a security harness that requires proof by exploit, and a refusal to count AI-authored lines of code.
Shopify at a glance
The company
| Sector | "A complete commerce platform to sell online or in person" 10 |
|---|---|
| Employees | ~7,600 worldwide, as of December 31, 2025 8 |
| Customers | Not disclosed as a discrete count |
| Revenue | $11.56bn revenue, $378.4bn GMV (fiscal 2025) 9 |
Shopify does not publicly disclose a merchant count; GMV is the closest disclosed proxy for scale 9.
The engineering organisation
| Engineering size | ~7,600 employees company-wide; no separate engineering headcount disclosed 8 |
|---|---|
| AI tooling since | April 2025, when the CEO memo made AI usage a stated expectation 4 |
| Tool policy | Mandated baseline expectation, mixed tool choice underneath it 4,7 |
| Agents in the SDLC | Slack-native coding agent coauthoring PRs; exploit-gated security scanning 1,2 |
| Reported adoption | River coauthored 1 in 8 merged PRs; 59,918 agent sessions in a 30-day window 2 |
| Published metrics | PR reversion rate, not AI-authored LOC or PR count 5 |
Shopify explicitly states it does not track AI-authored lines of code or PR counts as a success metric 5.
People
What they set out to solve
Two problems appear in the record, from two different altitudes. From the top, the CEO's April 2025 memo names a cultural one: reflexive AI usage was not yet the default, so Tobi Lütke made it an explicit expectation, tied to performance and peer reviews, and to how headcount requests get justified 4. That memo, alone, is a mandate, not a mechanism. It says nothing about how agents would actually touch the codebase.
The engineering blog describes three narrower, more concrete problems instead, each addressed by a different system built independently of the memo. Human security review of AI-generated and human-generated code alike couldn't scale, and a theoretical finding without proof wastes a responder's time, so Dispatch was built to treat a security finding as a claim that must be proven rather than asserted 1. Separately, River's authors describe it not as a tool but as a teaching surface: it runs only in public Slack channels so that a session becomes a searchable transcript, and, in Tobi Lütke's own framing, "a teaching workshop for all of Shopify" 2. And the risk of losing expensive intermediate work to a crash or a retry drove Roast, which lets deterministic code and AI steps interleave with every execution session-replayable from any step 3.
No source states a single unifying motivation connecting the memo to the engineering systems underneath it. The memo and the mechanisms are presented separately, in separate posts, by different authors, months apart.
Maturity level
Assigned by testing published facts against fixed criteria, and capped by what the sources state.
How Shopify's software factory works
Shopify's agent stack has no single entry point. It is four separately-built systems, each solving a different problem, running in production at once.
River lives only where it can be watched. It operates exclusively in public Slack channels, never direct messages, so that every session becomes a searchable transcript by construction 2. It reads code, runs tests, opens pull requests, queries the data warehouse, inspects production traces, and can push back on a plan it judges bad. In a 30-day window, Shopify recorded 59,918 agent sessions across 5,170 Slack channels, involving more than 7,000 employees, and 3,536 River-coauthored pull requests merged; across the wider record, 1 in 8 of all merged pull requests carries River as a coauthor 2.
Underneath River sits Aquifer, a purpose-built hosting platform, not agents bolted onto CI. Aquifer separates three concerns: a durable, Postgres-backed session (the canonical event log of what happened), a disposable harness (the agent loop itself, cheap to recreate), and a disposable sandbox (where code actually runs, filesystem and shell included). The harness "lives outside the sandbox," in the authors' words, which they credit with three properties: the agent can't destroy what it's isolated from, models can be swapped without disrupting the sandbox, and the entire decision stream stays observable in one place 2. Sessions run as ephemeral "session cells," described as "cattle, not pets": idle, a cell exits; the next interaction gets a fresh one, sometimes on a different host, with the actual work persisted in Postgres rather than the process 2. Agents work against World, Shopify's monorepo, which the source describes as containing not just code but skills, conventions, intent documents, runbooks, and AGENTS.md files 2.
Dispatch treats a security finding as a claim that must be proven, not asserted. Its pipeline runs nine stages: test bootstrap, architecture documentation, file cataloging, partitioning, hunting, verification, post-processing, reporting, and remediation 1. A finding that can't be backed by a generated integration test that actually exploits the issue is rejected or downgraded; theoretical findings don't count. The verification stage deliberately uses a different model than the hunting stage, an adversarial cross-check the authors credit with reducing noise and blind spots. Deterministic, non-agent code owns credentials, git operations, and storage, "so the agents don't have to" 1. Over an unspecified period, Dispatch produced more than 300 findings, including two the authors say would have rated as Critical, which the post values at more than $400,000 in bug-bounty-equivalent terms 1.
Roast, and the internal Boba workflow built on it, interleave deterministic and AI steps by convention rather than by framework lock-in. Roast is an open-sourced Ruby gem combining YAML configuration with Markdown prompts; every workflow execution is saved, so a crash or a retry doesn't repeat expensive steps 3. Boba applies this to Sorbet type-annotation backfill: deterministic cleanup handles what it can, and an AI step resolves the remaining type errors across thousands of test files 3.
A single internal LLM proxy sits underneath all of it. It routes every AI request, regardless of which tool originated it, through one gateway, giving Shopify bulk-provider leverage, automatic failover if a provider degrades, PII masking, and prompt-injection detection 5,6,7. Spend is not capped per team; instead, a long-running session burning tokens triggers an alert asking the user whether the spend was intentional, described in one account as a check at the 10-hour mark and in another as a $250-a-day threshold 5,6. A separate internal tool, referred to as UDP, distills a larger "teacher" model down to a smaller fine-tuned target using supplied training data and evals, which engineers reported yielding results "2x cheaper and faster" in typical cases and "30x cheaper and faster" in extreme ones 6.
Operating the software factory: teams, roles, and skills
No single team owns agents as a function within Shopify's engineering organization. River traces to a team that includes at least two Principal Engineers, Javier Moreno and Burke Libbey 2, while Dispatch, Roast, and the internal LLM proxy each trace to different posts by different authors, with no stated organizational chart connecting them 1,3,5.
Hiring and performance both treat AI fluency as a baseline skill rather than a specialization. Director-level and above candidates do live coding interviews in which AI tool use is explicitly permitted and expected: "You let them use whatever they want... If they don't use a copilot, they usually get creamed," in Farhan Thawar's account, with judgment, not raw output, as the thing being assessed: "I want you to be like 90 or 95%... able to go in and look at the code and say, oh yeah, there's a line that's wrong" 7. That standard for senior hiring sits inside the same company-wide expectation the April 2025 memo set 4.
The same permissiveness extends to spend. Thawar frames the absence of a per-team token budget as deliberate: "why AI tools aren't the place to pinch pennies" 6,7.
Implementation journey
- June 18, 2025Roast is open-sourced, an internal convention for interleaving deterministic and AI workflow steps 3.
- April 7, 2025Tobi Lütke's memo makes "reflexive AI usage" an explicit company-wide expectation, tied to performance reviews and headcount justification 4.
- April 2, 2026Bessemer Venture Partners publishes an account of Shopify's engineering playbook, quoting Farhan Thawar on PR reversion rate, comprehension debt, and a 20% productivity estimate 5.
- May 28, 2026"Under the River" is published, detailing River, Aquifer, and the World monorepo, alongside 30-day adoption figures 2.
- June 24, 2026VentureBeat publishes an account naming the UDP distillation pipeline and its cost/speed claims 6.
- July 29, 2026"Building an agentic harness that outlasts the model" is published, detailing Dispatch's nine-stage pipeline and its bug-bounty-equivalent findings 1.
Measuring the software factory
Shopify states directly which number it chose not to chase: no tracking of AI-authored lines of code or PR counts as a success metric, with leadership favoring weekly demos over throughput statistics instead 5. What it tracks instead is PR reversion rate: AI-assisted engineers ship more pull requests per week, and, in Thawar's account, "the reversion rate of those PRs has remained roughly the same" 5. That is a claim about stability, not output, and it is offered with no published baseline number or measurement window.
Where usage is counted at all, the counts run to sessions and merged pull requests rather than code volume, as the session, PR, and finding totals described above for River and Dispatch already show 1,2. Quick, a different kind of tool entirely, is cited by comparison at more than 50,000 sites 5. Thawar's estimate of a roughly 20% productivity gain carries an explicit caveat in his own words: "a humble estimate" 5.
Lessons learned
The richest operational detail in the record is also the least independently verifiable: proxy internals, hiring changes, and distillation economics come from press interviews, including a VC firm's content arm with a direct incentive to present the story as more mature than it is, rather than from Shopify's own engineering blog 5,6. That sourcing gap compounds a second one, since the self-reported figures throughout the record carry no disclosed methodology, and no source defines what counts as a "coauthored" pull request or a "verified" security finding 1,2,6.
Two further risks are about mistaken identity rather than missing data. Quick's adoption figures describe a company-wide, non-engineering-specific prototyping tool, but secondary coverage routinely cites it as evidence of engineering practice specifically 5. And a separate, unrelated product carries the same branding risk: the Shopify AI Toolkit is a merchant- and developer-facing product, not evidence of internal engineering practice, though secondary blogs sometimes conflate the two.
What's next for Shopify's software factory
No source lays out a roadmap for Shopify's agent stack. The clearest forward-looking statement in the record is philosophical rather than a shipped plan: Thawar's framing of "comprehension debt," the risk that engineers lose system understanding through over-reliance on AI, paired with a stated countermeasure of maintaining knowledge "two or three layers below" their normal working level 5. Whether that countermeasure is enforced by any system, or is a stated value only, is not described.
