Inside Cloudflare's software factory
An AI code reviewer running up to seven specialist agents per merge request across 5,169 repositories, built on the same developer platform Cloudflare ships to customers.
Cloudflare at a glance
The company
| Sector | A technology company that builds infrastructure for the internet 1 |
|---|---|
| Employees | "Approximately 5,500 at end of Q1 2026, before a cut of more than 1,100 roles in May 2026 5,6" |
| Customers | Not disclosed as a count |
| Revenue | $639.8 million in Q1 2026, up 34% year over year 5 |
The April 2026 engineering posts cited approximately 6,100 employees as context for adoption percentages 1. The May 2026 earnings call disclosed 5,500 headcount at Q1 end, before the 1,100+ layoffs announced that day 5,6. Both figures were stated by company representatives.
The engineering organisation
| Engineering size | "97% of engineering using AI coding tools per May 2026 earnings call 5; 3,683 active users of AI coding tools in April 2026 1" |
|---|---|
| AI tooling since | "The 11-month iMARS project started in approximately May 2025; November 2025 marked the internal tipping point when Opus 4.5 drove visible productivity gains 5,10" |
| Tool policy | Built in-house on shipping products, with open-source components (OpenCode, Backstage) and third-party model providers routed through AI Gateway 1,2 |
| Agents in the SDLC | "AI Code Reviewer (CI-native, up to seven specialist agents per merge request), Spec Reviewer (~600 specs since May 2026), Incident Report Reviewer (200+ reports), AGENTS.md generation pipeline across 4,000+ repositories, MCP Server Portal with 27 servers 1,2,4,10" |
| Reported adoption | "97% of engineering using AI coding tools; 100% of production contributions reviewed by autonomous AI agents; AI usage increased 600% in three months ending May 2026 5" |
| Published metrics | Cost per review, review duration at multiple percentiles, finding counts by severity and reviewer, token usage by model tier, cache hit rate, break-glass rate, acted-on rate, merge request volume trends 2,3 |
The 97% figure from the May 2026 earnings call refers to engineering specifically, while the April 2026 blog's 93% figure covers all of R&D. Both describe AI coding tool usage, not only the AI Code Reviewer.
People
What they set out to solve
Eleven months before publishing, Cloudflare formed a tiger team called iMARS (Internal MCP Agent/Server Rollout Squad) to integrate AI into its engineering stack. The sustained work landed with the Dev Productivity team, which also owns CI/CD, build systems, and automation 1. The problem they started from was not code generation speed but code review latency: across internal projects, the median wait time for a first review was often measured in hours 2.
The first attempt was the path most people take: try existing AI code review tools. The tools worked, and some offered good customisation, but none offered enough flexibility for an organisation Cloudflare's size. The next attempt, shoving a git diff into a half-baked prompt, produced exactly the noise expected: vague suggestions, hallucinated syntax errors, and helpful advice to "consider adding error handling" on functions that already had it 2.
The fix was not to build a better single prompt but to build orchestration. Instead of one model with a massive generic prompt, Cloudflare launches up to seven specialised reviewers covering security, performance, code quality, documentation, release management, Codex compliance, and AGENTS.md currency. A coordinator agent deduplicates their findings, judges severity, and posts a single structured comment 2.
A second problem sat underneath the reviewer: point-to-point integrations do not scale. Making MCP servers useful at Cloudflare meant building the portal, the access layer, and the tooling for agents to reach internal systems through one connection rather than wiring each agent to each system directly 1.
Maturity level
Assigned by testing published facts against fixed criteria, and capped by what the sources state.
How Cloudflare's software factory works
Every merge request at Cloudflare gets an AI code review, and 100% of production contributions are reviewed by autonomous AI agents before any human reviewer can approve them 5,10. Teams add a single CI component to their pipeline, and from that point every MR is reviewed automatically. The reviewer is implemented as a GitLab CI component that runs OpenCode v2, deployed in October 2026 after 98% approval agreement in shadow mode 2,3.
Risk tiers: don't send the dream team to review a typo fix
The system classifies every MR into one of three risk tiers based on the size and nature of the diff. Security-sensitive files, anything touching authentication, cryptography, or related paths, always trigger a full review 2.
Trivial MRs (≤10 lines, ≤20 files) get 2 agents at $0.20 average; lite MRs (≤100 lines, ≤20 files) get 4 agents at $0.67; full MRs (>100 lines or >50 files) get 7+ agents at $1.68. The trivial tier also downgrades the coordinator from Opus to Sonnet 2.
What the specialist agents look for
Each agent has tightly scoped instructions telling it what to flag and, more importantly, what to ignore. The security reviewer, for example, is instructed to flag only exploitable or concretely dangerous issues: injection vulnerabilities, authentication and authorisation bypasses, hardcoded secrets, insecure cryptographic usage, and missing input validation at trust boundaries. The flip side is equally specific: the reviewer is told not to flag theoretical risks with unlikely preconditions, defense-in-depth suggestions when primary defenses are adequate, or issues in code that the merge request does not touch 2.
Telling an LLM what not to do is where the actual prompt engineering value resides. Without these boundaries, the output is a firehose of speculative theoretical warnings that developers immediately learn to ignore 2.
The coordinator's judgment pass
After spawning all sub-reviewers, the coordinator performs a judge pass: deduplicating findings where the same issue was flagged by multiple agents, re-categorising findings to the section where they fit best, and filtering out speculative issues, nitpicks, and false positives. If the coordinator is not sure whether a finding holds, it reads the source code to verify 2.
The approval decision follows a strict rubric: all LGTM or only trivial suggestions gets approved; suggestion-severity items or warnings with no production risk get approved_with_comments; multiple warnings suggesting a risk pattern triggers minor_issues (unapproval); any critical item or production safety risk triggers significant_concerns (blocks merge). A single warning in an otherwise clean MR still gets approved_with_comments rather than a block. When a human reviewer comments "break glass," the system forces an approval regardless of what the AI found, an escape hatch used 288 times across 48,095 merge requests (0.6%) in the April 2026 measurement window 2.
The platform underneath
Every LLM request routes through AI Gateway, which handles provider keys, cost tracking, and data retention policies. A proxy Worker sits in front, serving the shared config, injecting API keys server-side so no keys exist on user machines, and mapping each user's email to an anonymous UUID so model providers never see identities 1.
The entire setup starts with a single login command against an internal endpoint. That command triggers a discovery flow that configures providers, models, MCP servers, agents, commands, and permissions without the user touching a config file. Making changes to what 3,000+ people get in their coding environment is a single wrangler deploy 1.
The MCP Server Portal grew from 13 production MCP servers in April 2026 to 27 by August, with teams shipping more servers every month 1,8. WriteGuard, a policy layer introduced in August 2026, adds risk tiers (read-only, minimal impact, contained write, critical), agent attribution so downstream systems can distinguish human from agent actions, and centralized audit logging 8.
Cloudflare Codex: machine-readable engineering standards
The Cloudflare Codex is a governed set of engineering standards built for people and agents 4. Each standard is an RFC using SHOULD and MUST keywords from RFC 2119, with a separate lifecycle step for enforcement so teams have time to absorb new requirements. An approved RFC produces non-blocking findings; an enforced RFC blocks merges that violate MUST statements 4.
A purpose-built agent extracts and compacts the SHOULD and MUST statements into a JSON structure with stable slug identifiers for tracking. Over four months, the AI code reviewer flagged close to 230,000 violations and blocked 16,000 merges 4,7. Codex linter packages provide oxlint-based TypeScript enforcement; Rust is in progress, Go is planned. A local CLI lets engineers run the AI code reviewer without waiting for CI 4.
Spec and incident report reviewers
The spec reviewer evaluates technical designs against relevant Codex requirements before implementation begins. Since May 2026, it has reviewed ~600 unique open specs with over 3,200 review invocations; 65% of findings are major severity, 29% minor, 6% critical 4.
The incident report reviewer applies the same approach to incident reports. Since May 2026, it has assessed more than 200 reports and is mandatory for high-severity incidents 4.
AGENTS.md: getting thousands of repos ready for AI
A common failure mode in the early rollout: agents produced changes that looked plausible and were still wrong for the repo, usually because the model didn't know the right test command, the team's conventions, or which parts of the codebase were off-limits. The fix was AGENTS.md: a short, structured file telling agents how the codebase actually works 1.
The generator pipeline pulls metadata from Backstage, analyses the repository structure, maps the stack to relevant Codex standards, and has a capable model generate the document. The system opens a merge request for the owning team to review. About 4,000+ repositories have been processed this way 1,10.
The AI Code Reviewer then closes the loop: a dedicated AGENTS.md reviewer flags when repository changes suggest the file should be updated. It classifies changes into materiality tiers: high (package manager changes, test framework changes, major directory restructures), medium (major dependency bumps, new linting rules), and low (bug fixes using existing patterns) 2.
Operating Cloudflare's software factory: teams, roles, and skills
The sustained work landed with the Dev Productivity team, which also owns CI/CD, build systems, and automation. The 11-month project started with the iMARS tiger team, engineers pulled from across the company 1. Ayush Thakur built the AGENTS.md system and AI Gateway integration, Scott Roe-Meschke is Engineering Manager of Developer Productivity and co-authored WriteGuard, and Rajesh Bhatia leads the Productivity Platform function 1,8,10. Ryan Skidmore, a Systems Engineer, built the AI Code Reviewer 2,3,10. Sam Rhea, the Chief Information Officer, authored the Cloudflare OS post 7. No headcount for the Dev Productivity team or the iMARS tiger team is disclosed 1,2.
Cloudflare engineers have contributed over 45 pull requests upstream to OpenCode 2.
Implementation journey
- June–July 2025Windsurf IDE rollout begins; Claude Code is piloted alongside it 10.
- Approximately May 2025The iMARS tiger team forms to integrate AI into the engineering stack 1.
- November 2025
- March 19, 2026Workers AI launches Kimi K2.5 as a daily driver inside OpenCode 9.
- March 10–April 9, 2026The 30-day measurement window for the initial AI Code Reviewer metrics: 131,246 review runs across 48,095 merge requests in 5,169 repositories 2.
- Week of March 23, 2026Merge request volume hits 10,952, nearly double the Q4 baseline of ~5,600/week 1.
- April 20, 2026
- May 2026Spec reviewer begins evaluating technical designs; incident report reviewer becomes mandatory for high-severity incidents 4.
- May 7, 2026Q1 2026 earnings call discloses 97% engineering AI adoption, 100% autonomous review gate, and 1,100+ layoffs 5.
- August 2026
- Early October 2026OpenCode v2 deployed after 98% approval agreement in shadow mode 3.
- October 19, 2026Optimization post reports $0.77 average review cost (down 68% from $2.43), 1m32s median duration (down from 4m09s), 97% cache hit rate (up from 85.7%) 3.
Measuring Cloudflare's software factory
Volume and adoption
In the 30 days ending April 15, 2026: 3,683 internal users actively using AI coding tools (60% company-wide, 93% across R&D), 295 teams with AI activity, 47.95 million AI messages. The 4-week rolling average of merge requests climbed from ~5,600/week to over 8,700/week 1.
By May 2026, 97% of engineering used AI coding tools and 100% of production contributions were reviewed by autonomous AI agents; internal AI usage had increased 600% in three months 5.
AI Gateway throughput
241.37 billion tokens routed through AI Gateway, 51.83 billion tokens processed on Workers AI in the April 2026 window. Model provider distribution: Frontier Labs (OpenAI, Anthropic, Google) at 91.16% of requests, Workers AI at 8.84% 1.
Code reviewer performance (April 2026 baseline)
Over 30 days (March 10–April 9, 2026): 131,246 review runs, 48,095 merge requests, 5,169 repositories. Average MR reviewed 2.7 times (initial plus re-reviews as the engineer pushes fixes) 2.
Median cost was $0.98 with 3m39s duration; P90 was $2.36 at 6m27s; P95 was $2.93 at 7m29s; P99 was $4.45 at 10m21s. 159,103 total findings at about 1.2 per review, deliberately low because the system is biased for signal over noise. Code quality produced the most findings by volume (74,898); security flagged the highest proportion of critical issues at 4% (484 critical of 11,985 total) 2.
Cache hit rate: 85.7%, saving an estimated five figures compared to full input token pricing. 120 billion tokens processed in total, with top-tier models (Claude Opus 4.7, GPT-5.4) handling 51.8% and standard-tier (Claude Sonnet 4.6, GPT-5.3 Codex) handling 46.2% 2.
Code reviewer performance (October 2026 optimized)
After aggressive optimization starting mid-September 2026: average review cost dropped from $2.43 (peak in late August) to $0.77, a 68% reduction. Median review duration fell from 4m09s to 1m32s. Reviews taking longer than 5 minutes dropped from 37.9% to 0.7%. Cache hit rate increased from 85.7% to 97%. Tokens per review rose from 0.91 million to 2.99 million, reflecting expanded context but offset by better caching 3.
About 40% of savings came from code changes (sharing cached prefixes across repositories, reducing LLM calls per session) and 60% from model and effort tuning. All reviewers moved to Anthropic models in early August: the coordinator runs Claude Opus 5.5 at medium effort, sub-reviewers run Sonnet 5.5, and the Codex reviewer runs Opus 4.7 3.
Acted-on rate
Before October 2026 optimization, authors acted on 34% of the code quality reviewer's serious findings. After optimization, 59% of serious findings are acted on. Serious findings that led to a change increased from 229 to 321 per 1,000 reviews across all reviewers 3.
A blinded judge study of 657 findings from the three Sonnet reviewers found that important findings per review stayed roughly constant (0.28 before, 0.30 after). The bigger change was severity calibration: findings marked more severe than deserved dropped from 0.46 to 0.14 per review, a 70% reduction. Much of what disappeared was minor findings, duplicates, and inflated severity, not real problems 3.
The break-glass rate
Engineers needed to override the reviewer 288 times across 48,095 merge requests in the April 2026 window: 0.6%. This measures how often a human forced an approval against the AI's concerns, not how often the reviewer was wrong 2.
Codex enforcement
Over four months, the AI code reviewer flagged close to 230,000 violations of Codex standards and blocked 16,000 merges for enforced MUST statements 4,7.
Lessons learned
Telling an LLM what not to do is where the actual prompt engineering value resides. Without explicit "What NOT to Flag" sections, the security reviewer produced a flood of theoretical warnings that developers learned to ignore 2.
One URL to configure everything. Centralising through a Worker from day one meant later adding per-user attribution, model catalog management, and permission enforcement without touching any client configs. The proxy pattern gives a control plane that direct connections don't 1.
Code Mode is not optional at scale. The GitLab MCP server's 34 tool schemas consumed 15,000 tokens per request, 7.5% of a 200K context window before asking a question. Collapsing to two portal-level discovery tools keeps the gateway usable 1.
The reviewer is not a replacement for human code review. It struggles with architectural awareness, cross-system impact, and subtle concurrency bugs: it sees the diff and surrounding code but not why a system was designed a certain way or whether a change is moving the architecture in the right direction 2.
AGENTS.md files rot fast. If a team migrates from Jest to Vitest but forgets to update instructions, the AI stubbornly keeps trying to write Jest tests. The fix was a dedicated reviewer that flags when changes suggest the file should be updated 2.
AI-written patches can introduce regressions. During security testing with Mythos Preview, the team observed AI-written patches that "fixed the original bug while quietly breaking something else the code depended on." Regression testing cannot be skipped even under a two-hour patch SLA 11.
What's next for Cloudflare's software factory
The next evolution is background agents: agents that can be spun up on demand with the same tools available locally (MCP portal, git, test runners) but running entirely in the cloud. The architecture uses Durable Objects and the Agents SDK for orchestration, delegating to Sandbox containers when the job requires a full development environment like cloning a repo, installing dependencies, or running tests 1. The October 2026 optimization post mentions ongoing experiments with running the reviewer solely on Durable Objects using pi-durable 3.
