Cloudflare's Internal AI Engineering Stack: A Technical Overview

Cloudflare has developed an internal AI engineering stack, built on their own platform, to integrate AI into their engineering workflow. This stack has been adopted by 93% of their R&D organization, with 3,683 internal users actively using AI coding tools daily. The stack consists of several key components, including:

• MCP Servers: The foundation of the stack, providing a centralized platform for AI tooling and access control.
• AI Gateway: A critical component that handles authentication, routing, and inference for AI requests, ensuring security and visibility across multiple clients and use cases.
• Workers AI: A platform for on-platform inference with open-weight models, allowing for sandboxed execution of agent-generated code.
• Code Reviewer: An AI-powered code review tool that helps maintain quality at scale.
• Engineering Codex: A knowledge layer that provides a structured understanding of Cloudflare's systems and processes.

Key Technical Details

• The stack uses Zero Trust authentication and centralized LLM routing, cost tracking, BYOK, and Zero Data Retention controls.
• AI Gateway handles 688.46k requests per day, routing to four providers through one endpoint.
• Workers AI provides on-platform inference with open-weight models, allowing for sandboxed execution of agent-generated code.
• Code Reviewer uses AI to help maintain quality at scale, with 20.18 million AI Gateway requests per month.

Practical Implications for Developers

• The internal AI engineering stack has significantly improved developer velocity, with a 4-week rolling average of merge requests increasing from ~5,600/week to over 8,700.
• The stack provides a scalable and secure platform for AI tooling and access control, enabling developers to focus on building high-quality software.
• The use of AI-powered code review and knowledge management tools helps maintain quality at scale, reducing the risk of errors and improving the overall development process.