Anthropic rewrites core API with Claude, reducing CPU usage by 40% to 50%

Beating AI News Flash: Katelyn Lesse, Head of Platform Engineering at Anthropic, revealed in an interview on Latent Space that the company has migrated its Messages API from Python to Rust. After the overhaul, CPU usage for the API service is estimated to have decreased by approximately 40% to 50%, and first-token response latency improved by about 10% to 20%.

The Messages API is the primary interface for developers to call Claude. As request volume continued to grow, the original Python monolith service consumed increasingly more CPU, and the team decided to rewrite it directly in Rust. However, when the project kicked off, the migration team did not even have a single Rust engineer.

In the end, Claude took on the bulk of code writing, while human engineers handled architecture design and validation. The team first built a complete Rust version, had the old and new systems process identical requests, compared outputs one by one, and then gradually switched production traffic over. During the process, they also hired several engineers with Rust experience.

Lesse stated that in the past, a migration of this scale would typically be split into multiple phases and take years. This time, the team completed the migration of the entire API in one go, and it went fully live in September.