Skip to content

GitHub Rebuilds Git Foundation: 5,000 AIs Working in Parallel, Processing 7,200 Code Pushes per Second in a Single Repository

Nov 9, 18:47

Beating AI News Flash: GitHub has unveiled a new-generation Git storage architecture codenamed Rubicon. Internal testing shows that 5,000 AI Agents can work simultaneously in a single code repository, completing 7,200 code pushes per second while maintaining low read latency.

In the past, even GitHub's busiest code repositories only saw brief bursts of a few dozen writes per second. Now, large numbers of AI Agents can develop in parallel around the clock, continuously submitting new code. Each commit may also trigger automated tests, prompting other Agents to download the latest version, which further increases read and write pressure on the repository.

GitHub's solution is to separate "storing code" from "updating branches."

Every time Git pushes code, it must accomplish two things. The first step is saving the newly added or modified code content. Git does not directly overwrite old versions; instead, it stores changed content as new objects, while unchanged parts can continue to be reused. The second step is updating the branch pointer, recording which latest version the branch now points to.

For example, 1,000 Agents are developing software simultaneously—some modifying login functionality, others modifying payment functionality. As long as they work on different branches, new code can be saved simultaneously and branches can each be updated independently. Only when two Agents compete to update the same branch at the same time does the system need to determine whose update succeeds first.

GitHub initially had all branches share a single update record. Even if different AI Agents were modifying different branches, they still had to compete to update the same file, resulting in only hundreds of pushes per second. GitHub then tried splitting the record into multiple parts so different branches could update simultaneously. But this meant that every check for repository changes required querying each part individually, which instead slowed down read speed.

Ultimately, GitHub developed a dedicated Ref-CAS service for managing branch updates. It allows different branches to update simultaneously, and conflicts only need to be handled when modifying the same branch. At the same time, it maintains a unified version number for the entire repository. Other programs only need to check this version number to know whether the repository has changed, without having to query every branch individually.

GitHub also arranged independent machines to merge accumulating code data packs in the background, preventing data packs from growing in number and slowing down reads. The new architecture ultimately achieved 7,200 pushes per second in internal testing.

GitHub's Rubicon is currently still in a limited private testing phase and plans to expand its launch in early 2027.

Source