Cloudflare AI User Isolation Failure: GPU Cache Insufficient Yet Responses Continue, 155 Replies Leak Other Customers' Content

Beating AI News Flash: Cloudflare discloses an AI data leak incident. The GLM-5.3 model on its Workers AI platform had a user isolation vulnerability, causing 155 responses to be mixed with content previously submitted by other customers. In other words, the information you send to the AI may directly appear in a stranger's response.

The problem lies in GPU cache management. Cloudflare opened a context window of approximately 1.05 million tokens for the model, but the GPU cache responsible for processing input can only hold about 680,000 tokens. Oversized requests should have directly returned an error, but the underlying inference engine SGLang incorrectly continued execution, allowing the model to read residual cache from other users. All 155 affected responses came from oversized requests exceeding the cache capacity.

Cloudflare admitted that it knew the cache capacity was insufficient before launch, but during testing it only checked whether the interface returned normally and did not verify the response content, thereby missing the data cross-contamination issue.

During joint investigation, the two parties also discovered another type of cache vulnerability. After a user cancels a request midway, the system may allocate the cache to someone else prematurely, but the original data has not finished transferring and subsequently overwrites the new user's content. The SGLang team reproduced this with two H200 GPUs: the AI was counting normally from 1, and after reaching 80, it suddenly began outputting the prompt from another request. Such errors do not even leave abnormal logs.

Cloudflare fixed the actual leak incident on September 9, confirmed that other Workers AI models were not affected by this issue, and found no evidence of malicious exploitation. SGLang has successively fixed related cache management defects and recommends that relevant deployments upgrade to v0.5.22 or later versions.