The Wall Between Customers Sits in the Server

Cloudflare disclosed on Oct. 26 that between Aug. 27 and Sept. 9, 155 responses from GLM-5.3 on its Workers AI service, all to very long prompts, contained pieces of other users’ prompts and replies. The figure counts responses, not people. The company knew before launch that the model’s million-token context window exceeded the memory pool for a prompt’s working state, and expected SGLang, its open-source serving engine, to refuse oversized prompts. Instead the engine passed a placeholder onward, and replies drew on cache pages earlier requests had left behind.

The engine handed one request’s leftovers to another as if they were its own. Whatever a model can be taught about where its input came from, separating one customer’s data from another’s should not be left to the model’s behavior. It has to be enforced by the serving system that decides which memory each request may touch.

Cloudflare had argued something close to that. On Aug. 3, three weeks before the launch, it wrote that hundreds of requests now share pages of one GPU’s cache, an arrangement that depends on “getting the bookkeeping exactly right,” and introduced a check that tags each page and halts a request before it reads the wrong one. The incident report says the prototype was switched off after trials on an earlier model and does not claim it would have caught this bug.

The flaw was also known upstream. On GitHub, a user’s July 6 report that over-length prefill requests could leave decoding to run on incomplete cache was closed on Sept. 5 by an automated bot for inactivity, three days before anyone alerted Cloudflare. A proposed fix opened July 8 remained unmerged on Oct. 27. A fix for a related path shipped in SGLang 0.5.20 on Sept. 18.

The report is candid about the testing gap: “we only validated HTTP response codes and engine performance at long contexts.” Those tests confirmed that answers came back, and never checked whose words were inside. Gadi Evron of the security firm Knostic reported the leak on Sept. 8; Cloudflare fixed it the next day and says it saw no sign of deliberate exploitation. Publication waited for upstream fixes, giving other operators time to patch. I found no separate statement from Knostic or RadixArk, the company whose engineers maintain SGLang.

The pattern is not new, even for Cloudflare. In 2017, Tavis Ormandy of Google’s Project Zero found a parser bug that spilled memory from unrelated customer sites into Cloudflare’s responses; the company published its report five days later and worked with search engines to purge cached copies. In March 2023, a redis-py bug let some ChatGPT users see titles from other users’ chat histories, and OpenAI said it contacted users whose payment details may have been exposed, as Help Net Security also reported. In nine years the faulty layer has moved from a parser to a connection library to a model’s working memory, and in 2017 as in 2026 the alarm came from outside.

What the new report omits is the people at either end. It does not say whether customers whose prompts surfaced in strangers’ answers, or those who received them, were told; when to publish technical details and whether to inform those affected are separate questions. Cloudflare has fixed the configuration and the rejection path; SGLang’s separate cache-corruption checks, it says, do not yet cover GLM-5.3. Long-context models are sold on how much a customer can put into one prompt. That makes the wall between customers a part of the product to be tested in its own right: the configuration, the rejection path and, above all, whether one customer’s answer ever contains another customer’s words.