Modal says it has rebuilt its sandbox infrastructure to do something Kubernetes was never designed to handle comfortably: create and manage 1 million concurrent sandboxes in seconds. In a recent article, staff engineers Colin Weld and Connor Adams described a ground-up redesign aimed at supporting millions of sandboxes and tens of thousands of sandbox creations per second, with benchmark results showing 1 million sandboxes created in under a minute and median startup-to-code time under 0.5 seconds.
Why Modal moved beyond Kubernetes
Weld and Adams argue that traditional container orchestration systems run into structural limits at this scale. The issue is not only the number of sandboxes, but also the number of compute nodes required to host them, which drives a large amount of O(nodes) and O(containers) work across the platform.
In Kubernetes’ case, they said the scheduling algorithm and the central durable store, etcd, both grow with the number of nodes and pods. They also noted that pods and nodes write to etcd multiple times, which can become a serious problem when pod creation rates or churn are very high, especially since etcd is not natively shardable within a keyspace.
They acknowledged that these limits can be addressed, but said doing so would require “serious work,” including rewriting or replacing etcd and parallelizing the scheduling algorithm.
How Modal rebuilt the sandbox layer
The key design shift was to stop coordinating globally. Instead of treating a central datastore as the source of truth, Modal made each worker its own source of truth. Scheduling also changed from a single serialized process into a fleet of scheduling servers that operate in parallel, which lets the scheduling layer scale horizontally.
In practical terms, a scheduling server chooses a worker, then contacts that worker directly via RPC to request sandbox creation. The worker accepts the request if it has free resources or rejects it if it does not. Modal said this makes the sandbox creation path as simple as possible while keeping everything else secondary.
A single remaining bottleneck
The company says the architecture has one main bottleneck left: all workers publish their state to a single Redis stream. Even so, Modal said load testing suggests the design remains viable until well over 100,000 workers.
That tradeoff appears deliberate. Rather than pushing all coordination through a strongly consistent central store, Modal has shifted much of the system toward horizontally scalable components and direct worker interactions.
What the benchmark shows
Modal’s benchmark is the headline result in the announcement. The company said it created 1 million sandboxes in under a minute, while keeping median startup-to-code time below half a second.
- 1 million sandboxes created in under a minute
- Median startup-to-code time under 0.5 seconds
- Tens of thousands of sandbox creations per second
Those figures matter because they point to a system optimized not just for scale, but for responsiveness at scale. In workloads tied to AI and agentic applications, that combination can be as important as raw throughput.
Why the result drew attention
The announcement also prompted reactions from other industry figures. On LinkedIn, Hopsworks CEO Jim Dowling said “new technical problems arise at every order of magnitude increase in scale,” adding that the team likely had to iterate multiple times to reach a reliable 50,000 sandbox creations per second.
AWS principal AI engineer Alex Jones argued that the most important part of Modal’s approach was not trying to extend Kubernetes, but “walking around the whole thing” after understanding its limitations. He called it “the first credible signal that Kubernetes isn’t adapting fast enough to what GenAI infrastructure actually needs.”
Jones said the industry may be heading toward a split between coordination and execution. In his view, the execution plane needs fast isolation boundaries, while coordination systems for multi-agent workflows still need the shared memory and overlapping security boundaries that Kubernetes-style systems are good at.
What this means for AI infrastructure
Modal is a serverless compute platform built for AI workloads, offering programmable access to CPUs, GPUs, containers, inference, training, batch jobs, and isolated sandboxes. Its redesign reflects a broader push to build cloud infrastructure around low-latency startup and large-scale isolation rather than around the traditional orchestration model alone.
It is not the only project pursuing that direction. The source article notes similar efforts from Unikraft, Google Substrate, and Overdrive, all of which are trying to rebuild parts of the cloud around high scalability and sub-10-millisecond cold starts.
For teams building AI agents, ephemeral compute, or bursty sandboxed workloads, Modal’s approach suggests a possible future in which the job of scheduling and the job of execution are more cleanly separated. Whether that becomes a broader replacement for Kubernetes or remains a specialized pattern for AI platforms will depend on how well these new designs perform under real-world pressure.
Source: Original report
Was this helpful?
Explore more: DevOps Services More Cloud & DevOps Tech News
Last Modified: September 23, 2026 at 10:34 pm
0 views
