Docker has added support for Harbor, the open-source agent evaluation framework, through Docker Sandboxes, giving AI agent benchmarks a more isolated and reproducible execution environment. The integration runs each eval task inside its own microVM, so work that once shared a container can now be separated at the kernel level while still following the same Harbor workflow locally or in Docker Cloud Sandboxes.
What the Harbor and Docker Sandboxes integration changes
Harbor is used to orchestrate agent evaluations: it launches tasks, gathers scores, and stores logs. That makes the runtime underneath it especially important, because a benchmark may repeat the same task thousands of times while letting an agent install packages, edit files, and execute commands.
Docker says the new setup is designed for those conditions. Each task runs in a Docker Sandbox, which is described as a microVM with its own kernel. In practical terms, that keeps agent activity inside the sandbox and away from the host machine, other tasks, or anything outside the evaluation environment.
Why isolation matters for AI agent evals
Docker’s engineers frame the problem as running untrusted, AI-written code at scale. They also call out reward hacking, where an agent can try to pass an eval by tampering with tests or overwriting the file that records its score rather than solving the actual task.
Better isolation does not eliminate every evaluation trick, but it reduces one major source of inconsistency: shared runtime state. By giving each task a dedicated microVM, Harbor can run benchmarks under more uniform conditions, which is especially important for comparing results across repeated runs.
Network controls aim to improve reproducibility
In Docker Cloud Sandboxes, Harbor tasks can use configurable network policies. A task can pull dependencies during setup and then have internet access restricted while the agent works. Docker says this helps every agent run under the same network conditions, which addresses a common cause of non-reproducible benchmark results.
Docker Compose is also supported, allowing Harbor to run multi-container tasks inside a single cloud sandbox, with services and sidecars sharing the same microVM.
Availability and setup details
The integration is already available in the nightly release of Docker Sandboxes and is scheduled to reach the stable release on October 23. Harbor can be installed with the sbx extra, and Docker says the nightly build is pinned to version 0.24.1.dev202610150414.
For cloud runs, users need a Docker account with Cloud access enabled and a personal access token with the Cloud Sandboxes permission. Harbor reads those credentials from the SBX_DOCKER_USERNAME and SBX_DOCKER_PAT environment variables.
Docker gave an example command showing how to select the sandbox environment:
- harbor run -t hello-world/hello-world -a oracle -e sbx
For local runs, the post says the setup goes through the Docker Sandboxes CLI after sbx login, with –ek target=local added to the Harbor command. The local path currently supports only public and no-network modes, and it follows the machine’s existing sandbox policy.
What Docker has and has not disclosed
Docker did not publish benchmark data on startup overhead, per-task cost, or score variance. The only concrete scale number included in the announcement is that Docker tested Harbor with 64 concurrent tasks in the cloud. The company says higher limits are available on request through its contact channel.
That means teams evaluating the integration will still need to measure latency, throughput, and cost on their own workloads before deciding whether the cloud path fits their needs.
Harbor’s broader sandbox ecosystem
Docker’s integration is the latest example of Harbor connecting to multiple sandbox backends. The framework already supports horizontal scaling through providers including Daytona, Modal, E2B, Runloop, Tensorlake, LangSmith, Blaxel, Novita Sandbox, EC2, and Beam.
According to the source material, LangSmith was the most recent addition before Docker. Harbor’s cloud deployment documentation also points to Daytona as the best choice for multi-container deployments, since it supports docker-compose.yaml, while Modal, E2B, and Runloop do not.
Docker’s wider push into agent runtimes
Harbor is not the only agent platform Docker has been targeting with Sandboxes. NanoCo and Docker have collaborated so that NanoClaw, a claw-based agent platform, can be deployed in Docker’s MicroVM sandbox with a single command. In July 2026, GitHub Agentic Workflows also added Docker Sandboxes as a supported agent runtime in gh-aw version 0.82.9.
Docker’s earlier Cloud Sandboxes launch on September 24 extended local isolation into Docker-managed cloud infrastructure. The company said that lets agent workflows continue even if a laptop shuts down. Third-party coverage of that launch reported a one-hour default session with a 24-hour maximum, and paused sandboxes carrying no compute charge. That release also introduced Kits as OCI images, and Docker said it would submit the Kits specification to the CNCF.
For teams already using Harbor, the new Docker Sandbox backend offers a cleaner path to repeatable agent testing, especially when the benchmark involves untrusted code, package installs, or multi-step tool use. The trade-off is that the most controlled network behavior is tied to Docker’s cloud offering, while local use remains simpler and more limited.
Source: Original report
Was this helpful?
Explore more: AI Automation Services More AI & Automation Tech News
Last Modified: October 22, 2026 at 10:34 pm
1 views
