Runloop vs CodeSandbox SDK
Both give each sandbox its own micro-VM. Runloop is built around coding agents, with hostname egress rules, arm64 and deploys into your own cloud, while CodeSandbox SDK is strongest at VMs that hibernate with their memory and wake on the next HTTP request.
The companies and products
Runloop
Runloop is a US platform of sandboxes, called devboxes, for building, running and testing AI coding agents.
You get Linux devboxes built from Dockerfile-style blueprints, disk snapshots, HTTPS tunnels, agent gateways that hide API keys, and Axons, event streams that drive agents such as Claude Code. Each devbox runs in a micro-VM on Runloop's own bare-metal hypervisor, with a container layer inside. It is built for teams shipping coding agents, plus evaluation and RL fine-tuning work on the Enterprise plan.
- Compliance
- SOC 2 Type 2
- Architectures
- x86_64 and arm64 base blueprints
- Own-cloud deploy
- Deploy to VPC on AWS, GCP and Azure (Enterprise)
- Regions
- Not documented or selectable
- Company
- Runloop AI, Inc.
CodeSandbox SDK
CodeSandbox SDK is an API from CodeSandbox, now a Together AI company, that starts sandboxed development VMs on the same infrastructure as the CodeSandbox editor.
Each sandbox is a Firecracker microVM built from a template or any Dockerfile. When it goes idle it hibernates with a memory snapshot, and it resumes or forks from that snapshot in seconds. You drive it from a TypeScript SDK through terminals, commands, files and HTTPS hosts. Together's docs now call it the legacy offering and point new users to its successor, the together-sandbox SDK.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “20th cheapest of 367” means 19 of the other 366 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Runloop: price rank | CodeSandbox SDK: price rank | boat: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month | 20th cheapest of 367 $8,152 a month | 178th of 367, with limits $1,939 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB | 32nd cheapest of 366 $10,194 a month | 215th of 366, with limits $2,478 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month Runloop and CodeSandbox SDK both fit short code runs and sit close together here, while boat's start limits queue very large bursts. | 44th cheapest of 345 $525 a month | 46th cheapest of 345 $642 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month CodeSandbox SDK wakes a user's sandbox on their next HTTP request but caps Scale at 250 at once; Runloop and boat fit 300 at once. | 26th cheapest of 361 $3,301 a month | 140th of 361, with limits $1,707 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month Runloop fits large waves of rollouts with no published concurrency cap, while CodeSandbox SDK's Scale plan stops at 250 VMs at once. | 4th cheapest of 360 $15,922 a month | 348th of 360, with limits $3,861 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 81st cheapest of 364 $2,562 a month | 64th cheapest of 364 $1,335 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month CI jobs that build Android apps or VM images need /dev/kvm, which boat has and neither Runloop nor CodeSandbox SDK documents. | 121st cheapest of 377 $1,554 a month | 92nd cheapest of 377 $860 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB Runloop needs an idle policy to run all month and has no public IP, CodeSandbox SDK hibernates after an idle timeout, and boat keeps one machine with SSH and its disk. | 113th cheapest of 368 $410 a month | 310th of 368, with limits $240 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Runloop ships a desktop and a browser your agent can control, CodeSandbox SDK has no desktop, and boat streams its desktop at 60fps. | 39th cheapest of 369 $2,306 a month | 206th of 369, with limits $691 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month On CodeSandbox SDK you build a headless browser template yourself; Runloop and boat ship a browser in the sandbox. | 40th cheapest of 367 $1,153 a month | 251st of 367, with limits $443 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 68th cheapest of 346 $781 a month | 281st of 346, with limits $508 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 266th of 338, with limits $15,611 a month | 193rd of 338, with limits $7,390 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month Runloop can lock each devbox to a hostname allowlist, which CodeSandbox SDK does not offer; boat gives each one a full kernel with nested virtualization. | 10th cheapest of 360 $770 a month | 240th of 360, with limits $326 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month | Can't run it | Can't run it | Can't run it |
Key features
| Feature | Runloop | CodeSandbox SDK | boat |
|---|---|---|---|
| Isolation | Micro-VM on a custom bare-metal hypervisor, plus a container layer | Firecracker microVM with its own kernel | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts |
| CPU speed (Node.js build-and-test, 4 vCPU / 8 GB) | 8.29 runs a second | Not measured | 26.69 runs a second |
| Sleep and wake | Suspend after idle with the disk kept, wake on an HTTP request to the tunnel (Pro plan) | Hibernates with a memory snapshot; an HTTP request wakes it with processes still running | Disk saved every minute and on stop; resume boots the machine again, started by API |
| Fork | Fork from disk snapshots | Fork a hibernated sandbox from its memory snapshot, or a running one sharing its memory | Fork a running sandbox with its disk |
| Nested virtualization | Not documented; Docker-in-Docker blueprints only | Not documented | Yes: /dev/kvm, Android emulator, VMs inside |
| Docker and root | Yes, on the Docker-in-Docker blueprints, as root | Docker and root; built from any Dockerfile | Docker, systemd and root |
| Images and CPU architecture | Blueprints built from Dockerfiles, x86_64 and arm64 | Templates or any Dockerfile, x86_64 only | Templates from named snapshots, no Dockerfile builds, x86_64 only |
| Shell access | SSH through a TLS proxy | Terminals, commands and files through the SDK; no SSH | SSH, plus exec and files over the API |
| Inbound networking | One HTTPS tunnel per devbox covering its ports, open or bearer token; no public IP | HTTPS host per port, public by default, signed tokens for private hosts; no raw TCP or public IP | HTTPS URLs for up to 50 ports, token-protected by default, raw TCP, dedicated IPv6 or IPv4 |
| Egress rules | Hostname allowlist with wildcards, or block all | No provider-side allowlist documented | Open internet, no provider-side allowlist |
| Keeping API keys out of the sandbox | Agent gateways for any API endpoint with bearer, header or basic auth | Secrets passed in as environment variables | Credentials proxy for model providers and any HTTPS API; switch a credential to another account when one runs out |
| Desktop and browser | Desktop and browser your agent can control | No desktop; a headless Chromium template | Linux desktop streamed at 60fps, Chromium and computer-use tools |
| Coding agents | Claude Code, Codex, OpenCode, Gemini CLI, DeepAgents, driven through Axons and Broker | Run your own agent through the command API | Claude Code, Codex, pi, OpenCode, Prime Agent, Kimi Code, Mistral Vibe, with a prompt, steer and events API |
| Where it runs | Regions not documented; deploy to your own AWS, GCP or Azure account on Enterprise | A country hint steers scheduling, not guaranteed; no self-hosting | EU only (Germany, Finland, France), EU data residency |
Every feature Battleships tracks (43)
| Feature | Runloop | CodeSandbox SDK | boat |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | No | No | Yes |
| Root | Yes | Yes | Yes |
| Systemd | Not published | Not published | Yes |
| Desktop GUI | Yes | No | Yes |
| Browser | Yes | No | Yes |
| Computer-use API | Yes | No | Yes |
| SSH | Yes | No | Yes |
| Persistent disk | Yes | Yes | Yes |
| Pause/resume | Yes | Yes | Yes |
| Automatic snapshots | Yes | Yes | Yes |
| Fork/clone | Yes | Yes | Yes |
| Live fork (no pause) | Yes | Yes | Yes |
| Memory fork | No | Yes | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | Yes | Yes | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | Yes | Yes | No |
| HTTPS preview URLs | Yes | Yes | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | Yes | No | No |
| Secret proxy | Yes | No | Yes |
| Secret proxy for any API | Yes | No | Yes |
| Preinstalled agents | Yes | No | Yes |
| Agent harness API | Yes | No | Yes |
| MCP server | Yes | No | No |
| GPU | No | No | No |
| Windows | No | No | No |
| Arm64 | Yes | No | No |
| Self-hosting | No | No | No |
| BYOC | Yes | No | No |
| Open source | No | No | No |
| Scoped API keys | Yes | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | No | No | Yes |
| Audit logs | Not published | Not published | No |
| SSO | Yes | Yes | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | Not published | No | No |
| EU data residency | No | No | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
On the StarSling hpc-sandbox-benchmarks Node.js build-and-test loop at 4 vCPU / 8 GB (run 36498291449, 2026-09-29), boat scored 26.69 runs a second and Runloop 8.29, about 3.2 times apart. CodeSandbox SDK was not part of that run, so there is no comparable CPU measurement for it. The test measures CPU work only, not start time or network speed.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. CodeSandbox SDK has no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Runloop, a month | CodeSandbox SDK, a month | boat, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $113 · Basic | $193 · Scale | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $429 · Basic | $341 · Scale | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $5,268 · Basic | $2,500 · Scale | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $1,441 · Basic | $815 · Scale · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $3,184 · Basic | $1,632 · Scale · with limits | $2,000 · $2000/mo · with limits |
Runloop: Runloop bills per second for the CPU and memory you allocate, while a devbox is starting, running, suspending or resuming, and bills its disk for the devbox's whole life. Basic has no plan fee; Pro adds a $250 fee each month that is not usage credit and unlocks suspend and resume.
CodeSandbox SDK: CodeSandbox SDK sells VM credits, and each size burns a fixed number per hour, billed per started minute while the VM runs, including the idle time before it hibernates. Scale adds a $170 platform fee each month that is not usage credit, and a hibernated VM costs nothing.
CodeSandbox SDK: Plans start at $20 a month and the whole fee is usage credit, billed per second while a sandbox runs. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
CodeSandbox SDK's Scale plan allows 250 VMs at once, so the 300-user and 1,000-job scenarios do not fit it without Enterprise, and each VM has a 20 GB disk with no published price for more. Runloop publishes no concurrency cap on paid plans and fits both, but its fixed lifetime tops out at 48 hours unless an idle policy replaces it, and its disk tops out at 64 GiB. Neither publishes an egress price. boat's limit that bites is start rate: even the $2,000 plan queues bursts above 90 starts a minute, so 1,000 jobs at once does not fit there.
| Scenario | Runloop | CodeSandbox SDK | boat |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Runloop when
you launch large waves of agents or rollouts, need hostname allowlists or arm64, or want devboxes inside your own AWS, GCP or Azure account.
- Lock down the network. Useful if a sandbox must only reach a fixed list of hosts: Runloop allows listed hostnames, with wildcards, or blocks everything.
- Run it in your own cloud. On the Enterprise plan, Runloop can run devboxes inside your own AWS, GCP or Azure account.
- Big waves and arm64. Runloop publishes no concurrency cap on paid plans and offers arm64, so large rollout waves and arm64 builds fit.
Use CodeSandbox SDK when
your sandboxes sleep between visits and should wake on the next HTTP request with their memory intact, and you are fine building on what Together now calls its legacy SDK.
- Memory hibernation and wake on request. Useful for sandboxes that sleep between visits: an HTTP request wakes a hibernated VM with its processes still running.
- Forks that share memory. Good for branching many runs from one warm state, such as rollouts or test variants.
- Bigger machines on Enterprise. Teams that need very large single machines can get up to 64 cores and 128 GB through Enterprise sales.
Where boat fits
If your agents do real work for hours, such as installs, builds, Docker, emulators and servers other people reach, look at boat. Each sandbox is a full Linux machine with /dev/kvm, SSH and raw TCP, on CPUs about three times faster than Runloop's in the same benchmark, with the disk saved every minute on every plan.
Emulators and VMs start
Neither Runloop nor CodeSandbox SDK documents nested virtualization, so an Android emulator or a VM inside the sandbox does not start. Runloop's workaround is Docker-in-Docker, which does not cover them. boat exposes /dev/kvm.
Reach the machine any way
Runloop reaches a devbox through one HTTPS tunnel and SSH over a TLS proxy, and CodeSandbox SDK through HTTPS hosts and its SDK with no SSH. Neither gives a public IP, so a database or server speaking raw TCP has no way in.
Data that stays in the EU
Runloop does not document regions, and running in a place you choose means its Enterprise deploy into your own cloud. CodeSandbox SDK's country hint is not a guarantee. boat runs only in Germany, Finland and France.
Use boat when your agents do real work for hours: installs, builds, Docker, emulators and servers other people reach, on a full machine with faster CPUs and files kept on every plan. Compare it one to one: boat vs Runloop and boat vs CodeSandbox SDK.
Questions people ask
Is Runloop cheaper than CodeSandbox SDK?
It depends on the shape. One coding agent on workdays costs $113 on Runloop and $193 on CodeSandbox SDK, but 50 agents in parallel cost $5,268 a month on Runloop and $2,500 a month on CodeSandbox SDK. On boat those two cost $20 and $317.
Which is faster, Runloop or CodeSandbox SDK?
Runloop scored 8.29 runs a second on the StarSling Node.js build-and-test benchmark at 4 vCPU / 8 GB, and CodeSandbox SDK was not measured in that run. boat scored 26.69 on the same test.
Can Runloop or CodeSandbox SDK run Docker and Android emulators?
Both run Docker inside the sandbox, Runloop on its Docker-in-Docker blueprints. Neither documents nested virtualization, so Android emulators and VMs are not covered. boat exposes /dev/kvm for both.
Can I SSH into the sandbox?
Runloop offers SSH through a TLS proxy. CodeSandbox SDK connects through its SDK, with terminals, commands and files but no SSH. Every boat sandbox has SSH as well as an exec and files API.
Which one can sleep and wake up on a request?
Both. CodeSandbox SDK hibernates with a memory snapshot and an HTTP request wakes it with processes still running. Runloop can suspend an idle devbox and wake it on an HTTP request, but only on the Pro plan.
Is CodeSandbox SDK still maintained?
CodeSandbox is now part of Together AI. Together's docs call the CodeSandbox-based SDK deprecated and name together-sandbox as its successor, while self-serve sign-up still runs through CodeSandbox.
Try boat
Install the CLI, then run boat new.
7-day free trial