Runloop vs Agent 37
Runloop gives each coding agent a micro-VM devbox, with hostname egress rules, arm64 and deploys into your own cloud. Agent 37 gives each user a lighter gVisor sandbox that sleeps when idle, keeps its memory and costs far less per hour.
The companies and products
Runloop
Runloop is a US platform of sandboxes, called devboxes, for building, running and testing AI coding agents.
You get Linux devboxes built from Dockerfile-style blueprints, disk snapshots, HTTPS tunnels, agent gateways that hide API keys, and Axons, event streams that drive agents such as Claude Code. Each devbox runs in a micro-VM on Runloop's own bare-metal hypervisor, with a container layer inside. It is built for teams shipping coding agents, plus evaluation and RL fine-tuning work on the Enterprise plan.
- Compliance
- SOC 2 Type 2
- Architectures
- x86_64 and arm64 base blueprints
- Own-cloud deploy
- Deploy to VPC on AWS, GCP and Azure (Enterprise)
- Regions
- Not documented or selectable
- Company
- Runloop AI, Inc.
Agent 37
Agent 37 hosts AI agents, one isolated, always-on sandbox per user, behind one API.
You create an instance from a template (Hermes, OpenClaw, Claude Code, Codex, OpenCode, Pi, Grok or your own Docker image) and talk to it through a chat API, files, exec, SSH and HTTPS URLs. The default instance is a container enforced by gVisor; a VM type gives a full Linux machine with its own kernel for Docker, Kubernetes and Android emulators. It is built for companies that give each of their customers an agent.
- Where it runs
- Dedicated servers in the United States
- Instance types
- Default (shared), Performance (dedicated cores), VM
- Compliance
- SOC 2 Type I, Type II in progress
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “20th cheapest of 367” means 19 of the other 366 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Runloop: price rank | Agent 37: price rank | boat: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month Agent 37 ranks 2nd and Runloop ranks 20th; boat ranks 3rd and gives every agent Docker, systemd and /dev/kvm on a full VM without an extra instance type. | 20th cheapest of 367 $8,152 a month | 2nd cheapest of 367 $551 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB Agent 37 ranks 1st here: an always-on assistant per user sleeps when idle, keeps its memory and wakes on the next message, while Runloop's suspend needs the Pro plan. | 32nd cheapest of 366 $10,194 a month | Cheapest of 366 $279 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month Agent 37's small default sandboxes suit short code runs; boat's start limits make very large bursts a poor fit. | 44th cheapest of 345 $525 a month | Cheapest of 345 $22 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month boat ranks 1st; Agent 37 is a compromise because its cap of 200 instances counts sleeping and stopped ones, and Runloop fits at a higher bill. | 26th cheapest of 361 $3,301 a month | 30th of 361, with limits $79 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month Runloop ranks 4th for RL rollouts: it starts large waves without published start limits, while Agent 37's self-serve tiers stop at 200 instances. | 4th cheapest of 360 $15,922 a month | 6th of 360, with limits $363 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 81st cheapest of 364 $2,562 a month | 90th of 364, with limits $58 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month Agent 37 ranks 2nd and boat 4th; Docker builds on Agent 37 need its VM type, and Runloop's CPU ran about three times slower than boat's in the same benchmark. | 121st cheapest of 377 $1,554 a month | 2nd cheapest of 377 $84 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB | 113th cheapest of 368 $410 a month | 3rd cheapest of 368 $12 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month All three ship a desktop and a browser an agent can control; boat streams its desktop at 60fps and you can embed it in your app. | 39th cheapest of 369 $2,306 a month | Cheapest of 369 $53 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 40th cheapest of 367 $1,153 a month | 48th of 367, with limits $29 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 68th cheapest of 346 $781 a month | Cheapest of 346 $32 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 266th of 338, with limits $15,611 a month | 36th of 338, with limits $648 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month Agent 37 ranks 1st and boat 2nd; Runloop can lock each devbox to a hostname allowlist, which neither of the others offers. | 10th cheapest of 360 $770 a month | Cheapest of 360 $43 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month | Can't run it | Can't run it | Can't run it |
Key features
| Feature | Runloop | Agent 37 | boat |
|---|---|---|---|
| Isolation | Micro-VM on a custom bare-metal hypervisor, plus a container layer | gVisor by default; full VM as an opt-in instance type | Every sandbox is a full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts |
| Docker inside | Yes, on the Docker-in-Docker blueprints, as root | Only on the VM type, at 2.5x the default compute rate | Yes, in every sandbox, with root and systemd |
| Nested virtualization | Not documented | /dev/kvm on the VM type, with a ready Android template | /dev/kvm in every sandbox; Android emulators and VMs run inside |
| CPU speed (Node.js benchmark, 4 vCPU / 8 GB) | 8.29 runs a second | No published result; shared cores by default, dedicated cores on the Performance type at 4x | 26.69 runs a second |
| CPU architecture | x86_64 and arm64 | linux/amd64 images | x86_64 only |
| Images | Blueprints built from Dockerfiles, plus disk snapshots | Your own linux/amd64 Docker image as a template | Templates from named snapshots of a running sandbox, no Dockerfile builds |
| Sleep when idle | Suspend after idle, wake on an HTTP request (Pro plan) | Auto-sleep after a quiet period, memory checkpointed, wakes on the next request | No idle timer; a time limit you set (default 1 hour), or no limit |
| Keeping files | Disk kept through suspend (Pro) or manual disk snapshots; shutdown ends the devbox | Disk kept while stopped or asleep; daily backups plus one on-demand backup | Snapshot every minute and on stop, on every plan; resume with the same disk |
| Long runs | Fixed lifetime 1 hour by default, 48 hours max; an idle policy replaces it | Runs until you stop it or it sleeps | No session cap on paid plans |
| Fork a running machine | Disk snapshots of a devbox to start new ones from | No fork; backups restore in place only | Fork a running sandbox; named snapshots as shareable templates |
| Inbound networking | One HTTPS tunnel per devbox; SSH through a TLS proxy; no public IP | HTTPS URL per port, custom domains; no public IP; SSH through a WebSocket tunnel | HTTPS URLs for up to 50 ports, raw TCP, dedicated IPv6 or IPv4 |
| Egress rules | Hostname allowlist with wildcards, or block all | Open internet, no allowlist | Open internet, no provider-side allowlist |
| Keys kept out of the sandbox | Agent gateways for any API endpoint | Managed LLM, search and Composio; your own keys go in as env vars | Credentials proxy for model providers and any HTTPS API; switch to another account when one runs out of credits |
| Where it runs | Regions not documented; Deploy to VPC on AWS, GCP or Azure (Enterprise) | Dedicated servers in the United States | EU only (Germany, Finland, France), EU data residency |
Every feature Battleships tracks (43)
| Feature | Runloop | Agent 37 | boat |
|---|---|---|---|
| Full VM (own kernel) | Yes | No | Yes |
| Docker inside | Yes | No | Yes |
| Nested virtualization | No | No | Yes |
| Root | Yes | Yes | Yes |
| Systemd | Not published | Not published | Yes |
| Desktop GUI | Yes | Yes | Yes |
| Browser | Yes | Yes | Yes |
| Computer-use API | Yes | Yes | Yes |
| SSH | Yes | Yes | Yes |
| Persistent disk | Yes | Yes | Yes |
| Pause/resume | Yes | Yes | Yes |
| Automatic snapshots | Yes | Yes | Yes |
| Fork/clone | Yes | Yes | Yes |
| Live fork (no pause) | Yes | Yes | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | Yes | Yes | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | Yes | Yes | No |
| HTTPS preview URLs | Yes | Yes | Yes |
| Raw TCP inbound | No | Not published | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | Yes | No | No |
| Secret proxy | Yes | Yes | Yes |
| Secret proxy for any API | Yes | No | Yes |
| Preinstalled agents | Yes | Yes | Yes |
| Agent harness API | Yes | Yes | Yes |
| MCP server | Yes | No | No |
| GPU | No | No | No |
| Windows | No | No | No |
| Arm64 | Yes | No | No |
| Self-hosting | No | No | No |
| BYOC | Yes | No | No |
| Open source | No | No | No |
| Scoped API keys | Yes | Not published | Yes |
| Spend limits | Not published | Yes | Yes |
| Usage per sandbox (API) | No | Yes | Yes |
| Audit logs | Not published | Not published | No |
| SSO | Yes | No | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | Not published | No | No |
| EU data residency | No | No | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
On the hpc-sandbox-benchmarks Node.js build-and-test loop at 4 vCPU / 8 GB (run 36498291449, 2026-09-29), boat scored 26.69 runs a second and Runloop 8.29, about 3.2 times apart. Agent 37 has no published result on that benchmark, so there is no comparable measurement. Agent 37 says its default instances share host resources, and sells a Performance type on dedicated cores.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. Agent 37 has no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Runloop, a month | Agent 37, a month | boat, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $113 · Basic | $6 · Wallet (any top-up) | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $429 · Basic | $12 · Wallet (any top-up) | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $5,268 · Basic | $329 · Wallet ($100 lifetime top-ups) | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $1,441 · Basic | $33 · Wallet ($250 lifetime top-ups) · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $3,184 · Basic | $73 · Wallet ($250 lifetime top-ups) · with limits | $2,000 · $2000/mo · with limits |
Runloop: Runloop bills per second for the CPU and memory you allocate, while a devbox is starting, running, suspending or resuming, and bills its disk for the devbox's whole life. Basic has no plan fee; Pro adds $250 a month that is not usage credit and unlocks suspend and resume.
Agent 37: Agent 37 uses a prepaid wallet with no plan fee, metered per minute: $0.80 per vCPU and $0.70 per GB of RAM per month while running, and $0.09 per GB of disk per month even while stopped or asleep. Performance compute costs 4x and VM compute 2.5x the default rate.
Agent 37: Plans start at $20 a month and the whole fee is usage credit, billed per second while a sandbox runs. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
Agent 37's instance cap is 10 after your first top-up, 50 at $100 of total top-ups and 200 at $250, and it counts sleeping and stopped instances, so sandboxes for your users and 1,000 jobs at once do not fit. Its 50-agent price uses the VM type, since that scenario needs Docker. Runloop publishes no concurrency cap on paid plans and no egress price, and its fixed lifetime tops out at 48 hours unless an idle policy replaces it. boat's limit that bites is start rate, which stops it in the 1,000-at-once burst.
| Scenario | Runloop | Agent 37 | boat |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Runloop when
you need hostname allowlists, arm64, devboxes inside your own AWS, GCP or Azure account, or very large bursts of short jobs.
- Lock down the network. Useful if a sandbox must only reach a fixed list of hosts: Runloop allows listed hostnames, with wildcards, or blocks everything.
- Run it in your own cloud. On the Enterprise plan, Runloop can run devboxes inside your own AWS, GCP or Azure account.
- arm64 and big bursts. Runloop offers arm64 blueprints and publishes no concurrency cap on paid plans, so 1,000 short jobs at once fit.
Use Agent 37 when
you host one mostly idle assistant per customer in the US that should sleep with its memory kept and wake on the next message.
- Sleeps when idle and keeps its memory. Useful for assistants that sit idle most of the day: an instance sleeps, keeps its memory and wakes on the next message.
- A low bill for light, steady work. Agent 37 meters per minute from a prepaid wallet with no plan fee, and its default rates are the lowest of the three.
- Runs in the United States. Agent 37 runs on dedicated servers in the US, closer to US users.
Where boat fits
If your agents do real work for hours, installing packages, building, testing and running Docker, look at boat. Every boat sandbox is a full VM with Docker, systemd and /dev/kvm, on CPUs that ran about three times faster than Runloop's, with files kept on every plan.
A full Linux machine in every sandbox
Runloop documents Docker-in-Docker but not /dev/kvm, so Android emulators and VMs do not start. Agent 37 runs Docker only on its VM type, at 2.5x the compute rate, and with no init system.
Faster CPUs for builds and tests
boat scored 26.69 runs a second and Runloop 8.29 on the same benchmark, so every install and test waits about three times longer there. Agent 37 has no published score and shares cores unless you pay 4x for Performance.
Real hosting, not just tunnels
Runloop and Agent 37 give no public IP and reach SSH through a proxy or tunnel, so a database or server speaking raw TCP has no way in. boat gives HTTPS on up to 50 ports, raw TCP and a dedicated IP.
Use boat when your agents do real work for hours: installs, builds, tests, Docker and emulators on a full VM with fast CPUs, forks of running machines and raw TCP hosting. Compare it one to one: boat vs Runloop and boat vs Agent 37.
Questions people ask
Is Runloop cheaper than Agent 37?
No. In our scenarios Agent 37 is far lower: one coding agent on workdays costs $6 a month on Agent 37 and $113 on Runloop, and one agent that never stops costs $12 versus $429. Runloop fits sandboxes for your users and 1,000 jobs at once, where Agent 37's 200-instance cap does not.
Which is faster, Runloop or Agent 37?
We can't say. Runloop scored 8.29 runs a second on the Node.js build-and-test benchmark at 4 vCPU / 8 GB, and Agent 37 has no published result. boat scored 26.69 on the same test.
Can Agent 37 or Runloop run Docker?
Runloop runs Docker on its Docker-in-Docker blueprints. Agent 37 runs Docker only on its VM type, which bills compute at 2.5x; the default gVisor type cannot run a Docker daemon. On boat, Docker works in every sandbox.
Can I run Android emulators on Runloop or Agent 37?
Agent 37's VM type exposes /dev/kvm and has a ready Android template. Runloop does not document nested virtualization. boat exposes /dev/kvm in every sandbox.
Which one sleeps an idle agent and wakes it?
Both can. Runloop suspends an idle devbox and wakes it on an HTTP request, on the Pro plan. Agent 37 sleeps on idle, keeps the memory, and wakes on the next request on any wallet.
Where do Runloop, Agent 37 and boat run?
Agent 37 runs on dedicated servers in the United States. Runloop does not document regions, and on Enterprise it can deploy into your own AWS, GCP or Azure account. boat runs only in the EU, in Germany, Finland and France.
Try boat
Install the CLI, then run boat new.
7-day free trial