Runloop vs AWS AgentCore
Runloop gives each coding agent its own devbox, a micro-VM you size and build from a Dockerfile. AWS AgentCore hosts agent sessions and managed code and browser tools inside your AWS account, with small 2 vCPU microVMs that stop after 8 hours.
The companies and products
Runloop
Runloop is a US platform of sandboxes, called devboxes, for building, running and testing AI coding agents.
You get Linux devboxes built from Dockerfile-style blueprints, disk snapshots, HTTPS tunnels, agent gateways that hide API keys, and Axons, event streams that drive agents such as Claude Code. Each devbox runs in a micro-VM on Runloop's own bare-metal hypervisor, with a container layer inside. It is built for teams shipping coding agents, plus evaluation and RL fine-tuning work on the Enterprise plan.
- Compliance
- SOC 2 Type 2
- Architectures
- x86_64 and arm64 base blueprints
- Own-cloud deploy
- Deploy to VPC on AWS, GCP and Azure (Enterprise)
- Regions
- Not documented or selectable
- Company
- Runloop AI, Inc.
AWS AgentCore
Amazon Bedrock AgentCore is a set of AWS services to deploy and run AI agents: a Runtime that hosts your agent code, plus built-in Code Interpreter and Browser tools.
AgentCore is not a sandbox product first. It is a hyperscaler's agent platform: you push a container image or code, and each Runtime session gets its own Firecracker microVM that you reach through AgentCore APIs. Around it sit managed tools (Code Interpreter, Browser) and services for memory, identity, gateways and evaluations, and an Instances compute type runs longer sessions on EC2 in your own account. It is built for teams who already build on AWS and want agents to stay inside their account, IAM and VPC.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “20th cheapest of 367” means 19 of the other 366 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Runloop: price rank | AWS AgentCore: price rank | boat: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month Runloop fits a coding agent fully, while AgentCore is a compromise at 2 vCPU per microVM and 8 hours per session; boat ranks 3rd with a full machine that keeps its disk. | 20th cheapest of 367 $8,152 a month | 100th of 367, with limits $888 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB An agent that runs all month works on Runloop with an idle policy, but on AgentCore it outlasts the 8-hour microVM lifecycle and needs the Instances type on EC2. | 32nd cheapest of 366 $10,194 a month | 64th of 366, with limits $1,300 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month AWS AgentCore ranks 7th here: Code Interpreter is built for short code runs and bills only CPU actually used, ahead of Runloop and of boat's start limits. | 44th cheapest of 345 $525 a month | 7th cheapest of 345 $114 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month AgentCore sessions have no open ports, so your users cannot open what their sandbox runs; Runloop gives each devbox an HTTPS tunnel, and boat ranks 1st with URLs on up to 50 ports. | 26th cheapest of 361 $3,301 a month | 210th of 361, with limits $506 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month Runloop ranks 4th for RL rollouts: it starts large waves without published start limits and sells RL fine-tuning on Enterprise. | 4th cheapest of 360 $15,922 a month | 125th of 360, with limits $8,075 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 81st cheapest of 364 $2,562 a month | 33rd cheapest of 364 $650 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month | 121st cheapest of 377 $1,554 a month | 52nd cheapest of 377 $399 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB Both are compromises for a dev box: Runloop has no public IP or raw TCP, and AgentCore has no SSH and resets packages outside its storage mount after a stop. | 113th cheapest of 368 $410 a month | 228th of 368, with limits $52 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Runloop ships a desktop your agent can control, while AgentCore Browser drives only a browser; boat streams a full Linux desktop at 60fps. | 39th cheapest of 369 $2,306 a month | 104th of 369, with limits $202 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month AWS AgentCore ranks 4th here: its managed Chromium has live view, session replay, profiles and proxies, billed only for CPU used. | 40th cheapest of 367 $1,153 a month | 4th cheapest of 367 $144 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 68th cheapest of 346 $781 a month | 27th cheapest of 346 $287 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 266th of 338, with limits $15,611 a month | 165th of 338, with limits $5,458 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 10th cheapest of 360 $770 a month | 62nd of 360, with limits $294 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month | Can't run it | Can't run it | Can't run it |
Key features
| Feature | Runloop | AWS AgentCore | boat |
|---|---|---|---|
| What it is | A sandbox platform of devboxes for building, running and testing coding agents | An AWS agent platform: hosted agent sessions plus managed Code Interpreter and Browser tools | A sandbox platform: each agent gets its own full Linux machine |
| Isolation | Micro-VM on a custom bare-metal hypervisor, plus a container layer | Firecracker microVM per Runtime session; Code Interpreter and Browser in a containerized environment | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts |
| A 4 vCPU / 8 GB agent | Yes, as a devbox size on the Basic plan | Not on microVMs, which stop at 2 vCPU / 8 GB; only on the Instances type, EC2 in your account | Yes, the default size; up to 16 vCPU / 32 GB on the $100 plan and above |
| Session length | Fixed lifetime 1 hour by default, 48 hours max; an idle policy replaces it with no documented cap | Up to 8 hours per lifecycle on microVMs, 15 idle minutes by default; up to 14 days on Instances | No session cap on paid plans; you set the time to live |
| What survives a stop | The disk through suspend (Pro plan) or disk snapshots you take; shutdown ends the devbox | Only one storage mount under /mnt, up to 1 GB, in preview; EBS volumes on Instances; the rest resets | The whole disk, saved every minute and on stop, on every plan |
| Snapshots and fork | Disk snapshots on demand | No user snapshots or fork of a session | Automatic snapshots every minute; fork a running sandbox with its disk |
| Docker and /dev/kvm | Docker on the dnd blueprints, as root; nested virtualization not documented | Root inside the microVM; Docker and /dev/kvm not documented | Root, Docker, systemd and /dev/kvm for Android emulators and VMs |
| Inbound access | One HTTPS tunnel per devbox, open or bearer token; SSH through a TLS proxy; no public IP | No open ports or SSH; sessions are reached only through AgentCore APIs with IAM or JWT | HTTPS URLs for up to 50 ports, token-protected by default; raw TCP; SSH; dedicated IPv6 or IPv4 |
| Egress allowlist | Hostname allowlist with wildcards, or block all | Security groups in VPC mode | Open internet, no provider-side allowlist |
| Your own cloud and network | Deploy to VPC on AWS, GCP or Azure (Enterprise) | Runs in your AWS account: VPC mode, PrivateLink and a NAT gateway for a fixed outbound IP | boat's own EU data centers only |
| Browser and desktop | A desktop and a browser your agent can control | Managed Chromium with live view, session recording and replay, profiles and proxies; no full desktop | Full Linux desktop streamed at 60fps, with Chromium and computer use |
| Coding agents | Claude Code, Codex, OpenCode, Gemini CLI, DeepAgents, driven through Axons and Broker | AWS's own managed harness built on Strands Agents, or your own agent code | Claude Code, Codex, pi, OpenCode, Prime Agent, Kimi Code, Mistral Vibe, with a prompt, steer and events API |
| GPUs | No | NVIDIA G-series and Inferentia2 on the Instances type | No |
| Regions | Not documented or selectable | 17 commercial AWS regions plus GovCloud | EU only (Germany, Finland, France), EU data residency |
Every feature Battleships tracks (43)
| Feature | Runloop | AWS AgentCore | boat |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Not published | Yes |
| Nested virtualization | No | No | Yes |
| Root | Yes | Yes | Yes |
| Systemd | Not published | No | Yes |
| Desktop GUI | Yes | No | Yes |
| Browser | Yes | Yes | Yes |
| Computer-use API | Yes | Yes | Yes |
| SSH | Yes | No | Yes |
| Persistent disk | Yes | Yes | Yes |
| Pause/resume | Yes | No | Yes |
| Automatic snapshots | Yes | No | Yes |
| Fork/clone | Yes | No | Yes |
| Live fork (no pause) | Yes | No | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | Yes | No | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | Yes | Yes | No |
| HTTPS preview URLs | Yes | No | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | Yes | Yes | No |
| Secret proxy | Yes | No | Yes |
| Secret proxy for any API | Yes | No | Yes |
| Preinstalled agents | Yes | No | Yes |
| Agent harness API | Yes | No | Yes |
| MCP server | Yes | Yes | No |
| GPU | No | Yes | No |
| Windows | No | No | No |
| Arm64 | Yes | Yes | No |
| Self-hosting | No | No | No |
| BYOC | Yes | Yes | No |
| Open source | No | No | No |
| Scoped API keys | Yes | Not published | Yes |
| Spend limits | Not published | Yes | Yes |
| Usage per sandbox (API) | No | No | Yes |
| Audit logs | Not published | Yes | No |
| SSO | Yes | Yes | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | Not published | Yes | No |
| EU data residency | No | Yes | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
On the hpc-sandbox-benchmarks Node.js build-and-test loop at 4 vCPU / 8 GB (run 36498291449, 2026-09-29), boat scored 26.69 runs a second and Runloop 8.29, about 3.2 times apart. AWS AgentCore was not part of that run, so there is no comparable score; its microVM sessions stop at 2 vCPU, so a like-for-like test would need the Instances type on EC2.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. AWS AgentCore has no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Runloop, a month | AWS AgentCore, a month | boat, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $113 · Basic | $26 · Pay-as-you-go (Instances) · with limits | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $429 · Basic | $107 · Pay-as-you-go (Instances) · with limits | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $5,268 · Basic | $1,286 · Pay-as-you-go (Instances) · with limits | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $1,441 · Basic | $423 · Pay-as-you-go · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $3,184 · Basic | $1,615 · Pay-as-you-go (Instances) | $2,000 · $2000/mo · with limits |
Runloop: Runloop bills per second for the CPU and memory you allocate while a devbox is starting, running, suspending or resuming, and bills its disk for the devbox's whole life. Basic has no plan fee; Pro adds $250 a month that is not usage credit and unlocks suspend and resume.
AWS AgentCore: Runtime microVMs, Code Interpreter and Browser bill per second for CPU actually used and for peak memory. The 4 vCPU shape in these scenarios is the Instances type: the EC2 on-demand price plus a 12% management fee, with EBS storage and data transfer billed by AWS on top.
AWS AgentCore: Plans start at $20 a month and the whole fee is usage credit, billed per second while a sandbox runs. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
AgentCore's microVM sessions stop at 2 vCPU / 8 GB and 8 hours, so every 4 vCPU scenario here runs on the Instances type, and files outside the storage mount reset after a stop, so it does not fully fit four of the five. Runloop fits all five, but publishes no egress price, and keeping a devbox's disk while idle needs the Pro plan. boat's limit that bites is start rate: even the $2,000 plan queues bursts above 90 starts a minute, so 1,000 jobs at once does not fit there, while AgentCore and Runloop both fit it.
| Scenario | Runloop | AWS AgentCore | boat |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Runloop when
you ship coding agents that need a 4 vCPU devbox with Docker, hostname allowlists, arm64 or very large waves of RL rollouts, without running EC2 yourself.
- A sized devbox per coding agent. Teams shipping coding agents get 4 vCPU devboxes with Docker, built from Dockerfiles, without running EC2 in their own account.
- Lock a devbox to listed hosts. Useful if a sandbox must only reach a fixed list of hosts: Runloop allows listed hostnames, with wildcards, or blocks everything.
- Big waves of RL rollouts. For RL work, Runloop starts large waves without published start limits and sells RL fine-tuning on Enterprise.
Use AWS AgentCore when
your agents must stay inside your AWS account and VPC, need a GPU, or are short code and browser tasks that fit the managed tools.
- It lives inside your AWS account. Teams built on AWS keep agents behind their own IAM, VPC, security groups and PrivateLink, in 17 commercial regions.
- Managed tools that bill for what they use. For short code runs and browser tasks, Code Interpreter and Browser charge only for CPU actually used, with live view and replay.
- GPUs and two-week sessions. Agents that need a GPU or a session of up to 14 days can run on the Instances type, EC2 in your account.
Where boat fits
If your agents do real work on a machine for hours, installing, building, testing and running Docker or emulators, both Runloop and AgentCore make you work around their limits. boat gives each agent a full Linux VM on dedicated AMD Ryzen 9 9950X hosts that keeps its whole disk, runs as long as you want and serves URLs you can open. It ran about 3.2 times faster than Runloop on the same CPU benchmark.
A whole machine with /dev/kvm
Runloop documents Docker but not nested virtualization, and AgentCore documents neither and caps microVMs at 2 vCPU. On both, an Android emulator or a VM inside will not start; on boat it runs.
The disk stays, with no session cap
A Runloop devbox is gone once it shuts down unless you pay for the Pro plan's suspend or write snapshot code. AgentCore keeps only a 1 GB mount and stops after 8 hours. boat saves the whole disk every minute.
Open the app your agent builds
Runloop gives one HTTPS tunnel and no raw TCP, and AgentCore has no open ports, so you proxy traffic through your own code. boat gives HTTPS URLs on up to 50 ports, raw TCP and SSH.
Use boat when your agents do real work on a machine for hours: installs, builds, tests, Docker, emulators and a browser, on faster CPUs, with a disk that keeps everything and a URL you can open. Compare it one to one: boat vs Runloop and boat vs AWS AgentCore.
Questions people ask
Is Runloop cheaper than AWS AgentCore?
No, AgentCore is lower in every scenario on this page, for example $26 against $113 a month for one coding agent on workdays. But AgentCore only fully fits the 1,000 short jobs scenario, at $1,615 against $3,184 on Runloop; in the agent scenarios its Instances lose packages installed outside the storage mount after a stop. boat costs $20 a month for that workday agent.
Which is faster, Runloop or AWS AgentCore?
There is no like-for-like answer: AgentCore was not in the benchmark run. Runloop scored 8.29 runs a second on the Node.js build-and-test loop at 4 vCPU / 8 GB, and boat 26.69 on the same test.
Can Runloop or AWS AgentCore run Docker?
Runloop runs Docker inside devboxes built from its dnd blueprints, as root. AgentCore runs your container image as root in a microVM, but its docs do not say Docker works inside a session. boat runs Docker with systemd and also exposes /dev/kvm.
How long can a session run on each?
A Runloop devbox has a fixed lifetime of 1 hour by default and 48 hours at most, but an idle policy replaces that with no documented cap. An AgentCore microVM session lasts up to 8 hours, or up to 14 days on the Instances type. boat has no session cap on paid plans.
Do files survive when a session stops?
On Runloop, the disk survives a suspend on the Pro plan or lives on in disk snapshots; a shutdown ends the devbox. On AgentCore, only a storage mount under /mnt, up to 1 GB and in preview, survives, or EBS volumes on Instances. boat keeps the whole disk and saves it every minute.
Is boat SOC 2 compliant?
SOC 2 is in progress. We're open about where we stand: live status at status.ascii.dev and our data retention policy at docs.boat.dev/data-retention.
Try boat
Install the CLI, then run boat new.
7-day free trial