Runloop vs GitHub Codespaces
Runloop is a sandbox platform built for AI coding agents: devboxes you create and drive from code, with agent gateways, network rules and Docker inside. GitHub Codespaces is a dev environment for people working on a GitHub repository, opened in VS Code or the browser, and every codespace stops after 12 hours at most.
The companies and products
Runloop
Runloop is a US platform of sandboxes, called devboxes, for building, running and testing AI coding agents.
You get Linux devboxes built from Dockerfile-style blueprints, disk snapshots, HTTPS tunnels, agent gateways that hide API keys, and Axons, event streams that drive agents such as Claude Code. Each devbox runs in a micro-VM on Runloop's own bare-metal hypervisor, with a container layer inside. It is built for teams shipping coding agents, plus evaluation and RL fine-tuning work on the Enterprise plan.
- Compliance
- SOC 2 Type 2
- Architectures
- x86_64 and arm64 base blueprints
- Own-cloud deploy
- Deploy to VPC on AWS, GCP and Azure (Enterprise)
- Regions
- Not documented or selectable
- Company
- Runloop AI, Inc.
GitHub Codespaces
GitHub Codespaces is GitHub's cloud development environment: a ready-to-code Linux machine for a repository, opened in the browser, VS Code or the GitHub CLI.
Codespaces is a different kind of product from boat: a dev environment for people working on a GitHub repository, not a sandbox API for agents. Each codespace is a Docker container built from the repo's dev container config, running on its own newly built VM on Azure, with 2 to 32 cores. It is built for developers and teams who want every contributor to start from the same setup, with prebuilds, port forwarding and GitHub billing.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “20th cheapest of 367” means 19 of the other 366 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Runloop: price rank | GitHub Codespaces: price rank | boat: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month Runloop fits a software factory fully, while Codespaces is a compromise because every codespace stops at 12 hours; boat adds /dev/kvm for emulators and keeps the disk on every plan. | 20th cheapest of 367 $8,152 a month | 274th of 367, with limits $4,618 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB Runloop can keep a devbox running with an idle policy, while a codespace stops after 12 hours even in use, so an always-on assistant needs restarts there. | 32nd cheapest of 366 $10,194 a month | 309th of 366, with limits $6,970 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month Runloop fits many short code runs well, while each codespace builds a dev container on a new VM, which prebuilds speed up but do not remove. | 44th cheapest of 345 $525 a month | 288th of 345, with limits $619 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Runloop is built to hand devboxes to your own users from code, while Codespaces are made for GitHub users on a repository and GitHub does not publish how many can run at once. | 26th cheapest of 361 $3,301 a month | 281st of 361, with limits $1,964 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month Runloop ranks 4th for RL rollouts: it starts large waves without published start limits, and Codespaces is not built for it at all. | 4th cheapest of 360 $15,922 a month | 282nd of 360, with limits $9,489 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 81st cheapest of 364 $2,562 a month | 295th of 364, with limits $1,519 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month Docker works inside both, but neither Runloop nor Codespaces matches boat's measured CPU speed, and Codespaces has no snapshot API to start runs from a warm machine. | 121st cheapest of 377 $1,554 a month | 314th of 377, with limits $909 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB Codespaces is a polished VS Code box per repo but stops after 12 hours; Runloop can run all month with an idle policy; boat is a full VM with a public address. | 113th cheapest of 368 $410 a month | 130th of 368, with limits $270 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Runloop ships a desktop and a browser your agent can control, a codespace has no graphical desktop, and boat streams a 60fps desktop you can embed. | 39th cheapest of 369 $2,306 a month | 267th of 369, with limits $1,367 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 40th cheapest of 367 $1,153 a month | 275th of 367, with limits $678 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 68th cheapest of 346 $781 a month | 310th of 346, with limits $922 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 266th of 338, with limits $15,611 a month | 280th of 338, with limits $18,637 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 10th cheapest of 360 $770 a month | 267th of 360, with limits $445 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month | Can't run it | Can't run it | Can't run it |
Key features
| Feature | Runloop | GitHub Codespaces | boat |
|---|---|---|---|
| Isolation | Micro-VM on a custom hypervisor, with a container layer on top | A Docker container on its own newly built VM, with its own virtual network | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts |
| How long it runs | Lifetime 1 hour by default, 48 hours max; an idle policy replaces it, with no documented cap | Stops after 30 idle minutes by default and after 12 hours at most, even while in use | Runs until you stop it; no session cap on paid plans |
| CPU speed (Node.js build-and-test, 4 vCPU / 8 GB) | 8.29 runs a second | No comparable measurement | 26.69 runs a second |
| Docker inside | Yes, on the Docker-in-Docker blueprints, as root | Yes, inside the dev container, with sudo | Yes, with root and systemd |
| Nested virtualization | Not documented | /dev/kvm and nested virtualization are not documented | Yes: /dev/kvm, Android emulators and VMs inside |
| Desktop and computer use | Desktop and browser an agent can control | VS Code, JupyterLab and a terminal; no graphical desktop | 60fps desktop with Chromium, computer use and browser control, embeddable stream |
| Keeping files | Disk kept through suspend (Pro plan) or disk snapshots; shutdown ends the devbox | Files kept while stopped, storage billed; stopped codespaces deleted after 30 days by default | Snapshot every minute and on stop, on every plan; a stopped sandbox keeps its disk and costs nothing |
| Starting image | Blueprints built from Dockerfiles, plus disk snapshots | Dev container config in the repo, with prebuilds built by GitHub Actions | Templates from named snapshots, no Dockerfile builds |
| Coding agents | Claude Code, Codex, OpenCode, Gemini CLI and DeepAgents, driven through Axons | GitHub Copilot as a VS Code extension; other agents you install yourself | Claude Code, Codex, OpenCode and more preinstalled, with a prompt, steer and events API |
| Keeping API keys out | Agent gateways for any API endpoint with bearer, header or basic auth | Secrets are copied into the codespace as environment variables | Credentials proxy for model providers and any HTTPS API; switch a credential to another account when one runs out |
| Reaching the sandbox | One HTTPS tunnel per devbox; SSH through a TLS proxy; no public IP | A URL per forwarded port; SSH and CLI tunnels; no public IP | HTTPS URLs for up to 50 ports, token-protected, raw TCP, dedicated IPv6 or IPv4 |
| Outbound rules | Hostname allowlist with wildcards, or block all | Open internet; GitHub says there is no way to restrict it | Open internet, no provider-side allowlist |
| CPU architecture | x86_64 and arm64 | x86_64 only | x86_64 only |
| Where it runs | Regions not documented; deploy into your own AWS, GCP or Azure on Enterprise | US East, US West, Europe West and Southeast Asia | EU only (Germany, Finland, France), EU data residency |
Every feature Battleships tracks (43)
| Feature | Runloop | GitHub Codespaces | boat |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | No | No | Yes |
| Root | Yes | Yes | Yes |
| Systemd | Not published | Not published | Yes |
| Desktop GUI | Yes | No | Yes |
| Browser | Yes | No | Yes |
| Computer-use API | Yes | No | Yes |
| SSH | Yes | Yes | Yes |
| Persistent disk | Yes | Yes | Yes |
| Pause/resume | Yes | Yes | Yes |
| Automatic snapshots | Yes | No | Yes |
| Fork/clone | Yes | No | Yes |
| Live fork (no pause) | Yes | No | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | Yes | No | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | Yes | No | No |
| HTTPS preview URLs | Yes | Yes | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | Yes | No | No |
| Secret proxy | Yes | No | Yes |
| Secret proxy for any API | Yes | No | Yes |
| Preinstalled agents | Yes | No | Yes |
| Agent harness API | Yes | No | Yes |
| MCP server | Yes | No | No |
| GPU | No | No | No |
| Windows | No | No | No |
| Arm64 | Yes | No | No |
| Self-hosting | No | No | No |
| BYOC | Yes | No | No |
| Open source | No | No | No |
| Scoped API keys | Yes | Not published | Yes |
| Spend limits | Not published | Yes | Yes |
| Usage per sandbox (API) | No | No | Yes |
| Audit logs | Not published | Yes | No |
| SSO | Yes | Yes | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | Not published | No | No |
| EU data residency | No | Yes | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
On StarSling's Node.js build-and-test benchmark at 4 vCPU / 8 GB (run of 2026-09-29), boat did 26.69 runs a second and Runloop 8.29, about 3.2 times apart. GitHub Codespaces was not in that benchmark, so there is no comparable score for it. The test measures CPU work only, not start time or network speed.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. GitHub Codespaces has no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Runloop, a month | GitHub Codespaces, a month | boat, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $113 · Basic | $51 · Personal account: GitHub Free | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $429 · Basic | $270 · Personal account: GitHub Free · with limits | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $5,268 · Basic | $3,019 · Personal account: GitHub Free | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $1,441 · Basic | $850 · Personal account: GitHub Free · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $3,184 · Basic | $1,889 · Personal account: GitHub Free · with limits | $2,000 · $2000/mo · with limits |
Runloop: Runloop bills per second for the CPU and memory you allocate, including while a devbox starts, suspends or resumes, and bills its disk for the devbox's whole life. Basic has no plan fee; Pro adds $250 a month that is not usage credit and unlocks suspend and resume.
GitHub Codespaces: Codespaces bills $0.09 per core-hour while a codespace is active, plus $0.07 per GB-month for storage while it exists, stopped too. There is no 4-core machine with 8 GB, so a 4 vCPU / 8 GB job is priced on the 4-core 16 GB machine; personal accounts get a free monthly quota and organizations do not.
GitHub Codespaces: Plans start at $20 a month and the whole fee is usage credit, spent per second while a sandbox runs. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
Codespaces stops every codespace after 12 hours, so it does not fit the always-on scenario, and GitHub does not publish how many can run at once, so 300 user sandboxes and 1,000 short jobs show as not confirmed. Runloop fits every scenario we priced, but publishes no egress price and no concurrency cap for paid plans, and suspend needs Pro. boat's limit that bites is start rate: even the $2,000 plan queues bursts above 90 starts a minute, so 1,000 jobs at once does not fit.
| Scenario | Runloop | GitHub Codespaces | boat |
|---|---|---|---|
| One coding agent, workdays |
| None that bite | None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Runloop when
your code launches agents in large waves or long jobs, and you want hostname allowlists, arm64 or devboxes inside your own AWS, GCP or Azure account.
- Built to be driven by code. Useful if your product starts agents: Runloop devboxes come with an API, agent gateways for keys and Axons to drive Claude Code or Codex.
- Runs past 12 hours. Useful for long agent jobs: with an idle policy a Runloop devbox can keep running, while a codespace always stops at 12 hours.
- Network rules, arm64 and your own cloud. Useful for locked-down work: Runloop can allow only listed hostnames, offers arm64, and deploys into your AWS, GCP or Azure on Enterprise.
Use GitHub Codespaces when
your developers want a ready VS Code environment for a GitHub repository, opened from a branch or pull request, in the US, EU or Asia.
- Built into GitHub. Useful if your team lives on GitHub: open a codespace from any repo, branch or pull request, with the dev container already set up.
- Free hours for personal accounts. Useful for occasional personal work: a GitHub Free account includes 120 core-hours and 15 GB-month of storage each month.
- Regions in the US and Asia. Useful if your developers sit in the US or Asia: codespaces run in US East, US West, Europe West and Southeast Asia.
Where boat fits
If what you are really running is agents that work for hours, look at boat. Each sandbox is a full VM with /dev/kvm, raw TCP and a 60fps desktop, its disk is saved every minute, and paid plans have no session cap. You drive the built-in coding agents from outside, with real keys kept out of the sandbox.
A full VM with /dev/kvm and open ports
Neither Runloop nor Codespaces documents /dev/kvm, so Android emulators and VMs do not start. Both reach the machine through HTTPS tunnels or forwarded ports with no public IP, so a raw TCP service has no way in.
Work kept without restarts or extra plans
A codespace stops at 12 hours and its processes end, so you write a restart script. On Runloop a devbox is gone once it shuts down, so keeping its disk means the $250 Pro plan or snapshot code you maintain.
Faster CPUs, measured
On the same Node.js build-and-test benchmark, boat ran 26.69 runs a second and Runloop 8.29, so each install and test waits about three times longer there. Codespaces has no comparable measurement.
Use boat when your agents do real work for hours or days: full VMs with /dev/kvm, a desktop and open ports, CPUs about three times faster than Runloop's, and files kept on every plan. Compare it one to one: boat vs Runloop and boat vs GitHub Codespaces.
Questions people ask
Is Runloop cheaper than GitHub Codespaces?
Usually not. One coding agent on workdays is $113 a month on Runloop and $51 on Codespaces, and 50 agents in parallel are $5,268 against $3,019. boat comes to $20 and $317 for the same two scenarios.
Which is faster, Runloop or GitHub Codespaces?
We cannot say: Runloop scored 8.29 runs a second on the Node.js build-and-test benchmark, but Codespaces has no comparable measurement. boat scored 26.69 on the same test.
Can I run an AI agent 24/7 on Runloop or Codespaces?
On Runloop, yes: an idle policy replaces the 48-hour lifetime setting, and no cap is documented for it. A codespace stops after 12 hours at most, even while in use, so it needs a restart script. A boat sandbox runs until you stop it on paid plans.
Can Runloop and Codespaces run Docker?
Yes. Runloop runs Docker on its Docker-in-Docker blueprints as root, and a codespace runs Docker inside its dev container with sudo. Neither documents /dev/kvm, which boat has on every machine for emulators and VMs.
Can I limit which websites a sandbox can reach?
On Runloop, yes: network policies allow only the hostnames you list, or block all traffic. GitHub says there is no way to restrict outbound traffic from a codespace, and boat has no provider-side allowlist either.
Try boat
Install the CLI, then run boat new.
7-day free trial