Runloop vs Fly.io Machines

Runloop is a sandbox product for AI coding agents, with agent gateways, agent wrappers and hostname egress rules. Fly.io Machines are microVMs for hosting your own apps near users. On Fly you build the agent sandbox on top yourself, or use Fly's separate Sprites product.

The companies and products

Runloop

Runloop is a US platform of sandboxes, called devboxes, for building, running and testing AI coding agents.

You get Linux devboxes built from Dockerfile-style blueprints, disk snapshots, HTTPS tunnels, agent gateways that hide API keys, browser and computer use, and Axons, event streams that drive agents such as Claude Code. Each devbox runs in a micro-VM on Runloop's own bare-metal hypervisor, with a container layer inside. It is built for teams that ship coding agents, plus evaluation and RL fine-tuning work on the Enterprise plan.

Fly.io Machines

Fly.io is an app hosting platform, and Machines are the microVMs underneath it that you can also start and stop through a REST API.

A Fly Machine boots your Docker image as a Firecracker microVM on Fly's own servers, in one of 17 regions. You pick a shared or performance CPU preset and mount one Volume at a time for data that must last. Fly Proxy can stop, suspend and wake Machines as traffic comes and goes. Machines themselves are an app-hosting building block; for agent sandboxes Fly also sells Sprites (see boat vs Sprites), so this page compares only the Machines product.

Which use cases each one fits

For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “24th cheapest of 365” means 23 of the other 364 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.

Use case and its workloadRunloop: price rankFly.io Machines: price rankboat: price rank
Software factory
100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month
Runloop fits with its Docker-in-Docker blueprints. On Fly you set it up: install Docker in your image, and keep files on a Volume or turn on persist_rootfs. boat saves its disk every minute and at each stop and adds /dev/kvm.
24th cheapest of 365
$8,152 a month
123rd of 365, with limits
$836 a month
5th cheapest of 365
$634 a month
Always-on AI employees
50 machines 24/7 · 2 vCPU / 4 GB
38th cheapest of 361
$10,194 a month
186th of 361, with limits
$816 a month
6th cheapest of 361
$657 a month
Code interpreter bursts
100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month
Runloop fits and ranks higher than boat here. Fly bills per second with no plan fee, but its lowest rate needs a 1-year reservation. boat includes 12 / 60 / 200 starts per minute / hour / day on $20, and extra starts cost $0.03 to $0.07.
52nd cheapest of 336
$525 a month
83rd of 336, with limits
$94 a month
189th of 336, with limits
$2,000 a month
Sandboxes for your users
300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month
Runloop fits. Fly's Machines API takes about one create request per second per app, so big sign-up waves need pre-created Machines or several apps. boat allows 100 sandboxes at once on $20 and 2,000 on $2,000.
37th cheapest of 349
$3,301 a month
75th of 349, with limits
$600 a month
3rd cheapest of 349
$209 a month
RL rollouts
1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month
Runloop ranks highest of the three here. It publishes no start-rate limit for paid plans and sells RL fine-tuning on Enterprise.
8th cheapest of 349
$15,922 a month
52nd of 349, with limits
$2,838 a month
39th of 349, with limits
$2,000 a month
Agent evals
250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month
91st cheapest of 359
$2,562 a month
124th of 359, with limits
$457 a month
10th cheapest of 359
$200 a month
CI runners
40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month
Builds keep the CPU busy. Fly's lowest-priced shared vCPUs guarantee 6.25% of a core, and full cores cost more. In StarSling's run of 3 October 2026, boat did 28.06 runs a second and Runloop 8.93; Fly was not measured.
125th cheapest of 374
$1,554 a month
154th of 374, with limits
$288 a month
2nd cheapest of 374
$86 a month
24/7 dev boxes
1 machine 24/7 · 4 vCPU / 8 GB
Runloop can run 24/7 with an idle policy, reached through HTTPS tunnels and SSH. Fly keeps one Volume per Machine, and the root file system only with opt-in persist_rootfs. On boat, turn auto-stop off (default 1 hour) on a paid plan.
118th cheapest of 371
$410 a month
160th of 371, with limits
$35 a month
19th cheapest of 371
$26 a month
Computer-use agents
50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month
Runloop and boat ship a desktop and a browser that an agent can control; boat streams at 60 fps. On Fly you set it up: add a desktop, a browser and a VNC stream to your own image.
41st cheapest of 367
$2,306 a month
140th of 367, with limits
$173 a month
7th cheapest of 367
$200 a month
Browser-use agents
50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month
38th cheapest of 356
$1,153 a month
229th of 356, with limits
$90 a month
7th cheapest of 356
$200 a month
Scraping & bots (startup scale)
50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month
73rd cheapest of 343
$781 a month
233rd of 343, with limits
$67 a month
6th cheapest of 343
$90 a month
Mass scraping & bots (undetectable)
1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month
None of the three is a full match. Fly ranks highest of the three: it runs in 17 regions and bills per second. None ships anti-bot stealth, CAPTCHA solving or residential IPs.
267th of 327, with limits
$15,611 a month
47th of 327, with limits
$1,232 a month
73rd of 327, with limits
$1,800 a month
Parallel red teaming
200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month
16th cheapest of 351
$770 a month
168th of 351, with limits
$63 a month
3rd cheapest of 351
$200 a month
GPU eval jobs
10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month
Not ranked: misses a requirementNot ranked: misses a requirementNot ranked: misses a requirement
macOS agents
5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month
Not ranked: misses a requirementNot ranked: misses a requirementNot ranked: misses a requirement

Key features

FeatureRunloopFly.io Machinesboat
Kind of productAgent sandbox: devboxes for building, running and testing coding agentsApp hosting platform; you boot your own image as a microVM. Fly sells Sprites separately for agent sandboxesAgent sandbox: a full VM per agent, ready to use
IsolationMicro-VM on a custom bare-metal hypervisor, plus a container layerFirecracker microVM with its own kernelFull VM with its own kernel, on physical hosts dedicated to boat (AMD Ryzen 7 9700X and Ryzen 9 9950X); vCPUs are shared
Docker, systemd and nested virtualizationDocker on the Docker-in-Docker blueprints, as root; nested virtualization not documentedRoot and Docker you install yourself; Fly's own init by default; /dev/kvm not documentedDocker, root, real systemd and /dev/kvm on every sandbox
ImagesBlueprints built from Dockerfiles, x86_64 and arm64Your own Docker imageboat's image or a saved snapshot, x86_64 only; you cannot boot your own Docker or OCI image
Files after a stopDisk snapshots on every plan; idle suspend keeps the disk on Pro; shutdown deletes the diskRoot file system resets by default; opt-in persist_rootfs keeps it, but Fly staff say not to trust it with critical data. Lasting data goes on one Volume, billed for its full sizeDisk saved every minute and at each stop, kept at least 14 days; memory and processes are not saved; a machine failure loses writes after the last good snapshot
Long runsKeep-alive timer 1 hour by default, 48 hours max; an idle policy replaces it, with no documented capNo documented maximum runtimeAuto-stop 1 hour by default, up to 30 days, or off on a paid plan (then no session cap)
Sleep and wakeSuspend after idle, wake on an HTTP request to the tunnel (Pro)Fly Proxy stops or suspends idle Machines and wakes them on a request; memory kept for Machines up to 2GBNo idle timer; auto-stop counts from creation or resume; resume reboots from disk
Snapshots and forkDisk snapshots you can start new devboxes fromClone copies a Machine's config; restore a Volume snapshot or fork the Volume; the root file system is not copiedAutomatic disk snapshots; fork a running sandbox from its last 1-minute snapshot
Desktop and computer useDesktop and browser that an agent can controlNot built in; you set it up: add a desktop, a browser and a VNC stream to your own image1920x1080, 60 fps desktop with Chrome and computer tools; embeddable stream
Coding agentsClaude Code, Codex, OpenCode, Gemini CLI and DeepAgents through public agent wrappersNot built in on Machines; you install agents in your image. Fly's separate Sprites product ships Claude Code, Codex and GeminiSeven agents installed, with a prompt, steer and events API
Keeping keys out of the sandboxAgent gateways for any API endpoint with bearer, header or basic authSecrets arrive as environment variables; you set up key injection by deploying Fly's open-source tokenizer proxyOptional AI credentials proxy, off by default, $1 a day plus $1 per 1,000 requests; keys of supported model providers stay outside; other secrets still enter the sandbox
Reaching the sandboxOne HTTPS tunnel per devbox, open or bearer token; SSH through a TLS proxy; no public IP documentedPublic HTTPS and TCP services through Fly Proxy, Anycast IPs, custom domains, private network between MachinesUp to 50 HTTPS URLs, token-gated by default; SSH; public IPv6 address with raw ports (some sandboxes have no IPv4 of their own)
Outbound rulesHostname allowlist with wildcards, or block allNetwork policies allow chosen ports and protocolsOpen internet; no provider-side allowlist. You set it up: firewall rules inside the VM, as root
Where it runsRegions not documented; Deploy to VPC on AWS, GCP or Azure (Enterprise)17 regions on six continents, on Fly's own serversGermany, Finland and France; you cannot pick a region; no US or Asia region
Every feature Battleships tracks (43)
FeatureRunloopFly.io Machinesboat
Full VM (own kernel)YesYesYes
Docker insideYesYesYes
Nested virtualizationNo documented wayNot publishedYes
RootYesYesYes
SystemdNot publishedNot publishedYes
Desktop GUIYesNo documented wayYes
BrowserYesNo documented wayYes
Computer-use APIYesNo documented wayYes
SSHYesYesYes
Persistent diskYesYesYes
Pause/resumeYesYesYes
Automatic snapshotsYesYesYes
Fork/cloneYesYesYes
Live fork (no pause)YesYesYes
Memory forkNo documented wayNo documented wayNo documented way
Custom image (Docker or snapshot)YesYesYes
≥24 h sessionsYesYesYes
Idle auto-stopYesYesNo documented way
Wake on requestYesYesNo documented way
HTTPS preview URLsYesYesYes
Raw TCP inboundNo documented wayYesYes
Public IPv4No documented wayYesNo documented way
Egress allowlistYesYesNo documented way
Secret proxyYesNo documented wayYes
Secret proxy for any APIYesNo documented wayYes
Preinstalled agentsYesNo documented wayYes
Agent harness APIYesNo documented wayYes
MCP serverYesYesNo documented way
GPUNo documented wayNo documented wayNo documented way
WindowsNo documented wayNo documented wayNo documented way
Arm64YesNo documented wayNo documented way
Self-hostingNo documented wayNo documented wayNo documented way
BYOCYesNo documented wayNo documented way
Open sourceNo documented wayYesNo documented way
Scoped API keysYesNot publishedYes
Spend limitsNot publishedNo documented wayYes
Usage per sandbox (API)YesNo documented wayYes
Audit logsNot publishedNot publishedNo documented way
SSOYesNo documented wayNo documented way
SOC 2YesYesNo documented way
HIPAANot publishedYesNo documented way
EU data residencyNo documented wayYesNo documented way
Zero data retentionNot publishedNot publishedYes

Performance

Runloop8.93 runs/s
Fly.io MachinesNot measured
boat28.06 runs/s

In StarSling's run of 3 October 2026 (run 37073680030), on the V8 Web Tooling Benchmark at 4 vCPU / 8 GB, boat did 28.06 runs a second (second of 15) and Runloop 8.93 (15th of 15), about 3.1 times apart. Fly.io Machines are not in the public benchmarks we use, so we cannot say how Fly compares. Fly results would also depend on the CPU class: shared vCPUs guarantee 6.25% of a core, and performance vCPUs get the full core. Every boat sandbox in this run ran on an AMD Ryzen 9 9950X host. boat also places some sandboxes on AMD Ryzen 7 9700X hosts, which this run did not measure. Providers ran on different hardware in different regions.

V8 Web Tooling Benchmark at a 4 vCPU / 8 GB target, runs per second, higher is better. Source: StarSling's run of 3 October 2026 (run 37073680030), median of 5 sandboxes. Providers ran on different hardware in different regions. StarSling gives a shared rank when the run cannot separate two providers. Every boat sandbox in this run ran on an AMD Ryzen 9 9950X host. boat also places some sandboxes on AMD Ryzen 7 9700X hosts, which this run did not measure. Fly.io Machines is not in the public benchmarks we use, so we cannot say which is faster.

Pricing in five scenarios

Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario. Start limits, per-start fees and plan minimums are in the price, boat's included.

ScenarioRunloop, a monthFly.io Machines, a monthboat, a month
One coding agent, workdays
4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days
$113 · Basic$15 · Pay As You Go · with limits$20 · $20/mo
One agent that never stops
4 vCPU / 8 GB, 50 GB disk, running 24/7 all month
$429 · Basic$36 · Pay As You Go · with limits$26 · $20/mo
50 agents in parallel
50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside
$5,268 · Basic$487 · Pay As You Go · with limits$317 · $20/mo × 3 seats
Sandboxes for your users
2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once
$1,441 · Basic$110 · Pay As You Go · with limits$200 · $100/mo × 2 seats
1,000 short jobs at once
2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month
$3,184 · Basic$232 · Pay As You Go · with limits$8,800 · $2000/mo · with limits

Runloop: Runloop bills per second for allocated CPU ($0.108 per vCPU-hour) and memory ($0.0252 per GB-hour) while a devbox is initializing, running, suspending or resuming: $0.63 an hour at 4 vCPU / 8 GB. Its disk bills $0.2499 per GB-month for the devbox's whole life. Basic has no plan fee. Pro adds $250 a month that is not usage credit, and unlocks suspend and resume. New accounts get $50 of credit with no card.

Fly.io Machines: Fly bills each Machine per second while it runs, by CPU preset and memory, with no plan fee. Stopped Machines pay for their root file system, and Volumes are billed for their full size at $0.15 per GB a month. The table uses Fly's lowest rate: shared vCPUs in 1-year reservation blocks for 40% off. A performance-4x Machine with 8GB, which gets the full cores, is $0.1833 an hour on demand.

Limits in those scenarios

Runloop fits all five scenarios, but publishes no egress price, caps disk at 64 GiB, and needs an idle policy to run past 48 hours; suspend on idle needs Pro. The table marks Fly as a compromise in all five: its lowest figures need a 1-year reservation and use shared vCPUs that guarantee 6.25% of each core, and its Machines API takes about one create request per second per app. boat's limits: plans start at $20 a month, and the included time expires at the end of the month. A stopped sandbox has no compute charge, but saved snapshots above the included count and the AI credentials proxy still charge. For 1,000 short jobs at once, even the $2,000 plan charges $0.03 to $0.07 for starts above 90 a minute and refuses them above 270, so the burst does not fit boat.

ScenarioRunloopFly.io Machinesboat
One coding agent, workdays
  • egress price not published
  • different kind of product: app platform
  • files survive a stop only in one Fly Volume (root file system too with opt-in persist_rootfs): packages installed elsewhere reset
  • 1-year commitment
  • burstable CPU: 6.25% of each vCPU guaranteed, throttled above that
None for this scenario
One agent that never stops
  • egress price not published
  • different kind of product: app platform
  • files survive a stop only in one Fly Volume (root file system too with opt-in persist_rootfs): packages installed elsewhere reset
  • 1-year commitment
  • burstable CPU: 6.25% of each vCPU guaranteed, throttled above that
None for this scenario
50 agents in parallel
  • egress price not published
  • different kind of product: app platform
  • files survive a stop only in one Fly Volume (root file system too with opt-in persist_rootfs): packages installed elsewhere reset
  • 1-year commitment
  • burstable CPU: 6.25% of each vCPU guaranteed, throttled above that
None for this scenario
Sandboxes for your users
  • egress price not published
  • different kind of product: app platform
  • 1-year commitment
  • burstable CPU: 6.25% of each vCPU guaranteed, throttled above that
None for this scenario
1,000 short jobs at once
  • egress price not published
  • different kind of product: app platform
  • 1-year commitment
  • burstable CPU: 6.25% of each vCPU guaranteed, throttled above that
  • 2,400 starts/day included, paid above that (needs ~6,667)
  • 600 starts/hour included, paid above that (needs ~1,000)
  • starts above 90/min are paid, and starts above 270/min fail with 429

Which one should you use?

Use Runloop when

you want a ready agent sandbox with hostname egress rules, arm64 or Dockerfile images, deploys into your own cloud, or very large bursts of short jobs.

  • Made for coding agents. For agent tooling with no setup: Runloop ships agent wrappers, Axons event streams and agent gateways that keep API keys out of the devbox.
  • Hostname egress rules. For a sandbox that must reach only a fixed list of hosts: Runloop allows listed hostnames, with wildcards, or blocks everything.
  • Your own cloud and arm64. On Enterprise, Runloop runs devboxes in your own AWS, GCP or Azure account. It builds x86_64 or arm64 images from Dockerfiles.

Use Fly.io Machines when

you host web apps or light services near users worldwide that sleep between requests and wake on traffic, and you are happy to build the sandbox image yourself.

  • Lower bills in these scenarios. On a 1-year shared-CPU reservation, Fly costs less than Runloop in all five scenarios on this page. Its shared CPU guarantees 6.25% of each core.
  • 17 regions and wake on request. For services near users that sleep between requests: Fly runs in 17 regions, wakes a Machine on traffic, and resumes Machines up to 2GB with memory kept.
  • A full app platform. For hosting whole apps next to your sandboxes: Fly adds public TCP services, private networking, Anycast IPs and custom domains.

Where boat fits

Look at boat if your agents spend hours installing, building and testing on one machine. Each sandbox is a full VM with Docker, systemd and /dev/kvm, and boat saves its disk every minute and at each stop with no code from you.

The disk comes back with no setup

Runloop deletes the disk at shutdown, so you keep work with snapshot calls or Pro suspend. Fly keeps one Volume, and the root file system only with opt-in persist_rootfs. boat saves the disk every minute and at each stop and keeps it at least 14 days. Memory and running programs are not saved.

A fuller Linux machine

boat has root, real systemd, Docker and /dev/kvm in every sandbox, so Android emulators and nested VMs start. Neither Runloop nor Fly documents /dev/kvm. boat cannot boot your own image and runs x86_64 only.

More CPU work per hour, at a lower list price

In StarSling's run of 3 October 2026, boat did 28.06 runs a second and Runloop 8.93 on the V8 Web Tooling Benchmark; Fly was not measured. boat lists $0.036 an hour at 4 vCPU / 8 GB, from a $20 plan. Every boat sandbox in that run ran on an AMD Ryzen 9 9950X host; boat also uses other hosts.

Use boat when your agents do real work for hours: installs, builds, tests, Docker and emulators on a full VM whose disk is saved every minute and at each stop. boat runs only in Germany, Finland and France and has no GPU. Compare it one to one: boat vs Runloop and boat vs Fly.io Machines.

boat's own limits
  • Plans start at $20 a month. The plan price comes back as VM time for that month. Unused time does not carry over. Source
  • Starts (create, fork, resume): the $20 plan includes 12 a minute, 60 an hour and 200 a day. The $2,000 plan includes 90 a minute, 600 an hour and 2,400 a day. Extra starts cost $0.03 to $0.07. A start fails with 429 above 3 times the included starts per minute, or 6 times per hour or per day. Source
  • A stopped sandbox has no compute charge. boat keeps its disk for at least 14 days. A saved snapshot keeps it longer; extra saved snapshots and the AI credentials proxy are paid. Source
  • Auto-stop defaults to 1 hour. You can turn it off on a paid plan. The trial allows 2 hours at most. Source
  • boat saves the disk every minute and at each stop. Memory, running programs and /tmp are not saved: a resume is like a reboot. A machine failure loses writes after the last good snapshot. Source
  • Machines run in Germany, Finland and France only. Snapshot copies can be stored outside the EU. Source
  • Public IPv6 and outbound IPv4. No dedicated IPv4. The IP changes at each resume. Source
  • Hosts are AMD Ryzen 7 9700X and Ryzen 9 9950X. vCPUs are shared. No GPU. You cannot boot your own Docker image. Source

Questions people ask

Is Runloop cheaper than Fly.io Machines?

No. At Fly's lowest rate, a 1-year shared-CPU reservation, Fly costs less in all five scenarios on this page. One coding agent on workdays is $113 a month on Runloop and $15 on Fly. 50 agents in parallel are $5,268 and $487. Those Fly shared vCPUs guarantee 6.25% of each core. boat is $20 and $317 for the same two.

Which is faster, Runloop or Fly.io Machines?

We cannot say: Fly is not in the public benchmarks we use. In StarSling's run of 3 October 2026, Runloop did 8.93 runs a second on the V8 Web Tooling Benchmark and boat 28.06. On Fly, speed also depends on the CPU class you buy.

Can Runloop and Fly.io Machines run Docker?

Yes. Runloop has Docker-in-Docker blueprints, and on Fly you install Docker in your own image. Neither documents /dev/kvm for nested VMs or Android emulators. boat has Docker, systemd and /dev/kvm on every sandbox.

Do they keep my files when a sandbox stops?

Runloop keeps the disk through idle suspend on Pro or in disk snapshots on every plan; shutdown deletes it. Fly keeps one Volume, and the root file system with opt-in persist_rootfs, which Fly staff say not to trust with critical data. boat saves the disk every minute and at each stop and keeps it at least 14 days; memory and running programs are not saved.

Can I use Fly.io Machines as an AI agent sandbox?

Yes, but you build the sandbox yourself: the image with tools, browser and agents, a Volume for files, and the code that drives each agent. Fly's own agent sandbox is Sprites, sold separately. Runloop and boat ship agents and a desktop.

Where do they run?

Fly runs Machines in 17 regions on six continents. Runloop does not document its regions, but Enterprise can deploy into your own AWS, GCP or Azure account. boat runs in Germany, Finland and France, with no US or Asia region.

Try boat

Install the CLI, then run boat new.

7-day free trial

More comparisons