RunsOn vs Ubicloud
RunsOn is a GitHub Actions runner service you deploy in your own AWS account, where each job gets a fresh EC2 machine that is gone when the job ends. Ubicloud is an open source cloud that sells managed runners too, but also plain Linux VMs that stay up until you delete them.
The companies and products
RunsOn
RunsOn is self-hosted GitHub Actions runners on AWS: one CloudFormation or Terraform stack in your account, a yearly licence, and compute billed by AWS at cost.
You install RunsOn in your own AWS account, change the runs-on: label in your workflows, and every job then starts on a new EC2 instance that is terminated when the job ends. It covers Linux on x64 and arm64, Windows Server and GPU instances, with caches in S3 and sticky EBS disks. It is a CI runner, a different kind of product from boat, so this page compares the part where they overlap: a machine where an agent or a job does its work.
Ubicloud
Ubicloud is an open source cloud that runs VMs, managed Postgres, Kubernetes and GitHub Actions runners on leased bare-metal servers.
You rent Linux VMs (Standard with dedicated CPU, or cheaper Burstable with shared CPU) from Ubicloud's managed service, or run the same AGPL code on your own servers. Around the VMs it offers private subnets, an inbound firewall, load balancers, managed PostgreSQL, Kubernetes, GPU VMs, inference endpoints and cheap GitHub Actions runners. It is general infrastructure, not an agent sandbox, so this page compares the two where they overlap: giving an agent its own Linux machine.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “58th cheapest of 377” means 57 of the other 376 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | RunsOn: price rank | Ubicloud: price rank | boat: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month | 122nd of 367, with limits $1,114 a month | 126th of 367, with limits $1,136 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB An assistant that never stops is a compromise on both: GitHub ends RunsOn jobs after 5 days, and on a Ubicloud VM you build the agent setup, HTTPS link and backups yourself. | 188th of 366, with limits $1,724 a month | 154th of 366, with limits $1,139 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month Ubicloud ranks 59th against RunsOn's 229th for short bursts, because its burstable VMs bill per minute with no plan fee and start without a GitHub job; boat's start limits make big bursts a compromise. | 229th of 345, with limits $147 a month | 59th of 345, with limits $52 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Ubicloud can start a VM per user from its API, though 300 sandboxes of 2 vCPU need a quota raise past 256 vCPU; RunsOn only starts machines for your own workflows, and boat ranks 1st. | 216th of 361, with limits $585 a month | 53rd of 361, with limits $390 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month | 245th of 360, with limits $2,714 a month | 234th of 360, with limits $1,560 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month Ubicloud is a full fit for agent evals and ranks 31st, against 251st for RunsOn, whose machines only exist inside a GitHub Actions job. | 251st of 364, with limits $389 a month | 31st cheapest of 364 $624 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month Both are built for CI: RunsOn ranks 58th and Ubicloud ranks 28th of 377 here, and boat ranks 4th by running jobs on full VMs with Docker and /dev/kvm. | 58th cheapest of 377 $432 a month | 28th cheapest of 377 $300 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB A Ubicloud VM is a full fit for a 24/7 dev box because it stays up and keeps its disk, while a RunsOn runner is gone when its job ends. | 158th of 368, with limits $86 a month | 67th cheapest of 368 $91 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Both are headless: RunsOn runners are reached over SSM or SSH and Ubicloud VMs start as a plain OS image, while boat streams a 60fps desktop with Chromium. | 142nd of 369, with limits $353 a month | 132nd of 369, with limits $312 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 206th of 367, with limits $224 a month | 188th of 367, with limits $156 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 228th of 346, with limits $205 a month | 165th of 346, with limits $78 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 122nd of 338, with limits $3,539 a month | 60th of 338, with limits $1,573 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 178th of 360, with limits $157 a month | 129th of 360, with limits $81 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month Only Ubicloud is ranked here, since it rents Nvidia GPU VMs on request; RunsOn launches GPU instances for CI jobs only, and boat has no GPUs. | Can't run it | 93rd of 98, with limits $795 a month | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month | Can't run it | Can't run it | Can't run it |
Key features
| Feature | RunsOn | Ubicloud | boat |
|---|---|---|---|
| What it is | Self-hosted GitHub Actions runners: one EC2 machine per CI job | An open source cloud of VMs, managed Postgres, Kubernetes and GitHub Actions runners | A sandbox API for agents: full VMs you create, stop, fork and delete from code |
| Where it runs | In your own AWS account, in the region you pick | Managed in Germany, Finland and Virginia (US), GPUs also in Istanbul, or on your own servers | Managed by boat in the EU (Germany, Finland, France) |
| GitHub Actions runners | The whole product: change the runs-on: label, with S3 caches, sticky disks and warm pools | Managed x64 and arm64 runners billed per minute of job time, with a $2.50 monthly credit | Can run a runner on a full VM, but no managed runner service |
| Machines outside CI | No; a machine starts only when a GitHub Actions job asks for one | Yes; Linux VMs from an API, a CLI or a web console | Yes; any time, by API, CLI or SDK |
| How long it can run | One job; GitHub ends self-hosted jobs after 5 days | A VM runs until you stop or delete it | No session cap on paid plans; you set the time to live |
| Stopping and files | The root disk is thrown away; one sticky EBS disk per job is restored next job | A stopped VM keeps its disk, but billing continues as if it were running | Whole disk saved every minute and on stop; a stopped sandbox costs nothing |
| Snapshots and fork | No fork; every job starts from an AMI | Machine image of a stopped VM's single disk, up to 40 GiB, made by hand | Automatic snapshots every minute, and fork of a running sandbox in one call |
| Docker and nested virtualization | Docker and root; KVM on compatible Intel x64 families, not arm64 | Root over SSH, so Docker installs as on any Linux VM; nested virtualization not documented | Docker, root, systemd and /dev/kvm on every machine |
| Inbound access | SSH from a CIDR range you allow, and SSM; no HTTPS URLs | Free public IPv6, public IPv4 for $3 a month; HTTPS means running TLS on the VM yourself | HTTPS URLs for up to 50 ports, token-protected; raw TCP; dedicated IPv4 or IPv6 |
| Network rules | Your own AWS Network Firewall policy or security groups; static egress IPs through NAT | Stateful inbound firewall by IP range and port; outbound rules not supported | Open internet, no provider-side allowlist |
| Desktop and computer use | None; headless runners | None; you install a desktop and browser on the VM yourself | 60fps Linux desktop and Chromium, with computer use and browser control |
| Coding agents and keys | Bedrock through the instance role; Copilot's coding agent can use a runner | None; you install agents yourself, and secrets live on the VM | Claude Code, Codex and more preinstalled with a prompt, steer and events API; credentials proxy keeps real keys out |
| Platforms | Any EC2 size up to 96 vCPU: Linux x64 and arm64, Windows Server, NVIDIA or AMD GPUs | Linux x64 VMs, arm64 CI runners, Nvidia GPU VMs on request | Linux x64, up to 16 vCPU / 32 GB on the $100 plan and above; no GPUs |
| Source code | Not open source; full source for security review on the top licence | Open source under the AGPL v3; you can run it yourself on leased servers | Managed service only |
Every feature Battleships tracks (43)
| Feature | RunsOn | Ubicloud | boat |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | Yes | Not published | Yes |
| Root | Yes | Yes | Yes |
| Systemd | Not published | Not published | Yes |
| Desktop GUI | No | No | Yes |
| Browser | No | No | Yes |
| Computer-use API | No | No | Yes |
| SSH | Yes | Yes | Yes |
| Persistent disk | Yes | Yes | Yes |
| Pause/resume | Yes | No | Yes |
| Automatic snapshots | Yes | No | Yes |
| Fork/clone | No | Yes | Yes |
| Live fork (no pause) | No | No | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | Yes | Yes | Yes |
| Idle auto-stop | Yes | No | No |
| Wake on request | Yes | No | No |
| HTTPS preview URLs | No | No | Yes |
| Raw TCP inbound | No | Not published | Yes |
| Public IPv4 | Yes | Yes | No |
| Egress allowlist | Yes | No | No |
| Secret proxy | No | No | Yes |
| Secret proxy for any API | No | No | Yes |
| Preinstalled agents | No | No | Yes |
| Agent harness API | No | No | Yes |
| MCP server | No | No | No |
| GPU | Yes | Yes | No |
| Windows | Yes | No | No |
| Arm64 | Yes | Yes | No |
| Self-hosting | Yes | Yes | No |
| BYOC | Yes | Yes | No |
| Open source | No | Yes | No |
| Scoped API keys | Not published | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | No | No | Yes |
| Audit logs | Not published | Yes | No |
| SSO | No | Yes | No |
| SOC 2 | No | Yes | No |
| HIPAA | No | No | No |
| EU data residency | Yes | Yes | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
There is no comparable measurement for either: neither RunsOn nor Ubicloud is in the StarSling hpc-sandbox-benchmarks run where boat did 26.69 runs per second at 4 vCPU / 8 GB. RunsOn's speed depends on the EC2 family you pick, and Ubicloud's on whether you choose Standard VMs with dedicated CPU or Burstable VMs that get half of each vCPU.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. RunsOn and Ubicloud have no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | RunsOn, a month | Ubicloud, a month | boat, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $61 · Starter · with limits | $22 · Usage | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $159 · Starter · with limits | $91 · Usage | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $1,597 · Starter · with limits | $1,098 · Usage | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $515 · Starter · with limits | $156 · Usage | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $1,103 · Starter · with limits | $312 · Usage · with limits | $2,000 · $2000/mo · with limits |
RunsOn: AWS bills the EC2 instances and EBS disks on your own AWS bill, to the second, at spot or on-demand prices with no markup, and S3 cache traffic and data transfer are extra. RunsOn charges one licence a year per company, from EUR 300 or $350 for under 50,000 jobs a month, and is free for non-commercial use.
Ubicloud: Ubicloud bills each VM per minute for as long as it exists, running or stopped, and a public IPv4 adds $3 a month. Its GitHub Actions runners bill per minute of job time, with a $2.50 monthly credit for runner time only.
Ubicloud: You pick a monthly plan from $20, and all of it becomes usage credit spent per second on running sandboxes. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
None of the five scenarios is a clean fit for RunsOn: it starts machines only for GitHub Actions jobs, GitHub ends them after 5 days, and only the sticky disk survives between jobs. Ubicloud fits four of them, but its project quota is 256 vCPU once verified, so 300 user sandboxes and 1,000 short jobs at once need a raise from support, and the cheap burstable shape guarantees only half of each vCPU. On boat the 1,000-job burst gets queued and needs more starts per day than the $2,000 plan allows.
| Scenario | RunsOn | Ubicloud | boat |
|---|---|---|---|
| One coding agent, workdays |
| None that bite | None that bite |
| One agent that never stops |
| None that bite | None that bite |
| 50 agents in parallel |
| None that bite | None that bite |
| Sandboxes for your users |
| None that bite | None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use RunsOn when
you want faster, cheaper GitHub Actions runners inside your own AWS account, including Windows, arm64 or GPU jobs, with compute billed at AWS cost.
- Everything stays in your AWS account. Useful when code, secrets and logs must never leave your own AWS account, in whatever region you already use.
- Windows, arm64 and GPUs at any EC2 size. Useful for jobs that need Windows Server, native arm64 builds or NVIDIA GPUs, on machines up to 96 vCPU.
- Deep GitHub Actions tooling at AWS cost. Useful if you want a drop-in runner: change one label and get S3 caches, sticky disks and warm pools on EC2 at cost.
Use Ubicloud when
you want managed runners without an AWS stack, plain VMs that stay up next to Postgres and Kubernetes, GPU VMs, or an open source cloud you can run yourself.
- Open source you can run yourself. Useful if you want to run the cloud on your own servers: Ubicloud's whole stack is AGPL code.
- VMs that outlive a job. Useful when a machine has to stay up for days or months, next to managed Postgres and Kubernetes, not just for one CI job.
- Managed runners with no AWS account. Useful if you want cheaper x64 and arm64 GitHub Actions runners billed per minute, without running a stack yourself.
Where boat fits
If what you are really running is agents rather than CI jobs, neither RunsOn nor Ubicloud gives you an agent sandbox: one hands you a machine for one job, the other a plain VM you set up yourself. boat gives each agent a full VM with its own kernel, Docker and /dev/kvm, with coding agents, a desktop, HTTPS ports and free stops built in.
A machine that keeps its work
A RunsOn runner is terminated after its job, at most 5 days, with only the sticky disk kept. A stopped Ubicloud VM bills as if running, and copying it means stopping it first. boat saves the disk every minute, stops for free and forks while running.
Coding agents driven from outside
On RunsOn any agent besides Copilot's is something you install and watch from workflow steps, and a Ubicloud VM starts as an empty OS image. On both, keys sit on the machine. boat drives agents inside from outside and keeps keys in a proxy.
A desktop and links you can share
Both are headless: RunsOn is reached over SSM or SSH, and on Ubicloud HTTPS means running TLS on the VM yourself. boat streams a 60fps desktop with Chromium and serves up to 50 ports on token-protected HTTPS URLs, plus raw TCP.
Use boat when the machines are for agents rather than CI jobs: a full VM you start by API, keep with its whole disk, fork while running, reach over HTTPS and drive with built-in coding agents. Compare it one to one: boat vs RunsOn and boat vs Ubicloud.
Questions people ask
Is RunsOn cheaper than Ubicloud?
For CI runners in our model, Ubicloud comes to $300 a month against $432 on RunsOn, and boat to $86. For agent work Ubicloud is cheaper too: one agent that never stops costs $91 a month on Ubicloud against $159 on RunsOn, though RunsOn does not fit that scenario cleanly.
Which is faster, RunsOn or Ubicloud?
There is no comparable measurement, since neither is in the benchmark run where boat did 26.69 runs per second. RunsOn's speed depends on the EC2 family you pick, and Ubicloud's on Standard or Burstable VMs.
Can RunsOn and Ubicloud run Docker?
Yes. RunsOn runners have Docker and root, with KVM on compatible Intel x64 families. Ubicloud runners use images that ship Docker, and on a Ubicloud VM you have root, so Docker installs as on any Linux machine. boat has Docker, systemd and /dev/kvm on every machine.
Can I use RunsOn or Ubicloud outside GitHub Actions?
RunsOn only starts machines for GitHub Actions jobs. Ubicloud also rents plain Linux VMs from its API, CLI or console, which stay up until you delete them and keep billing while stopped.
Can either one be a sandbox for AI agents?
Only with work on your side. A RunsOn runner lives for one job, up to 5 days, and a Ubicloud VM is a plain OS image where you install and drive agents over SSH. boat has coding agents, a desktop and a credentials proxy in every sandbox.
Where does my data run?
RunsOn runs in your own AWS account and region, and your code, secrets and logs are never sent to RunsOn. Ubicloud runs in Germany, Finland and Virginia, with GPUs also in Istanbul, or on your own servers. boat runs only in the EU, in Germany, Finland and France.
Try boat
Install the CLI, then run boat new.
7-day free trial