GitHub Actions vs RunsOn

Both run GitHub Actions jobs, each on a fresh machine that is gone when the job ends. GitHub Actions hosts that machine for you on Azure or GitHub's Mac cloud, while RunsOn starts it as an EC2 instance in your own AWS account, billed by AWS at cost plus a yearly licence.

The companies and products

GitHub Actions

GitHub Actions is GitHub's CI/CD service: workflows in your repository run as jobs on runners that GitHub hosts or that you host yourself.

GitHub Actions is a CI service, not a sandbox API. Each job on a GitHub-hosted runner gets a new VM, Linux and Windows on Azure and macOS on GitHub's own Mac fleet, with passwordless sudo, and the VM is gone when the job ends. It is built for building, testing and deploying the code in a repository, and GitHub's own Copilot cloud agent and the Claude and Codex agents on GitHub also run on Actions runners.

Kind of product: CI/CD service; jobs are defined in workflow files in a GitHub repository
Isolation: A new VM for each job; Linux and Windows on Azure, macOS on GitHub's own Mac cloud
Job length: Each job can run for up to 6 hours
Concurrency: 20 jobs on Free, 40 on Pro, 60 on Team, 500 on Enterprise; macOS 5, or 50 on Enterprise
Allowed use: GitHub-hosted runners are for work on the repository's software project; reselling Actions as a service is not allowed
Self-hosted runners: Open-source runner under MIT; usage on self-hosted runners is free
Compliance: SOC 1 and SOC 2 Type 2 reports, ISO/IEC 27001:2022

RunsOn

RunsOn is self-hosted GitHub Actions runners on AWS: one CloudFormation or Terraform stack in your account, a yearly licence, and compute billed by AWS at cost.

You install RunsOn in your own AWS account, change the runs-on: label in your workflows, and every job then starts on a new EC2 instance that is terminated when the job ends. It covers Linux on x64 and arm64, Windows Server and GPU instances, with caches in S3 and sticky EBS disks. It is a CI runner, a different kind of product from boat, so this page compares the part where they overlap: a machine where an agent or a job does its work.

Product: Self-hosted GitHub Actions runners, deployed as one stack in your own AWS account
Isolation: Every job runs on an EC2 instance no other job has used, terminated when the job ends
Data: Code, secrets and logs stay in your AWS account; RunsOn gets a licence check and one event per job
Platforms: Linux x64 and arm64, Windows Server 2022 and 2025, NVIDIA and AMD GPUs; macOS not supported
Licence: From EUR 300 or $350 a year per company, sized by jobs per month; free for non-commercial use
Company: Bootstrapped and self-funded, built by Cyril Rohr; says 900+ companies and 3M+ jobs a day

Which use cases each one fits

For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “100th cheapest of 377” means 99 of the other 376 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.

Use case and its workload	GitHub Actions: price rank	RunsOn: price rank	boat: price rank
Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month A software factory needs agents that keep their checkout and Docker cache; GitHub Actions wipes the VM each job, RunsOn keeps only the sticky disk, and boat keeps the whole disk.	298th of 367, with limits $5,853 a month	122nd of 367, with limits $1,114 a month	3rd cheapest of 367 $634 a month
Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB An assistant that never stops does not fit either, since hosted jobs end after 6 hours and RunsOn jobs after GitHub's 5-day self-hosted limit.	333rd of 366, with limits $10,692 a month	188th of 366, with limits $1,724 a month	5th cheapest of 366 $657 a month
Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month RunsOn costs less than GitHub Actions for short bursts here because AWS bills spot minutes at cost, though each burst still has to arrive as a GitHub Actions job.	294th of 345, with limits $721 a month	229th of 345, with limits $147 a month	186th of 345, with limits $2,000 a month
Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Neither is built to give your own users sandboxes: GitHub's terms rule out offering Actions as a service, and RunsOn only starts machines for your own workflows.	308th of 361, with limits $2,892 a month	216th of 361, with limits $585 a month	Cheapest of 361 $209 a month
RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month	304th of 360, with limits $14,721 a month	245th of 360, with limits $2,714 a month	34th of 360, with limits $2,000 a month
Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month	303rd of 364, with limits $1,962 a month	251st of 364, with limits $389 a month	10th cheapest of 364 $200 a month
CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month Both fit: GitHub Actions needs nothing to set up, RunsOn runs the same jobs on EC2 at AWS cost, and boat runs CI on full VMs that keep a warm disk between runs.	100th cheapest of 377 $1,012 a month	58th cheapest of 377 $432 a month	4th cheapest of 377 $86 a month
24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB A dev box needs to stay up with its files and be reachable; hosted runners take no inbound traffic, RunsOn offers SSH or SSM for the life of a job, and a boat machine stays as long as you want.	279th of 368, with limits $159 a month	158th of 368, with limits $86 a month	18th cheapest of 368 $26 a month
Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Both run headless runners, so a computer-use agent needs a desktop you set up yourself, while boat streams a 60fps Linux desktop with Chromium.	292nd of 369, with limits $1,768 a month	142nd of 369, with limits $353 a month	6th cheapest of 369 $200 a month
Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month	135th of 367, with limits $942 a month	206th of 367, with limits $224 a month	7th cheapest of 367 $200 a month
Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month	134th of 346, with limits $1,184 a month	228th of 346, with limits $205 a month	6th cheapest of 346 $90 a month
Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month	301st of 338, with limits $28,971 a month	122nd of 338, with limits $3,539 a month	71st of 338, with limits $1,800 a month
Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month	70th of 360, with limits $467 a month	178th of 360, with limits $157 a month	2nd cheapest of 360 $200 a month
GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month	Can't run it	Can't run it	Can't run it
macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month GitHub Actions has hosted macOS runners on Apple silicon; RunsOn does not support macOS and boat runs Linux only.	25th of 36, with limits $1,178 a month	Can't run it	Can't run it

Key features

Feature	GitHub Actions	RunsOn	boat
What it is	GitHub's CI/CD service: jobs from workflow files run on runners GitHub hosts	Self-hosted GitHub Actions runners, deployed as one stack in your own AWS account	A sandbox API for agents: full VMs you create, stop, fork and delete from code
Where machines run	GitHub's Azure fleet for Linux and Windows, GitHub's own Mac cloud for macOS	EC2 in your AWS account, in the region you pick; code, secrets and logs stay there	Managed by boat in the EU (Germany, Finland, France)
Isolation	A new VM for each job, with passwordless sudo	A fresh EC2 instance per job, terminated when the job ends	Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts, with root and systemd
How long it can run	Up to 6 hours per job	One job; GitHub ends self-hosted jobs after 5 days	No session cap on paid plans; you set the time to live
Files between runs	Every job starts on a fresh VM; files carry over only through cache, artifacts or custom images	Root disk is thrown away; one sticky EBS disk per job is snapshotted and restored next job	Whole disk saved every minute and on stop; a stopped sandbox costs nothing
Fork	No; custom images are a template for new runners	No; every job starts from an AMI	Fork a running sandbox with its disk
Docker and nested virtualization	Docker; KVM on Linux runners after adding the runner user to the KVM group	Docker and root; KVM on compatible Intel x64 families, not arm64	Docker, root, systemd and /dev/kvm on every machine
Inbound access	None; GitHub says hosted runners never need inbound connections	SSH from a CIDR range you allow, and SSM; no HTTPS URLs	SSH, HTTPS URLs for up to 50 ports, raw TCP, dedicated IPv4 or IPv6
Desktop and computer use	Headless runners; browsers are preinstalled for tests	Headless runners reached over SSM or optional SSH	Linux desktop streamed at 60fps, Chromium, computer use and browser control
Coding agents and keys	Copilot cloud agent, plus Claude and Codex in public preview, on issues and pull requests; secrets enter as environment variables	Bedrock access through the instance role; Copilot's coding agent can use a runner; secrets live on the machine	Seven agents preinstalled, driven from outside with a prompt, steer and events API; credentials proxy keeps real keys out
Operating systems	Linux x64 and arm64, Windows x64 and arm64, macOS	Linux x64 and arm64, Windows Server 2022 and 2025; no macOS	Linux x64
GPUs and machine sizes	Larger runners, including Linux and Windows with an NVIDIA T4	Any EC2 size up to 96 vCPU, with NVIDIA or AMD GPUs from T4 up to H100 and H200	CPU only, up to 16 vCPU / 32 GB on the $100 plan and above
Networking	Larger runners can join your Azure network in US, EU and APAC regions, with static IP ranges	Your own AWS security groups or Network Firewall, private subnets, static egress IPs through NAT	Open internet, no provider-side allowlist
Billing	Per minute per job, rounded up, from $0.006 a minute for 2-core Linux; plans include minutes for standard runners	AWS bills EC2 and EBS to the second at cost; RunsOn charges a yearly licence from $350	Monthly plan from $20, all of it usage credit spent per second; a stopped sandbox costs nothing

Every feature Battleships tracks (43)

Feature	GitHub Actions	RunsOn	boat
Full VM (own kernel)	Yes	Yes	Yes
Docker inside	Yes	Yes	Yes
Nested virtualization	Yes	Yes	Yes
Root	Yes	Yes	Yes
Systemd	No	Not published	Yes
Desktop GUI	No	No	Yes
Browser	Yes	No	Yes
Computer-use API	No	No	Yes
SSH	No	Yes	Yes
Persistent disk	No	Yes	Yes
Pause/resume	No	Yes	Yes
Automatic snapshots	No	Yes	Yes
Fork/clone	No	No	Yes
Live fork (no pause)	No	No	Yes
Memory fork	No	No	No
Custom image (Docker or snapshot)	Yes	Yes	Yes
≥24 h sessions	No	Yes	Yes
Idle auto-stop	Yes	Yes	No
Wake on request	No	Yes	No
HTTPS preview URLs	No	No	Yes
Raw TCP inbound	No	No	Yes
Public IPv4	No	Yes	No
Egress allowlist	Yes	Yes	No
Secret proxy	No	No	Yes
Secret proxy for any API	No	No	Yes
Preinstalled agents	Yes	No	Yes
Agent harness API	Yes	No	Yes
MCP server	Yes	No	No
GPU	Yes	Yes	No
Windows	Yes	Yes	No
Arm64	Yes	Yes	No
Self-hosting	Yes	Yes	No
BYOC	No	Yes	No
Open source	Yes	No	No
Scoped API keys	Not published	Not published	Yes
Spend limits	Yes	Not published	Yes
Usage per sandbox (API)	Yes	No	Yes
Audit logs	Yes	Not published	No
SSO	Yes	No	No
SOC 2	Yes	No	No
HIPAA	No	No	No
EU data residency	Yes	Yes	Yes
Zero data retention	Not published	Not published	Yes

Performance

GitHub ActionsNot measured

RunsOnNot measured

boat26.69 runs/s

There is no comparable measurement for GitHub Actions hosted runners or for RunsOn in the hpc-sandbox-benchmarks CPU test. boat did 26.69 runs per second on it at 4 vCPU / 8 GB, on dedicated AMD Ryzen 9 9950X hosts. RunsOn's speed depends on the EC2 family you pick, and its pricing page lists its own CPU scores against GitHub-hosted runners, from a different test.

Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. GitHub Actions and RunsOn have no published run in this suite yet, so there is no like-for-like number to compare.

Pricing in five scenarios

Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.

Scenario	GitHub Actions, a month	RunsOn, a month	boat, a month
One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days	$80 · Team · with limits	$61 · Starter · with limits	$20 · $20/mo
One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month	$354 · Team · with limits	$159 · Starter · with limits	$26 · $20/mo
50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside	$3,814 · Team · with limits	$1,597 · Starter · with limits	$317 · $20/mo × 3 seats
Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once	$1,092 · Enterprise Cloud · with limits	$515 · Starter · with limits	$200 · $100/mo × 2 seats
1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month	$2,721 · Enterprise Cloud · with limits	$1,103 · Starter · with limits	$2,000 · $2000/mo · with limits

GitHub Actions: GitHub bills per minute per job, rounding each job up to the next whole minute, from $0.006 a minute for 2-core Linux. Plans include monthly minutes for standard runners only, and larger and GPU runners are always billed.

RunsOn: AWS bills the EC2 instances and EBS disks on your own AWS bill, to the second, at spot or on-demand prices with no markup, and S3 cache traffic and data transfer are extra. RunsOn charges one licence a year per company, from EUR 300 or $350 for under 50,000 jobs a month.

RunsOn: Plans start at $20 a month, and the whole fee is usage credit spent per second while a sandbox runs. A stopped sandbox costs nothing and keeps its disk.

Limits in those scenarios

Neither GitHub Actions nor RunsOn fits the five agent scenarios cleanly, because both are CI runners whose machines end with the job. GitHub-hosted jobs stop at 6 hours and keep nothing on disk, while RunsOn jobs can run up to GitHub's 5-day self-hosted limit and keep only the sticky disk. For 1,000 short jobs at once, GitHub Enterprise Cloud stops at 500 concurrent jobs, RunsOn is bound by your AWS quotas and launch pacing, and boat's $2,000 plan queues starts above 90 a minute.

Scenario	GitHub Actions	RunsOn	boat
One coding agent, workdays	Team: sessions capped at 6 h missing persistent disk different kind of product: CI runner	different kind of product: CI runner files survive a stop only in the sticky disk: packages installed elsewhere reset disk beyond 30 GiB: price unknown egress price not published	None that bite
One agent that never stops	Team: sessions capped at 6 h missing persistent disk different kind of product: CI runner	different kind of product: CI runner sessions end after 120 h: needs restarts to run 24/7 files survive a stop only in the sticky disk: packages installed elsewhere reset disk beyond 30 GiB: price unknown	None that bite
50 agents in parallel	Team: sessions capped at 6 h missing persistent disk different kind of product: CI runner	different kind of product: CI runner files survive a stop only in the sticky disk: packages installed elsewhere reset egress price not published	None that bite
Sandboxes for your users	different kind of product: CI runner	different kind of product: CI runner egress price not published	None that bite
1,000 short jobs at once	Enterprise Cloud: max 500 concurrent different kind of product: CI runner	different kind of product: CI runner egress price not published	max 2,400 starts/day (needs ~6,667) max 600 starts/hour (needs ~1,000) bursts above 90 starts/min are queued (a burst of 333 takes ~4 min to start)

Which one should you use?

Use GitHub Actions when

you want CI for a GitHub repository with nothing to host, especially for macOS builds, free public-repo minutes, or Copilot, Claude and Codex agents on your issues.

Nothing to run yourself. For teams that want CI with no cloud account to manage, GitHub hosts the runners and workflows start from pushes and pull requests.
macOS and Windows on arm64. For apps built for Apple platforms, GitHub hosts macOS runners on Apple silicon, which RunsOn does not offer.
Free minutes and built-in agents. Public repositories use standard runners for free, paid plans include minutes, and Copilot, Claude and Codex agents work on your issues.

Use RunsOn when

you run a lot of GitHub Actions jobs and want bigger, cheaper runners in your own AWS account, including Windows, arm64 or large GPUs, billed at AWS cost.

Bigger runners at AWS cost. For teams with heavy CI, changing one runs-on: label moves jobs to EC2 billed at cost, with S3 caches and sticky disks.
Large GPUs and any EC2 size. For jobs that need NVIDIA or AMD GPUs up to H100 and H200, or up to 96 vCPU, RunsOn launches them.
Your own AWS account and network. For teams whose code and logs must stay in their AWS account, RunsOn keeps them there under your own network rules.

Where boat fits

If what you are really running is agents rather than CI jobs, both GitHub Actions and RunsOn make each agent live inside one workflow job on a machine that is thrown away at the end. boat gives each agent its own full VM that you start from code, keep with its whole disk, reach from outside and drive from outside.

A machine that stays

GitHub Actions wipes the VM every job and stops it at 6 hours; RunsOn keeps only one sticky disk and GitHub ends its jobs after 5 days. A boat sandbox keeps its whole disk, has no session cap and can be forked while running.

Reach it and see it

Hosted runners take no inbound traffic, and RunsOn runners only take SSH or SSM, so a preview or webhook needs your own tunnel or load balancer. boat gives HTTPS URLs on 50 ports, raw TCP and a 60fps desktop.

Agents driven from outside, keys kept out

On both, an agent starts as a workflow step and its secrets sit on the machine it controls. boat lets you prompt, steer and follow seven preinstalled agents through an API, with real keys held in a credentials proxy for any HTTPS API.

Use boat when the work is agents rather than CI jobs: a full VM you start by API, keep with its whole disk, fork, reach over HTTPS and drive with built-in coding agents. Compare it one to one: boat vs GitHub Actions and boat vs RunsOn.

Questions people ask

Is GitHub Actions cheaper than RunsOn?

For heavy CI, usually not: in our CI runners model GitHub Actions comes to $1,012 a month against $432 on RunsOn, because AWS bills spot minutes at cost. For small or public-repo workloads, GitHub's included minutes and free standard runners can make it cheaper, since RunsOn adds a yearly licence.

Which is faster, GitHub Actions or RunsOn?

We have no comparable CPU measurement for either. RunsOn's speed depends on the EC2 family you choose, and its pricing page publishes its own CPU scores against GitHub-hosted runners from a different test.

How long can a job run on each?

Up to 6 hours on GitHub-hosted runners. RunsOn runners register as self-hosted, so a job can run up to GitHub's 5-day limit. A boat sandbox has no session cap on paid plans.

Can GitHub Actions and RunsOn run Docker and Android emulators?

Both run Docker. GitHub exposes KVM on Linux runners once you add the runner user to the KVM group, and RunsOn supports KVM on compatible Intel x64 instance families but not arm64. boat has Docker and /dev/kvm on every machine.

Do files persist between jobs?

On GitHub Actions, only through the cache, artifacts or a custom image. On RunsOn, only on the sticky disk, which is snapshotted after a job and restored before the next. A boat sandbox keeps its whole disk, saved every minute and on stop.

Can I use GitHub Actions or RunsOn as a sandbox for AI agents?

Only as workflow jobs. GitHub runs its Copilot, Claude and Codex agents on Actions runners, and RunsOn supports Bedrock and Copilot's coding agent, but neither has an API to create, keep or fork a sandbox on its own, and neither takes HTTPS traffic in.

Try boat

Install the CLI, then run boat new.

7-day free trial