GitHub Actions vs RunsOn

Both run GitHub Actions jobs, each on a fresh machine that ends with the job. GitHub Actions hosts that machine for you. RunsOn starts it as an EC2 instance in your own AWS account, billed by AWS at cost plus a yearly licence.

The companies and products

GitHub Actions

GitHub Actions is GitHub's CI/CD service: workflows in your repository run as jobs on runners that GitHub hosts or that you host yourself.

GitHub Actions is a CI service, not a general sandbox API. Each job on a GitHub-hosted runner gets a new VM with passwordless sudo, and GitHub removes the VM when the job ends. Windows and larger runners run on Azure, standard Linux runners run on a multi-cloud pool, and macOS runs on GitHub's own Mac fleet. The 1-CPU slim runner is a container, not a VM. Actions is built to build, test and deploy the code in a repository. The Copilot cloud agent runs on Actions runners when you start it from the web or Copilot Chat. The newer Copilot in the cloud runs in a Copilot cloud sandbox by default: an isolated Linux session that keeps a snapshot when it stops.

RunsOn

RunsOn is self-hosted GitHub Actions runners on AWS: one stack in your account, a yearly licence, and compute that AWS bills at cost.

You install RunsOn in your own AWS account with CloudFormation or Terraform (Flex), or with Terraform (Fleet). You change the runs-on: label in your workflows. Each job then starts on a new EC2 instance, and the instance is terminated when the job ends. RunsOn covers Linux on x64 and arm64, Windows Server and GPU instances, with caches in S3, sticky disks and warm pools. It is a CI runner, a different kind of product from boat. This page compares the part where they overlap: a machine where an agent or a job does its work.

Which use cases each one fits

For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “85th cheapest of 374” means 84 of the other 373 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.

Use case and its workloadGitHub Actions: price rankRunsOn: price rankboat: price rank
Software factory
100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month
A software factory needs agents that keep their checkout and Docker cache. You set it up on both: GitHub Actions starts each job on a fresh VM and carries files through the cache, artifacts or a larger-runner custom image; RunsOn keeps work on sticky disks and tools in stopped warm pools. boat keeps the sandbox disk, saved every minute and at each stop.
304th of 365, with limits
$5,553 a month
143rd of 365, with limits
$970 a month
5th cheapest of 365
$634 a month
Always-on AI employees
50 machines 24/7 · 2 vCPU / 4 GB
An assistant that never stops is a compromise on both: hosted jobs end after 6 hours, and RunsOn jobs after GitHub's 5-day self-hosted limit. On boat you turn auto-stop off on a paid plan (it defaults to 1 hour).
151st of 361, with limits
$12,882 a month
116th of 361, with limits
$1,724 a month
6th cheapest of 361
$657 a month
Code interpreter bursts
100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month
RunsOn costs less than GitHub Actions here ($147 against $721 a month) because AWS bills spot minutes at cost, though each burst still has to arrive as a GitHub Actions job.
288th of 336, with limits
$721 a month
226th of 336, with limits
$147 a month
189th of 336, with limits
$2,000 a month
Sandboxes for your users
300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month
Neither is built to give your own users sandboxes: GitHub's terms rule out offering Actions as a service, and RunsOn starts machines for your own workflows.
302nd of 349, with limits
$2,892 a month
211th of 349, with limits
$585 a month
3rd cheapest of 349
$209 a month
RL rollouts
1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month
297th of 349, with limits
$14,721 a month
240th of 349, with limits
$2,714 a month
39th of 349, with limits
$2,000 a month
Agent evals
250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month
299th of 359, with limits
$1,662 a month
250th of 359, with limits
$341 a month
10th cheapest of 359
$200 a month
CI runners
40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month
Both fit. GitHub Actions needs nothing to set up, and RunsOn runs the same jobs on EC2 at AWS cost. In our CI runners scenario GitHub Actions comes to $729 a month and RunsOn to $432. boat runs CI on full VMs that keep a warm disk between runs, for $86, but sells no managed runner service.
85th cheapest of 374
$729 a month
56th cheapest of 374
$432 a month
2nd cheapest of 374
$86 a month
24/7 dev boxes
1 machine 24/7 · 4 vCPU / 8 GB
A dev box needs to stay up with its files and be reachable. Hosted runners take no public inbound traffic, and RunsOn offers SSH or SSM for the life of a job. A boat sandbox runs until you stop it once auto-stop is off, and keeps its disk at least 14 days after a stop.
253rd of 371, with limits
$71 a month
175th of 371, with limits
$78 a month
19th cheapest of 371
$26 a month
Computer-use agents
50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month
Neither streams a desktop. You set it up: install a virtual display such as Xvfb in a GitHub job, or a desktop in your RunsOn runner image reached over SSH or SSM. boat streams a 1920x1080, 60 fps Linux desktop with Chrome.
306th of 367, with limits
$1,502 a month
173rd of 367, with limits
$310 a month
7th cheapest of 367
$200 a month
Browser-use agents
50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month
198th of 356, with limits
$942 a month
177th of 356, with limits
$224 a month
7th cheapest of 356
$200 a month
Scraping & bots (startup scale)
50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month
202nd of 343, with limits
$1,184 a month
185th of 343, with limits
$205 a month
6th cheapest of 343
$90 a month
Mass scraping & bots (undetectable)
1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month
296th of 327, with limits
$28,971 a month
124th of 327, with limits
$3,539 a month
73rd of 327, with limits
$1,800 a month
Parallel red teaming
200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month
123rd of 351, with limits
$467 a month
98th of 351, with limits
$157 a month
3rd cheapest of 351
$200 a month
GPU eval jobs
10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month
Not ranked: misses a requirementNot ranked: misses a requirementNot ranked: misses a requirement
macOS agents
5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month
GitHub Actions has hosted macOS runners. RunsOn does not support macOS, and boat runs Linux only.
23rd of 35, with limits
$1,178 a month
Not ranked: misses a requirementNot ranked: misses a requirement

Key features

FeatureGitHub ActionsRunsOnboat
What it isGitHub's CI/CD service: jobs from workflow files run on runners GitHub hostsSelf-hosted GitHub Actions runners, deployed as one stack in your own AWS accountA sandbox API for agents: full VMs you create, stop, fork and delete from code
Where machines runWindows and larger runners on Azure, standard Linux on a multi-cloud pool, macOS on GitHub's Mac fleetEC2 in your AWS account, in the region you pick; code, secrets and logs stay thereManaged by boat on machines in Germany, Finland and France; no region choice; snapshot copies can be stored outside the EU
IsolationA new VM for each job, with passwordless sudoA fresh EC2 instance per job, terminated when the job endsFull VM with its own kernel, root and systemd, on hosts dedicated to boat (AMD Ryzen 7 9700X and Ryzen 9 9950X); vCPUs are shared
How long it can runUp to 6 hours per jobOne job; GitHub ends self-hosted jobs after 5 daysAuto-stop defaults to 1 hour; set up to 30 days, or turn it off on a paid plan for no session cap
Files between runsEvery job starts on a fresh VM. You set it up: the cache, artifacts, or a larger runner's disk saved as a custom imageRoot disk is new each job; sticky EBS disks are snapshotted after a job and restored before the next; stopped warm pools keep dependencies installedDisk saved every minute and at each stop, kept at least 14 days after a stop; memory and running programs are not saved
ForkNo live fork; a larger runner's custom image is a template for new runnersNot documented; each job starts from an AMI, and sticky disks and warm pools reuse stateFork a running sandbox from its last 1-minute snapshot
Docker and nested virtualizationDocker; KVM on Linux runners after adding the runner user to the KVM groupDocker and root; KVM on compatible Intel x64 families, not arm64Docker, root, systemd and /dev/kvm on every sandbox
Inbound accessNo public inbound access; runners joined to your Azure network can be reached only from inside itSSH from a CIDR range you allow, and SSM; HTTPS previews you set up with your own load balancer or tunnel in AWSSSH, up to 50 token-gated HTTPS URLs, public IPv6; raw ports on sandboxes without their own IPv4 work over IPv6 only; no static IP
Desktop and computer useNo desktop stream; browsers preinstalled for tests. You set it up: a virtual display such as XvfbFull runner images include browsers for headless use; a desktop you set up in your image, reached over SSM or SSHLinux desktop and Chrome streamed at 1920x1080, 60 fps, with computer tools for agents
Coding agents and keysCopilot cloud agent, plus Claude and Codex in public preview, on issues and pull requests; secrets enter as inputs or environment variablesBedrock access through the instance role; Copilot's coding agent can use a runner; other agents you install in workflow stepsSeven agent harnesses installed, with a prompt, steer and events API; optional paid credentials proxy keeps supported providers' real keys out
Operating systemsLinux x64 and arm64, Windows x64 and arm64, macOSLinux x64 and arm64, Windows Server 2022 and 2025; macOS not supportedLinux x64 only; Windows programs through Wine
GPUs and machine sizesLarger runners, including Linux and Windows with an NVIDIA T4Any EC2 size up to 96 vCPU, with NVIDIA or AMD GPUs from T4 up to H100 and H200No GPU on any size; up to 16 vCPU / 32 GB on the $100 plan and above
NetworkingLarger runners can get static IP ranges or join your Azure network in US, EU and APAC regionsYour own AWS security groups or Network Firewall, private subnets, static egress IPs through NATOpen internet, no provider-side allowlist; egress not billed
BillingPer minute per job, rounded up, from $0.006 a minute for 2-core Linux; each plan includes a monthly compute amount ($12 on Free)AWS bills EC2 and EBS to the second at cost; RunsOn charges a yearly licence from EUR 300Plans from $20 a month, returned as VM time for that month; per second while running; a stopped sandbox has no compute charge, but extra saved snapshots and the credentials proxy still charge
Every feature Battleships tracks (43)
FeatureGitHub ActionsRunsOnboat
Full VM (own kernel)YesYesYes
Docker insideYesYesYes
Nested virtualizationYesYesYes
RootYesYesYes
SystemdNo documented wayNot publishedYes
Desktop GUINo documented wayNo documented wayYes
BrowserYesYesYes
Computer-use APINo documented wayNo documented wayYes
SSHNo documented wayYesYes
Persistent diskNo documented wayYesYes
Pause/resumeNo documented wayYesYes
Automatic snapshotsNo documented wayYesYes
Fork/cloneNo documented wayNo documented wayYes
Live fork (no pause)No documented wayNo documented wayYes
Memory forkNo documented wayNo documented wayNo documented way
Custom image (Docker or snapshot)YesYesYes
≥24 h sessionsNo documented wayYesYes
Idle auto-stopYesYesNo documented way
Wake on requestNo documented wayYesNo documented way
HTTPS preview URLsNo documented wayNo documented wayYes
Raw TCP inboundNo documented wayNo documented wayYes
Public IPv4No documented wayYesNo documented way
Egress allowlistYesYesNo documented way
Secret proxyNo documented wayNo documented wayYes
Secret proxy for any APINo documented wayNo documented wayYes
Preinstalled agentsYesNo documented wayYes
Agent harness APIYesNo documented wayYes
MCP serverYesNo documented wayNo documented way
GPUYesYesNo documented way
WindowsYesYesNo documented way
Arm64YesYesNo documented way
Self-hostingYesYesNo documented way
BYOCNo documented wayYesNo documented way
Open sourceYesNo documented wayNo documented way
Scoped API keysNot publishedNot publishedYes
Spend limitsYesNot publishedYes
Usage per sandbox (API)YesNo documented wayYes
Audit logsYesNot publishedNo documented way
SSOYesNo documented wayNo documented way
SOC 2YesNo documented wayNo documented way
HIPAANo documented wayNo documented wayNo documented way
EU data residencyYesYesNo documented way
Zero data retentionNot publishedNot publishedYes

Performance

GitHub ActionsNot measured
RunsOnNot measured
boat28.06 runs/s

Neither GitHub Actions hosted runners nor RunsOn is in the public benchmarks we use, so we cannot say which is faster. RunsOn's speed depends on the EC2 family you pick, and its pricing page lists its own CPU scores against GitHub-hosted runners, from a different test. For reference, boat did 28.06 runs/s on the V8 Web Tooling Benchmark at 4 vCPU / 8 GB, second of 15 in StarSling's run of 3 October 2026 (run 37073680030). Every boat sandbox in this run ran on an AMD Ryzen 9 9950X host. boat also places some sandboxes on AMD Ryzen 7 9700X hosts, which this run did not measure.

V8 Web Tooling Benchmark at a 4 vCPU / 8 GB target, runs per second, higher is better. Source: StarSling's run of 3 October 2026 (run 37073680030), median of 5 sandboxes. Providers ran on different hardware in different regions. StarSling gives a shared rank when the run cannot separate two providers. Every boat sandbox in this run ran on an AMD Ryzen 9 9950X host. boat also places some sandboxes on AMD Ryzen 7 9700X hosts, which this run did not measure. GitHub Actions and RunsOn are not in the public benchmarks we use, so we cannot say which is faster.

Pricing in five scenarios

Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario. Start limits, per-start fees and plan minimums are in the price, boat's included.

ScenarioGitHub Actions, a monthRunsOn, a monthboat, a month
One coding agent, workdays
4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days
$21 · Enterprise Cloud · with limits$61 · Starter · with limits$20 · $20/mo
One agent that never stops
4 vCPU / 8 GB, 50 GB disk, running 24/7 all month
$71 · Enterprise Cloud · with limits$159 · Starter · with limits$26 · $20/mo
50 agents in parallel
50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside
$3,565 · Enterprise Cloud · with limits$1,597 · Starter · with limits$317 · $20/mo × 3 seats
Sandboxes for your users
2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once
$1,092 · Enterprise Cloud · with limits$515 · Starter · with limits$200 · $100/mo × 2 seats
1,000 short jobs at once
2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month
$2,721 · Enterprise Cloud · with limits$1,103 · Starter · with limits$8,800 · $2000/mo · with limits

GitHub Actions: GitHub bills per minute per job, rounding each job up to the next whole minute, from $0.006 a minute for 2-core Linux. Each plan includes a monthly compute amount in dollars ($12 Free, about 2,000 minutes of 2-core Linux; $18 Pro and Team; $300 Enterprise Cloud), shared with Copilot cloud sandboxes. Larger runners draw from it on Team and Enterprise. Public repositories use standard runners for free.

RunsOn: AWS bills the EC2 instances and EBS disks on your own AWS bill, to the second, at spot or on-demand prices with no markup. S3 cache traffic, data transfer and a public IPv4 are extra. RunsOn charges one licence a year per company, from EUR 300 for under 50,000 jobs a month. Spot machines can be interrupted; RunsOn can fall back to on-demand at a higher rate.

RunsOn: Plans cost $20, $100, $500 or $2,000 a month. The whole plan price comes back as VM time for that month, and it does not carry over. A default sandbox (4 vCPU / 8 GB) costs $0.036 an hour, billed per second while it runs, busy or idle. A stopped sandbox has no compute charge; saved snapshots above the included count and the AI credentials proxy still charge. The $20 plan includes 200 starts a day; each extra start costs $0.03 to $0.07.

Limits in those scenarios

Neither GitHub Actions nor RunsOn fits the five agent scenarios cleanly, because both are CI runners whose machines end with the job. GitHub-hosted jobs stop at 6 hours and start on a fresh VM, while RunsOn jobs can run up to GitHub's 5-day self-hosted limit and keep only sticky disks. For 1,000 short jobs at once, GitHub Enterprise standard runners stop at 500 concurrent jobs; larger runners on Team and Enterprise allow 1,000, and GitHub Support can raise limits. RunsOn is bound by your AWS quotas and launch pacing. On boat that burst does not fit as priced either: the $2,000 plan includes 90 starts a minute, extra starts cost $0.03 to $0.07, and starts above 270 a minute fail with 429. boat never queues a start.

ScenarioGitHub ActionsRunsOnboat
One coding agent, workdays
  • Enterprise Cloud: sessions capped at 6 h
  • different kind of product: CI runner
  • different kind of product: CI runner
  • files survive a stop only in the sticky disk: packages installed elsewhere reset
  • disk beyond 30 GiB: price unknown
  • egress price not published
None for this scenario
One agent that never stops
  • Enterprise Cloud: sessions capped at 6 h
  • different kind of product: CI runner
  • different kind of product: CI runner
  • sessions end after 120 h: needs restarts to run 24/7
  • files survive a stop only in the sticky disk: packages installed elsewhere reset
  • disk beyond 30 GiB: price unknown
None for this scenario
50 agents in parallel
  • Enterprise Cloud: sessions capped at 6 h
  • different kind of product: CI runner
  • different kind of product: CI runner
  • files survive a stop only in the sticky disk: packages installed elsewhere reset
  • egress price not published
None for this scenario
Sandboxes for your users
  • different kind of product: CI runner
  • different kind of product: CI runner
  • egress price not published
None for this scenario
1,000 short jobs at once
  • Enterprise Cloud: max 500 concurrent
  • different kind of product: CI runner
  • needs talk to sales: above plan "Enterprise Cloud" limits: contact sales for more
  • different kind of product: CI runner
  • egress price not published
  • 2,400 starts/day included, paid above that (needs ~6,667)
  • 600 starts/hour included, paid above that (needs ~1,000)
  • starts above 90/min are paid, and starts above 270/min fail with 429

Which one should you use?

Use GitHub Actions when

you want CI for a GitHub repository with nothing to host, especially for macOS builds, free public-repo runners, or Copilot, Claude and Codex agents on your issues.

  • Nothing to run yourself. For teams that want CI with no cloud account to manage, GitHub hosts the runners and workflows start from pushes and pull requests.
  • macOS runners. For apps built for Apple platforms, GitHub hosts macOS runners, which RunsOn does not support.
  • Included usage and built-in agents. Public repositories use standard runners for free, every plan includes monthly compute, and Copilot, Claude and Codex agents work on your issues.

Use RunsOn when

you run a lot of GitHub Actions jobs and want bigger, cheaper runners in your own AWS account, including Windows, arm64 or large GPUs, billed at AWS cost.

  • Bigger runners at AWS cost. For teams with heavy CI, changing one runs-on: label moves jobs to EC2 billed at cost, with S3 caches and sticky disks.
  • Large GPUs and any EC2 size. For jobs that need NVIDIA or AMD GPUs up to H100 and H200, or up to 96 vCPU, RunsOn launches them.
  • Your own AWS account and network. For teams whose code and logs must stay in their AWS account, RunsOn keeps them there under your own network rules.

Where boat fits

If what you run is agents rather than CI jobs, both GitHub Actions and RunsOn put each agent inside one workflow job on a machine that ends with the job. boat gives each agent its own full VM that you start from code, keep with its disk, reach from outside and drive from outside.

A machine that stays

GitHub Actions starts each job on a fresh VM and stops it at 6 hours. RunsOn keeps sticky disks between jobs, and GitHub ends its jobs after 5 days. A boat sandbox saves its disk every minute and at each stop, can be forked while running, and has no session cap once you turn auto-stop off (default 1 hour). Memory and running programs are not saved.

Reach it and see it

Hosted runners take no public inbound traffic, and RunsOn runners take SSH or SSM, so a preview or webhook needs your own tunnel or load balancer. boat gives up to 50 token-gated HTTPS URLs, SSH and a 60 fps desktop.

Agents driven from outside

On both, an agent starts as a workflow step and its secrets sit on the machine it controls. boat lets you prompt, steer and follow seven installed agent harnesses through an API. Its optional AI credentials proxy ($1 per day on, plus $1 per 1,000 requests) keeps real keys of supported providers out; other secrets still enter the sandbox.

Use boat when the work is agents rather than CI jobs: a full Linux VM you start by API, keep with its disk between runs, fork, reach over HTTPS and drive with installed coding agents. Compare it one to one: boat vs GitHub Actions and boat vs RunsOn.

boat's own limits
  • Plans start at $20 a month. The plan price comes back as VM time for that month. Unused time does not carry over. Source
  • Starts (create, fork, resume): the $20 plan includes 12 a minute, 60 an hour and 200 a day. The $2,000 plan includes 90 a minute, 600 an hour and 2,400 a day. Extra starts cost $0.03 to $0.07. A start fails with 429 above 3 times the included starts per minute, or 6 times per hour or per day. Source
  • A stopped sandbox has no compute charge. boat keeps its disk for at least 14 days. A saved snapshot keeps it longer; extra saved snapshots and the AI credentials proxy are paid. Source
  • Auto-stop defaults to 1 hour. You can turn it off on a paid plan. The trial allows 2 hours at most. Source
  • boat saves the disk every minute and at each stop. Memory, running programs and /tmp are not saved: a resume is like a reboot. A machine failure loses writes after the last good snapshot. Source
  • Machines run in Germany, Finland and France only. Snapshot copies can be stored outside the EU. Source
  • Public IPv6 and outbound IPv4. No dedicated IPv4. The IP changes at each resume. Source
  • Hosts are AMD Ryzen 7 9700X and Ryzen 9 9950X. vCPUs are shared. No GPU. You cannot boot your own Docker image. Source

Questions people ask

Is GitHub Actions cheaper than RunsOn?

For heavy CI, usually not: in our CI runners scenario GitHub Actions comes to $729 a month against $432 on RunsOn, because AWS bills spot minutes at cost. For small or public-repo workloads, GitHub's included usage and free standard runners can make it cheaper, since RunsOn adds a yearly licence.

Which is faster, GitHub Actions or RunsOn?

Neither is in the public benchmarks we use, so we cannot say which is faster. RunsOn's speed depends on the EC2 family you choose, and its pricing page publishes its own CPU scores against GitHub-hosted runners from a different test.

How long can a job run on each?

Up to 6 hours on GitHub-hosted runners. RunsOn runners register as self-hosted, so a job can run up to GitHub's 5-day limit. A boat sandbox auto-stops after 1 hour by default. You can set up to 30 days, or turn auto-stop off on a paid plan for no session cap.

Can GitHub Actions and RunsOn run Docker and Android emulators?

Both run Docker. GitHub exposes KVM on Linux runners once you add the runner user to the KVM group, and RunsOn supports KVM on compatible Intel x64 instance families but not arm64. boat has Docker and /dev/kvm on every sandbox.

Do files persist between jobs?

On GitHub Actions, through the cache, artifacts or a larger-runner custom image. On RunsOn, on sticky disks, which RunsOn snapshots after a job and restores before the next, and in stopped warm pools. A boat sandbox saves its disk every minute and at each stop, and keeps it at least 14 days after a stop. Memory and running programs are not saved.

Can I use GitHub Actions or RunsOn as a sandbox for AI agents?

As workflow jobs, yes. GitHub runs its Copilot, Claude and Codex agents for work on issues and pull requests, and RunsOn supports Bedrock and Copilot's coding agent. Neither documents an API to create, keep or fork a sandbox outside a job. GitHub runners take no public inbound traffic, and on RunsOn you add your own load balancer or tunnel.

Try boat

Install the CLI, then run boat new.

7-day free trial

More comparisons