Bitrise vs RunsOn
Bitrise is a hosted CI/CD service for iOS and Android apps, with Apple silicon Macs, Linux machines and remote dev environments. RunsOn puts your GitHub Actions jobs on fresh EC2 machines inside your own AWS account, with Linux, Windows and GPUs but no macOS.
The companies and products
Bitrise
Bitrise is a mobile CI/CD platform that builds, tests and ships iOS and Android apps on its own Mac and Linux machines.
Bitrise is a different kind of product: a CI service. Each build runs in its own virtual machine from a Bitrise stack, and the machine is rolled back after the build. Since 2026 it also sells Remote Dev Environments, macOS and Linux sessions with a persistent disk where you can start Claude Code from a CLI or drive the machine through an MCP server. It is built for mobile teams.
- Funding
- $95M+ in total
RunsOn
RunsOn is self-hosted GitHub Actions runners on AWS: one CloudFormation or Terraform stack in your account, a yearly licence, and compute billed by AWS at cost.
You install RunsOn in your own AWS account, change the runs-on: label in your workflows, and every job then starts on a new EC2 instance that is terminated when the job ends. It covers Linux on x64 and arm64, Windows Server and GPU instances, with caches in S3 and sticky EBS disks. It is a CI runner, a different kind of product from boat, so this page compares the part where they overlap: a machine where an agent or a job does its work.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “38th cheapest of 367” means 37 of the other 366 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Bitrise: price rank | RunsOn: price rank | boat: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month A software factory needs machines that keep their work; Bitrise CI rolls back after each build, RunsOn keeps only the sticky disk, and boat keeps the whole disk. | 38th of 367, with limits $3,403 a month | 122nd of 367, with limits $1,114 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB An assistant that never stops is a compromise on both: Bitrise dev sessions stop after days idle and for maintenance, RunsOn jobs end after 5 days, and boat has no session cap on paid plans. | 73rd of 366, with limits $1,970 a month | 188th of 366, with limits $1,724 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month | 289th of 345, with limits $658 a month | 229th of 345, with limits $147 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Neither is built to hand sandboxes to your users: Bitrise caps machines per plan and RunsOn only starts machines for your own GitHub workflows, while boat starts sandboxes by API. | 269th of 361, with limits $1,604 a month | 216th of 361, with limits $585 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month | 271st of 360, with limits $6,818 a month | 245th of 360, with limits $2,714 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 289th of 364, with limits $1,274 a month | 251st of 364, with limits $389 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month RunsOn ranks 58th of 377 here and fits fully, since GitHub Actions runners are its whole product; Bitrise is built for mobile pipelines, and boat runs CI jobs on full VMs with Docker and /dev/kvm. | 181st of 377, with limits $859 a month | 58th cheapest of 377 $432 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB Bitrise dev environments keep one persistent disk that is not backed up, a RunsOn runner is gone when its job ends, and a boat machine keeps its whole disk with snapshots every minute. | 168th of 368, with limits $119 a month | 158th of 368, with limits $86 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Bitrise lets an agent control a macOS desktop through its MCP server, RunsOn runners are headless, and boat streams a 60fps Linux desktop with Chromium. | 68th of 369, with limits $1,186 a month | 142nd of 369, with limits $353 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 132nd of 367, with limits $746 a month | 206th of 367, with limits $224 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 131st of 346, with limits $862 a month | 228th of 346, with limits $205 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 256th of 338, with limits $13,088 a month | 122nd of 338, with limits $3,539 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 73rd of 360, with limits $537 a month | 178th of 360, with limits $157 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month None of the three fits in our model; RunsOn can launch NVIDIA and AMD GPU instances for CI jobs, while Bitrise and boat have no GPUs. | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month Only Bitrise fits: it rents Apple silicon Macs with iOS simulators, while RunsOn does not support macOS and boat runs Linux only. | 21st of 36, with limits $400 a month | Can't run it | Can't run it |
Key features
| Feature | Bitrise | RunsOn | boat |
|---|---|---|---|
| Kind of product | Hosted mobile CI/CD service, plus Remote Dev Environments for developers and agents | Self-hosted GitHub Actions runners: one EC2 machine per CI job | Sandbox API for agents: full VMs you create, stop, fork and delete from code |
| Where machines run | Managed by Bitrise in the US, Ireland, Frankfurt, Singapore and Sydney, or in your own AWS account, or self-hosted Runners | Always in your own AWS account, in the region you pick | Managed by boat in the EU (Germany, Finland, France) |
| Operating systems | macOS on Apple silicon with iOS simulators, and Linux with Android emulators | Linux x64 and arm64, Windows Server 2022 and 2025; no macOS | Linux only; Wine for Windows apps |
| Machine sizes and GPUs | Macs on M2 Pro, M4 and M4 Pro, Linux from 2 to 48 vCPUs; no GPUs | Any EC2 size up to 96 vCPU, with NVIDIA or AMD GPUs from T4 to H200 | Linux x64 up to 16 vCPU / 32 GB on the $100 plan and above; CPU only |
| How a machine starts | A build from a Bitrise workflow, or a dev session from the CLI or MCP server | A GitHub Actions job that asks for it with a runs-on: label | Any time, by API, CLI or SDK |
| How long a machine runs | CI builds time out at 210 minutes (up to 4 hours on Enterprise); dev sessions have no cap but stop after 5 days idle by default | One job; GitHub ends self-hosted jobs after 5 days | No session cap on paid plans; you set the time to live |
| Files after a stop | CI machines are rolled back; dev sessions keep one persistent disk that is not backed up | Root disk is thrown away; one sticky EBS disk is snapshotted and restored next job | Whole disk saved every minute and on stop; a stopped sandbox costs nothing |
| Fork | No fork; warm pools hand out fresh sessions from a template | No; every job starts from an AMI | Fork a running sandbox into an independent copy |
| Docker and nested virtualization | Docker and nested virtualization on Linux machines; neither on macOS sessions | Docker and root; KVM on compatible Intel x64 families, not arm64 | Docker, root, systemd and /dev/kvm on every machine |
| Desktop and computer use | VNC desktop on macOS sessions; an MCP server takes screenshots and controls it | Headless runners reached over SSM or optional SSH | 60fps Linux desktop and Chromium, with computer use and browser control |
| Reaching the machine | SSH, VNC and a web terminal; no public URL for a port | SSH from a CIDR range you allow, and SSM; no HTTPS URLs | HTTPS URLs for up to 50 ports, token-protected; raw TCP; dedicated IPv4 or IPv6 |
| Coding agents | Claude Code started from the CLI and used in a terminal; other agents through warmup scripts | Bedrock access through the instance role; Copilot's coding agent can use a runner | Claude Code, Codex, OpenCode and more preinstalled, driven from outside with a prompt, steer and events API |
| Keeping keys out of the machine | Secrets are environment variables, so the value enters the machine | Secrets reach the job on the runner; AWS access through the instance role | AI credentials proxy for model providers and any HTTPS API; real keys never enter the VM |
| CI features | Workflows, code signing, build cache, release management and over-the-air updates | Drop-in GitHub Actions runners with S3 caches, sticky disks and warm pools | None built in; you can run a runner on a full VM |
Every feature Battleships tracks (43)
| Feature | Bitrise | RunsOn | boat |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | Yes | Yes | Yes |
| Root | Yes | Yes | Yes |
| Systemd | No | Not published | Yes |
| Desktop GUI | Yes | No | Yes |
| Browser | Yes | No | Yes |
| Computer-use API | Yes | No | Yes |
| SSH | Yes | Yes | Yes |
| Persistent disk | Yes | Yes | Yes |
| Pause/resume | Yes | Yes | Yes |
| Automatic snapshots | No | Yes | Yes |
| Fork/clone | No | No | Yes |
| Live fork (no pause) | No | No | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | No | Yes | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | No | Yes | No |
| HTTPS preview URLs | No | No | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | Yes | No |
| Egress allowlist | No | Yes | No |
| Secret proxy | No | No | Yes |
| Secret proxy for any API | No | No | Yes |
| Preinstalled agents | Yes | No | Yes |
| Agent harness API | Yes | No | Yes |
| MCP server | Yes | No | No |
| GPU | No | Yes | No |
| Windows | No | Yes | No |
| Arm64 | Yes | Yes | No |
| Self-hosting | Yes | Yes | No |
| BYOC | Yes | Yes | No |
| Open source | No | No | No |
| Scoped API keys | Not published | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | No | No | Yes |
| Audit logs | Not published | Not published | No |
| SSO | Yes | No | No |
| SOC 2 | Yes | No | No |
| HIPAA | No | No | No |
| EU data residency | Yes | Yes | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
There is no comparable measurement for either: neither Bitrise nor RunsOn is in the public sandbox benchmark we use, so we do not compare speeds. boat did 26.69 Node.js build-and-test runs per second at 4 vCPU / 8 GB in StarSling's run of 2026-09-29. RunsOn's speed depends on the EC2 family you pick.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. Bitrise and RunsOn have no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Bitrise, a month | RunsOn, a month | boat, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $260 · Pro · with limits | $61 · Starter · with limits | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $119 · RDE Pro · with limits | $159 · Starter · with limits | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $2,309 · Pro · with limits | $1,597 · Starter · with limits | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $812 · Pro · with limits | $515 · Starter · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $1,538 · Pro · with limits | $1,103 · Starter · with limits | $2,000 · $2000/mo · with limits |
Bitrise: Bitrise CI is sold by plan: Starter is a flat fee with unlimited minutes on Medium machines, and Pro adds per-minute machine time and a monthly build allowance on top of its fee. Remote Dev Environments charge a base fee with included minutes, then per minute up to a monthly cap per OS.
RunsOn: AWS bills the EC2 instances and EBS disks on your own AWS bill, to the second, at spot or on-demand prices with no markup; S3 cache traffic and data transfer are extra. RunsOn charges one licence a year per company, from EUR 300 or $350 for under 50,000 jobs a month.
RunsOn: You pick a monthly plan from $20, and all of it becomes usage credit spent per second on running sandboxes. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
None of the five scenarios is a clean fit for either, because both are CI runners. Bitrise Pro runs at most 10 builds at once, caps each at 3.5 hours and does not allow the Linux M size, so 50 agents and 300 user sandboxes do not fit. RunsOn has no cap of its own on machines, but each one lives for one GitHub Actions job of at most 5 days, only the sticky disk survives, and your AWS quotas and launch pacing set the real limit. On boat the 1,000-job burst gets queued and needs more starts per day than the $2,000 plan allows.
| Scenario | Bitrise | RunsOn | boat |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Bitrise when
you build and ship iOS or Android apps, need macOS on Apple silicon, or want a hosted CI/CD service with code signing and store releases built in.
- Macs and iOS simulators. Useful if you build or test iOS apps: Bitrise rents Apple silicon Macs with iOS simulators, which RunsOn does not offer.
- A full mobile pipeline. Useful if you ship mobile apps: builds, code signing, caching, store releases and over-the-air updates come in one hosted service.
- Dev environments for agents. Useful if you want a cloud Mac or Linux machine with a persistent disk where you start Claude Code from a CLI.
Use RunsOn when
your CI is GitHub Actions and you want cheaper, bigger runners in your own AWS account, including Windows, arm64 or GPU jobs.
- GitHub Actions at AWS cost. Useful if your CI lives in GitHub Actions: change one label and jobs run on EC2, billed by AWS with no markup.
- Windows, arm64 and GPUs. Useful for jobs that need Windows Server, native arm64 builds or NVIDIA GPUs, on any EC2 size up to 96 vCPU.
- Everything stays in your AWS account. Useful when code, secrets and logs must never leave your own AWS account, in the region you choose.
Where boat fits
Bitrise and RunsOn are CI services, and boat is not. If what you are really running is agents, not builds, you need a machine you start from code, keep for days with its whole disk, fork, and reach over HTTPS. That is what boat is built for.
A machine that keeps running and keeps its files
Bitrise CI builds stop at 210 minutes and roll back, and RunsOn jobs end after 5 days with only the sticky disk kept. boat has no session cap on paid plans and saves the whole disk every minute.
Agents you drive from outside
On Bitrise you work with Claude Code in a terminal, and on RunsOn you wrap each agent in workflow steps. boat ships coding agents in every sandbox with a prompt, steer and events API, and keeps real keys in a credentials proxy.
Fork, open ports and a desktop
Neither can fork a machine or give a port a public HTTPS URL, so an agent's web app preview needs your own tunnel. boat forks running sandboxes, serves HTTPS on up to 50 ports with raw TCP, and streams a 60fps Linux desktop.
Use boat when you need machines for agents rather than CI jobs: a full VM you start by API, keep for days with its whole disk, fork, reach over HTTPS and drive with built-in coding agents. Compare it one to one: boat vs Bitrise and boat vs RunsOn.
Questions people ask
Is Bitrise cheaper than RunsOn?
In our scenarios RunsOn costs less in four of five: one coding agent on workdays comes to $61 on RunsOn and $260 on Bitrise, and 50 agents in parallel to $1,597 against $2,309. One agent that never stops is the exception, at $119 on Bitrise against $159 on RunsOn. None of these fits cleanly on either, and RunsOn adds a yearly licence.
Which is faster, Bitrise or RunsOn?
We cannot say: neither is in the public sandbox benchmark we use. RunsOn's speed depends on the EC2 family you choose, and Bitrise's on the machine size your plan allows.
Does RunsOn support macOS or iOS builds?
No. RunsOn covers Linux x64 and arm64, Windows Server and GPU instances, but not macOS. For iOS builds and simulators, Bitrise rents Apple silicon Macs.
Can Bitrise and RunsOn run Docker?
Both run Docker on Linux. Bitrise also offers nested virtualization on Linux, but macOS sessions have neither Docker nor nested virtualization. RunsOn has KVM on compatible Intel x64 families, not arm64.
Can I run AI coding agents on Bitrise or RunsOn?
On Bitrise, yes, in Remote Dev Environments, where the CLI starts Claude Code in a session. On RunsOn an agent has to run inside a GitHub Actions job, with Bedrock through the instance role or Copilot's coding agent. boat ships agents preinstalled and lets you drive them through an API.
Where does my data run?
RunsOn runs in your own AWS account, in the region you choose. Bitrise runs in the US, Ireland, Frankfurt, Singapore and Sydney, or in your own AWS account. boat runs only in the EU (Germany, Finland and France), with EU data residency.
Try boat
Install the CLI, then run boat new.
7-day free trial