Buildkite vs Codemagic
Buildkite is a general CI platform: its hosted agents run pipeline jobs of up to 8 hours on Linux and 4 on macOS, and you can add your own machines. Codemagic is built for mobile apps: each build gets a fresh Mac, Linux or Windows machine for at most 120 minutes on self-serve plans, with code signing and store publishing built in.
The companies and products
Buildkite
Buildkite is a CI and software delivery platform, and hosted agents are the build machines it runs for you.
You write Buildkite pipelines, and each job runs on a fresh virtual machine that Buildkite starts for that job and destroys when it ends; only cache volumes carry files to the next job. Hosted agents come in Linux and macOS on Apple M4, with Windows in private preview, and you can also run Buildkite's agent on your own machines. It is a CI service, not an agent sandbox, so this page compares the part where they overlap: a machine where an agent or a job does its work.
- Where it runs
- Hosted agents run only in US East Coast data centers
- Compliance
- Maintains a SOC 2 Type II report
Codemagic
Codemagic is a cloud CI/CD service built for mobile developers: it builds, tests and publishes iOS, Android, Flutter and React Native apps.
You describe a workflow in codemagic.yaml, and each build runs on a new virtual machine: an Apple silicon Mac mini with Xcode, an Ubuntu Linux machine, or a Windows machine. The machine is destroyed when the build finishes, and the result can go straight to App Store Connect, Google Play and other stores. It is built for app teams that want builds and code signing handled for them; it is a different kind of product from an agent sandbox, and the two overlap only where an agent needs a machine for a short, scripted job.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “43rd cheapest of 367” means 42 of the other 366 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Buildkite: price rank | Codemagic: price rank | boat: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month A coding agent that builds a whole app needs one machine for hours; Buildkite jobs end after 8 hours and Codemagic builds after 120 minutes, and both machines are destroyed. | 43rd of 367, with limits $11,648 a month | 362nd of 367, with limits $32,718 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB An always-on assistant does not fit either CI service, since every job or build ends and its machine is deleted; boat has no session cap on paid plans. | 351st of 366, with limits $17,564 a month | 364th of 366, with limits $98,550 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month | 315th of 345, with limits $1,587 a month | 337th of 345, with limits $9,147 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Your users' sandboxes need hundreds at once and a URL to open; Buildkite Pro caps at 128 concurrent jobs, Codemagic sells build slots one at a time, and neither exposes ports. | 326th of 361, with limits $4,984 a month | 349th of 361, with limits $28,497 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month | 316th of 360, with limits $23,881 a month | 336th of 360, with limits $135 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 324th of 364, with limits $3,884 a month | 363rd of 364, with limits $10,947 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month Both are CI services and land close together here, Buildkite at 203 and Codemagic at 207; boat ranks 4th because a full VM keeps its disk warm between jobs. | 203rd of 377, with limits $2,318 a month | 207th of 377, with limits $3,387 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB A dev box has to keep its files overnight; Buildkite keeps only cache volume paths and Codemagic keeps only dependency caches, while a stopped boat sandbox keeps its whole disk. | 354th of 368, with limits $715 a month | 191st of 368, with limits $333 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Both offer a desktop only for a person debugging a build, with no computer-use API; boat streams a 60fps Linux desktop an agent can drive. | 70th of 369, with limits $3,500 a month | 71st of 369, with limits $10,047 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 142nd of 367, with limits $1,772 a month | 151st of 367, with limits $10,947 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 142nd of 346, with limits $2,354 a month | 148th of 346, with limits $13,310 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 317th of 338, with limits $46,814 a month | 337th of 338, with limits $263 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 85th of 360, with limits $1,184 a month | 95th of 360, with limits $6,672 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month Both run macOS on Apple silicon with Xcode, Codemagic on Mac mini M2 and M4 and Buildkite on Apple M4; boat runs Linux only, so it has no entry. | 35th of 36, with limits $2,310 a month | 32nd of 36, with limits $1,952 a month | Can't run it |
Key features
| Feature | Buildkite | Codemagic | boat |
|---|---|---|---|
| Kind of product | General CI platform; hosted agents run jobs from Buildkite pipelines | Mobile CI/CD; builds start from codemagic.yaml or the API | Agent sandboxes you create and drive by API, CLI or SDK |
| How long a machine runs | Up to 8 hours per job on Linux and 4 on macOS, longer on request | 120 minutes per build on Free, Pay as you go and Fixed price; custom on Enterprise | No session cap on paid plans; you set the time to live |
| Files after a run | VM destroyed; cache volumes keep chosen paths after successful jobs, best effort, up to 14 days | Machine and disk deleted; only per-workflow dependency caches carry over | Whole disk saved every minute and on stop; a stopped sandbox costs nothing |
| Fork | Each job gets its own copy of the last committed cache volume | No fork; every build starts from a stock image | Fork a running sandbox with its disk |
| Docker and nested virtualization | Docker on Linux; macOS has the Docker CLI but no daemon; /dev/kvm not documented | Docker and Android emulators on Linux; no nested virtualization on macOS | Docker, root, systemd and /dev/kvm on every machine |
| Operating systems | Linux (ARM64 on Enterprise), macOS on Apple M4, Windows Server 2022 in private preview | macOS on Apple silicon, Linux (arm64 on request) and Windows | Linux only (x86-64) |
| Mac machines | Apple M4 with Xcode, up to 12 vCPU / 56 GB | Mac mini M2 (8-core / 12 GB) and M4 (10-core / 16 GB) with Xcode and iOS simulators | None |
| Biggest Linux machine | 8 vCPU / 32 GB on Pro, up to 64 vCPU / 256 GB on Enterprise | One fixed size: Linux X2 at 8 vCPU / 32 GB | Up to 16 vCPU / 32 GB on the $100 plan and above |
| App store publishing | Not built in; you script it in pipeline steps | Code signing and publishing to App Store Connect, TestFlight, Google Play and more | Not built in; you script it yourself |
| Your own machines | Run Buildkite's agent on your own machines next to hosted ones | Hosted machines only | Hosted sandboxes on dedicated AMD Ryzen 9 9950X hosts |
| Reaching the machine | No exposed ports; a terminal into a running job through the UI or bk job ssh | Temporary SSH, VNC or RDP through a gateway, turned on per build | HTTPS URLs on up to 50 ports, token-protected; raw TCP; SSH |
| Desktop and computer use | Browser desktop to debug a running macOS job; no computer-use API | VNC or RDP for a person to debug a build; no computer-use API | 60fps Linux desktop with Chromium, computer use and browser control |
| Coding agents and keys | Claude Code runs as a pipeline step; an Anthropic proxy sets the API key for the job | Agents run as a build step; keys arrive as encrypted environment variables | Claude Code, Codex and more preinstalled with a prompt, steer and events API; credentials proxy for any HTTPS API |
| Regions | US East Coast only | US data centers; dedicated macOS hosts in the EU on request | EU only (Germany, Finland, France), with EU data residency |
Every feature Battleships tracks (43)
| Feature | Buildkite | Codemagic | boat |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | No | Yes | Yes |
| Root | Not published | Not published | Yes |
| Systemd | No | No | Yes |
| Desktop GUI | Yes | Yes | Yes |
| Browser | Yes | Yes | Yes |
| Computer-use API | No | No | Yes |
| SSH | Yes | Yes | Yes |
| Persistent disk | Yes | No | Yes |
| Pause/resume | No | No | Yes |
| Automatic snapshots | Yes | No | Yes |
| Fork/clone | Yes | No | Yes |
| Live fork (no pause) | No | No | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | No | Yes |
| ≥24 h sessions | No | No | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | No | No | No |
| HTTPS preview URLs | No | No | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | No | No | No |
| Secret proxy | Yes | No | Yes |
| Secret proxy for any API | No | No | Yes |
| Preinstalled agents | No | No | Yes |
| Agent harness API | No | No | Yes |
| MCP server | Yes | No | No |
| GPU | No | No | No |
| Windows | Yes | Yes | No |
| Arm64 | Yes | Yes | No |
| Self-hosting | Yes | No | No |
| BYOC | No | No | No |
| Open source | No | No | No |
| Scoped API keys | Not published | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | No | No | Yes |
| Audit logs | Not published | Yes | No |
| SSO | Yes | Yes | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | No | No | No |
| EU data residency | No | Yes | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
There is no comparable measurement for Buildkite or Codemagic: neither was part of the StarSling hpc-sandbox-benchmarks run. On that Node.js build-and-test benchmark at 4 vCPU / 8 GB, boat did 26.69 runs per second on dedicated AMD Ryzen 9 9950X (Zen 5) hosts. Codemagic also sells only fixed sizes, so it has no 4 vCPU / 8 GB machine to compare.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. Buildkite and Codemagic have no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Buildkite, a month | Codemagic, a month | boat, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $166 · Pro · with limits | $333 · Free (individual) · with limits | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $715 · Pro · with limits | $333 · Free (individual) · with limits | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $7,677 · Pro · with limits | $21,531 · Pay as you go (team, 4 concurrent) · with limits | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $2,204 · Pro · with limits | $12,297 · Pay as you go (team, 4 concurrent) · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $4,787 · Pro · with limits | $27,147 · Pay as you go (team, 4 concurrent) · with limits | $2,000 · $2000/mo · with limits |
Buildkite: Pro costs $30 per active user a month with 4,000 Linux vCPU minutes included; after that Linux is $0.004 per vCPU minute and macOS $0.02, metered to the second. Enterprise pricing is custom.
Codemagic: Codemagic bills build minutes: $0.095 a minute on a Mac mini M2, $0.114 on an M4 and $0.045 on Linux X2 or Windows, with 500 free M2 minutes for individuals. Each extra build at once costs $49 a month, and fixed annual plans start at $3,990 a year.
Codemagic: You pick a monthly plan from $20, and all of it becomes usage credit spent per second on running sandboxes. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
Neither CI service fits any of the five agent scenarios. On Buildkite the job is the limit: it stops after 8 hours, everything outside cache volume paths resets, and Pro shares one pool of 256 Linux vCPU, so 1,000 short jobs mostly wait in the queue. On Codemagic builds stop within 120 minutes with nothing kept, Pay as you go adds parallel builds one $49 slot at a time, and the smallest Linux machine is 8 vCPU / 32 GB. On boat only the 1,000-job burst does not fit: it needs more starts per day than the $2,000 plan allows, so it gets queued.
| Scenario | Buildkite | Codemagic | boat |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Buildkite when
you want a general CI platform with pipelines, hosted Linux and macOS agents next to your own machines, or builds larger and longer than Codemagic allows.
- A general CI platform. Useful if you build more than mobile apps: pipelines, test tooling and cache volumes for any language or stack.
- Hosted and self-hosted agents together. Useful if you already own build machines: run Buildkite's agent on them next to the agents Buildkite hosts.
- Bigger and longer jobs. Useful for heavy builds: Linux jobs run up to 8 hours, on shapes up to 64 vCPU and 256 GB on Enterprise.
Use Codemagic when
you build, sign and publish iOS, Android or Flutter apps and want Macs with Xcode and iOS simulators, with store publishing handled for you.
- A full mobile release pipeline. Useful if you ship mobile apps: Codemagic signs builds and publishes to App Store Connect, TestFlight and Google Play.
- Macs with iOS simulators ready. Useful for iOS testing: Mac mini M2 and M4 machines come with current Xcode and iOS simulator runtimes preinstalled.
- Windows and Android emulators today. Useful for cross-platform apps: Codemagic offers Windows build machines, and its Linux machines run Android emulators.
Where boat fits
Buildkite and Codemagic are CI services: a job or build gets a machine, runs a script and the machine is destroyed. If what you are really running is agents that work for hours, keep their files and serve what they build, boat gives each one a full Linux VM instead. For iOS builds and store releases, keep a Mac CI service.
A machine that keeps its state
A Buildkite job ends after 8 hours and a Codemagic build after 120 minutes, and both machines are deleted. Caches save only chosen folders, so an agent reinstalls its tools every run. boat saves the whole disk every minute and on stop.
Coding agents you drive from outside
On both services an agent is a step you start and read through logs, with no way to steer it mid-run. Buildkite's key proxy covers Anthropic only, and Codemagic hands keys in as variables. boat's credentials proxy works for any HTTPS API.
A full Linux computer you can reach
Neither exposes ports, and their desktops are for a person debugging a build. boat gives root, systemd, Docker and /dev/kvm, a 60fps desktop for computer use, and HTTPS URLs on up to 50 ports plus raw TCP.
Use boat when what you are really running is agents on Linux: one full VM that keeps its disk, Docker and /dev/kvm, a 60fps desktop, URLs you can reach, and coding agents you drive from outside. Compare it one to one: boat vs Buildkite and boat vs Codemagic.
Questions people ask
Is Buildkite cheaper than Codemagic?
It depends on the job. In our agent scenarios Buildkite came out lower for 50 agents in parallel ($7,677 against $21,531) and 300 user sandboxes ($2,204 against $12,297), but Codemagic was lower for one agent running all month ($333 against $715). Neither service fits these scenarios, and boat comes to $317 and $200 for the first two.
Which is better for iOS builds?
Both run macOS on Apple silicon with Xcode. Codemagic adds iOS simulators, code signing and publishing to App Store Connect and TestFlight out of the box, while Buildkite gives you larger M4 machines inside a general pipeline you script yourself. boat runs Linux only.
Which is faster?
We have no comparable CPU measurement for either Buildkite or Codemagic. boat did 26.69 runs per second on the StarSling Node.js benchmark at 4 vCPU / 8 GB.
Can Buildkite and Codemagic run Docker and Android emulators?
Both run Docker on Linux. Codemagic runs Android emulators on Linux but not on macOS, and Buildkite's macOS agents have the Docker CLI but no daemon, with /dev/kvm not documented. On boat every machine has Docker and /dev/kvm.
How long can a job run on each?
Buildkite hosted jobs run up to 8 hours on Linux and 4 on macOS, longer on request. Codemagic builds stop within 120 minutes on Free, Pay as you go and Fixed price plans. boat has no session cap on paid plans.
Can I run AI coding agents on Buildkite or Codemagic?
Only as a step inside a job or build, on a machine that is destroyed when it ends. Buildkite's Anthropic proxy lets Claude Code call the API with the job token; on Codemagic keys arrive as environment variables. On boat coding agents are preinstalled and you prompt and steer them through an API.
Try boat
Install the CLI, then run boat new.
7-day free trial