CircleCI vs Buildkite
Both are CI services that run each job on a fresh machine and throw it away. CircleCI bills in credits and adds GPUs, Windows and a self-hosted server edition, while Buildkite bills per user plus vCPU minutes, lets a Linux job run up to 8 hours and keeps chosen paths in cache volumes.
The companies and products
CircleCI
CircleCI is a CI and CD platform that runs your pipeline jobs on machines it hosts, or on runners you host.
You describe jobs in a config file, and each job runs on a fresh Docker container or a machine VM (Linux, Arm, Windows, macOS on Apple M4 Pro, or GPU on Scale), then the machine is discarded. Files move between jobs only through caches, workspaces and artifacts. It is a CI service, not an agent sandbox, so this page compares where they overlap: a machine where a job or an agent does its work. CircleCI also has Chunk, a toolkit that starts its own agent tasks in CircleCI Cloud from a prompt.
Buildkite
Buildkite is a CI and software delivery platform, and hosted agents are the build machines it runs for you.
You write Buildkite pipelines, and each job runs on a fresh virtual machine that Buildkite starts for that job and destroys when it ends; only cache volumes carry files to the next job. Hosted agents come in Linux and macOS on Apple M4, with Windows in private preview, and you can also run Buildkite's agent on your own machines. It is a CI service, not an agent sandbox, so this page compares the part where they overlap: a machine where an agent or a job does its work.
- Where it runs
- Hosted agents run only in US East Coast data centers
- Compliance
- Maintains a SOC 2 Type II report
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “136th cheapest of 377” means 135 of the other 376 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | CircleCI: price rank | Buildkite: price rank | boat: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month A coding agent that builds an app for hours outlives a CircleCI job, which stops after 3 hours on Performance; Buildkite allows 8 hours on Linux, and boat has no session cap on paid plans. | 333rd of 367, with limits $8,692 a month | 43rd of 367, with limits $11,648 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB An assistant that lives on its machine all month fits neither CI service, since both destroy the machine after the job; boat keeps one VM and its disk running as long as you need. | 340th of 366, with limits $13,137 a month | 351st of 366, with limits $17,564 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month | 285th of 345, with limits $572 a month | 315th of 345, with limits $1,587 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Your users' sandboxes need a URL they can open, and neither CircleCI nor Buildkite exposes ports on its jobs; boat gives each sandbox HTTPS on up to 50 ports and raw TCP. | 316th of 361, with limits $3,702 a month | 326th of 361, with limits $4,984 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month | 310th of 360, with limits $17,882 a month | 316th of 360, with limits $23,881 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 318th of 364, with limits $2,877 a month | 324th of 364, with limits $3,884 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month CI is what both are built for; CircleCI fits our CI runner scenario as priced at $4,302 a month while Buildkite comes to $2,318 as a partial match, and boat runs CI jobs on full VMs with no pipeline service. | 136th cheapest of 377 $4,302 a month | 203rd of 377, with limits $2,318 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB A dev box has to keep what you installed; CircleCI restores only cached paths, Buildkite only cache volume paths, and boat saves the whole disk every minute. | 197th of 368, with limits $508 a month | 354th of 368, with limits $715 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Buildkite has browser desktop access for a person debugging a macOS job and CircleCI runs headless, but neither has a computer-use API; boat streams a 60fps Linux desktop with computer use. | 320th of 369, with limits $2,589 a month | 70th of 369, with limits $3,500 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 116th of 367, with limits $1,293 a month | 142nd of 367, with limits $1,772 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 70th cheapest of 346 $860 a month | 142nd of 346, with limits $2,354 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 274th of 338, with limits $17,532 a month | 317th of 338, with limits $46,814 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 80th of 360, with limits $852 a month | 85th of 360, with limits $1,184 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month Only CircleCI offers GPUs, on its custom-priced Scale plan; Buildkite hosted agents and boat are CPU only. | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month Both run macOS with Xcode, CircleCI on Apple M4 Pro and Buildkite on Apple M4, at close prices; boat runs Linux only. | 34th of 36, with limits $2,262 a month | 35th of 36, with limits $2,310 a month | Can't run it |
Key features
| Feature | CircleCI | Buildkite | boat |
|---|---|---|---|
| Kind of product | CI and CD platform; jobs start from a config file on cloud executors or your own runners | CI platform; hosted agents run jobs from Buildkite pipelines, next to agents you host | Agent sandboxes you create and drive by API, CLI or SDK |
| Isolation | Containers on the Docker executor; a fresh VM per job on machine, Windows and macOS | A virtual machine per job, destroyed after the job; Apple Virtualization framework on macOS | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X (Zen 5) hosts |
| How long a job or machine runs | 1 hour on Free, 3 on Performance, 5 on Scale | Up to 8 hours on Linux and 4 on macOS, longer on request | No session cap on paid plans; you set the time to live |
| Files after a run | Machine discarded; caches, workspaces and artifacts copy chosen paths out | VM destroyed; cache volumes keep chosen paths after successful jobs, best effort, up to 14 days | The whole disk is saved every minute and on stop; a stopped sandbox costs nothing |
| Fork | No fork of a job | Each job gets its own copy of the last committed cache volume | Fork a running sandbox with its disk |
| Docker, root and nested virtualization | Docker and sudo on machine executors; nested virtualization on the Android machine image | Docker on Linux; macOS has the Docker CLI but no daemon; /dev/kvm not documented | Docker, root, systemd and /dev/kvm on every machine |
| Desktop and computer use | Headless jobs; browsers in images for tests | Browser-based desktop access to debug a running macOS job; no computer-use API | 60fps Linux desktop with Chromium, computer use and browser control, embeddable stream |
| Reaching the machine | No exposed ports; rerun a job with SSH, open 10 minutes after it ends | No exposed ports; a terminal into a running job through the UI or bk job ssh | HTTPS URLs for up to 50 ports, token-protected; raw TCP; dedicated IPv4 or IPv6 |
| Coding agents and API keys | Chunk starts CircleCI's own agent tasks; secrets reach jobs as environment variables | You install Claude Code as a step; an Anthropic proxy sets the API key header for the job | Claude Code, Codex and more preinstalled with a prompt, steer and events API; credentials proxy for any HTTPS API |
| Operating systems | Linux on x86 and Arm, Windows, macOS on Apple M4 Pro with Xcode | Linux (ARM64 on Enterprise), macOS on Apple M4 with Xcode, Windows in private preview | Linux only (x86) |
| GPUs | Nvidia GPU classes such as T4 and V100 on the Scale plan | None on hosted agents | None |
| Biggest machine | Classes from 2xlarge up are on the Scale plan only | Linux up to 8 vCPU / 32 GB on Pro and 64 vCPU / 256 GB on Enterprise; macOS up to 12 vCPU / 56 GB | Up to 16 vCPU / 32 GB on the $100 plan and above |
| Self-hosting | Self-hosted runners, and CircleCI Server on your own Kubernetes on AWS or GCP | Self-hosted agents next to hosted ones | Hosted only |
| Regions | US only: AWS us-east-1 and us-east-2, GCP us-east1 and us-central1 | US East Coast only | EU only (Germany, Finland, France), with EU data residency and zero data retention |
Every feature Battleships tracks (43)
| Feature | CircleCI | Buildkite | boat |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | Yes | No | Yes |
| Root | Not published | Not published | Yes |
| Systemd | No | No | Yes |
| Desktop GUI | No | Yes | Yes |
| Browser | Yes | Yes | Yes |
| Computer-use API | No | No | Yes |
| SSH | Yes | Yes | Yes |
| Persistent disk | No | Yes | Yes |
| Pause/resume | No | No | Yes |
| Automatic snapshots | No | Yes | Yes |
| Fork/clone | No | Yes | Yes |
| Live fork (no pause) | No | No | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | No | No | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | No | No | No |
| HTTPS preview URLs | No | No | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | No | No | No |
| Secret proxy | No | Yes | Yes |
| Secret proxy for any API | No | No | Yes |
| Preinstalled agents | No | No | Yes |
| Agent harness API | Yes | No | Yes |
| MCP server | Yes | Yes | No |
| GPU | Yes | No | No |
| Windows | Yes | Yes | No |
| Arm64 | Yes | Yes | No |
| Self-hosting | Yes | Yes | No |
| BYOC | No | No | No |
| Open source | No | No | No |
| Scoped API keys | Not published | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | Yes | No | Yes |
| Audit logs | Yes | Not published | No |
| SSO | Yes | Yes | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | No | No | No |
| EU data residency | No | No | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
We have no comparable measurement for CircleCI or Buildkite hosted agents: neither was part of the StarSling hpc-sandbox-benchmarks run. On that Node.js build-and-test benchmark at 4 vCPU / 8 GB, boat did 26.69 runs per second on dedicated AMD Ryzen 9 9950X (Zen 5) hosts.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. CircleCI and Buildkite have no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | CircleCI, a month | Buildkite, a month | boat, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $96 · Performance · with limits | $166 · Pro · with limits | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $508 · Performance · with limits | $715 · Pro · with limits | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $5,714 · Performance · with limits | $7,677 · Pro · with limits | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $1,617 · Performance · with limits | $2,204 · Pro · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $3,562 · Performance · with limits | $4,787 · Pro · with limits | $2,000 · $2000/mo · with limits |
CircleCI: CircleCI bills in credits per minute for the machine class a job uses: Performance starts at $15 a month with 30,000 credits, and more credits cost $15 per 25,000. A 4 vCPU / 15 GB Linux machine uses 20 credits a minute, about $0.72 an hour, and Scale is custom and billed annually.
Buildkite: Buildkite Pro costs $30 per active user a month and includes 4,000 Linux vCPU minutes; after that Linux is $0.004 per vCPU minute and macOS $0.02, metered to the second. A 4 vCPU / 16 GB Linux agent comes to $0.96 an hour, and Enterprise pricing is custom.
Buildkite: You pick a monthly plan from $20, and all of it becomes usage credit spent per second on running sandboxes. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
On CircleCI the job is the limit: it stops after 3 hours on Performance, and Performance runs at most 80 jobs at once. On Buildkite a Linux job stops after 8 hours, files outside cache volume paths reset, and Pro shares one pool of 256 Linux vCPU, so 1,000 short jobs at once mostly wait in the queue on both. On boat the limits are a sandbox count per plan and starts per minute, so the same burst is queued and needs more starts per day than the $2,000 plan allows.
| Scenario | CircleCI | Buildkite | boat |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use CircleCI when
you need GPUs for tests, Windows or Arm machines, macOS on Apple M4 Pro, or the whole CI platform on your own Kubernetes.
- GPUs and Windows. Useful if your tests need a GPU or Windows: the Scale plan adds Nvidia classes such as T4 and V100, and it also runs Windows VMs.
- The whole platform on your own cluster. Useful if CI must run inside your cloud: CircleCI Server installs the full platform on your own Kubernetes on AWS or GCP.
- Credits you can start small with. Useful for small teams: Performance starts at $15 a month with 30,000 credits, and five active users are included.
Use Buildkite when
your Linux jobs run longer than 3 to 5 hours, need machines larger than 8 vCPU, or you want per-user plus per-minute billing with your own agents alongside hosted ones.
- Longer jobs. Useful for long builds: a Linux job can run up to 8 hours, against 3 on CircleCI Performance and 5 on Scale.
- Bigger Linux machines. Useful for heavy builds: Pro reaches 8 vCPU and 32 GB, and Enterprise Linux shapes go up to 64 vCPU and 256 GB.
- Simple per-minute metering. Useful if you want clear compute bills: Linux is $0.004 per vCPU minute, metered to the second with no minimums.
Where boat fits
CircleCI and Buildkite are CI services, and boat is not one: it has no pipelines. If what you are really running is AI agents that work for hours or days, a preview your users open, or a desktop an agent clicks through, both CI services throw the machine away after each job, and boat keeps one full Linux VM going.
A machine that keeps its state
A CircleCI job stops after 3 hours on Performance and a Buildkite job after 8 on Linux, and both discard the machine. The workaround is caches or cache volumes, so the agent rebuilds the rest every run; boat saves the whole disk every minute.
Agents you drive from outside
On both services an agent is a job step you read through logs, with no way to steer it mid-run. Buildkite's key proxy covers Anthropic only and CircleCI passes secrets as environment variables; boat's proxy covers any HTTPS API.
A computer you can reach and see
Neither exposes ports, and neither has a desktop an agent can drive; Buildkite's desktop is for a person debugging a macOS job. boat gives HTTPS on up to 50 ports, raw TCP, /dev/kvm and a 60fps desktop with computer use.
Use boat when what you are really running is AI agents on Linux for hours or days: one full VM that keeps its disk, Docker and /dev/kvm, a 60fps desktop, URLs you can reach, and coding agents you drive from outside. Compare it one to one: boat vs CircleCI and boat vs Buildkite.
Questions people ask
Is CircleCI cheaper than Buildkite?
It depends on the work. For our CI runner scenario Buildkite comes to $2,318 a month against $4,302 on CircleCI, though Buildkite is only a partial match there. For machines that run like agents, CircleCI comes out lower: one agent on workdays is $96 a month on CircleCI and $166 on Buildkite, but neither fits as priced.
Which is faster, CircleCI or Buildkite?
We have no comparable CPU measurement for either: neither was in the StarSling hpc-sandbox-benchmarks run. boat did 26.69 runs per second on that benchmark at 4 vCPU / 8 GB.
How long can a job run on CircleCI and Buildkite?
CircleCI allows 1 hour on Free, 3 hours on Performance and 5 hours on Scale. Buildkite hosted agents allow up to 8 hours on Linux and 4 on macOS, longer on request. boat has no session cap on paid plans.
Can CircleCI and Buildkite run Docker?
Yes. CircleCI runs Docker with sudo on machine executors and documents nested virtualization on its Android image. Buildkite runs Docker on Linux, but macOS agents have only the Docker CLI with no daemon. boat has Docker, root, systemd and /dev/kvm on every machine.
Do CircleCI and Buildkite run macOS?
Both do, with Xcode: CircleCI on Apple M4 Pro and Buildkite on Apple M4, up to 12 vCPU and 56 GB on Buildkite. boat runs Linux only.
Can I run Claude Code on CircleCI or Buildkite?
Yes, as a job step on either: you install it in your image or step. Buildkite's Anthropic proxy lets it call the API with the job token, and on CircleCI you pass the key as a secret. On boat Claude Code is preinstalled, and you prompt and steer it through an API while it works.
Try boat
Install the CLI, then run boat new.
7-day free trial