CircleCI vs Blacksmith
CircleCI is a whole CI platform: you write its own config, and it runs pipelines, caches and test splitting on machines it hosts or you host. Blacksmith keeps your GitHub Actions workflows and runs each job on its own faster microVMs, after you change one runner label.
The companies and products
CircleCI
CircleCI is a CI and CD platform that runs your pipeline jobs on machines it hosts, or on runners you host.
You describe jobs in a config file, and each job runs on a fresh Docker container or a machine VM (Linux, Arm, Windows, macOS on Apple M4 Pro, or GPU on Scale), then the machine is discarded. Files move between jobs only through caches, workspaces and artifacts. It is a CI service, not an agent sandbox, so this page compares where they overlap: a machine where a job or an agent does its work. CircleCI also has Chunk, a toolkit that starts its own agent tasks in CircleCI Cloud from a prompt.
Blacksmith
Blacksmith is a CI service: it runs your GitHub Actions jobs on its own faster, cheaper runners instead of GitHub's.
You change the runner label in your workflow file, and each job runs in a fresh Firecracker microVM that boots in under 3 seconds and is destroyed when the job ends. It adds Docker layer caching and sticky disks that carry chosen folders from one run to the next, plus a Testbox CLI in early beta that lets a local coding agent run commands inside a CI job. It is built for teams that want quicker, cheaper GitHub Actions, not as a general sandbox you drive by API; a separate Sandboxes product is marked coming soon.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “136th cheapest of 377” means 135 of the other 376 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | CircleCI: price rank | Blacksmith: price rank | boat: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month An agent building a whole app needs one machine for hours; a CircleCI job stops after 3 hours on Performance and a Blacksmith machine ends with its job. | 333rd of 367, with limits $8,692 a month | 248th of 367, with limits $3,607 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB An assistant that lives on its machine all month does not fit either CI service; boat has no session cap on paid plans. | 340th of 366, with limits $13,137 a month | 78th of 366, with limits $5,463 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month | 285th of 345, with limits $572 a month | 281st of 345, with limits $488 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Your users' sandboxes need to start from your backend and expose a URL; neither CI service exposes ports, while boat gives HTTPS on up to 50 ports. | 316th of 361, with limits $3,702 a month | 268th of 361, with limits $1,563 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month | 310th of 360, with limits $17,882 a month | 276th of 360, with limits $7,488 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 318th of 364, with limits $2,877 a month | 354th of 364, with limits $1,188 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month Both are built for CI: CircleCI if you want its own pipeline platform, Blacksmith if your CI already lives in GitHub Actions; boat runs CI jobs on full VMs but has no pipeline service. | 136th cheapest of 377 $4,302 a month | 78th cheapest of 377 $708 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB A dev box has to stay up and keep what you installed; CircleCI keeps only cached paths and Blacksmith only sticky disk folders, while boat saves the whole disk. | 197th of 368, with limits $508 a month | 180th of 368, with limits $207 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Both CI services run headless jobs with no desktop to drive; boat streams a 60fps Linux desktop with Chromium and computer use. | 320th of 369, with limits $2,589 a month | 243rd of 369, with limits $1,088 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 116th of 367, with limits $1,293 a month | 131st of 367, with limits $588 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 70th cheapest of 346 $860 a month | 130th of 346, with limits $719 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 274th of 338, with limits $17,532 a month | 262nd of 338, with limits $14,613 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 80th of 360, with limits $852 a month | 64th of 360, with limits $350 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month Only CircleCI has GPUs, on its custom-priced Scale plan, so none of the three is priced here; Blacksmith and boat have no GPUs. | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month Both run macOS, CircleCI on Apple M4 Pro with Xcode and Blacksmith on Apple Silicon, while boat runs Linux only. | 34th of 36, with limits $2,262 a month | 28th of 36, with limits $1,508 a month | Can't run it |
Key features
| Feature | CircleCI | Blacksmith | boat |
|---|---|---|---|
| What it is | A full CI and CD platform; jobs start from pipelines in its own config file | Drop-in runners for GitHub Actions; jobs start from your GitHub workflows | A full VM per agent that you create, stop and start by API, CLI or SDK |
| Isolation | Containers on the Docker executor; a fresh VM per job on machine, Windows and macOS executors | A Firecracker microVM per job, destroyed with its state when the job ends | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X (Zen 5) hosts |
| How long a machine runs | 1 hour per job on Free, 3 on Performance, 5 on Scale | Until the job ends; Testbox stops after 30 idle minutes by default | No session cap on paid plans; you set the time to live |
| Files after a run | The machine is discarded; caches, workspaces and artifacts copy chosen paths out | Only sticky disk folders, up to 5 per job, evicted after 7 days unused | The whole disk, saved every minute and on stop; a stopped sandbox costs nothing |
| Fork | No fork of a job | No; every machine starts fresh from a job | Fork a running sandbox with its disk |
| Docker and nested virtualization | Docker and sudo on machine executors; nested virtualization documented on the Android machine image | Docker inside; nested virtualization on x64 Linux runners, not on ARM | Docker, root, systemd and /dev/kvm on every machine |
| Operating systems | Linux on x86 and Arm, Windows, and macOS on Apple M4 Pro with Xcode | Ubuntu on x64 and ARM, Windows Server 2025 (beta), macOS on Apple Silicon | Linux only (x86) |
| GPUs | Nvidia T4, V100 and other GPU classes on the Scale plan | None | None |
| Jobs at once | 30 on Free, 80 on Performance (15 for macOS), unlimited on Scale | Docs say Blacksmith imposes no concurrency limits | A sandbox count per plan, with starts per minute queued above 90 |
| Reaching the machine | No exposed ports; rerun a job with SSH, open for 10 minutes after it ends | Opt-in debug SSH for the user who triggered the job; no ports or URLs | HTTPS URLs for up to 50 ports, token-protected; raw TCP; dedicated IP |
| Desktop and computer use | Headless jobs; no desktop or computer-use API | Headless runners | 60fps Linux desktop with Chromium, computer use and browser control |
| Coding agents | Chunk starts CircleCI's own agent tasks from a prompt and returns a run ID | Its own [code]smith agent from Slack, GitHub or a web UI; Testbox CLI (early beta) for local agents | Claude Code, Codex and more preinstalled, with a prompt, steer and events API |
| Keeping API keys out | Contexts inject secrets as environment variables; OIDC tokens for cloud logins | GitHub Actions secrets and OIDC tokens available inside the job | Credentials proxy for model providers and any HTTPS API; switch a credential to another account when credits run out |
| Regions and self-hosting | US only (AWS and GCP); self-hosted runners and CircleCI Server on your own Kubernetes | US east, US west, Amsterdam and Frankfurt; no self-hosting | EU only (Germany, Finland, France), with EU data residency |
Every feature Battleships tracks (43)
| Feature | CircleCI | Blacksmith | boat |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | Yes | Yes | Yes |
| Root | Not published | Yes | Yes |
| Systemd | No | No | Yes |
| Desktop GUI | No | No | Yes |
| Browser | Yes | Yes | Yes |
| Computer-use API | No | No | Yes |
| SSH | Yes | Yes | Yes |
| Persistent disk | No | Yes | Yes |
| Pause/resume | No | No | Yes |
| Automatic snapshots | No | No | Yes |
| Fork/clone | No | No | Yes |
| Live fork (no pause) | No | No | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | No | Yes |
| ≥24 h sessions | No | Yes | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | No | No | No |
| HTTPS preview URLs | No | No | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | No | No | No |
| Secret proxy | No | No | Yes |
| Secret proxy for any API | No | No | Yes |
| Preinstalled agents | No | No | Yes |
| Agent harness API | Yes | No | Yes |
| MCP server | Yes | No | No |
| GPU | Yes | No | No |
| Windows | Yes | Yes | No |
| Arm64 | Yes | Yes | No |
| Self-hosting | Yes | No | No |
| BYOC | No | No | No |
| Open source | No | No | No |
| Scoped API keys | Not published | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | Yes | Yes | Yes |
| Audit logs | Yes | Not published | No |
| SSO | Yes | No | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | No | No | No |
| EU data residency | No | Yes | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
We have no comparable measurement for CircleCI or Blacksmith: neither was part of the StarSling hpc-sandbox-benchmarks run. On that Node.js benchmark at 4 vCPU / 8 GB, boat did 26.69 runs per second on dedicated AMD Ryzen 9 9950X (Zen 5) hosts. Blacksmith says its runners use fast gaming CPUs, but there is no number measured the same way to set beside it.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. CircleCI and Blacksmith have no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | CircleCI, a month | Blacksmith, a month | boat, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $96 · Performance · with limits | $36 · Pay-as-you-go · with limits | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $508 · Performance · with limits | $207 · Pay-as-you-go · with limits | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $5,714 · Performance · with limits | $2,364 · Pay-as-you-go · with limits | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $1,617 · Performance · with limits | $663 · Pay-as-you-go · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $3,562 · Performance · with limits | $1,488 · Pay-as-you-go · with limits | $2,000 · $2000/mo · with limits |
CircleCI: CircleCI bills in credits per minute for the machine class a job uses: Performance starts at $15 a month with 30,000 credits, and more credits cost $15 per 25,000. Seats are paid from the same credits, and Scale is custom and billed annually.
Blacksmith: Blacksmith bills per minute of job time, scaled by vCPU and OS: Ubuntu x64 is $0.004 a minute for 2 vCPU, ARM $0.0025, Windows $0.008 and macOS $0.08 for 6 vCPU, with 3,000 free minutes a month. Sticky disks cost $0.50 per GB a month.
Blacksmith: Plans start at $20 a month, and the whole fee is usage credit spent per second on running sandboxes. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
None of the agent scenarios fit either CI service as priced, because both are CI runners. On CircleCI the job stops after 3 hours on Performance and the machine is discarded, and Performance runs at most 80 jobs at once, so 300 user sandboxes and 1,000 short jobs wait in a queue. On Blacksmith only sticky disk folders survive between runs, so packages installed anywhere else are gone; its prices here use ARM runners, the cheapest shape. boat fits every scenario except the 1,000-job burst, where starts above 90 a minute are queued.
| Scenario | CircleCI | Blacksmith | boat |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use CircleCI when
you want a CI platform of its own with pipelines and test splitting, need to host CI yourself, or need GPUs or macOS on Apple M4 Pro.
- A CI platform of its own. Useful if you want pipelines, caches, test splitting and orbs that do not depend on GitHub Actions.
- Self-hosting and GPUs. Useful if you must run CI on your own Kubernetes with CircleCI Server, or need Nvidia GPU classes on Scale.
- macOS on Apple M4 Pro. Useful for iOS and Mac builds: it runs macOS VMs on Apple M4 Pro with many Xcode versions, even on Free.
Use Blacksmith when
your CI already runs on GitHub Actions and you want faster jobs by changing one label, with no concurrency limit and US or EU regions.
- A one-label switch. If your CI already lives in GitHub Actions, you keep your workflows and change the runner label.
- No concurrency limit. Its docs say it imposes no concurrency limits, so a big test matrix can start all at once.
- US and EU regions. It runs in Ashburn, Phoenix, Amsterdam and Frankfurt, with Ubuntu x64 runners up to 32 vCPU and 128 GB.
Where boat fits
CircleCI and Blacksmith both run CI jobs: a machine starts for one job and is thrown away when it ends. If what you are really running is agents that work for hours, keep their files, host what they build or need a screen, a CI job is the wrong shape on both. boat gives each agent its own full VM that you start by API and keep as long as the work lasts.
A machine that stays and keeps its disk
A CircleCI job stops after 3 hours on Performance, and a Blacksmith machine is destroyed when its job ends. Both make the agent copy chosen folders into caches or sticky disks and rebuild everything else next run.
A way in: URLs and a desktop
Neither CI service exposes ports; the only way in is a short SSH session for the person who started the job. boat gives HTTPS on up to 50 ports, raw TCP and a 60fps desktop for computer use.
Agents you drive from outside
On both, an agent is a job step you read through logs, and secrets sit inside the job as environment variables. boat ships coding agents with a prompt, steer and events API, and a credentials proxy keeps real keys out.
Use boat when what you are running is agents on Linux: one full VM that keeps its disk, Docker and /dev/kvm, URLs you can reach, a 60fps desktop and coding agents you drive from outside. Compare it one to one: boat vs CircleCI and boat vs Blacksmith.
Questions people ask
Is CircleCI cheaper than Blacksmith?
In what we priced, Blacksmith comes out lower: for a CI runner workload it is $708 a month against $4,302 on CircleCI, and one always-on 4 vCPU machine is $207 against $508. Blacksmith's estimates use ARM runners, its cheapest shape.
Which is faster, CircleCI or Blacksmith?
We have no comparable CPU measurement for either one, so we cannot say. Blacksmith says its runners use fast gaming CPUs. boat did 26.69 runs per second on the StarSling Node.js benchmark at 4 vCPU / 8 GB.
Can CircleCI and Blacksmith run Docker?
Yes. CircleCI has a Docker executor and runs Docker with sudo on its machine executors, and Blacksmith runs Docker inside its microVMs with a managed Docker layer cache. boat runs Docker, root, systemd and /dev/kvm on every machine.
How long can a job run on CircleCI and Blacksmith?
A CircleCI job stops after 1 hour on Free, 3 on Performance and 5 on Scale. A Blacksmith machine lives until its job ends and is then destroyed. boat has no session cap on paid plans.
Do files survive between runs?
On CircleCI only what you save to caches, workspaces and artifacts. On Blacksmith only sticky disk folders, up to 5 per job, evicted after 7 days unused. boat saves the whole disk every minute and on stop.
Can I run an AI agent on CircleCI or Blacksmith?
As a job step, yes, until the job ends. CircleCI's Chunk starts its own agent tasks, and Blacksmith's Testbox CLI, in early beta, lets a local agent run commands inside a CI job. On boat, coding agents come preinstalled and you prompt and steer them through an API.
Try boat
Install the CLI, then run boat new.
7-day free trial