Blacksmith vs Ubicloud
Blacksmith only does CI: it runs your GitHub Actions jobs on fresh microVMs that are destroyed when each job ends, on Linux, Windows and macOS. Ubicloud is a whole open source cloud that also sells cheap Linux runners, next to plain VMs you keep, managed Postgres, Kubernetes and GPUs.
The companies and products
Blacksmith
Blacksmith is a CI service: it runs your GitHub Actions jobs on its own faster, cheaper runners instead of GitHub's.
You change the runner label in your workflow file, and each job runs in a fresh Firecracker microVM that boots in under 3 seconds and is destroyed when the job ends. It adds Docker layer caching and sticky disks that carry chosen folders from one run to the next, plus a Testbox CLI in early beta that lets a local coding agent run commands inside a CI job. It is built for teams that want quicker, cheaper GitHub Actions, not as a general sandbox you drive by API; a separate Sandboxes product is marked coming soon.
Ubicloud
Ubicloud is an open source cloud that runs VMs, managed Postgres, Kubernetes and GitHub Actions runners on leased bare-metal servers.
You rent Linux VMs (Standard with dedicated CPU, or cheaper Burstable with shared CPU) from Ubicloud's managed service, or run the same AGPL code on your own servers. Around the VMs it offers private subnets, an inbound firewall, load balancers, managed PostgreSQL, Kubernetes, GPU VMs, inference endpoints and cheap GitHub Actions runners. It is general infrastructure, not an agent sandbox, so this page compares the two where they overlap: giving an agent its own Linux machine.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “78th cheapest of 377” means 77 of the other 376 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Blacksmith: price rank | Ubicloud: price rank | boat: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month | 248th of 367, with limits $3,607 a month | 126th of 367, with limits $1,136 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB An assistant that lives on its machine all month needs a VM that stays up, which Ubicloud gives as a plain VM you set up yourself, boat gives ready to work, and a Blacksmith job does not. | 78th of 366, with limits $5,463 a month | 154th of 366, with limits $1,139 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month Ubicloud fits short bursts better than Blacksmith or boat here: its burstable VMs bill per minute with no plan fee. | 281st of 345, with limits $488 a month | 59th of 345, with limits $52 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Giving each of your users a sandbox means starting machines from your own backend: Ubicloud does it by API within a vCPU quota, boat by API with agents built in, and Blacksmith only through GitHub workflows. | 268th of 361, with limits $1,563 a month | 53rd of 361, with limits $390 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month | 276th of 360, with limits $7,488 a month | 234th of 360, with limits $1,560 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 354th of 364, with limits $1,188 a month | 31st cheapest of 364 $624 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month Both sell GitHub Actions runners and both fit; Ubicloud ranks 28th and Blacksmith 78th on price, Blacksmith adds macOS, Windows and a Docker layer cache, and boat ranks 4th for CI you drive by API. | 78th cheapest of 377 $708 a month | 28th cheapest of 377 $300 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB A Ubicloud VM can be a 24/7 dev box with dedicated CPU, while a Blacksmith machine is destroyed when its job ends; boat keeps the disk and snapshots it every minute. | 180th of 368, with limits $207 a month | 67th cheapest of 368 $91 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Computer-use agents need a screen; Blacksmith runners are headless, a Ubicloud VM needs you to install a desktop yourself, and boat streams a 60fps Linux desktop. | 243rd of 369, with limits $1,088 a month | 132nd of 369, with limits $312 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 131st of 367, with limits $588 a month | 188th of 367, with limits $156 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 130th of 346, with limits $719 a month | 165th of 346, with limits $78 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 262nd of 338, with limits $14,613 a month | 60th of 338, with limits $1,573 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 64th of 360, with limits $350 a month | 129th of 360, with limits $81 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month Only Ubicloud fits, with Nvidia GPU VMs enabled on request; Blacksmith and boat have no GPUs. | Can't run it | 93rd of 98, with limits $795 a month | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month Only Blacksmith fits here, with macOS runners on Apple Silicon; Ubicloud and boat run Linux only. | 28th of 36, with limits $1,508 a month | Can't run it | Can't run it |
Key features
| Feature | Blacksmith | Ubicloud | boat |
|---|---|---|---|
| What it is | Drop-in GitHub Actions runners; a machine lives for one workflow job | An open source cloud: VMs, managed Postgres, Kubernetes, load balancers, GPU VMs and GitHub Actions runners | A full VM per agent that you create, stop, start and fork by API |
| Isolation | Firecracker microVM per job with its own kernel, destroyed after the job | Full KVM guest under Cloud Hypervisor, each VMM in its own Linux namespaces | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts |
| How a machine starts | A GitHub Actions job; Testbox warmup dispatches a workflow | API, CLI or web console for VMs; GitHub Actions jobs for runners | One API call, SDK or CLI, from anywhere |
| Operating systems | Ubuntu x64 and ARM, Windows Server 2025 (beta), macOS on Apple Silicon | Linux on x64, plus arm64 CI runners | Linux (x64) |
| What survives between runs | Only sticky disk folders, up to 5 per job, evicted after 7 days unused | A VM keeps its whole disk while it exists; machine images of a stopped VM by hand | The whole disk, saved every minute and on stop |
| Stopping | The microVM is destroyed when the job ends | A stopped VM stays reserved and keeps billing as if it were running | A stopped sandbox costs nothing and keeps its disk |
| Fork | No; every machine starts fresh from a job | Stop the VM, capture a machine image (single disk up to 40 GiB), boot new VMs from it | Fork a running sandbox in one call (disk, no memory) |
| Docker and nested virtualization | Docker in jobs; nested virtualization on x64 Linux runners, not on ARM | Root over SSH, so Docker installs as on any Linux VM; nested virtualization not documented | Docker, root, systemd and /dev/kvm on every machine |
| Reaching the machine | Opt-in debug SSH for the user who started the job; no ports or URLs | Free public IPv6, public IPv4 for $3 a month; HTTPS means running TLS on the VM yourself | HTTPS URLs for up to 50 ports, token-protected; raw TCP; dedicated IP |
| Desktop and computer use | Headless runners | None; you install a desktop and browser yourself | 60fps Linux desktop with Chromium and computer use |
| Coding agents | Its own [code]smith agent from Slack, GitHub or a web UI; Testbox CLI (early beta) for local agents | None; you install and drive agents yourself, for example over SSH | Seven agents preinstalled, with a prompt, steer and events API |
| Keeping API keys out of the machine | GitHub Actions secrets and OIDC tokens inside the job | No credentials proxy; secrets live on the VM | Credentials proxy for model providers and any HTTPS API; switch to another account when credits run out |
| GPUs and self-hosting | No GPUs; managed service only | Nvidia A100, B200 and RTX PRO 6000 VMs on request; AGPL code you can run yourself | No GPUs; managed service only |
| Regions | Ashburn, Phoenix, Amsterdam and Frankfurt | Falkenstein (Germany), Helsinki (Finland) and Manassas (Virginia), plus GPUs in Istanbul | EU only: Germany, Finland, France |
Every feature Battleships tracks (43)
| Feature | Blacksmith | Ubicloud | boat |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | Yes | Not published | Yes |
| Root | Yes | Yes | Yes |
| Systemd | No | Not published | Yes |
| Desktop GUI | No | No | Yes |
| Browser | Yes | No | Yes |
| Computer-use API | No | No | Yes |
| SSH | Yes | Yes | Yes |
| Persistent disk | Yes | Yes | Yes |
| Pause/resume | No | No | Yes |
| Automatic snapshots | No | No | Yes |
| Fork/clone | No | Yes | Yes |
| Live fork (no pause) | No | No | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | No | Yes | Yes |
| ≥24 h sessions | Yes | Yes | Yes |
| Idle auto-stop | Yes | No | No |
| Wake on request | No | No | No |
| HTTPS preview URLs | No | No | Yes |
| Raw TCP inbound | No | Not published | Yes |
| Public IPv4 | No | Yes | No |
| Egress allowlist | No | No | No |
| Secret proxy | No | No | Yes |
| Secret proxy for any API | No | No | Yes |
| Preinstalled agents | No | No | Yes |
| Agent harness API | No | No | Yes |
| MCP server | No | No | No |
| GPU | No | Yes | No |
| Windows | Yes | No | No |
| Arm64 | Yes | Yes | No |
| Self-hosting | No | Yes | No |
| BYOC | No | Yes | No |
| Open source | No | Yes | No |
| Scoped API keys | Not published | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | Yes | No | Yes |
| Audit logs | Not published | Yes | No |
| SSO | No | Yes | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | No | No | No |
| EU data residency | Yes | Yes | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
There is no comparable measurement for Blacksmith or Ubicloud: neither was part of the same CPU benchmark run. boat did 26.69 runs per second on the StarSling hpc-sandbox-benchmarks Node.js loop at 4 vCPU / 8 GB, on dedicated AMD Ryzen 9 9950X hosts. Blacksmith says its runners use fast gaming CPUs, and Ubicloud's speed depends on the family: Standard VMs have dedicated CPU, Burstable VMs get half of each vCPU with short bursts.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. Blacksmith and Ubicloud have no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Blacksmith, a month | Ubicloud, a month | boat, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $36 · Pay-as-you-go · with limits | $22 · Usage | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $207 · Pay-as-you-go · with limits | $91 · Usage | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $2,364 · Pay-as-you-go · with limits | $1,098 · Usage | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $663 · Pay-as-you-go · with limits | $156 · Usage | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $1,488 · Pay-as-you-go · with limits | $312 · Usage · with limits | $2,000 · $2000/mo · with limits |
Blacksmith: Blacksmith bills per minute of job time, scaled by vCPU and OS: Ubuntu x64 is $0.004 a minute for 2 vCPU, ARM $0.0025, Windows $0.008 and macOS $0.08 for 6 vCPU, with 3,000 free minutes a month. Sticky disks cost $0.50 per GB a month and a static IP $100 a month.
Ubicloud: Ubicloud bills runners per minute of job time and VMs per minute for as long as they exist, running or stopped; a VM's public IPv4 adds $3 a month, and runners include one. Every account gets a $2.50 monthly credit that Ubicloud says equals 1,250 minutes of runner time.
Ubicloud: Plans start at $20 a month, and the whole fee is usage credit spent per second on running sandboxes. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
None of the agent scenarios fit Blacksmith cleanly, because a machine only exists while a GitHub Actions job runs and only sticky disk folders survive; its prices here use ARM runners. Ubicloud fits most of them as plain VMs, but limits the total vCPU a project can run: 32 for a new project and 256 once it has paid an invoice over $5, with more from support. Its cheap burstable shape guarantees only half of each vCPU, so the 1,000-job burst gets throttled, and on boat that burst is queued at 90 starts a minute.
| Scenario | Blacksmith | Ubicloud | boat |
|---|---|---|---|
| One coding agent, workdays |
| None that bite | None that bite |
| One agent that never stops |
| None that bite | None that bite |
| 50 agents in parallel |
| None that bite | None that bite |
| Sandboxes for your users |
| None that bite | None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Blacksmith when
your CI lives in GitHub Actions and you want faster jobs by changing one label, with a Docker layer cache, no concurrency limit, or macOS and Windows runners.
- macOS and Windows runners. Teams that build iOS, macOS or Windows apps can run those jobs on Apple Silicon and Windows Server runners.
- Faster CI by changing one label. If your CI lives in GitHub Actions, you change a runner label and get a Docker layer cache and sticky disks.
- No concurrency limit. Its docs say it imposes no concurrency limits, so a big test matrix can start all at once.
Use Ubicloud when
you want cheap Linux runners and long-lived VMs in one open source cloud you could also run yourself, with managed Postgres, Kubernetes, GPU VMs or a US region.
- Cheaper Linux runners. For plain Linux CI, Ubicloud's x64 and arm64 runners come out cheaper here, with a public IPv4 included.
- VMs that stay, next to the rest of a cloud. Useful when CI sits beside long-lived VMs, managed Postgres, Kubernetes or GPU VMs in one account.
- Open source you can run yourself. The whole stack is AGPL code, so you can run the same cloud on your own leased servers.
Where boat fits
Neither Blacksmith nor Ubicloud is an agent sandbox: one runs CI jobs, the other rents plain VMs. If what you are really running is agents, boat gives each one a full VM that you start by API, that keeps its disk, hosts what it builds, shows a desktop and comes with coding agents you drive from outside.
A machine that stays, and stops for free
A Blacksmith microVM is destroyed after each job, keeping only sticky disk folders. A Ubicloud VM stays, but bills while stopped. boat saves the whole disk every minute, a stopped sandbox costs nothing, and you can fork a running one.
Reach in: HTTPS ports and a desktop
Blacksmith runners have no ports, only debug SSH, and a Ubicloud VM needs you to run TLS and install a desktop yourself. boat serves up to 50 ports on HTTPS URLs, raw TCP, and a 60fps desktop for computer use.
Agents driven from outside, keys kept out
Blacksmith machines start from GitHub workflows, and a Ubicloud VM is an empty OS where agents read any key you put on it. boat has seven coding agents preinstalled, a prompt and steer API, and a credentials proxy for any HTTPS API.
Use boat when what you are running is agents: each one on its own full VM that you start by API, that keeps its disk with free stops, hosts what it builds, shows a desktop and runs coding agents you drive from outside. Compare it one to one: boat vs Blacksmith and boat vs Ubicloud.
Questions people ask
Is Blacksmith cheaper than Ubicloud?
For the CI runner workload in our table, no: it comes to $708 a month on Blacksmith against $300 on Ubicloud. Blacksmith gives 3,000 free minutes a month, and Ubicloud a $2.50 runner credit. Blacksmith also covers macOS and Windows, which Ubicloud does not offer.
Which is faster, Blacksmith or Ubicloud?
We have no comparable CPU measurement for either. Blacksmith says its runners use fast gaming CPUs; Ubicloud Standard VMs have dedicated CPU and Burstable VMs get half of each vCPU. boat measured 26.69 runs per second at 4 vCPU / 8 GB.
Can Blacksmith and Ubicloud run Docker?
Yes. Blacksmith jobs run Docker and nested virtualization on x64 Linux runners, not on ARM, and its Windows runners cannot run Linux containers. Ubicloud's runners ship Docker, and a Ubicloud VM gives you root to install it; nested virtualization is not documented.
Does either keep files between runs?
Blacksmith keeps only sticky disks: up to 5 folders per job, evicted after 7 days unused. A Ubicloud VM keeps its whole disk while it exists, but a stopped VM keeps billing as if it were running. boat saves a sandbox's whole disk every minute, and a stopped sandbox costs nothing.
Can I run AI agents on Blacksmith or Ubicloud?
Partly. Blacksmith's Testbox CLI, in early beta, lets a local coding agent run commands inside a CI job, and its Sandboxes product is marked coming soon. On Ubicloud you install agents on a plain VM and drive them over SSH. boat comes with coding agents, a desktop and a credentials proxy in every sandbox.
Is Ubicloud open source?
Yes. Ubicloud's code is under the AGPL v3, and you can run it yourself on leased bare-metal servers or use its managed service. Blacksmith and boat are managed services only.
Try boat
Install the CLI, then run boat new.
7-day free trial