CircleCI vs GitLab.com hosted runners
CircleCI is a CI platform of its own, with jobs that run up to 1, 3 or 5 hours depending on the plan. GitLab.com hosted runners are the machines built into GitLab that run a GitLab project's pipelines, with every job cut off at 3 hours.
The companies and products
CircleCI
CircleCI is a CI and CD platform that runs your pipeline jobs on machines it hosts, or on runners you host.
You describe jobs in a config file, and each job runs on a fresh Docker container or a machine VM (Linux, Arm, Windows, macOS on Apple M4 Pro, or GPU on Scale), then the machine is discarded. Files move between jobs only through caches, workspaces and artifacts. It is a CI service, not an agent sandbox, so this page compares where they overlap: a machine where a job or an agent does its work. CircleCI also has Chunk, a toolkit that starts its own agent tasks in CircleCI Cloud from a prompt.
GitLab.com hosted runners
GitLab.com hosted runners are the machines that run GitLab CI/CD jobs for projects on GitLab.com, paid for in compute minutes.
This is a CI service, not a sandbox API. Each job gets a new VM on Google Cloud (Linux and Windows) or on AWS Mac hardware (macOS), with passwordless sudo, and the VM is deleted right after the job. It is built for building, testing and deploying the code in a GitLab project, and GitLab's Duo Agent Platform also runs Claude Code and Codex agents as CI/CD jobs on these runners.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “136th cheapest of 377” means 135 of the other 376 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | CircleCI: price rank | GitLab.com hosted runners: price rank | boat: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month A coding agent that builds a whole app needs one machine for hours with its files in place; CircleCI stops a job after 3 hours on Performance, GitLab after 3 hours on any plan, and both discard the machine. | 333rd of 367, with limits $8,692 a month | 347th of 367, with limits $14,454 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB An assistant that lives on its machine all month fits neither CI service, since jobs end within 3 to 5 hours and start clean; boat has no session cap on paid plans. | 340th of 366, with limits $13,137 a month | 353rd of 366, with limits $21,858 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month | 285th of 345, with limits $572 a month | 320th of 345, with limits $1,896 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Neither CI service lets your users open a URL on a job, while boat gives each sandbox HTTPS on up to 50 ports and raw TCP. | 316th of 361, with limits $3,702 a month | 331st of 361, with limits $6,133 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month | 310th of 360, with limits $17,882 a month | 318th of 360, with limits $29,762 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 318th of 364, with limits $2,877 a month | 330th of 364, with limits $4,758 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month Both are built for CI: CircleCI as a standalone pipeline service and GitLab's runners as the native choice for GitLab projects, while boat runs CI jobs on full VMs with Docker and /dev/kvm but has no pipeline service. | 136th cheapest of 377 $4,302 a month | 128th cheapest of 377 $2,089 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB A dev box has to stay up and keep what you installed; both services start each job from an image, with only cached paths restored. | 197th of 368, with limits $508 a month | 358th of 368, with limits $805 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month CircleCI and GitLab's runners both run headless jobs with no desktop to drive; boat streams a 60fps Linux desktop with Chromium and computer use. | 320th of 369, with limits $2,589 a month | 342nd of 369, with limits $4,278 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month | 116th of 367, with limits $1,293 a month | 144th of 367, with limits $2,118 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 70th cheapest of 346 $860 a month | 143rd of 346, with limits $2,854 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 274th of 338, with limits $17,532 a month | 319th of 338, with limits $58,429 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 80th of 360, with limits $852 a month | 87th of 360, with limits $1,383 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month None of the three is priced here: CircleCI's GPU classes are on its custom-priced Scale plan, GitLab has a Linux runner with an NVIDIA T4 on Premium and Ultimate, and boat has no GPUs. | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month Both run macOS, CircleCI on Apple M4 Pro even on the Free plan one job at a time, GitLab on Apple silicon in beta for Premium and Ultimate; boat runs Linux only. | 34th of 36, with limits $2,262 a month | 24th of 36, with limits $1,069 a month | Can't run it |
Key features
| Feature | CircleCI | GitLab.com hosted runners | boat |
|---|---|---|---|
| Kind of product | Standalone CI and CD platform; jobs start from a config file | CI runners built into GitLab; jobs run from a project's .gitlab-ci.yml | Agent sandboxes you create, stop, fork and reach by API, CLI or SDK |
| Isolation | Containers on the Docker executor; a fresh VM per job on machine, Windows and macOS executors | A new VM per job on Google Compute Engine, deleted right after; Linux jobs run in a privileged Docker container | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X (Zen 5) hosts, with root and systemd |
| How long a job or machine runs | 1 hour per job on Free, 3 on Performance, 5 on Scale | 3 hours per job on every plan, whatever the project setting | No session cap on paid plans; you set the time to live |
| Files after a run | The machine is discarded; chosen paths carry over through caches, workspaces and artifacts | The VM is deleted; files carry over through a shared cache (5 GB per archive, removed after 14 days unused) or artifacts | The whole disk is saved every minute and on stop; a stopped sandbox costs nothing |
| Fork | No fork of a job | No; every job starts from a fresh VM and a container image | Fork a running sandbox with its disk |
| Docker and nested virtualization | Docker and sudo on machine executors; nested virtualization documented on the Android machine image | Docker-in-Docker in privileged mode with passwordless sudo; nested virtualization not documented | Docker, root, systemd and /dev/kvm on every machine |
| Getting into a running job | Rerun a job with SSH, open for 10 minutes after it ends, for the user who started it; no exposed ports | None; inbound traffic is blocked and instance runners have no interactive web terminal | SSH, HTTPS URLs for up to 50 ports, raw TCP, dedicated IPv4 or IPv6 |
| Desktop and computer use | Headless jobs; browsers in images for tests | Headless; macOS runners do not support jobs that need UI interaction | 60fps Linux desktop with Chromium, computer use and browser control |
| Coding agents | Chunk starts CircleCI's own agent tasks from a prompt; other agents you install in a job yourself | Duo Agent Platform runs Claude Code, Codex, Amazon Q and Gemini agents from issues and merge requests, on Premium and Ultimate | Claude Code, Codex and more preinstalled, driven from outside with a prompt, steer and events API |
| Keeping API keys out | Contexts inject secrets as environment variables; OIDC tokens for cloud logins | GitLab-managed credentials for its Claude and Codex agents only; other secrets enter as CI/CD variables | Credentials proxy for model providers and any HTTPS API; switch a credential to another account when credits run out |
| Operating systems | Linux on x86 and Arm, Windows, and macOS on Apple M4 Pro with Xcode | Linux x64 and Arm64, Windows (beta), macOS on Apple silicon (beta, Premium and Ultimate) | Linux only (x86) |
| GPUs | Nvidia T4, V100 and other GPU classes on the Scale plan | A Linux runner with an NVIDIA T4 on Premium and Ultimate | None |
| Self-hosting | Self-hosted runners, and CircleCI Server on your own Kubernetes on AWS or GCP | Open-source GitLab Runner (MIT) on your own machines, with no compute minutes used | No |
| Regions | US only: AWS us-east-1 and us-east-2, GCP us-east1 and us-central1 | US (Google Cloud us-east1 by default for Linux) | EU only (Germany, Finland, France), with EU data residency |
Every feature Battleships tracks (43)
| Feature | CircleCI | GitLab.com hosted runners | boat |
|---|---|---|---|
| Full VM (own kernel) | Yes | Yes | Yes |
| Docker inside | Yes | Yes | Yes |
| Nested virtualization | Yes | No | Yes |
| Root | Not published | Yes | Yes |
| Systemd | No | No | Yes |
| Desktop GUI | No | No | Yes |
| Browser | Yes | Yes | Yes |
| Computer-use API | No | No | Yes |
| SSH | Yes | No | Yes |
| Persistent disk | No | No | Yes |
| Pause/resume | No | No | Yes |
| Automatic snapshots | No | No | Yes |
| Fork/clone | No | No | Yes |
| Live fork (no pause) | No | No | Yes |
| Memory fork | No | No | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | No | No | Yes |
| Idle auto-stop | Yes | Yes | No |
| Wake on request | No | No | No |
| HTTPS preview URLs | No | No | Yes |
| Raw TCP inbound | No | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | No | No | No |
| Secret proxy | No | Yes | Yes |
| Secret proxy for any API | No | No | Yes |
| Preinstalled agents | No | Yes | Yes |
| Agent harness API | Yes | Yes | Yes |
| MCP server | Yes | No | No |
| GPU | Yes | Yes | No |
| Windows | Yes | Yes | No |
| Arm64 | Yes | Yes | No |
| Self-hosting | Yes | Yes | No |
| BYOC | No | No | No |
| Open source | No | Yes | No |
| Scoped API keys | Not published | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | Yes | No | Yes |
| Audit logs | Yes | Yes | No |
| SSO | Yes | Yes | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | No | No | No |
| EU data residency | No | No | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
We have no comparable measurement for CircleCI or GitLab.com hosted runners: neither was part of the StarSling hpc-sandbox-benchmarks run. On that Node.js build-and-test benchmark at 4 vCPU / 8 GB, boat did 26.69 runs per second on dedicated AMD Ryzen 9 9950X (Zen 5) hosts.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. CircleCI and GitLab.com hosted runners have no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | CircleCI, a month | GitLab.com hosted runners, a month | boat, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $96 · Performance · with limits | $119 · Premium · with limits | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $508 · Performance · with limits | $805 · Premium · with limits | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $5,714 · Performance · with limits | $9,491 · Premium · with limits | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $1,617 · Performance · with limits | $2,658 · Premium · with limits | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $3,562 · Performance · with limits | $5,896 · Premium · with limits | $2,000 · $2000/mo · with limits |
CircleCI: CircleCI bills credits per minute for the machine class a job uses: Performance starts at $15 a month with 30,000 credits, and more credits cost $15 per 25,000. A 4 vCPU / 15 GB Linux machine uses 20 credits a minute, and Scale is custom and billed annually.
GitLab.com hosted runners: GitLab counts compute minutes as run time times a cost factor, 1 for small Linux and 2 for medium, up to 12 for the biggest machines. Premium is $29 per user a month billed yearly with 10,000 minutes per group, and extra minutes cost $10 per 1,000, bought ahead of time.
GitLab.com hosted runners: Plans start at $20 a month, and the whole fee is usage credit spent per second on running sandboxes. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
The limit that bites on both is the job itself: CircleCI stops it after 3 hours on Performance and GitLab after 3 hours on every plan, and neither keeps the disk, so the workday agent, the agent that never stops and the 50-agent fleet do not fit as priced. CircleCI Performance runs at most 80 jobs at once, and GitLab publishes no limit on runner VMs at once, so 300 user sandboxes and 1,000 short jobs are not covered on either. GitLab's included minutes are a hard quota: when they run out, new jobs stop until you buy more. On boat, the 1,000-job burst is queued above 90 starts a minute and needs more starts per day than the $2,000 plan allows.
| Scenario | CircleCI | GitLab.com hosted runners | boat |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use CircleCI when
you want CI as its own service, need macOS on Apple M4 Pro, jobs longer than 3 hours on Scale, GPU classes such as the V100, or the whole platform on your own Kubernetes.
- macOS on Apple M4 Pro, even on Free. For iOS and Mac builds, CircleCI runs macOS on Apple M4 Pro with Xcode, one job at a time even on Free.
- Longer jobs on Scale. If a job needs more than 3 hours, CircleCI's Scale plan allows 5, where GitLab's hosted runners stop every job at 3.
- A CI service for any team. Teams that want CI as its own service get pipelines, caches, test splitting, orbs, and CircleCI Server on their own Kubernetes.
Use GitLab.com hosted runners when
your code lives on GitLab.com, you want pipelines with no extra service, runners you host for free of minutes, or Claude Code and Codex working on your issues.
- Native CI for GitLab projects. If your code lives on GitLab.com, hosted runners are on by default and run pipelines with no extra service to wire up.
- Your own runners, free of minutes. The open-source GitLab Runner runs on your own machines, and those jobs use none of your compute minutes.
- Agents on issues and merge requests. On Premium and Ultimate, Claude Code and Codex can work on GitLab issues and merge requests as CI jobs.
Where boat fits
CircleCI and GitLab's runners are CI services, and for pipelines either one is a sound choice. If what you are really running is agents that work for hours on one machine, both make you squeeze that work into a short job on a machine that is thrown away. boat gives each agent a full Linux VM that keeps its disk, runs as long as the work does and can be reached from outside.
A machine that stays
A CircleCI job stops at 3 hours on Performance and a GitLab job at 3 hours on any plan, and both throw the machine away. The agent then reinstalls and reclones each time; boat saves the whole disk every minute and has no session cap on paid plans.
Reach it from outside
GitLab blocks all inbound traffic and CircleCI only allows an SSH rerun for the person who started it, so a preview server or webhook needs a tunnel inside the job. boat gives HTTPS URLs on up to 50 ports, raw TCP and a 60fps desktop.
Agents you drive, keys kept out
Chunk runs CircleCI's own agent and GitLab's Duo agents start from issues and merge requests, while other secrets sit in the job as variables. boat lets you drive agents inside from outside through one API, with real keys held in a credentials proxy.
Use boat when your agents work on Linux for hours or days: one full VM that keeps its disk, Docker and /dev/kvm, a 60fps desktop, URLs you can reach, and coding agents you drive from outside. Compare it one to one: boat vs CircleCI and boat vs GitLab.com hosted runners.
Questions people ask
Is CircleCI cheaper than GitLab.com hosted runners?
In our CI runners use case CircleCI comes to $4,302 a month and GitLab's hosted runners to $2,089, so GitLab is cheaper there. In the agent scenarios CircleCI is cheaper, for example $96 against $119 for one coding agent on workdays, but neither fits, because jobs stop after 3 hours.
Which is faster, CircleCI or GitLab runners?
We have no comparable CPU measurement for either one. boat did 26.69 runs per second on the StarSling Node.js benchmark at 4 vCPU / 8 GB.
How long can a job run on CircleCI and on GitLab?
CircleCI allows 1 hour on Free, 3 hours on Performance and 5 hours on Scale. GitLab.com hosted runners stop every job at 3 hours, whatever the project sets. A boat sandbox has no session cap on paid plans.
Can I SSH into a CircleCI or GitLab job?
On CircleCI you can rerun a job with SSH, and it stays open for 10 minutes after the job ends, for the user who started it. GitLab blocks inbound traffic to hosted runners and has no web terminal on them. You can SSH into a boat sandbox and open HTTPS URLs on up to 50 ports.
Can I run Claude Code on CircleCI or GitLab?
On CircleCI you install it as a step in a job and pass your key as a secret. On GitLab Premium and Ultimate, the Duo Agent Platform runs Claude Code and Codex as CI jobs from issues and merge requests. On boat, Claude Code is preinstalled and you prompt and steer it through an API.
Do CircleCI and GitLab run macOS?
Yes. CircleCI runs macOS on Apple M4 Pro with Xcode, even on the Free plan one job at a time. GitLab has macOS runners on Apple silicon in beta for Premium and Ultimate. boat runs Linux only.
Try boat
Install the CLI, then run boat new.
7-day free trial