Google Agent Runtime vs Upstash Box
Google Agent Runtime hosts the agent code you build with ADK or LangGraph inside your Google Cloud project, with locked sandboxes for code, shell and browser work. Upstash Box gives each agent its own container with a coding agent built in, billed only for the CPU it uses and paused when idle.
The companies and products
Google Agent Runtime
Google Agent Runtime is the agent hosting part of Gemini Enterprise Agent Platform on Google Cloud, renamed from Vertex AI Agent Engine in April 2026.
It is a hyperscaler service, not a sandbox company. Agent Runtime deploys agents you build with ADK, LangGraph, LangChain, AG2 or LlamaIndex and bills them only while they work on a turn. Next to it, Agent Sandbox gives those agents a place to run code: a Code Execution sandbox, a shell or custom container sandbox, and a Computer Use browser. All of them run as secure containers inside your Google Cloud project.
Upstash Box
Upstash Box is a cloud computer for AI agents from Upstash, the company known for serverless Redis, Vector and QStash.
Each box is a Docker container with its own filesystem, shell, git and an optional coding agent such as Claude Code, Codex or OpenCode, which you drive from TypeScript or Python SDKs. A box pauses when it sits idle, keeps its files, and you pay for the CPU it actually uses. Upstash built it to run its own Context7 service and opened it to everyone in March 2026.
Which use cases each one fits
For each use case, Battleships prices every sandbox provider on the same workload and sorts them from the cheapest monthly bill up. “28th cheapest of 366” means 27 of the other 365 providers run that workload, with every feature it needs, for less. “With limits” means the provider runs it but misses something, so it is listed after every full match. Open a use case for its whole ranking and what each provider misses.
| Use case and its workload | Google Agent Runtime: price rank | Upstash Box: price rank | boat: price rank |
|---|---|---|---|
| Software factory 100 at once · 4 vCPU / 8 GB · 2,200 sessions of 8 h a month Neither runs Docker inside: Google's sandboxes run as an unprivileged user and an Upstash box is itself a container, while boat runs Docker and systemd on every VM. | 295th of 367, with limits $5,801 a month | 61st of 367, with limits $464 a month | 3rd cheapest of 367 $634 a month |
| Always-on AI employees 50 machines 24/7 · 2 vCPU / 4 GB Google fits fully but deletes a sandbox when its TTL ends (7 days at most for a custom container), Upstash keeps a keep-alive box on all month, and boat has no session cap on paid plans. | 28th cheapest of 366 $7,519 a month | 103rd of 366, with limits $475 a month | 5th cheapest of 366 $657 a month |
| Code interpreter bursts 100 at once · 1 vCPU / 2 GB · 200,000 sessions of 1 min a month Both suit short code runs: Google's Code Execution sandbox is a ready Python interpreter, and an idle Upstash box costs no CPU. | 254th of 345, with limits $275 a month | 10th cheapest of 345 $119 a month | 186th of 345, with limits $2,000 a month |
| Sandboxes for your users 300 at once · 2 vCPU / 4 GB · 30,000 sessions of 23 min a month Upstash wakes a user's paused box on the next request to its URL, Google offers no public URL, and boat gives each user a VM that serves HTTPS on up to 50 ports. | 280th of 361, with limits $1,916 a month | 2nd cheapest of 361 $222 a month | Cheapest of 361 $209 a month |
| RL rollouts 1,000 at once · 2 vCPU / 4 GB · 1,000,000 sessions of 3 min a month | 128th of 360, with limits $8,240 a month | 352nd of 360, with limits $8,068 a month | 34th of 360, with limits $2,000 a month |
| Agent evals 250 at once · 4 vCPU / 8 GB · 15,000 sessions of 20 min a month | 79th cheapest of 364 $2,060 a month | 52nd cheapest of 364 $1,011 a month | 10th cheapest of 364 $200 a month |
| CI runners 40 at once · 8 vCPU / 16 GB · 6,000 sessions of 12 min a month CI jobs that build containers need Docker, which neither Google's sandboxes nor Upstash boxes offer; boat runs Docker on every VM. | 98th cheapest of 377 $989 a month | 120th cheapest of 377 $1,520 a month | 4th cheapest of 377 $86 a month |
| 24/7 dev boxes 1 machine 24/7 · 4 vCPU / 8 GB Upstash has SSH through a gateway and a flat-price keep-alive box, while Google has no SSH and each command runs in a new shell; boat gives you SSH and a desktop. | 311th of 368, with limits $241 a month | 10th cheapest of 368 $20 a month | 18th cheapest of 368 $26 a month |
| Computer-use agents 50 at once · 4 vCPU / 8 GB · 20,000 sessions of 15 min a month Google gives a browser with a VNC view and a click and type API, Upstash a headless Chromium with no desktop, and boat a full 60fps Linux desktop. | 283rd of 369, with limits $1,648 a month | 73rd of 369, with limits $76 a month | 6th cheapest of 369 $200 a month |
| Browser-use agents 50 at once · 2 vCPU / 4 GB · 30,000 sessions of 10 min a month Both let an agent drive Chromium over CDP; Google keeps outbound internet off unless the template turns it on, and Upstash adds AI page actions. | 102nd of 367, with limits $824 a month | 49th of 367, with limits $38 a month | 7th cheapest of 367 $200 a month |
| Scraping & bots (startup scale) 50 at once · 1 vCPU / 2 GB · 100,000 sessions of 3 min a month | 53rd cheapest of 346 $515 a month | 37th cheapest of 346 $400 a month | 6th cheapest of 346 $90 a month |
| Mass scraping & bots (undetectable) 1,000 at once · 1 vCPU / 2 GB · 3,000,000 sessions of 2 min a month | 210th of 338, with limits $8,240 a month | 200th of 338, with limits $8,000 a month | 71st of 338, with limits $1,800 a month |
| Parallel red teaming 200 at once · 2 vCPU / 4 GB · 5,000 sessions of 30 min a month | 260th of 360, with limits $412 a month | 225th of 360, with limits $259 a month | 2nd cheapest of 360 $200 a month |
| GPU eval jobs 10 at once · 8 vCPU / 32 GB · 500 sessions of 30 min a month | Can't run it | Can't run it | Can't run it |
| macOS agents 5 at once · 4 vCPU / 8 GB · 1,000 sessions of 20 min a month | Can't run it | Can't run it | Can't run it |
Key features
| Feature | Google Agent Runtime | Upstash Box | boat |
|---|---|---|---|
| Isolation | Secure container sandboxing inside your Google Cloud project | One Docker container per box on AWS | Full VM with its own kernel on dedicated AMD Ryzen 9 9950X hosts |
| Root and Docker inside | Runs as appuser with no sudo; images must not need root; no Docker | Not documented; the box is itself a container, custom images coming soon | Root, systemd, Docker and /dev/kvm on every machine |
| Shell access | execute_bash() API calls, each in a new shell; no SSH | Shell through the SDK, plus SSH through a gateway | SSH, plus an API for commands |
| Hosting your agent code | Deploys ADK, LangGraph, LangChain, AG2 and LlamaIndex agents; idle time between turns is not billed | You run your agent or a built-in coding agent inside the box | You run your own agent loop inside a sandbox |
| Coding agents | Managed Agents API (Preview) runs Google's Antigravity harness | Claude Code, Codex and OpenCode built in, with run and stream APIs, webhooks and scheduled prompts | Claude Code, Codex, OpenCode and more preinstalled, with a prompt, steer and events API |
| Desktop and computer use | A browser with a VNC live view, a click, type and screenshot API, and CDP | Managed headless Chromium with AI page actions and CDP; no desktop | 60fps Linux desktop and Chromium, computer-use tools, embeddable stream |
| Files after a stop | Pause keeps files at a lower, unpublished rate; deleted when the TTL ends | Pauses itself when idle and keeps its files; storage keeps billing | Disk saved every minute and on stop; a stopped sandbox costs nothing |
| Snapshots and branching | Snapshots keep memory and files; start many sandboxes from one | Snapshot a running box's files and start many new boxes from it | Fork a running sandbox with its disk; resume reboots from disk |
| How long a sandbox lives | TTL of up to 7 days for a custom container, 14 days for Code Execution | No cap documented; keep-alive boxes stay on all month | No session cap on paid plans |
| Reaching the sandbox | Through Google's API or a private endpoint in your VPC; no public URL documented | HTTPS URL per port, public unless you add a token; a request wakes a paused box; no raw TCP | HTTPS URLs for up to 50 ports, token-protected by default; raw TCP; own IP address |
| Outbound network rules | Off unless the template turns it on; route through your VPC with Cloud NAT, a static IP or Cloud NGFW | Allow all, deny all, or allow and deny by domain and CIDR, changeable on a running box | Open internet on every plan; no provider-side allowlist |
| Keeping keys out of the sandbox | You provision and scope the credentials the agent uses | Host proxy injects headers into outbound HTTPS; set when the box is created | Credentials proxy for model providers and any HTTPS API; switch to another account when credits run out |
| Machine size | Runtime up to 8 vCPU and 32 GiB; sandbox maximum not published | Up to 8 vCPU / 16 GB, with a 5, 10 or 20 GB disk | Up to 16 vCPU / 32 GB from the $100 plan; 50 GB disk at the default size |
| Regions | 23 locations in the US, Europe, Asia, Canada, Australia and Brazil | AWS us-east-1 only | EU only: Germany, Finland, France |
Every feature Battleships tracks (43)
| Feature | Google Agent Runtime | Upstash Box | boat |
|---|---|---|---|
| Full VM (own kernel) | No | No | Yes |
| Docker inside | No | No | Yes |
| Nested virtualization | No | No | Yes |
| Root | No | Not published | Yes |
| Systemd | Not published | Not published | Yes |
| Desktop GUI | No | No | Yes |
| Browser | Yes | Yes | Yes |
| Computer-use API | Yes | No | Yes |
| SSH | No | Yes | Yes |
| Persistent disk | Yes | Yes | Yes |
| Pause/resume | Yes | Yes | Yes |
| Automatic snapshots | Yes | No | Yes |
| Fork/clone | Yes | Yes | Yes |
| Live fork (no pause) | Yes | Yes | Yes |
| Memory fork | Yes | No | No |
| Custom image (Docker or snapshot) | Yes | Yes | Yes |
| ≥24 h sessions | Yes | Yes | Yes |
| Idle auto-stop | No | Yes | No |
| Wake on request | No | Yes | No |
| HTTPS preview URLs | No | Yes | Yes |
| Raw TCP inbound | Not published | No | Yes |
| Public IPv4 | No | No | No |
| Egress allowlist | Yes | Yes | No |
| Secret proxy | No | Yes | Yes |
| Secret proxy for any API | No | Yes | Yes |
| Preinstalled agents | Yes | Yes | Yes |
| Agent harness API | Yes | Yes | Yes |
| MCP server | No | Yes | No |
| GPU | No | No | No |
| Windows | No | No | No |
| Arm64 | Yes | Yes | No |
| Self-hosting | No | No | No |
| BYOC | No | No | No |
| Open source | No | No | No |
| Scoped API keys | Not published | Not published | Yes |
| Spend limits | Not published | Not published | Yes |
| Usage per sandbox (API) | Yes | Yes | Yes |
| Audit logs | Not published | Not published | No |
| SSO | Yes | No | No |
| SOC 2 | Yes | Yes | No |
| HIPAA | Yes | No | No |
| EU data residency | Yes | No | Yes |
| Zero data retention | Not published | Not published | Yes |
Performance
Neither Google Agent Runtime nor Upstash Box has a result in the StarSling hpc-sandbox-benchmarks CPU test, so there is no comparable measurement between them. boat did 26.69 Node.js build-and-test runs per second at 4 vCPU / 8 GB in that run of 2026-09-29, on dedicated AMD Ryzen 9 9950X hosts.
Node.js build-and-test loop at 4 vCPU / 8 GB, higher is better. Measured with the open hpc-sandbox-benchmarks suite. Google Agent Runtime and Upstash Box have no published run in this suite yet, so there is no like-for-like number to compare.
Pricing in five scenarios
Each bill is priced from the providers' public price lists by the open Battleships engine, with the cheapest plan that runs the scenario.
| Scenario | Google Agent Runtime, a month | Upstash Box, a month | boat, a month |
|---|---|---|---|
| One coding agent, workdays 4 vCPU / 8 GB, 50 GB disk, 8 hours a day, 22 days a month, files kept between days | $73 · Pay as you go | $21 · Pay as You Go | $20 · $20/mo |
| One agent that never stops 4 vCPU / 8 GB, 50 GB disk, running 24/7 all month | $301 · Pay as you go | $21 · Pay as You Go | $26 · $20/mo |
| 50 agents in parallel 50 machines of 4 vCPU / 8 GB at once, 8-hour sessions, 1,100 sessions a month, Docker inside | $3,626 · Pay as you go · with limits | $900 · Pay as You Go · with limits | $317 · $20/mo × 3 seats |
| Sandboxes for your users 2 vCPU / 4 GB, 30,000 sessions of 10 minutes a month, up to 300 at once | $1,030 · Pay as you go | $306 · Pay as You Go | $200 · $100/mo × 2 seats |
| 1,000 short jobs at once 2 vCPU / 4 GB, 1,000 at once, 3 minutes each, 200,000 jobs a month | $2,060 · Pay as you go | $1,822 · Pay as You Go | $2,000 · $2000/mo · with limits |
Google Agent Runtime: Google bills $0.085 per vCPU-hour and $0.009 per GiB-hour for what you allocate, with the first 50 vCPU-hours and 100 GiB-hours free each month. Sandboxes are billed while they exist, and disk, egress and paused rates are not published.
Upstash Box: Upstash bills only the CPU a box actually uses, at $0.10, $0.20 or $0.40 per active CPU hour for Small, Medium and Large, plus $0.10 per GB-month of storage, and RAM is free. A keep-alive box costs a flat $8, $16 or $32 a month.
Upstash Box: Plans start at $20 a month, and the whole fee becomes usage credit spent per second while a sandbox runs. A stopped sandbox costs nothing and keeps its disk.
Limits in those scenarios
The 50-agent fleet fits neither Google nor Upstash, because neither runs Docker inside a sandbox. Google leaves disk and egress unpriced in every scenario and deletes a sandbox when its TTL ends, while Upstash's egress price is not published and its keep-alive boxes bill every month whether used or not. On boat the 1,000-job burst is the problem: starts are capped at 90 a minute and 2,400 a day, so the jobs get queued.
| Scenario | Google Agent Runtime | Upstash Box | boat |
|---|---|---|---|
| One coding agent, workdays |
|
| None that bite |
| One agent that never stops |
|
| None that bite |
| 50 agents in parallel |
|
| None that bite |
| Sandboxes for your users |
|
| None that bite |
| 1,000 short jobs at once |
|
|
|
Which one should you use?
Use Google Agent Runtime when
you build agents with ADK or LangGraph on Google Cloud and want them hosted inside your own project, network rules and regions, with short code runs in a locked sandbox.
- A managed home for your agent code. Useful if you build agents with ADK or LangGraph: Agent Runtime hosts them and skips billing for idle time between turns.
- Inside your Google Cloud network. Useful if agents must stay in a Google Cloud perimeter, with egress off by default and routed through your own firewall.
- Memory snapshots in 23 locations. Useful if you need to branch a sandbox with its memory, or run close to users in Asia, Canada, Australia or Brazil.
Use Upstash Box when
your agents mostly wait, should wake on an incoming request, run a built-in coding agent, and can live in a container in the US.
- Idle costs no CPU. Good for agents that wait a lot: you pay only for CPU used, RAM is free, and a request wakes a paused box.
- Coding agents ready in a box. Useful if you want Claude Code, Codex or OpenCode running in a box with webhooks and scheduled prompts.
- Public URLs and SSH. Useful when a person needs to open what an agent built, or SSH in, without setting up a VPC endpoint.
Where boat fits
Both Google's sandboxes and Upstash boxes are containers that cannot run Docker, so they suit agents that only run code and call APIs. If your agents install packages with sudo, run docker compose, boot an emulator or use a desktop, look at boat: every sandbox is a full VM that keeps its disk.
Root, Docker and a real kernel
Google runs commands as appuser with no sudo, and an Upstash box is itself a container, so neither runs Docker. That is why the 50-agent scenario fits neither; boat runs Docker, systemd and /dev/kvm on every VM.
A desktop the agent can see
Google offers only a browser with a VNC view, and Upstash a headless Chromium with no desktop, so desktop apps and native dialogs are out of reach. boat streams a full Linux desktop at 60fps with computer-use tools.
A machine that keeps going
Google deletes a sandbox when its TTL ends, 7 days at most for a custom container, and Upstash disks stop at 20 GB. boat has no session cap on paid plans, a 50 GB default disk saved every minute, and free stops.
Use boat when your agents need a real computer: root, Docker, SSH, a desktop and public URLs on a full VM that keeps its disk, with Claude Code, Codex and other coding agents built in. Compare it one to one: boat vs Google Agent Runtime and boat vs Upstash Box.
Questions people ask
Is Google Agent Runtime cheaper than Upstash Box?
No, Upstash costs less in all five scenarios we priced. One coding agent on workdays comes to $73 a month on Google and $21 on Upstash, and one agent running all month to $301 against $21. Sandboxes for your users come to $1,030 against $306; boat comes to $20, $26 and $200 for those three.
Which is faster, Google Agent Runtime or Upstash Box?
We cannot say: neither has a result in the StarSling CPU benchmark. boat did 26.69 Node.js build-and-test runs per second at 4 vCPU / 8 GB in that test.
Can Google Agent Runtime or Upstash Box run Docker?
No. Google's shell sandbox runs as the unprivileged user appuser with no sudo, and an Upstash box is itself a Docker container with no Docker offered inside. On boat every sandbox is a VM with root, Docker and /dev/kvm.
Can I open a web app an agent built?
On Upstash, yes: each port gets an HTTPS URL, and a request wakes a paused box. Google's docs describe reaching a sandbox through its API or a private VPC endpoint, not a public URL. boat serves HTTPS on up to 50 ports and accepts raw TCP.
Where do they run?
Google runs in 23 locations across the US, Europe, Asia, Canada, Australia and Brazil. Upstash Box runs only in AWS us-east-1. boat runs only in the EU, in Germany, Finland and France.
Is boat SOC 2 compliant?
SOC 2 is in progress. We're open about where we stand: live status at status.ascii.dev and our data retention policy at docs.boat.dev/data-retention.
Try boat
Install the CLI, then run boat new.
7-day free trial
More comparisons
- boat vs Google Agent Runtime
- boat vs Upstash Box
- E2B vs Google Agent Runtime
- Daytona vs Google Agent Runtime
- Modal vs Google Agent Runtime
- Vercel Sandbox vs Google Agent Runtime
- Cloudflare Sandbox vs Google Agent Runtime
- Blaxel vs Google Agent Runtime
- Runloop vs Google Agent Runtime
- Tensorlake vs Google Agent Runtime
- Morph Cloud vs Google Agent Runtime
- CodeSandbox SDK vs Google Agent Runtime