Why Kubernetes is the wrong tool for agentic workloads

📝 摘要 #

文章通过基准测试对比了 Kubernetes 与 Daytona 沙箱在 AI 智能体工作负载下的环境启动性能。Kubernetes 的设计初衷是维护和扩缩稳定服务的副本，而 AI 智能体平台需要按需从零构建多样化的短生命周期环境（如机器人仿真、语音 Agent、用户自定义容器等）。在 500 次环境启动的测试中，Daytona 首次命令延迟中位数比 Kubernetes 快 4 倍（1.05s vs 4.17s），累计启动时间快约 3 倍（16-31 分钟 vs 47-50 分钟）。Kubernetes 的优势在于更稳定的尾部延迟（方差更小），Daytona 典型延迟更快但尾部更重。

核心概念： Kubernetes 适合稳定长期服务的运维，但不是 AI 智能体按需启动多样化短生命周期环境的正确工具——沙箱才是更合适的原语。

关键技术栈 / 关键要点：

1. 工作负载差异 — K8s 专为「已知服务的副本维护与扩缩」设计，而 AI 智能体需要「按需从零构建多种不同环境」
2. 基准测试方法 — 500 次环境启动，覆盖 1/10/100 种不同工作集，K8s 镜像预拉取（排除下载时间）
3. 中位数延迟对比 — Daytona 1.05-1.13s vs Kubernetes 4.17-4.37s，Daytona 快 4 倍
4. 累计启动时间 — Daytona 16-31 分钟 vs Kubernetes 47-50 分钟，快约 3 倍
5. 尾部延迟特征 — Daytona 尾部更重（少数启动较慢），K8s 更紧凑（4-5s 集中），方差更小
6. 运维开销 — K8s 需要 registry mirrors、custom AMIs、local image caches、lazy loading 等持续调优，每项优化都使系统更偏向特定工作负载形态

关键结论 / 实践建议： 需要可预测最坏情况延迟选 Kubernetes；需要快速启动大量不同短生命周期环境（RL 训练、Agent 评测）选沙箱。构建 AI Agent 基础设施时，不要因为熟悉 K8s 就默认使用，应基于数据选择合适的原语。

At some point, everyone building ai infra wonders whether they should use Kubernetes. There’s no straight answer for it because it depends on what you're trying to do.

Kubernetes is exceptional at what it was built for: maintaining and scaling replicas of a known service. Long-running, stable, and predictable. But modern AI platforms have a different problem.

Consider three jobs arriving at the same compute platform simultaneously:

- a robotics simulation in a custom container - a voice agent requiring a precise CUDA/Python/model-serving combination - a black-box container a customer supplied

These are three completely different environments that need to be materialized on demand from scratch. That's the kind of workload AI infra typically faces. Kubernetes was never designed for this.

We ran the benchmark with 500 total environment launches across three different working set sizes: 1, 10, and 100 distinct environments.

We gave Kubernetes every advantage possible. Images pre-pulled onto the node, no image download in the measured path, and the strongest practical comparison available.

Results:

Daytona median time-to-first-command: - 1 environment: 1.05 seconds - 10 environments: 1.13 secs - 100 environments: 1.10 secs

Kubernetes median time-to-first-command: - 1 environment: 4.17 secds - 10 environments: 4.27 secs - 100 environments: 4.37 secs

At the median, Daytona was 4x faster. In every single scenario. Even when Kubernetes had its images pre-loaded locally.

The compounding effect is where it gets interesting. Small differences per launch become consequential at scale.

Total cumulative startup time across 500 sequential launches:

1 environment working set: - Daytona: 16 mins - Kubernetes: 47 mins

10 environment working set: - Daytona: 26 mins - Kubernetes: 48 mins

100 environment working set: - Daytona: 31 minutes - Kubernetes: 50 mins

Jobs start sooner, compute sits idle less, and infra costs drop.

And this is before you factor in the operational overhead of keeping Kubernetes tuned, scaled, and ready for diverse workloads. Registry mirrors, custom AMIs, local image caches, lazy loading, and so on. The tuning path is effectively endless, and every optimization makes the system more specialized to one particular workload shape.

Honestly, Daytona has a heavier tail. Most launches are fast. A small fraction takes longer. Kubernetes is slower overall but tighter. Clustered around 4-5 seconds with less variance.

So the rule is simple: if you need predictable worst-case latency above everything else, go for Kubernetes. But if you need fast typical latency for many different short-lived environments on demand, you need sandboxes.

For RL training, agent evals, or anything where you're spinning up hundreds or thousands of distinct environments and speed compounds across every run, sandboxes win. By 4x at the median and 3x on total cumulative time.

Kubernetes is the right tool to maintain and scale stable services. It is the wrong choice for launching multiple distinct short-lived environments on demand.

If you're building AI agent infrastructure and you're still reaching for Kubernetes because it's what you know, run this benchmark yourself. Look at the data and use the right primitive for the job.

P.S. Shoutout to @lilgrav from the @daytonaio team for running this benchmark!