Agentic Kernels in Production
TL;DR: We’ve built an agentic kernel development framework that identifies model-level optimization opportunities, generates improved kernel...
Topic
#SGLang
6 篇笔记
SemiAnalysis 发布 AgentX InferenceXv3 研报，Mooncake 成为关键基础设施
SemiAnalysis 发布 AgentX InferenceXv3，用大规模真实 Agentic workload 重新审视推理系统性能。随着长上下文、多轮交互和 KV Cache 复用成为常态，Agentic Inference 正从“拼算力”走向系统级优化，而 Mooncake 也正在其中扮演越来越重要的基础设施角色。
Profiling LLM Inference with SGLang and Torch Profiler
This blog is my attempt to learn how to profile an llm served using sglang and its in built torch profiler integration that comes out of box...
LLM Inference: Past, Now, and Future — Where Does the Value Move?
LLM Inference — Past, Now, and Future: Where Does the Value Move?For the past two years, LLM inference has been one of the most contested la...
LLM Inference: Past, Now, and Future — Where Does the Value Move?
LLM Inference — Past, Now, and Future: Where Does the Value Move?For the past two years, LLM inference has been one of the most contested la...
AI INFRA 全景分析与展望
AI INFRA 全景分析与展望 深度调研日期：2026-06-24 | 来源：多源 Web 搜索 + 对抗验证（24 个搜索代理，15+ 源站，70+ 可验证 claims） 📐 全景架构定位图 ---|------| | 核心创新 | PagedAttention — OS 式虚拟内存分页管理 KV Cache，将显存碎片率从 40% 降至 ⚠️ 注意：vLLM 上游无法提供 day-0 R…