AI Chip Architectures
A look at AI Chip Architectures. NVIDIA, AMD, TPUs, Trainium, Groq, Cerebras.
Topic
#MoE
11 篇笔记
LLM Inference: Past, Now, and Future — Where Does the Value Move?
LLM Inference — Past, Now, and Future: Where Does the Value Move?For the past two years, LLM inference has been one of the most contested la...
LLM Inference: Past, Now, and Future — Where Does the Value Move?
LLM Inference — Past, Now, and Future: Where Does the Value Move?For the past two years, LLM inference has been one of the most contested la...
22580: From GPT2 to Kimi3, Explained
Twenty-two thousand five hundred and eighty. That’s how many GPT-2 (2019) models fit inside KimiK3 (2026). We scaled up by a factor of 22,58...
Kimi K3 Is Here: Efficient Day-0 Support on vLLM
vLLM delivers day-0 Kimi K3 serving with hybrid KDA prefix caching, DSpark speculative decoding, production-scale disaggregation, and optimized kernels across N
DSpark 深度解读——置信度调度投机解码与半自回归生成
DSpark：置信度调度投机解码与半自回归生成 DeepSpec 开源仓库: deepseek-ai/DeepSpec — 包含 Eagle3、DFlash、DSpark 的算法驱动训练代码与 checkpoints DSpark 是 DeepSeek-AI 提出的投机解码框架，核心创新： - 半自回归生成：并行 backbone + 轻量 sequential head，兼得并行首 token…
Optimizing distributed AI inference: Advanced deployment patterns
Learn about the three optimization levers for distributed AI inference: prefill/decode disaggregation, KV cache strategy, and speculative decoding.
Designing distributed AI inference: Core concepts and scaling dimensions
Learn about the five-dimensional design space in modern LLM serving, including tensor, pipeline, expert, data, and context parallelism.
Qwen Studio
Qwen Studio offers comprehensive functionality spanning chatbot, image and video understanding, image generation, document processing, web search integration, tool utilization, and artifacts.
本地大模型：从跑起来到跑得好
0. 太长不读很多人玩本地大模型，卡住的地方不是模型太复杂，而是不知道自己卡在哪一层。你以为是在跑一个模型，其实是在同时做五个决定：硬件、内存、Runtime、模型选择、量化。每一层都有坑，每一层都可以单独优化，但它们又互相约束。后面所有问题，基本都能归到这五层里。两条读法：如果...
NanoWhale 项目深度解析：从零学习 DeepSeek-V4 架构
NanoWhale 项目深度解析：从零学习 DeepSeek-V4 架构 一句话总结：NanoWhale 是 HuggingFace 发布的一个教学项目，它用约 1.1 亿参数（110M）实现了一个"迷你版" DeepSeek-V4 大语言模型，从零开始训练，让你能用一张 H100 显卡就跑通完整的大模型训练流程。 一、这个项目在做什么？ 大模型训练的一般流程 训练一个大语言模型（LLM）通常分为…