AI Chip Architectures
A look at AI Chip Architectures. NVIDIA, AMD, TPUs, Trainium, Groq, Cerebras.
Topic
#multi-token-prediction
3 篇笔记
Accelerating Gemini Nano models on Pixel with frozen Multi-Token Prediction
📝 摘要 Google Research 发布了一种将 Multi-Token Prediction (MTP) 增层改造（retrofit）到已冻结的 Gemini Nano v3 生产模型上的方法，在不修改基座模型权重的前提下，大幅加速 Pixel 设备上的端侧 LLM 推理。该方法已在 Pixel 9 和 10 系列实机部署，使 AI Notification Summaries、Proo…
Optimizing distributed AI inference: Advanced deployment patterns
Learn about the three optimization levers for distributed AI inference: prefill/decode disaggregation, KV cache strategy, and speculative decoding.