Inference batching Knowledge graph concept · also referred to as batching, batch size, batch Mentioned in Write-up of Reiner Pope's Lecture: How GPT, Claude, and Gemini Are Actually Trained and Served post · 2026-07-14 Related concepts AI inference pricing 1 KV cache 1 LLM inference 1 Memory bandwidth 1 Mixture-of-experts model 1 Roofline model 1 Speculative decoding 1