LLM inference

Knowledge graph concept · also referred to as serving, decode, decode step, speed of inference, LM Studio, llama.cpp, vLLM, inference workload, run models, output tok/s, aggregate throughput, local LLM inference, LLM serving, local inference, autoregressive generation, serving an LLM, local models, large language model serving, large language model, model serving, LLM performance

Mentioned in

• Upper Bounds for Local Model Throughput X thread · 2026-07-17
• Write-up of Reiner Pope's Lecture: How GPT, Claude, and Gemini Are Actually Trained and Served post · 2026-07-14
• Theoretical Upper Bounds for LLM Performance post · 2026-07-14
• Open source AI needs cheap local speed X thread · 2026-07-01
• Local models for real-time triage X thread · 2026-06-23
• The localening is here X thread · 2026-06-17
• Towards 1-click setup for local models in OpenClaw X thread · 2026-05-15
• Agentic Engineering needs rigor, not just intuition X thread · 2026-02-28

Related concepts

• Local model deployment 4
• Inference throughput 3
• KV cache 3
• Local language model 3
• Memory bandwidth 3
• Speculative decoding 3
• AI coding agent 2
• AI inference pricing 2