Most LLM tutorials teach you to call model.generate(). Most production tutorials teach you to run vllm serve. Very few explain what happens between those two commands. Yet that gap is exactly where the serious operational problems in LLM serving live: slow time-to-first-token, low GPU utilisation, KV-cache pressure, long-context failures, batching inefficiency, quality regressions, and cost-per-token surprises. Practical LLMOps Deep Dive is a first-principles guide to the machinery behind production LLM systems. It is not mainly about prompt engineering, chatbot demos, or calling a closed-source API. It is about what happens after a request arrives and before tokens reach the user. You will learn how production LLM systems actually work underneath: tokenisation, prefill, decode, KV cache layout, batching, scheduling, quantisation, speculative decoding, deployment, observability, cost, and evaluation. What you will learn Instead of memorising configuration flags, you will understand why those flags exist and what production serving engines are doing underneath: 1. The serving engineTokenisation, prefill, decode, sampling, stop conditions, and the inference loop that sits at the centre of every LLM service. 2. Attention and the KV cacheStandard attention, why caching works, KV-cache memory formulas, fragmentation, PagedAttention, prefix caching, and FlashAttention. 3. Batching and schedulingWhy continuous batching changes the economics of serving, how chunked prefill protects inter-token latency, and how admission control prevents memory collapse. 4. Quantisation and compressionFP32, FP16, BF16, FP8, INT8, INT4, post-training quantisation, calibration, GPTQ, AWQ, and a symmetric INT8 quantiser built from scratch. 5. Speculative decodingDraft-model speculation, Medusa-style decoding, EAGLE-style decoding, acceptance rate, rollback, and when speculation is worth the operational complexity. 6. Deployment at scaleData parallelism, tensor parallelism, pipeline parallelism, context parallelism, expert parallelism, replica scaling, fault tolerance, and deploying a custom engine on Azure ML. 7. Observability, cost, and evaluationMLflow, OpenTelemetry, Azure Monitor, dashboards, alerts, cost-per-token, GPU utilisation, offline evaluation, online evaluation, drift detection, and LLM-as-judge workflows. Why Qwen3-0.6B? Every example in the book is anchored on Qwen3-0.6B. That choice is deliberate. The model is small enough to run on accessible hardware, including a 16 GB T4, but large enough to expose the same serving problems that appear at 7B and 70B scale. The point is not that Qwen3-0.6B is the only model you should use. The point is that it is small enough to make the serving stack visible. Once you understand the runtime on a small model, the same ideas scale upward: KV-cache economics, batching behaviour, quantisation trade-offs, parallelism choices, deployment patterns, observability, and evaluation. Who this book is for This book is for: engineers who are no longer satisfied with “just call the API”; data scientists moving from notebooks into production LLM systems; ML engineers responsible for serving, monitoring, or tuning LLM workloads; platform and infrastructure engineers trying to understand why LLM systems behave differently from ordinary web services; technical leads and CTOs evaluating whether their teams should host, operate, or customise their own LLM systems. This is not a beginner Python book, and it is not a high-level GenAI strategy overview. It is written for technical readers who want to understand how production LLM systems behave when latency, memory, throughput, quality, and cost all matter at the same time. What is included The complete 331-page bookA full technical guide to the five layers of LLMOps: compute, model assets, serving runtime, observability, and application integration. Three end-to-end projectsBuild a custom inference engine from scratch, fine-tune Qwen3-0.6B on a custom domain, and assemble a production serving system. Mathematical foundationsClear explanations of numerically stable softmax, online softmax, scaled dot-product attention, KV-cache memory formulas, decoding throughput, sampling, and LoRA parameter counts. Reference implementation scaffoldAppendix E contains a production-shaped scaffold with type hints, structured logging, OpenTelemetry spans, retry boundaries, deployment assets, and evaluation utilities. Practical operating guidanceThe book connects low-level serving mechanics to production concerns: TTFT, inter-token latency, GPU utilisation, cost-per-token, quality monitoring, rollback, and release decisions. Why this book exists A model that works in a notebook is not yet a production system. When an LLM service is slow, expensive, or unreliable, the problem is often not “model intelligence.” It is usually the runtime: how prompts are tokenised, how prefill and decode are scheduled, how KV cache is allocated, how batches are formed, how quantisation changes quality, how replicas are routed, how costs are measured, and how evaluation catches regressions that dashboards cannot see. This book is for readers who want to understand that machinery from first principles. By the end, you should be able not only to deploy the reference system, but also to reason about the serving stack well enough to debug it, tune it, explain it, and defend it in a real production environment.
Related
Related