Building and Operating Production AI Systems from a Single GPU to Multi-Node Frontier Models
Buch, Englisch, Format (B × H): 178 mm x 254 mm
ISBN: 979-8-8688-3544-5
Verlag: APRESS L.P.
Running a large language model on hardware you control is no longer exotic, but the gap between a model that loads and a service that holds up under load is where most self-hosting efforts stall. This book closes that gap by building the thing, one working system that grows across all fifteen chapters.
You begin with an 8B model on a single card, the kind of hardware you can rent for pennies an hour or already own. From there the book adds what real traffic demands: an inference engine chosen on its merits, a control plane that provisions GPUs and tears them down again, a router and queue that decide what runs next, and cache-aware routing that stops a conversation being re-read from scratch on every turn. Part III moves past what one card can hold, placing a 70B deployment across a pool of GPUs selected on the network between them rather than the sum of their memory, and shopping for that capacity across several providers at once.
The final part goes to the frontier. You will serve a 753-billion-parameter sparse model on eight rented H200s, then measure what it costs per session as concurrency climbs, and find the point where a larger batch stops making each session cheaper. That result explains how commercial inference providers price what they sell, and it tells you when self-hosting wins and when it does not.
Costs are measured rather than estimated throughout. Every figure in the final part is regenerated from a committed data file, and the book says plainly which numbers are measurements and which are predictions. The complete control plane is available as an open-source companion repository, written in Go, so you can run what you read.
You Will:
- Build a working inference control plane in Go that provisions GPUs across providers, deploys engines onto them, routes and queues requests, and reports what it is doing.
- Learn how inference actually consumes a GPU, covering KV cache growth, quantization formats and their real cost, tensor and expert parallelism, and why a fast interconnect matters more than total memory.
- Measure the economics from real runs, working out per-session cost as batch size and context length grow, and identifying the point where self-hosting stops beating a commercial API.
This book is for:
Backend, systems and platform engineers who want to run LLMs on infrastructure they control. Also useful to infrastructure and DevOps engineers building internal AI platforms, and to technical founders with data-residency or cost constraints that rule out third-party APIs. No machine learning background is assumed. Readers should be comfortable with a systems language such as Go or Python, with containers, and with HTTP APIs.
Zielgruppe
Professional/practitioner
Autoren/Hrsg.
Fachgebiete
Weitere Infos & Material
- PART I: FOUNDATIONS.- Chapter 1: Introduction to Self-Hosted Inference.- Chapter 2: How LLMs Work.- Chapter 3: Hardware Fundamentals.- Chapter 4: Model Formats and Quantization.- Chapter 5: Inference Engines.- Chapter 6: Building the Control Plane (v0.1).- PART II: SERVING REAL WORKLOADS.- Chapter 7: Routing, Queueing, and the Control Plane in the Data Path.- Chapter 8: Stateful Routing and Prefix-Cache Reuse.- Chapter 9: Scaling to 30B+ Models.- PART III: DISTRIBUTED INFERENCE.- Chapter 10: Multi-GPU and Distributed Inference.- Chapter 11: Capacity Across Providers.- Chapter 12: 70B Deployment.- PART IV: LET’S GO BIG.- Chapter 13: Serving a Frontier Model, and What the Market Will Sell You.- Chapter 14: What It Costs, and Why Batching Stops Helping.- Chapter 15: The Ceiling, and What Comes Next.
