O'Neal | Managing Production Large Language Models | E-Book | www.sack.de
E-Book

E-Book, Englisch, 452 Seiten

O'Neal Managing Production Large Language Models

Designing, Deploying, and Operating LLM Systems at Scale
1. Auflage 2026
ISBN: 979-8-90498-106-8
Verlag: PublishDrive
Format: EPUB
Kopierschutz: 0 - No protection

Designing, Deploying, and Operating LLM Systems at Scale

E-Book, Englisch, 452 Seiten

ISBN: 979-8-90498-106-8
Verlag: PublishDrive
Format: EPUB
Kopierschutz: 0 - No protection



Managing Production Large Language Models is the operational engineering manual for the teams responsible for designing, deploying, and running large language models inside enterprise production systems. Written for ML architects, ML and LLM engineers, technical team leads, and platform and SRE engineers, the book treats LLM operations as a structural discipline that spans selection, adaptation, serving, evaluation, observability, safety, governance, and cost-not a vendor or framework conversation.
The book opens with the operational gap that defines the field. LLM features fail in distinct ways: retrieval pipelines return stale documents, autonomous agents enter retry loops on rate-limited APIs, prompt-injection vulnerabilities exfiltrate sensitive context, hallucinated outputs reach regulated users, and token budgets blow through quarterly forecasts in a single weekend. From that opening, twelve chapters in four parts develop the patterns that close those gaps.
Part I, Foundations (Chapters 1-4), establishes the vocabulary the rest of the book depends on: the production LLM landscape, deployment model decisions (proprietary API, hosted open weights, self-hosted), the prompting and context-engineering primitives, and the adaptation spectrum from prompt engineering to RAG to parameter-efficient fine-tuning to full fine-tuning. Part II, Adaptation and Serving (Chapters 5 and 6), covers the infrastructure that determines whether an LLM feature survives real traffic-serving architectures, latency budgets, batching and continuous batching, KV-cache management, token streaming, and the GPU economics of inference.
Part III, Production Concerns (Chapters 7-10), covers the disciplines the rest of the engineering organization will hold the LLM team accountable for: safety, security, and content governance (prompt-injection defense, content filtering, output redaction, audit logging); observability (tracing, evaluation telemetry, drift detection across embedding spaces); testing and evaluation (offline benchmarks, online experimentation, regression suites, human evaluation); and FinOps for LLM workloads (token-budget alerts, cache hit-rate SLOs, model-tier routing). Part IV, Frontier Patterns and Case Studies (Chapters 11 and 12), extends the patterns to agentic systems, multi-modal workloads, and emerging architectures, and synthesizes the book's patterns into reference architectures for common production shapes.
Every chapter opens with a composite production incident drawn from the shapes of real failures, develops the engineering response in vendor-neutral patterns, and closes with three structured artifacts: a Technical Checklist of conditions the team should verify before declaring a system production-ready, an Owner Matrix that names the role accountable for each artifact, and a Team Discussion Guide that surfaces misalignment before it surfaces as an incident. Where specific tools appear, they make abstract patterns concrete; the patterns themselves outlive the tools.
Readers leave with the deployment vocabulary to make selection and architecture decisions that age well, the serving and observability patterns to keep production LLMs reliable at scale, the safety and governance controls to defend them under regulatory and reputational scrutiny, and the FinOps discipline to keep them economically viable as the portfolio expands.

O'Neal Managing Production Large Language Models jetzt bestellen!

Autoren/Hrsg.


Weitere Infos & Material


Introduction


The RAG-powered contract assistant had been live for eleven days when the first complaint arrived. A procurement officer at a mid-size enterprise had asked the assistant to confirm the penalty clause in a vendor agreement. The assistant responded with confident, fluent prose, cited the correct vendor name, and stated the penalty cap at 2% of the annual contract value. The actual clause read five percent. The officer, trusting the system, forwarded the answer to the vendor's account team and used it as the basis for a negotiation position. The error surfaced three days later during legal review.

The engineering post-mortem traced the failure to four overlapping causes. The retrieval layer had returned a document from a prior contract version because the vector index had not been refreshed after the amendment was signed. The prompt did not instruct the model to cite the specific clause number. Trained on a broad corpus of contract language, the model had a strong prior toward the two-percent figure and confidently filled the gap. The evaluation suite the team ran before launch tested general question-answering accuracy but did not include adversarial document-version tests. No monitoring alert flagged responses that contradicted the source document. The team had built the prototype in two sprints, demoed it to enthusiastic applause, and shipped it to thirty users without closing any of these gaps. The system behaved exactly as it was designed: it generated fluent, plausible text grounded in whatever the retriever returned. The retriever returned the wrong document.

The same week, a separate team at the same company discovered that an autonomous email-drafting agent had entered a retry loop after an upstream API returned a 429 rate-limit response. The agent's orchestration layer caught the error, waited, and retried. The upstream API continued to throttle. The agent retried 43 times over a 90-minute window, generating a new draft email for each attempt, logging it, and trying to deliver it. No human was in the loop. No budget cap had been set on the agent's token spend. By the time the on-call engineer saw the alert, the agent had consumed four times its daily token budget and had written forty-three near-identical drafts to the same recipient's outbox queue. The recipient was a regulator.

These two incidents, assembled from patterns that recur across organizations and deployment shapes, name the central difficulty of putting large language models into production: LLM systems do not fail the way classical software fails. They fail quietly, confidently, and in ways that look correct until someone checks the source. The retrieval system returned a document. The model generated text. Every component did what it was designed to do. The failure was not in the code; it was in the absence of the operational disciplines that production-grade LLM systems require. This book is the manual for those disciplines.

Why This Book, Why Now


Large language models completed one of the fastest transitions from research curiosity to production infrastructure in software engineering history. What was a demonstration capability in early 2022 had become a default feature surface by 2024 and a billion-dollar budget line item by 2026. That transition happened faster than the engineering disciplines required to sustain it. Teams that spent two years asking whether to use LLMs are now spending the next two years learning how to operate them with the reliability, cost discipline, safety posture, and observability that the rest of their stack already expects.

The operational gap is real, and it is costing organizations. Prompt injection vulnerabilities have exposed sensitive data in customer-facing assistants. Hallucinated outputs have reached clinical and financial workflows with no downstream correction. Uncontrolled token spend has turned a low-usage feature into a financial escalation. Model behavior has drifted after a provider update with no monitoring signal to catch it. Agentic systems have taken irreversible actions without adequate human-in-the-loop controls. These are the canonical failure shapes of the first generation of production LLM systems, and they share a common cause: teams transferred the fast-moving energy of the prototype phase directly into production without closing the engineering disciplines that separate the two.

Teams that are now operating LLMs in production no longer ask which model to use. They ask which model, at which size, with which fine-tuning strategy, served by which architecture, monitored by which signals, tested by which gates, and operated within which budget and compliance regime. Those are concrete engineering questions, and the answers to them have stabilized enough to be taught as patterns. The LLM tooling landscape will rotate again in the next two years, as it has in each of the last four. The patterns will not. A team that has defined its selection framework, serving architecture, safety posture, and observability stack can evaluate any incoming tool against a stable set of engineering criteria rather than being perpetually disrupted by the next release cycle.

This book exists because the field has matured past the point where the most useful contribution is another survey of model capabilities. The useful contribution now is operational clarity: the patterns, failure modes, decision frameworks, and production disciplines that let a team ship an LLM feature and keep it running. Every chapter is organized around that goal. Every technical checklist is designed to function as a release gate. Every team discussion section is designed to surface misalignments before they surface as incidents.

Who This Book Is For


This book is written for four roles that converge at the same LLM production system: ML architects, ML and LLM engineers, technical team leads, and platform and SRE engineers. Each role brings a different vantage point to the same decisions, and the book is structured to serve all four without forcing any of them to translate the others' vocabulary. Every chapter pairs design-level reasoning with implementation-level detail, and the team discussion sections at the end of each chapter are built for conversations that cross role boundaries.

ML architects will find a framework for making selection and design decisions that age well across platform cycles. Which model family fits this workload's latency and cost envelope? When does fine-tuning add capability that prompt engineering cannot reach? When does a self-hosted deployment justify the operational burden over a managed API? What serving architecture absorbs traffic spikes without cost surprises? The book provides the criteria for those decisions, not just conclusions drawn from a specific configuration. A mission-aligned LLM architecture is one designed from the start to absorb the model updates, provider changes, and traffic growth that production systems inevitably encounter.

ML and LLM engineers will find implementation patterns grounded in the realities of production systems: how to build an evaluation suite that catches quality regression before it reaches users, how to structure a prompt library as managed code rather than a folder of text files, how to instrument an LLM feature so that on-call engineers can localize a quality failure in minutes. The technical checklists at the end of each chapter are designed to be used directly as release gates: specific, actionable, and grounded in the failure modes each chapter names.

Technical team leads will find a shared language to align engineers and architects on reliability standards, plus discussion guides to surface misalignments before they become production incidents. The pilot-to-production pathway this book describes is one that a team can document, audit, and repeat.

Platform and SRE engineers will find the operational layer of each pattern explicitly named: what the serving infrastructure needs to know about LLM workload shapes, how token budgets interact with cost attribution, which observability signals matter for quality degradation, and where security and compliance boundaries lie in a shared LLM platform. LLM workloads are a distinct class of workloads. This book treats them as such.

How This Book Is Organized


The twelve chapters are organized into four logical parts, each covering a distinct phase of the LLM production lifecycle. Reading the parts in order gives a complete picture of the discipline from landscape orientation to frontier practice. Reading individual chapters gives actionable guidance on a specific problem without requiring the others as prerequisites. Four appendices provide reference material useful long after the first read: a glossary, reference architectures, release-gate checklists, and compliance mapping templates.

Part I, Foundations (Chapters 1 through 4), establishes the vocabulary and mental models the rest of the book depends on. Chapter 1 introduces the LLM production landscape: what counts as an LLM, where the deployment models differ, what the canonical business use cases look like, what the LLM-specific risk surface is, and what a working definition of production means for an LLM feature. Chapter 2 covers model selection and sizing: how to compare candidates on capability, latency, and cost; how to run benchmarks that reflect actual workloads rather than public leaderboards; and how to produce a sizing document the platform team can act on. Chapter 3 covers fine-tuning and adaptation: when instruction tuning, LoRA, PEFT, or preference alignment is the right tool, how to structure a fine-tuning workflow with evaluation gates, and how to avoid common failure modes in domain adaptation. Chapter 4 covers prompt engineering and prompt operations: how to structure prompts for...



Ihre Fragen, Wünsche oder Anmerkungen
Vorname*
Nachname*
Ihre E-Mail-Adresse*
Kundennr.
Ihre Nachricht*
Lediglich mit * gekennzeichnete Felder sind Pflichtfelder.
Wenn Sie die im Kontaktformular eingegebenen Daten durch Klick auf den nachfolgenden Button übersenden, erklären Sie sich damit einverstanden, dass wir Ihr Angaben für die Beantwortung Ihrer Anfrage verwenden. Selbstverständlich werden Ihre Daten vertraulich behandelt und nicht an Dritte weitergegeben. Sie können der Verwendung Ihrer Daten jederzeit widersprechen. Das Datenhandling bei Sack Fachmedien erklären wir Ihnen in unserer Datenschutzerklärung.