Raina / Manvar Operational AI with Docker

Deploy, scale, and operate agentic AI services with Docker and Kubernetes
1. Auflage 2026
ISBN: 978-1-80730-108-8
Verlag: De Gruyter
Format: EPUB
Kopierschutz: 0 - No protection

E-Book, Englisch, 390 Seiten

Modern AI systems don't fail at modeling; they fail in production. Moving from experiments to reliable, scalable systems requires more than notebooks and scripts. It requires infrastructure.
Operational AI with Docker shows you how to build, deploy, and operate AI systems that work beyond a single machine. You'll learn how to use Docker as a consistent runtime for machine learning workflows, package models as reproducible artifacts, and run them reliably across environments.
Starting with containerized machine learning, you'll progress to model serving, AI deployment, and scalable infrastructure using Kubernetes. You'll implement production-ready patterns for resource management, autoscaling, observability, and performance tuning, ensuring your AI workloads remain stable under real-world conditions.
The book goes beyond traditional MLOps by introducing agentic AI systems, including autonomous agents, multi-agent architectures, and secure execution environments. You'll also explore modern integration patterns using the Model Context Protocol (MCP), enabling AI systems to interact safely with tools, APIs, and data sources.
By the end of this book, you'll be able to design and operate production AI systems that are reproducible, scalable, and ready for real-world deployment using Docker and Kubernetes.

Autoren/Hrsg.

Raina, Ajeet Singh

Manvar, Harsh

Fachgebiete

• Mathematik | Informatik EDV | Informatik Programmierung | Softwareentwicklung Software Engineering

Leseproben

Preface

The AI landscape is undergoing a fundamental shift. Models that once existed only in research papers and cloud APIs now run locally on developer laptops. Autonomous agents orchestrate complex multi-step workflows without human intervention. Multi-model systems collaborate like specialized teams, each bringing a distinct capability to solve problems no single model could tackle alone.

Yet for all this progress, a critical gap remains: the infrastructure challenge. How do you move an AI model from a Jupyter notebook to a production system that can be scaled, monitored, updated, and trusted? How do you give an agent access to tools and data sources without exposing secrets or bypassing security boundaries? How do you orchestrate dozens of specialized agents across a Kubernetes cluster while keeping costs under control?

This is the gap that was written to close.

We have spent years at the intersection of containerization and AI, building Docker communities, creating hands-on content, and working alongside engineers at some of the world's largest enterprises as they wrestled with exactly these questions. What we found, consistently, was that the Docker ecosystem already contained most of the answers. Docker Model Runner for local inference. Docker MCP Gateway for secure tool integration. Docker Compose for declarative multi-agent architectures. Kubernetes and kagent for production-grade orchestration at scale.

This book weaves those tools into a coherent, end-to-end story.

Two capabilities deserve a special mention. Docker Agent brings a native agentic layer to the Docker platform, enabling developers to build, run, and manage AI agents with the same familiar workflows they already use for containers. Docker Sandboxes takes security a step further by executing agent-generated code inside lightweight microVM boundaries, giving autonomous agents the freedom to act while ensuring they can never escape their designated environment. Together, these capabilities represent Docker's vision for safe, production-ready agentic AI.

Who this book is for

This book is written for Cloud Engineers, DevOps Engineers, SREs, Platform Engineers, and software developers who are responsible for deploying, operating and scaling AI workloads in modern infrastructure environments. If you have ever successfully trained or fine-tuned a model only to get stuck on the "now what?" question, this book is for you.

Readers should be comfortable with the command line, have a foundational understanding of containers or a willingness to build one, and have some exposure to machine learning concepts. No prior experience with Docker Model Runner, MCP, or agent frameworks is assumed - every concept is introduced from first principles before being pushed to production-grade complexity.

What this book covers

, , introduces Docker and explains why it has become essential to modern AI and machine learning workflows. You will learn the core Docker architecture — images, containers, and registries — and run your first containerized workload.

, , bridges Docker fundamentals with the practical realities of working with AI models. You will explore OCI artifacts, GGUF model formats, quantization trade-offs, and Docker Compose's model provider syntax, laying the foundation for everything that follows.

, , introduces Docker Model Runner, Docker's integrated solution for running and serving large language models locally. You will pull models from Docker Hub, call the OpenAI-compatible API, build streaming chat applications, configure GPU acceleration, and set up Prometheus and Grafana dashboards for observability.

, , covers the pattern of using Docker to isolate and offload resource-intensive workloads from your main application, keeping systems fast and predictable. You will learn when and how to apply offloading to real AI and ML scenarios.

, , focuses on deploying machine learning models on Kubernetes clusters in a practical, beginner-friendly way. You will package models into containers, define Kubernetes objects, configure resource limits, and handle automatic scaling and restarts.

, , introduces the Model Context Protocol and Docker's MCP Gateway. You will learn how AI applications can securely access external tools, services, and data sources — from file systems and databases to web APIs and OAuth-protected services — through a standardized, containerized security boundary.

, , transitions from serving models and connecting tools to building agents that can reason, plan, and execute multi-step tasks. You will design container-isolated agents, implement communication patterns using message queues and APIs, and establish security sandboxing and monitoring for production deployments.

, , advances from single agents to sophisticated systems where multiple models and agents collaborate. You will orchestrate specialized models, implement hierarchical and peer-to-peer coordination patterns, manage shared state with Redis, and build a complete multi-agent document processing pipeline.

, , tackles three tools that progressively eliminate the infrastructure complexity from and . Docker Sandboxes solves the security problem by running coding agents. Docker Agent solves the simplicity problem: the nine-file, 400-line multi-agent projects from become a single YAML file, with built-in tools for filesystem access, shell execution, persistent memory, and structured reasoning, plus sub-agent delegation and OCI-based distribution through Docker Hub. Finally, kagent solves the scale problem, bringing agent orchestration to Kubernetes with custom resource definitions, Horizontal Pod Autoscaling, Prometheus, and Jaeger observability, and network policies for production-grade security isolation.

To get the most out of this book

The code examples in this book are designed to be run on a machine with Docker Desktop installed (version 4.40 or later is recommended for full Docker Model Runner and MCP Toolkit support). For Kubernetes chapters, a local cluster via Docker Desktop's built-in Kubernetes or a cloud-based cluster works equally well.

GPU acceleration is demonstrated in but is entirely optional — every example runs on CPU. assumes access to a Kubernetes cluster; the examples are tested against both local Docker Desktop Kubernetes and managed cloud clusters.

Software/hardware covered in this book

OS requirements

Docker Desktop 4.40+

macOS, Windows 10/11, Linux

Kubernetes 1.28+

macOS, Windows, Linux (cloud or local)

Python 3.10+

macOS, Windows, Linux

Node.js 18+

macOS, Windows, Linux

Download the example code files

The code bundle for the book is hosted on GitHub at https://github.com/PacktPublishing/Operational-AI-with-Docker. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://packt.link/gbp/9781807301095.

Conventions used

There are a number of text conventions used throughout this book.

: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. For example: "A Docker CLI extension () that provides commands to interact with models."

A block of code is set as follows:

Any command-line input or output is written as follows:

Bold: Indicates a new term, an important word, or words that you see on the screen. For instance, words in menus or dialog boxes appear in the text like this. For example: "Docker Model Runner, or DMR, is a native Docker tool for running AI models...