Minimizing Data Movement and Parameter Count Across the Machine Learning Stack

Everything is a Matrix
Erscheinungsjahr 2026
ISBN: 978-3-032-23099-7
Verlag: Springer

Buch, Englisch, 110 Seiten, Format (B × H): 168 mm x 240 mm

Reihe: Synthesis Lectures on Computer Science

This book provides a focused, research-forward guide to making large AI models efficient in practice and also presents an array of novel techniques to reduce memory footprint, accelerate computation, and improve overall hardware utilization. The author demonstrates that substantial efficiency gains can be achieved by rethinking how data is computed, stored, and compressed, with a special focus on matrices, the core computational structure underpinning both scientific computing and neural networks. Modern AI models run on huge grids of numbers (matrices/tensors), and their speed and affordability depend on how those numbers are arranged and processed on real hardware (GPUs/TPUs/CPUs). This book explains practical methods to skip unnecessary work (structured sparsity), move data efficiently (gather/scatter), and shrink models without losing accuracy (block distillation) so that AI systems can use less memory, less time, and less energy without sacrificing quality. In addition, the book shows how to turn algorithmic ideas into hardware-aware speedups on GPUs/TPUs. Readers will learn when sparsity pays off, how to schedule irregular workloads, and how to recover accuracy in compressed models. Case studies illustrate end-to-end design choices, evaluation, and pitfalls. The result is a coherent perspective that bridges theory, compilers/run times, and real-world deployment.

Zielgruppe

Professional/practitioner

Autoren/Hrsg.

Sabot, Andrew

Fachgebiete

• Mathematik | Informatik EDV | Informatik Informatik Künstliche Intelligenz Computer Vision
• Mathematik | Informatik Mathematik Numerik und Wissenschaftliches Rechnen Optimierung
• Mathematik | Informatik EDV | Informatik Informatik Bildsignalverarbeitung
• Mathematik | Informatik EDV | Informatik Informatik Künstliche Intelligenz Maschinelles Lernen
• Mathematik | Informatik EDV | Informatik Informatik Rechnerarchitektur

Inhaltsverzeichnis

Introduction and Roadmap.- CAKE: Memory-Aware Block Shaping for GEMM.- mCAKE: From Matrices to Tensors.- Rosko: Structured Sparsity for ML Workloads.- Gather/Scatter for Rank-Sliced Activations.- Low-Rank Models via SVD.- Blockwise Knowledge Distillation.- Privacy-Preserving Split Inference (Edge/Cloud).- Conclusion: Design Rules, Evaluation, and Outlook.

Über Autor(innen)

Andrew Sabot, Ph.D., is a Software Engineer working on Machine Learning at Google. He received his Ph.D. (2025) and M.S. (2021) in Computer Science from Harvard University. Dr. Sabot’s work focuses on the intersection of hardware-aware kernels,  model compression, and transformer inference acceleration to enable the sustainable deployment of state-of-the-art AI.