PhD / Jeno Federated Learning with Python

Design and implement a federated learning system and develop applications using existing frameworks
1. Auflage 2024
ISBN: 978-1-80324-875-2
Verlag: De Gruyter
Format: EPUB
Kopierschutz: 0 - No protection

E-Book, Englisch, 326 Seiten

Federated learning (FL) is a paradigm-shifting technology in AI that enables and accelerates machine learning (ML), allowing you to work on private data. It has become a must-have solution for most enterprise industries, making it a critical part of your learning journey. This book helps you get to grips with the building blocks of FL and how the systems work and interact with each other using solid coding examples.
FL is more than just aggregating collected ML models and bringing them back to the distributed agents. This book teaches you about all the essential basics of FL and shows you how to design distributed systems and learning mechanisms carefully so as to synchronize the dispersed learning processes and synthesize the locally trained ML models in a consistent manner. This way, you'll be able to create a sustainable and resilient FL system that can constantly function in real-world operations. This book goes further than simply outlining FL's conceptual framework or theory, as is the case with the majority of research-related literature.
By the end of this book, you'll have an in-depth understanding of the FL system design and implementation basics and be able to create an FL system and applications that can be deployed to various local and cloud environments.

Autoren/Hrsg.

PhD, Kiyoshi Nakayama

Jeno, George

Fachgebiete

• Mathematik | Informatik EDV | Informatik Informatik Theoretische Informatik
• Mathematik | Informatik EDV | Informatik Informatik Künstliche Intelligenz

Inhaltsverzeichnis

Table of Contents - Challenges in Big Data and Traditional AI
- What Is Federated Learning?
- Workings of the Federated Learning System
- Federated Learning Server Implementation with Python
- Federated Learning Client-Side Implementation

- Running the Federated Learning System and Analyzing the Results
- Model Aggregation
- Introducing Existing Federated Learning Frameworks
- Case Studies with Key Use Cases of Federated Learning Applications
- Future Trends and Developments
- Appendix, Exploring Internal Libraries

Leseproben

Preface

Federated learning (FL) is becoming a paradigm-shifting technology in AI because it is often said that with the FL framework, it is the machine learning model that needs to move around across the Internet, not the data itself, for the intelligence to continuously evolve and grow. Therefore, people call FL a model-centric approach, compared to the traditional data-centric approach, thus it is considered a game-changing technology. The idea of the model-centric approach can create an intelligence-centric platform to pioneer the wisdom-driven world.

By adopting FL, you can overcome the challenges that big data AI has been facing for a long time, such as data privacy, training cost and efficiency, and delay in the delivery of the most updated intelligence. However, FL is not a magic solution to resolve all the issues in big data just by aggregating machine learning models blindly. We need to design the distributed systems and learning mechanisms very carefully to synchronize all the distributed learning processes and synthesize all the locally trained machine learning models consistently. That way, we can create a sustainable and resilient FL system that can continuously function even in a real operation at scale.

Therefore, this book goes beyond just describing the conceptual and theoretical aspects of FL as seen in many research projects with simulators or prototypes that have been introduced in most of the literature related to this field. Rather, you will learn about the entire design and implementation principles by looking into the codes of the simplified federated learning system to validate the workings and results of the framework.

By the end of this book, you will create your first application based on federated learning that can be installed and tested in various settings in both local and cloud environments.

Who this book is for

This book is for machine learning engineers, data scientists, and AI enthusiasts who want to learn about creating machine learning applications empowered by FL. You will need basic knowledge of Python programming and machine learning concepts to get started with this book.

What this book covers

, , is all about orienting you to understand the current issues with big data systems and the traditional centralized machine learning approach and how FL could resolve those problems, such as data privacy, bias and silos, model drifts, and performance degradation. This will prepare you to dive deeper into the design and implementation of the FL system.

, , continues the introduction to FL and will help you understand the FL concept with the current trend of distributed learning as well as machine learning basics. You will also learn about the benefits of FL as a model-centric machine learning approach in terms of data privacy and security as well as efficiency and scalability.

, , gives you a solid understanding of how the FL system will work and interact among distributed components within the FL system itself. You will learn about the fundamental architecture of the FL system together with its procedure flow, state transition, and sequence of messaging toward the continuous operation of the FL system.

, , guides you through learning the implementation principles of the FL server-side system, including database server and aggregator modules. The chapter also talks about communication among the FL components and model aggregation, as well as how to manage the states of the distributed learning agents and local and global models.

, , describes the FL client-side functionalities as well as libraries that can be used by local machine learning engines and processes. The core client-side functionalities include registration of a distributed learning agent with the FL platform, global model receipt, and local model upload and sharing.

, , will help you run a simplified FL framework to understand the behavior of the FL system and procedure as well as the most standard model aggregation method of federated averaging. You will also be able to validate the outcome of running the FL framework by analyzing the results of two example test cases.

, , is an important chapter to understand model aggregation, which is the foundation of FL. You will understand how different characterizations of an FL scenario call for various aggregation methods and should have an idea of how these algorithms can actually be implemented.

, , explains the existing FL projects and frameworks, such as PySyft, TFF, Flower, OpenFL, and STADLE, and their APIs. There are many useful FL projects with different design philosophies, and you will understand the features and differences among those frameworks.

, , introduces some of the major use cases of FL in different industries. You’ll become familiar with some of the applications of FL in various fields, such as medical and healthcare, the financial sector, edge computing, the Internet of Things, and distributed learning for big data, in which FL has shown significant potential to overcome many important technological challenges.

, , describes the future direction of AI technologies driven by ongoing research and development of FL. You will be introduced to the new perspective of the Internet of Intelligence and wisdom-centric platforms. Thus, you will be ready to welcome the world of collective intelligence.

, , provides an overview of internal libraries, including enumeration classes for implementing the FL systems, communication protocol, data structure handler, and helper and supporting functions.

To get the most out of this book

You will need Python version 3.7+ installed on your computer. To create a virtual environment to easily run the code examples in the book, Anaconda is recommended to be installed on macOS or Linux.

Software/hardware covered in the book

Operating system requirements

Python 3.7+

macOS, or Linux

Anaconda environment

GitHub

You can install the server-side code provided in the GitHub repo in any cloud environment, such as Amazon Web Services (AWS) or Google Cloud Platform (GCP), with proper security settings to set up distributed learning environments.

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Federated-Learning-with-Python. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Please also check the repository for the most updated code at https://github.com/tie-set/simple-fl.

Note

You can use the code files for personal or educational purposes. Please note that we will not support the deployment for commercials use and will not be responsible for any errors, issues, or damages caused by using the code.

Download the color images

We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://packt.link/qh1su.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “The server code imports StateManager and Aggregator for the FL processes.”

A block of code is set as follows:

import tensorflow as tf from tensorflow import keras from sst_model import SSTModel

Any command-line input or output is written as follows:

fx envoy start -n envoy_1 - -disable-tls --envoy-config-path envoy_config_1.yaml -dh localhost -dp 50051

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always...

Über Autor(innen)

PhD Kiyoshi Nakayama :

Kiyoshi Nakayama, PhD, is the founder and CEO of TieSet Inc., which leads the development and dissemination of one of the most advanced distributed and federated learning platforms in the world. Before founding TieSet, he was a research scientist at NEC Laboratories America, renowned for having the world's top-notch machine learning research group of researchers. He was also a postdoctoral researcher at Fujitsu Laboratories of America, where he implemented a distributed system for smart energy. He has published several international articles and patents and received the best paper award twice in his career. Kiyoshi received his PhD in computer science from the University of California, Irvine.Jeno George :

George Jeno is a co-founder of TieSet Inc. and has been a tech lead for the development of the STADLE federated learning platform. He has a deep understanding of machine learning theory and system architecture design, and he has leveraged this knowledge to research new algorithms and applications for distributed and federated learning. He holds a master's degree in computer science (with a specialization in machine learning) from Georgia Tech.