Archetti / Candelieri Bayesian Optimization and Data Science

1. Auflage 2019
ISBN: 978-3-030-24494-1
Verlag: Springer International Publishing
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 137 Seiten

Reihe: SpringerBriefs in Optimization

This volume brings together the main results in the field of Bayesian Optimization (BO), focusing on the last ten years and showing how, on the basic framework, new methods have been specialized to solve emerging problems from machine learning, artificial intelligence, and system optimization. It also analyzes the software resources available for BO and a few selected application areas. Some areas for which new results are shown include constrained optimization, safe optimization, and applied mathematics, specifically BO's use in solving difficult nonlinear mixed integer problems.  The book will help bring readers to a full understanding of the basic Bayesian Optimization framework and gain an appreciation of its potential for emerging application areas. It will be of particular interest to the data science, computer science, optimization, and engineering communities.

Autoren/Hrsg.

Archetti, Francesco

Candelieri, Antonio

Inhaltsverzeichnis

1;Preface;5
1.1;Who should read this book?;8
2;Contents;11
3;1 Automated Machine Learning and Bayesian Optimization;14
3.1;1.1 Automated Machine Learning;14
3.1.1;1.1.1 Motivation;14
3.1.2;1.1.2 Model Selection;16
3.1.3;1.1.3 Hyperparameter Optimization;17
3.1.4;1.1.4 Combined Algorithm Selection and Hyperparameter Optimization;19
3.1.5;1.1.5 Why Hyperparameter Optimization Is Important?;19
3.2;1.2 The Basic Structure of Bayesian Optimization;21
3.2.1;1.2.1 Sequential Model-Based Optimization;21
3.2.2;1.2.2 Surrogate Model;23
3.2.3;1.2.3 Acquisition Function;25
3.3;1.3 Automated Machine Learning for Predictive Analytics;27
3.4;References;30
4;2 From Global Optimization to Optimal Learning;32
4.1;2.1 A Priori Analysis of Global Optimization Strategies;33
4.2;2.2 Lipschitz Global Optimization (LGO);34
4.3;2.3 Random Search;37
4.3.1;2.3.1 General Properties of Uniform Sampling;39
4.3.2;2.3.2 Cluster Analysis;39
4.3.3;2.3.3 Stopping Rules;39
4.4;2.4 Bandits, Active Learning and Bayesian Optimization;41
4.5;References;45
5;3 The Surrogate Model;49
5.1;3.1 Gaussian Processes;49
5.1.1;3.1.1 Gaussian Processes Regression;49
5.1.2;3.1.2 Kernel: The Data Geometry of Bayesian Optimization;52
5.1.3;3.1.3 Embedding Derivative Observations in the Gaussian Process;55
5.1.4;3.1.4 Numerical Instability;58
5.2;3.2 Thompson Sampling;59
5.3;3.3 Alternative Models;62
5.3.1;3.3.1 Random Forest;63
5.3.2;3.3.2 Neural Networks: Feedforward, Deep and Bayesian;65
5.4;References;67
6;4 The Acquisition Function;69
6.1;4.1 Traditional Acquisition Functions;69
6.1.1;4.1.1 Probability of Improvement;69
6.1.2;4.1.2 Expected Improvement;70
6.1.3;4.1.3 Upper/Lower Confidence Bound;72
6.2;4.2 New Acquisition Functions;74
6.2.1;4.2.1 Scaled Expected Improvement;74
6.2.2;4.2.2 Portfolio Allocation;74
6.2.3;4.2.3 Thompson Sampling;75
6.2.4;4.2.4 Entropy-Based Acquisition Functions;77
6.2.5;4.2.5 Knowledge Gradient;78
6.2.6;4.2.6 Look-Ahead;80
6.2.7;4.2.7 K-Optimality;81
6.3;4.3 Optimizing the Acquisition Function;82
6.4;References;83
7;5 Exotic Bayesian Optimization;85
7.1;5.1 Constrained Global Optimization;85
7.2;5.2 Support Vector Machine—Constrained Bayesian Optimization;88
7.3;5.3 Safe Bayesian Optimization;98
7.4;5.4 Parallel Bayesian Optimization;102
7.5;5.5 Multi-objective Bayesian Optimization;103
7.6;5.6 Multi-source and Multi-fidelity Bayesian Optimization;105
7.7;References;106
8;6 Software Resources;109
8.1;6.1 Open Source Software;109
8.2;6.2 Bayesian Optimization as a Service;113
8.3;6.3 Bayesian Optimization-Based Services for Hyperparameters Optimization;114
8.4;6.4 Test Functions and Generators;115
8.4.1;6.4.1 Survey and Site/Repository of Test Functions;115
8.4.2;6.4.2 Test Functions Generators;119
8.5;6.5 Non-Bayesian Global Optimization Software;119
8.6;References;120
9;7 Selected Applications;122
9.1;7.1 Overview of Applications;122
9.2;7.2 Smart Water;127
9.2.1;7.2.1 Leakage Localization;127
9.2.2;7.2.2 Pump Scheduling Optimization;129
9.3;References;133