Beysolow Ii Applied Reinforcement Learning with Python

With OpenAI Gym, Tensorflow, and Keras
1. ed
ISBN: 978-1-4842-5127-0
Verlag: Apress
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 177 Seiten

Delve into the world of reinforcement learning algorithms and apply them to different use-cases via Python. This book covers important topics such as policy gradients and Q learning, and utilizes frameworks such as Tensorflow, Keras, and OpenAI Gym. Applied Reinforcement Learning with Python introduces you to the theory behind reinforcement learning (RL) algorithms and the code that will be used to implement them. You will take a guided tour through features of OpenAI Gym, from utilizing standard libraries to creating your own environments, then discover how to frame reinforcement learning problems so you can research, develop, and deploy RL-based solutions.
What You'll LearnImplement reinforcement learning with Python Work with AI frameworks such as OpenAI Gym, Tensorflow, and KerasDeploy and train reinforcement learning-based solutions via cloud resourcesApply practical applications of reinforcement learning   Who This Book Is For  Data scientists, machine learning engineers and software engineers familiar with machine learning and deep learning concepts.

Taweh Beysolow II is a data scientist and author currently based in the United States. He has a Bachelor of Science degree in economics from St. Johns University and a Master of Science in Applied Statistics from Fordham University. After successfully exiting the startup he co-founded, he now is a Director at Industry Capital, a San Francisco based Private Equity firm, where he helps lead the Cryptocurrency and Blockchain platforms.

Autoren/Hrsg.

Beysolow Ii, Taweh

Inhaltsverzeichnis

1;Table of Contents;5
2;About the Author;9
3;About the Technical Reviewer;10
4;Acknowledgments;11
5;Introduction;12
6;Chapter 1: Introduction to Reinforcement Learning;13
6.1;History of Reinforcement Learning;14
6.2;MDPs and their Relation to  Reinforcement Learning;15
6.3;Reinforcement Learning Algorithms and RL Frameworks;19
6.4;Q Learning;22
6.4.1;Actor-Critic Models;23
6.5;Applications of Reinforcement Learning;24
6.5.1;Classic Control Problems;24
6.5.2;Super Mario Bros.;25
6.5.3;Doom;26
6.5.4;Reinforcement-Based Marketing Making;27
6.6;Sonic the Hedgehog;28
6.7;Conclusion;29
7;Chapter 2: Reinforcement Learning Algorithms;30
7.1;OpenAI Gym;30
7.2;Policy-Based Learning;31
7.3;Policy Gradients Explained Mathematically;33
7.4;Gradient Ascent Applied to Policy Optimization;35
7.5;Using Vanilla Policy Gradients on the Cart Pole Problem;36
7.6;What Are Discounted Rewards and Why Do We Use Them?;40
7.7;Drawbacks to Policy Gradients;47
7.8;Proximal Policy Optimization (PPO) and Actor-Critic Models;48
7.9;Implementing PPO and Solving Super Mario Bros.;49
7.9.1;Overview of Super Mario Bros.;50
7.9.2;Installing Environment Package;51
7.9.3;Structure of the Code in Repository;51
7.9.4;Model Architecture;52
7.10;Working with a More Difficult Reinforcement Learning Challenge;58
7.11;Dockerizing Reinforcement Learning Experiments;61
7.12;Results of the Experiment;63
7.13;Conclusion;64
8;Chapter 3: Reinforcement Learning Algorithms: Q Learning and Its Variants;65
8.1;Q Learning;65
8.2;Temporal Difference (TD) Learning;67
8.3;Epsilon-Greedy Algorithm;69
8.4;Frozen Lake Solved with Q Learning;70
8.5;Deep Q Learning;75
8.6;Playing Doom with Deep Q Learning;76
8.6.1;Simple Doom Level;81
8.7;Training and Performance;83
8.8;Limitations of Deep Q Learning;84
8.9;Double Q Learning and Double Deep Q Networks;84
8.10;Conclusion;85
9;Chapter 4: Market Making via Reinforcement Learning;87
9.1;What Is Market Making?;87
9.2;Trading Gym;91
9.3;Why Reinforcement Learning for This Problem?;92
9.4;Synthesizing Order Book Data with Trading Gym;94
9.5;Generating Order Book Data with Trading Gym;95
9.6;Experimental Design;97
9.6.1;RL Approach 1: Policy Gradients;100
9.6.2;RL Approach 2: Deep Q Network;101
9.7;Results and Discussion;103
9.8;Conclusion;104
10;Chapter 5: Custom OpenAI Reinforcement Learning Environments;105
10.1;Overview of Sonic the Hedgehog;105
10.2;Downloading the Game;106
10.3;Writing the Code for the Environment;108
10.4;A3C Actor-Critic;113
10.5;Conclusion;121
11;Appendix A: Source Code;123
11.1;Market Making Model Utilities;123
11.2;Policy Gradient Utilities;125
11.3;Models;126
11.4;Chapter 1;135
11.4.1;OpenAI Example;135
11.5;Chapter 2;135
11.5.1;Cart Pole Example;135
11.5.2;Super Mario Example;140
11.6;Chapter 3;144
11.6.1;Frozen Lake Example;144
11.6.2;Doom Example;149
11.7;Chapter 4;156
11.7.1;Market Making Example;156
11.8;Chapter 5;168
11.8.1;Sonic Example;168
12;Index;174