Ketkar Deep Learning with Python

A Hands-on Introduction
1. ed
ISBN: 978-1-4842-2766-4
Verlag: Apress
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 235 Seiten

Discover the practical aspects of implementing deep-learning solutions using the rich Python ecosystem. This book bridges the gap between the academic state-of-the-art and the industry state-of-the-practice by introducing you to deep learning frameworks such as Keras, Theano, and Caffe. The practicalities of these frameworks is often acquired by practitioners by reading source code, manuals, and posting questions on community forums, which tends to be a slow and a painful process. Deep Learning with Python allows you to ramp up to such practical know-how in a short period of time and focus more on the domain, models, and algorithms.
This book briefly covers the mathematical prerequisites and fundamentals of deep learning, making this book a good starting point for software developers who want to get started in deep learning. A brief survey of deep learning architectures is also included.
Deep Learning with Python also introduces you to key concepts of automatic differentiation and GPU computation which, while not central to deep learning, are critical when it comes to conducting large scale experiments. 
What You Will Learn Leverage deep learning frameworks in Python namely, Keras, Theano, and Caffe 
Gain the fundamentals of deep learning with mathematical prerequisites 
Discover the practical considerations of large scale experiments 
Take deep learning models to production
Who This Book Is For
Software developers who want to try out deep learning as a practical solution to a particular problem. Software developers in a data science team who want to take deep learning models developed by data scientists to production.


Nikhil S. Ketkar currently leads the Machine Learning Platform team at Flipkart, India's largest e-commerce company. He received his Ph.D. from Washington State University. Following that he conducted postdoctoral research at University of North Carolina at Charlotte, which was followed by a brief stint in high frequency trading at Transmaket in Chicago. More recently he led the data mining team in Guavus, a startup doing big data analytics in the telecom domain and Indix, a startup doing data science in the e-commerce domain. His research interests include machine learning and graph theory.

Autoren/Hrsg.

Ketkar, Nikhil

Inhaltsverzeichnis

1;Contents at a Glance;5
2;Contents;6
3;About the Author;11
4;About the Technical Reviewer;12
5;Acknowledgments;13
6;Chapter 1: Introduction to Deep Learning;14
6.1;Historical Context;14
6.2;Advances in Related Fields;16
6.3;Prerequisites;16
6.4;Overview of Subsequent Chapters;17
6.5;Installing the Required Libraries;18
7;Chapter 2: Machine Learning Fundamentals;19
7.1;Intuition;19
7.2;Binary Classification;19
7.3;Regression;20
7.4;Generalization;21
7.5;Regularization;26
7.6;Summary;28
8;Chapter 3: Feed Forward Neural Networks;29
8.1;Unit;29
8.1.1;Overall Structure of a Neural Network;31
8.1.2;Expressing the Neural Network in Vector Form;32
8.1.3;Evaluating the output of the Neural Network;33
8.1.4;Training the Neural Network;35
8.2;Deriving Cost Functions using Maximum Likelihood;36
8.2.1;Binary Cross Entropy;37
8.2.2;Cross Entropy;37
8.2.3;Squared Error;38
8.2.4;Summary of Loss Functions;39
8.3;Types of Units/Activation Functions/Layers;39
8.3.1;Linear Unit;40
8.3.2;Sigmoid Unit;40
8.3.3;Softmax Layer;41
8.3.4;Rectified Linear Unit (ReLU);41
8.3.5;Hyperbolic Tangent;42
8.4;Neural Network Hands-on with AutoGrad;45
8.5;Summary;45
9;Chapter 4: Introduction to Theano;46
9.1;What is Theano;46
9.2;Theano Hands-On;47
9.3;Summary;72
10;Chapter 5: Convolutional Neural Networks;73
10.1;Convolution Operation;73
10.2;Pooling Operation;80
10.3;Convolution-Detector-Pooling Building Block;82
10.4;Convolution Variants;86
10.5;Intuition behind CNNs;87
10.6;Summary;88
11;Chapter 6: Recurrent Neural Networks;89
11.1;RNN Basics;89
11.2;Training RNNs;94
11.3;Bidirectional RNNs;101
11.4;Gradient Explosion and Vanishing;102
11.5;Gradient Clipping;103
11.6;Long Short Term Memory;105
11.7;Summary;106
12;Chapter 7: Introduction to Keras;107
12.1;Summary;121
13;Chapter 8: Stochastic Gradient Descent;122
13.1;Optimization Problems;122
13.2;Method of Steepest Descent;123
13.3;Batch, Stochastic (Single and Mini-batch) Descent;124
13.3.1;Batch;125
13.3.2;Stochastic Single Example;125
13.3.3;Stochastic Mini-batch;125
13.3.4;Batch vs. Stochastic;125
13.4;Challenges with SGD;125
13.4.1;Local Minima;125
13.4.2;Saddle Points;126
13.4.3;Selecting the Learning Rate;127
13.4.4;Slow Progress in Narrow Valleys;128
13.5;Algorithmic Variations on SGD;128
13.5.1;Momentum;129
13.5.2;Nesterov Accelerated Gradient (NAS);130
13.5.3;Annealing and Learning Rate Schedules;130
13.5.4;Adagrad;130
13.5.5;RMSProp;131
13.5.6;Adadelta;132
13.5.7;Adam;132
13.5.8;Resilient Backpropagation;132
13.5.9;Equilibrated SGD;133
13.6;Tricks and Tips for using SGD;133
13.6.1;Preprocessing Input Data;133
13.6.2;Choice of Activation Function;133
13.6.3;Preprocessing Target Value;134
13.6.4;Initializing Parameters;134
13.6.5;Shuffling Data;134
13.6.6;Batch Normalization;134
13.6.7;Early Stopping;134
13.6.8;Gradient Noise;134
13.7;Parallel and Distributed SGD;135
13.7.1;Hogwild;135
13.7.2;Downpour;135
13.8;Hands-on SGD with Downhill;136
13.9;Summary;141
14;Chapter 9: Automatic Differentiation;142
14.1;Numerical Differentiation;142
14.2;Symbolic Differentiation;143
14.3;Automatic Differentiation Fundamentals;144
14.3.1;Forward/Tangent Linear Mode;145
14.3.2;Reverse/Cotangent/Adjoint Linear Mode;149
14.3.3;Implementation of Automatic Differentiation;152
14.3.3.1;Source Code Transformation;152
14.3.3.2;Operator Overloading;153
14.4;Hands-on Automatic Differentiation with Autograd;154
14.5;Summary;157
15;Chapter 10: Introduction to GPUs;158
15.1;Summary;167
16;Chapter 11: Introduction to Tensorflow;168
16.1;Summary;203
17;Chapter 12: Introduction to PyTorch;204
17.1;Summary;217
18;Chapter 13: Regularization Techniques;218
18.1;Model Capacity, Overfitting, and Underfitting;218
18.2;Regularizing the Model;219
18.3;Early Stopping;219
18.4;Norm Penalties;221
18.5;Dropout;222
18.6;Summary;223
19;Chapter 14: Training Deep Learning Models;224
19.1;Performance Metrics;224
19.2;Data Procurement;227
19.3;Splitting Data for Training/Validation/Test;228
19.4;Establishing Achievable Limits on the Error Rate;228
19.5;Establishing the Baseline with Standard Choices;229
19.6;Building an Automated, End-to-End Pipeline;229
19.6.1;Orchestration for Visibility;229
19.7;Analysis of Overfitting and Underfitting;229
19.8;Hyper-Parameter Tuning;231
19.9;Summary;231
20;Index;232