El-Amir / Hamdy Deep Learning Pipeline

Building a Deep Learning Model with TensorFlow
1. ed
ISBN: 978-1-4842-5349-6
Verlag: Apress
Format: PDF
Kopierschutz: 1 - PDF Watermark

E-Book, Englisch, 563 Seiten

Build your own pipeline based on modern TensorFlow approaches rather than outdated engineering concepts. This book shows you how to build a deep learning pipeline for real-life TensorFlow projects. You'll learn what a pipeline is and how it works so you can build a full application easily and rapidly. Then troubleshoot and overcome basic Tensorflow obstacles to easily create functional apps and deploy well-trained models. Step-by-step and example-oriented instructions help you understand each step of the deep learning pipeline while you apply the most straightforward and effective tools to demonstrative problems and datasets.  You'll also develop a deep learning project by preparing data, choosing the model that fits that data, and debugging your model to get the best fit to data all using Tensorflow techniques. Enhance your skills by accessing some of the most powerful recent trends in data science. If you've ever considered building your own image or text-tagging solution or entering a Kaggle contest, Deep Learning Pipeline is for you!
What You'll LearnDevelop a deep learning project using dataStudy and apply various models to your dataDebug and troubleshoot the proper model suited for your dataWho This Book Is For

Developers, analysts, and data scientists looking to add to or enhance their existing skills by accessing some of the most powerful recent trends in data science. Prior experience in Python or other TensorFlow related languages and mathematics would be helpful.


Hisham Elamir? is a data scientist with expertise in machine learning, deep learning, and statistics. He currently lives and works in Cairo, Egypt. In his work projects, he faces challenges ranging from natural language processing (NLP), behavioral analysis, and machine learning to distributed processing. He is very passionate about his job and always tries to stay updated about the latest developments in data science technologies, attending meetups, conferences, and other events.  Mahmoud Hamdy is a machine learning engineer who works in Egypt and lives in Egypt, His primary area of study is the overlap between knowledge, logic, language, and learning. He works helping train machine learning, and deep learning models to distil large amounts of unstructured, semi-structured, and structured data into new knowledge about the world by using methods ranging from deep learning to statistical relational learning. He applies strong theoretical and practical skills in several areas of machine learning to finding novel and effective solutions for interesting and challenging problems in such interconnections

Autoren/Hrsg.

El-Amir, Hisham

Hamdy, Mahmoud

Inhaltsverzeichnis

1;Table of Contents;4
2;About the Authors;15
3;About the Technical Reviewer;16
4;Introduction;17
5;Part I: Introduction;24
5.1;Chapter 1: A Gentle Introduction;25
5.1.1;Information Theory, Probability Theory, and Decision Theory;26
5.1.1.1;Information Theory;26
5.1.1.2;Probability Theory;28
5.1.1.3;Decision Theory;30
5.1.2;Introduction to Machine Learning;32
5.1.2.1;Predictive Analytics and Its Connection with Machine learning;33
5.1.2.2;Machine Learning Approaches;34
5.1.2.2.1;Supervised Learning;34
5.1.2.2.2;Unsupervised Learning;36
5.1.2.2.3;Semisupervised Learning;38
5.1.2.2.4;Checkpoint;38
5.1.2.2.5;Reinforcement Learning;39
5.1.3;From Machine Learning to Deep Learning;41
5.1.4;Lets’ See What Some Heroes of Machine Learning Say About the Field;41
5.1.4.1;Connections Between Machine Learning and Deep Learning;42
5.1.4.2;Difference Between ML and DL;43
5.1.4.2.1;In Machine Learning;43
5.1.4.2.2;In Deep Learning;44
5.1.4.2.3;What Have We Learned Here?;44
5.1.4.3;Why Should We Learn About Deep Learning (Advantages of Deep learning)?;45
5.1.4.4;Disadvantages of Deep Learning (Cost of Greatness);46
5.1.5;Introduction to Deep Learning;47
5.1.6;Machine Learning Mathematical Notations;50
5.1.7;Summary;58
5.2;Chapter 2: Setting Up Your Environment;59
5.2.1;Background;59
5.2.1.1;Python 2 vs. Python 3;60
5.2.2;Installing Python;60
5.2.2.1;Python Packages;62
5.2.2.2;IPython;63
5.2.2.2.1;Installing IPython;64
5.2.2.3;Jupyter;65
5.2.2.3.1;Installing Jupyter;67
5.2.2.3.1.1;What Is an ipynb File?;69
5.2.3;Packages Used in the Book;72
5.2.3.1;NumPy;72
5.2.3.2;SciPy;72
5.2.3.3;Pandas;73
5.2.3.4;Matplotlib;73
5.2.3.5;NLTK;74
5.2.3.6;Scikit-learn;74
5.2.3.7;Gensim;75
5.2.3.8;TensorFlow;75
5.2.3.8.1;Installing on Mac or Linux distributions;76
5.2.3.8.2;Installing on Windows;78
5.2.3.9;Keras;78
5.2.4;Summary;78
5.3;Chapter 3: A Tour Through the Deep Learning Pipeline;79
5.3.1;Deep Learning Approaches;80
5.3.1.1;What Is Deep Learning;80
5.3.1.2;Biological Deep Learning;80
5.3.1.3;What Are Neural Networks Architectures?;84
5.3.2;Deep Learning Pipeline;90
5.3.2.1;Define and Prepare Problem;91
5.3.2.2;Summarize and Understand Data;92
5.3.2.3;Process and Prepare Data;93
5.3.2.4;Evaluate Algorithms;94
5.3.2.5;Improve Results;95
5.3.3;Fast Preview of the TensorFlow Pipeline;96
5.3.3.1;Tensors—the Main Data Structure;97
5.3.3.2;First Session;98
5.3.3.3;Data Flow Graphs;100
5.3.3.4;Tensor Properties;103
5.3.3.4.1;Tensor Rank;104
5.3.3.4.2;Tensor Shape;105
5.3.4;Summary;105
5.4;Chapter 4: Build Your First Toy TensorFlow app;107
5.4.1;Basic Development of TensorFlow;107
5.4.1.1;Hello World with TensorFlow;108
5.4.1.2;Simple Iterations;109
5.4.1.3;Prepare the Input Data;110
5.4.1.4;Doing the Gradients;113
5.4.2;Linear Regression;115
5.4.2.1;Why Linear Regression?;115
5.4.2.2;What Is Linear Regression?;115
5.4.2.3;Dataset Description;116
5.4.2.4;Full Source Code;121
5.4.3;XOR Implementation Using TensorFlow;123
5.4.3.1;Full Source Code;129
5.4.4;Summary;131
6;Part II: Data;132
6.1;Chapter 5: Defining Data;133
6.1.1;Defining Data;134
6.1.2;Why Should You Read This Chapter?;134
6.1.3;Structured, Semistructured, and Unstructured Data;135
6.1.4;Tidy Data;137
6.1.5;Divide and Conquer;138
6.1.6;Tabular Data;139
6.1.6.1;Quantitative? vs. ?Qualitative? Data;139
6.1.6.2;Example—the Titanic;139
6.1.6.3;Divide and Conquer;141
6.1.6.4;Making a Checkpoint;142
6.1.6.5;The Four Levels of Data;142
6.1.6.5.1;Measure of Center;143
6.1.6.6;The Nominal Level;143
6.1.6.6.1;Mathematical Operations Allowed for Nominal;144
6.1.6.6.2;Measures of Center for Nominal;144
6.1.6.6.3;What Does It Mean to be a Nominal Level Type?;145
6.1.6.7;The Ordinal Level;145
6.1.6.7.1;Examples of Being Ordinal;146
6.1.6.7.2;What Data Is Like at the Ordinal Level;146
6.1.6.7.3;Mathematical Operations Allowed for Ordinal;147
6.1.6.7.4;Measures of Center for Ordinal;148
6.1.6.8;Quick Recap and Check;149
6.1.6.9;The Interval Level;150
6.1.6.10;Examples of Interval Level Data;150
6.1.6.11;What Data Is Like at the Interval Level;151
6.1.6.12;Mathematical Operations Allowed for Interval;151
6.1.6.12.1;Measures of Center for Interval;151
6.1.6.12.2;Measures of Variation for Interval;152
6.1.6.12.2.1;Standard Deviation;153
6.1.6.13;The Ratio Level;154
6.1.6.13.1;Examples;154
6.1.6.13.2;Measures of Center for Ratio;155
6.1.6.13.3;Problems with the Ratio Level;155
6.1.6.14;Summarizing All Levels Table 5-1;156
6.1.7;Text Data;157
6.1.7.1;What Is Text Processing and What Is the Level of Importance of Text Processing?;157
6.1.7.2;IMDB—Example;158
6.1.7.3;Images Data;159
6.1.7.3.1;Type of Images (2-D, 3-D, 4-D);160
6.1.7.3.1.1;2-D Data;160
6.1.7.3.1.2;3-D Data;160
6.1.7.3.1.3;4-D Data;161
6.1.7.3.2;Example—MNIST;162
6.1.7.3.3;Example—CIFAR-10;163
6.1.8;Summary;164
6.2;Chapter 6: Data Wrangling and Preprocessing;166
6.2.1;The Data Fields Pipelines Revisited;167
6.2.1.1;Giving You a Reason;167
6.2.1.2;Where Is Data Cleaning in the Process?;168
6.2.2;Data Loading and Preprocessing;169
6.2.2.1;Fast and Easy Data Loading;169
6.2.3;Missing Data;177
6.2.3.1;Empties;178
6.2.3.2;Is It Ever Useful to Fill Missing Data Using a Zero Instead of an Empty or Null?;178
6.2.3.3;Managing Missing Features;179
6.2.4;Dealing with Big Datasets;180
6.2.5;Accessing Other Data Formats;182
6.2.6;Data Preprocessing;183
6.2.7;Data Augmentation;188
6.2.7.1;Image Crop;191
6.2.7.2;Crop and Resize;191
6.2.7.3;Crop to Bounding Box;193
6.2.7.4;Flipping;194
6.2.7.5;Rotate Image;196
6.2.7.6;Translation;197
6.2.7.7;Transform;198
6.2.7.8;Adding Salt and Pepper Noise;199
6.2.7.9;Convert RGB to Grayscale;200
6.2.7.10;Change Brightness;200
6.2.7.11;Adjust Contrast;201
6.2.7.12;Adjust Hue;202
6.2.7.13;Adjust Saturation;203
6.2.8;Categorical and Text data;204
6.2.8.1;Data Encoding;205
6.2.8.2;Performing One-Hot Encoding on Nominal Features;207
6.2.8.3;Can You Spot the Problem?;208
6.2.8.4;A Special Type of Data: Text;209
6.2.8.5;So Far, Everything Has Been Pretty Good, Hasn’t It?;214
6.2.8.6;Tokenization, Stemming, and Stop Words;220
6.2.8.6.1;What Are Tokenizing and Tokenization?;220
6.2.8.6.2;The Bag-of-Words (BoW) Model;223
6.2.8.6.2.1;What is the BoW?;223
6.2.9;Summary;225
6.3;Chapter 7: Data Resampling;226
6.3.1;Creating Training and Test Sets;227
6.3.2;Cross-Validation;228
6.3.2.1;Validation Set Technique;229
6.3.2.2;Leave-One-Out Cross-Validation (LOOCV);232
6.3.2.3;K-Fold Cross-Validation;235
6.3.3;Bootstrap;236
6.3.3.1;Bootstrap in Statistics;237
6.3.3.2;Tips to Use Bootstrap (Resampling with Replacement);239
6.3.4;Generators;242
6.3.4.1;What Are Keras Generators?;242
6.3.4.2;Data Generator;244
6.3.4.3;Callback;245
6.3.5;Summary;250
6.4;Chapter 8: Feature Selection and Feature Engineering;251
6.4.1;Dataset Used in This Chapter;252
6.4.2;Dimensionality Reduction—Questions to Answer;254
6.4.2.1;What Is Dimensionality Reduction?;255
6.4.2.2;When Should I Use Dimensionality Reduction?;257
6.4.3;Unsupervised Dimensionality Reduction via Principal Component Analysis (PCA);258
6.4.3.1;Total and Explained Variance;261
6.4.4;Feature Selection and Filtering;261
6.4.5;Principal Component Analysis;265
6.4.5.1;Nonnegative Matrix Factorization;274
6.4.6;Sparse PCA;275
6.4.7;Kernel PCA;277
6.4.8;Atom Extraction and Dictionary Learning;279
6.4.9;Latent Dirichlet Allocation (LDA);280
6.4.9.1;Latent Dirichlet Allocation (LDA in NLP);281
6.4.9.2;Code Example Using gensim;285
6.4.10;LDA vs. PCA;287
6.4.11;ZCA Whitening;290
6.4.12;Summary;294
7;Part III: TensorFlow;295
7.1;Chapter 9: Deep Learning Fundamentals;296
7.1.1;Perceptron;297
7.1.1.1;Single Perceptron;307
7.1.1.2;Multilayer Perceptron;308
7.1.1.3;Recap;309
7.1.2;Different Neural Network Layers;310
7.1.2.1;Input Layer;311
7.1.2.2;Hidden Layer(s);311
7.1.2.3;Output Layer;312
7.1.3;Shallow vs. Deep Neural Networks;312
7.1.4;Activation Functions;314
7.1.4.1;Types of Activation Functions;316
7.1.4.2;Recap;322
7.1.5;Gradient Descent;322
7.1.5.1;Recap;324
7.1.6;Batch vs. Stochastic vs. Mini-Batch Gradient Descent;325
7.1.6.1;Batch Gradient Descent;325
7.1.6.2;Stochastic Gradient Descent;326
7.1.6.3;Mini-batch Gradient Descent;327
7.1.6.4;Recap;328
7.1.7;Loss function and Backpropagation;329
7.1.7.1;Loss Function;333
7.1.7.2;Backpropagation;336
7.1.7.2.1;The Four Fundamental Equations Behind Backpropagation;339
7.1.8;Exploding Gradients;347
7.1.8.1;Re-Design the Network Model;349
7.1.8.2;Use Long Short-Term Memory Networks;349
7.1.8.3;Use Gradient Clipping;349
7.1.8.4;Use Weight Regularization;350
7.1.9;Vanishing Gradients;350
7.1.9.1;Vanishing Gradients Problem;351
7.1.10;TensorFlow Basics;353
7.1.10.1;Placeholder vs. Variable vs. Constant;354
7.1.10.2;Gradient-Descent Optimization Methods from a Deep-Learning Perspective;355
7.1.10.3;Learning Rate in the Mini-batch Approach to Stochastic Gradient Descent;360
7.1.11;Summary;360
7.2;Chapter 10: Improving Deep Neural Networks;361
7.2.1;Optimizers in TensorFlow;361
7.2.1.1;The Notation to Use;362
7.2.2;Momentum;363
7.2.2.1;Nesterov Accelerated Gradient;364
7.2.2.2;Adagrad;365
7.2.2.3;Adadelta;366
7.2.2.4;RMSprop;367
7.2.2.5;Adam;368
7.2.2.6;Nadam (Adam + NAG);369
7.2.3;Choosing the Learning Rate;370
7.2.4;Dropout Layers and Regularization;373
7.2.5;Normalization Techniques;375
7.2.5.1;Batch Normalization;376
7.2.5.2;Weight Normalization;377
7.2.5.3;Layer Normalization;378
7.2.5.4;Instance Normalization;379
7.2.5.5;Group Normalization;380
7.2.6;Summary;381
7.3;Chapter 11: Convolutional Neural Network;383
7.3.1;What is a Convolutional Neural Network;384
7.3.2;Convolution Operation;385
7.3.2.1;One-Dimensional Convolution;385
7.3.2.2;Two-Dimensional Convolution;387
7.3.2.3;Padding and Stride;388
7.3.3;Common Image-Processing Filters;391
7.3.3.1;Mean and Median Filters;391
7.3.3.2;Gaussian Filter;398
7.3.3.3;Sobel Edge-Detection Filter;401
7.3.3.4;Identity Transform;406
7.3.4;Convolutional Neural Networks;406
7.3.5;Layers of Convolutional Neural Networks;407
7.3.5.1;Input Layer;409
7.3.5.2;Convolutional layer;409
7.3.5.3;Pooling Layer;412
7.3.6;Backpropagation Through the Convolutional and Pooling Layers;413
7.3.7;Weight Sharing Through Convolution and Its Advantages;415
7.3.8;Translation Equivariance and Invariance;416
7.3.9;Case Study—Digit Recognition on the CIFAR-10 Dataset;419
7.3.10;Summary;429
7.4;Chapter 12: Sequential Models;430
7.4.1;Recurrent Neural Networks;430
7.4.2;Language Modeling;435
7.4.3;Backpropagation Through Time;439
7.4.4;Vanishing and Exploding Gradient Problems in RNN;444
7.4.5;The Solution to Vanishing and Exploding Gradients Problems in RNNs;447
7.4.6;Long Short-Term Memory;448
7.4.7;Case Study—Digit Identification on the MNIST Dataset;453
7.4.8;Gated Recurrent Unit;453
7.4.9;Bidirectional RNN (Bi-RNN);460
7.4.10;Summary;461
8;Part IV: Applying What You’ve Learned;462
8.1;Chapter 13: Selected Topics in Computer Vision;463
8.1.1;Different Architectures in Convolutional Neural Networks;464
8.1.1.1;LeNet;465
8.1.1.2;AlexNet;467
8.1.1.3;VGG;470
8.1.1.4;ResNet;472
8.1.2;Transfer Learning;474
8.1.2.1;What Is a Pretrained Model, and Why Use It?;475
8.1.2.2;How to Use a Pretrained Model?;476
8.1.2.3;Ways to Fine-Tune the Model;477
8.1.2.4;Pretrained VGG19;478
8.1.3;Summary;483
8.2;Chapter 14: Selected Topics in Natural Language Processing;484
8.2.1;Vector Space Model;485
8.2.2;Vector Representation of Words;488
8.2.3;Word2Vec;489
8.2.3.1;Continuous Bag of Words;489
8.2.3.1.1;Implementing Continuous Bag of Words;492
8.2.3.2;Skip-Gram Model for Word Embeddings;499
8.2.3.2.1;Implementing Skip-Gram;502
8.2.3.3;GloVe;505
8.2.4;Summary;507
8.3;Chapter 15: Applications;508
8.3.1;Case Study—Tabular Dataset;508
8.3.1.1;Understanding the Dataset;508
8.3.1.1.1;Scratching the Surface;510
8.3.1.1.2;Digging Deeper;514
8.3.1.2;Preprocessing Dataset;518
8.3.1.3;Building the Model;523
8.3.2;Case Study—IMDB Movie Review Data with Word2Vec;528
8.3.3;Case Study—Image Segmentation;538
8.3.4;Summary;548
9;Index;549