Python: Advanced Guide to Artificial Intelligence

Python: Advanced Guide to Artificial Intelligence

by Giuseppe Bonaccorso, Armando Fandango, Rajalingappaa Shanmugamani
Released December 2018
Publisher(s): Packt Publishing
ISBN: 9781789957211

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Demystify the complexity of machine learning techniques and create evolving, clever solutions to solve your problems

Key Features

  • Master supervised, unsupervised, and semi-supervised ML algorithms and their implementation
  • Build deep learning models for object detection, image classification, similarity learning, and more
  • Build, deploy, and scale end-to-end deep neural network models in a production environment

Book Description

This Learning Path is your complete guide to quickly getting to grips with popular machine learning algorithms. You'll be introduced to the most widely used algorithms in supervised, unsupervised, and semi-supervised machine learning, and learn how to use them in the best possible manner. Ranging from Bayesian models to the MCMC algorithm to Hidden Markov models, this Learning Path will teach you how to extract features from your dataset and perform dimensionality reduction by making use of Python-based libraries.

You'll bring the use of TensorFlow and Keras to build deep learning models, using concepts such as transfer learning, generative adversarial networks, and deep reinforcement learning. Next, you'll learn the advanced features of TensorFlow1.x, such as distributed TensorFlow with TF clusters, deploy production models with TensorFlow Serving. You'll implement different techniques related to object classification, object detection, image segmentation, and more.

By the end of this Learning Path, you'll have obtained in-depth knowledge of TensorFlow, making you the go-to person for solving artificial intelligence problems

This Learning Path includes content from the following Packt products:

  • Mastering Machine Learning Algorithms by Giuseppe Bonaccorso
  • Mastering TensorFlow 1.x by Armando Fandango
  • Deep Learning for Computer Vision by Rajalingappaa Shanmugamani

What you will learn

  • Explore how an ML model can be trained, optimized, and evaluated
  • Work with Autoencoders and Generative Adversarial Networks
  • Explore the most important Reinforcement Learning techniques
  • Build end-to-end deep learning (CNN, RNN, and Autoencoders) models

Who this book is for

This Learning Path is for data scientists, machine learning engineers, artificial intelligence engineers who want to delve into complex machine learning algorithms, calibrate models, and improve the predictions of the trained model.

You will encounter the advanced intricacies and complex use cases of deep learning and AI. A basic knowledge of programming in Python and some understanding of machine learning concepts are required to get the best out of this Learning Path.

Downloading the example code for this book You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Publisher resources

View/Submit Errata

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Python: Advanced Guide to Artificial Intelligence
  3. About Packt
    1. Why subscribe?
    2. Packt.com
  4. Contributors
    1. About the authors
    2. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Conventions used
    4. Get in touch
      1. Reviews
  6. Machine Learning Model Fundamentals
    1. Models and data
      1. Zero-centering and whitening
      2. Training and validation sets
        1. Cross-validation
    2. Features of a machine learning model
      1. Capacity of a model
        1. Vapnik-Chervonenkis capacity
      2. Bias of an estimator
        1. Underfitting
      3. Variance of an estimator
        1. Overfitting
        2. The Cramér-Rao bound
    3. Loss and cost functions
      1. Examples of cost functions
        1. Mean squared error
        2. Huber cost function
        3. Hinge cost function
        4. Categorical cross-entropy
      2. Regularization
        1. Ridge
        2. Lasso
        3. ElasticNet
        4. Early stopping
    4. Summary
  7. Introduction to Semi-Supervised Learning
    1. Semi-supervised scenario
      1. Transductive learning
      2. Inductive learning
      3. Semi-supervised assumptions
        1. Smoothness assumption
        2. Cluster assumption
        3. Manifold assumption
    2. Generative Gaussian mixtures
      1. Example of a generative Gaussian mixture
        1. Weighted log-likelihood
    3. Contrastive pessimistic likelihood estimation
      1. Example of contrastive pessimistic likelihood estimation
    4. Semi-supervised Support Vector Machines (S3VM)
      1. Example of S3VM
    5. Transductive Support Vector Machines (TSVM)
      1. Example of TSVM
    6. Summary
  8. Graph-Based Semi-Supervised Learning
    1. Label propagation
      1. Example of label propagation
      2. Label propagation in Scikit-Learn
    2. Label spreading
      1. Example of label spreading
    3. Label propagation based on Markov random walks
      1. Example of label propagation based on Markov random walks
    4. Manifold learning
      1. Isomap
        1. Example of Isomap
      2. Locally linear embedding
        1. Example of locally linear embedding
      3. Laplacian Spectral Embedding
        1. Example of Laplacian Spectral Embedding
      4. t-SNE
        1. Example of t-distributed stochastic neighbor embedding 
    5. Summary
  9. Bayesian Networks and Hidden Markov Models
    1. Conditional probabilities and Bayes' theorem
    2. Bayesian networks
      1. Sampling from a Bayesian network
        1. Direct sampling
          1. Example of direct sampling
        2. A gentle introduction to Markov chains
        3. Gibbs sampling
        4. Metropolis-Hastings sampling
          1. Example of Metropolis-Hastings sampling
      2. Sampling example using PyMC3
    3. Hidden Markov Models (HMMs)
      1. Forward-backward algorithm
        1. Forward phase
        2. Backward phase
        3. HMM parameter estimation
          1. Example of HMM training with hmmlearn
      2. Viterbi algorithm
        1. Finding the most likely hidden state sequence with hmmlearn
    4. Summary
  10. EM Algorithm and Applications
    1. MLE and MAP learning
    2. EM algorithm
      1. An example of parameter estimation
    3. Gaussian mixture
      1. An example of Gaussian Mixtures using Scikit-Learn
    4. Factor analysis
      1. An example of factor analysis with Scikit-Learn
    5. Principal Component Analysis
      1. An example of PCA with Scikit-Learn
    6. Independent component analysis
      1. An example of FastICA with Scikit-Learn
    7. Addendum to HMMs
    8. Summary
  11. Hebbian Learning and Self-Organizing Maps
    1. Hebb's rule
      1. Analysis of the covariance rule
        1. Example of covariance rule application
      2. Weight vector stabilization and Oja's rule
    2. Sanger's network
      1. Example of Sanger's network
    3. Rubner-Tavan's network
      1. Example of Rubner-Tavan's network
    4. Self-organizing maps
      1. Example of SOM
    5. Summary
  12. Clustering Algorithms
    1. k-Nearest Neighbors
      1. KD Trees
      2. Ball Trees
      3. Example of KNN with Scikit-Learn
    2. K-means
      1. K-means++
      2. Example of K-means with Scikit-Learn
        1. Evaluation metrics
          1. Homogeneity score
          2. Completeness score
          3. Adjusted Rand Index
          4. Silhouette score
    3. Fuzzy C-means
      1. Example of fuzzy C-means with Scikit-Fuzzy
    4. Spectral clustering
      1. Example of spectral clustering with Scikit-Learn
    5. Summary
  13. Advanced Neural Models
    1. Deep convolutional networks
      1. Convolutions
        1. Bidimensional discrete convolutions
          1. Strides and padding
        2. Atrous convolution
        3. Separable convolution
        4. Transpose convolution
      2. Pooling layers
      3. Other useful layers
      4. Examples of deep convolutional networks with Keras
        1. Example of a deep convolutional network with Keras and data augmentation
    2. Recurrent networks
      1. Backpropagation through time (BPTT)
      2. LSTM
      3. GRU
      4. Example of an LSTM network with Keras
    3. Transfer learning
    4. Summary
  14. Classical Machine Learning with TensorFlow
    1. Simple linear regression
      1. Data preparation
      2. Building a simple regression model
        1. Defining the inputs, parameters, and other variables
        2. Defining the model
        3. Defining the loss function
        4. Defining the optimizer function
        5. Training the model
      3. Using the trained model to predict
    2. Multi-regression
    3. Regularized regression
      1. Lasso regularization
      2. Ridge regularization
      3. ElasticNet regularization
    4. Classification using logistic regression
      1. Logistic regression for binary classification
      2. Logistic regression for multiclass classification
    5. Binary classification
    6. Multiclass classification
    7. Summary
  15. Neural Networks and MLP with TensorFlow and Keras
    1. The perceptron
    2. MultiLayer Perceptron
    3. MLP for image classification
      1. TensorFlow-based MLP for MNIST classification
      2. Keras-based MLP for MNIST classification
      3. TFLearn-based MLP for MNIST classification
      4. Summary of MLP with TensorFlow, Keras, and TFLearn
    4. MLP for time series regression
    5. Summary
  16. RNN with TensorFlow and Keras
    1. Simple Recurrent Neural Network
    2. RNN variants
    3. LSTM network
    4. GRU network
    5. TensorFlow for RNN
      1. TensorFlow RNN Cell Classes
      2. TensorFlow RNN Model Construction Classes
      3. TensorFlow RNN Cell Wrapper Classes
    6. Keras for RNN
    7. Application areas of RNNs
    8. RNN in Keras for MNIST data
    9. Summary
  17. CNN with TensorFlow and Keras
    1. Understanding convolution
    2. Understanding pooling
    3. CNN architecture pattern - LeNet
    4. LeNet for MNIST data
      1. LeNet CNN for MNIST with TensorFlow
      2. LeNet CNN for MNIST with Keras
    5. LeNet for CIFAR10 Data
      1. ConvNets for CIFAR10 with TensorFlow
      2. ConvNets for CIFAR10 with Keras
    6. Summary
  18. Autoencoder with TensorFlow and Keras
    1. Autoencoder types
    2. Stacked autoencoder in TensorFlow
    3. Stacked autoencoder in Keras
    4. Denoising autoencoder in TensorFlow
    5. Denoising autoencoder in Keras
    6. Variational autoencoder in TensorFlow
    7. Variational autoencoder in Keras
    8. Summary
  19. TensorFlow Models in Production with TF Serving
    1. Saving and Restoring models in TensorFlow
      1. Saving and restoring all graph variables with the saver class
      2. Saving and restoring selected  variables with the saver class
    2. Saving and restoring Keras models
    3. TensorFlow Serving
      1. Installing TF Serving
      2. Saving models for TF Serving
      3. Serving models with TF Serving
    4. TF Serving in the Docker containers
      1. Installing Docker
      2. Building a Docker image for TF serving
      3. Serving the model in the Docker container
    5. TensorFlow Serving on Kubernetes
      1. Installing Kubernetes
      2. Uploading the Docker image to the dockerhub
      3. Deploying in Kubernetes
    6. Summary
  20. Deep Reinforcement Learning
    1. OpenAI Gym 101
    2. Applying simple policies to a cartpole game
    3. Reinforcement learning 101
      1. Q function (learning to optimize when the model is not available)
      2. Exploration and exploitation in the RL algorithms
      3. V function (learning to optimize when the model is available)
      4. Reinforcement learning techniques
    4. Naive Neural Network policy for Reinforcement Learning
    5. Implementing Q-Learning
      1. Initializing and discretizing for Q-Learning
      2. Q-Learning with Q-Table
      3. Q-Learning with Q-Network  or Deep Q Network (DQN) 
    6. Summary
  21. Generative Adversarial Networks
    1. Generative Adversarial Networks 101
    2. Best practices for building and training GANs
    3. Simple GAN with TensorFlow
    4. Simple GAN with Keras
    5. Deep Convolutional GAN with TensorFlow and Keras
    6. Summary
  22. Distributed Models with TensorFlow Clusters
    1. Strategies for distributed execution
    2. TensorFlow clusters
      1. Defining cluster specification
      2. Create the server instances
      3. Define the parameter and operations across servers and devices
      4. Define and train the graph for asynchronous updates
      5. Define and train the graph for synchronous updates
    3. Summary
  23. Debugging TensorFlow Models
    1. Fetching tensor values with tf.Session.run()
    2. Printing tensor values with tf.Print()
    3. Asserting on conditions with tf.Assert()
    4. Debugging with the TensorFlow debugger (tfdbg)
    5. Summary
  24. Tensor Processing Units
  25. Getting Started
    1. Understanding deep learning
      1. Perceptron
      2. Activation functions
        1. Sigmoid
        2. The hyperbolic tangent function
        3. The Rectified Linear Unit (ReLU)
      3. Artificial neural network (ANN)
        1. One-hot encoding
        2. Softmax
        3. Cross-entropy
        4. Dropout
        5. Batch normalization
        6. L1 and L2 regularization
      4. Training neural networks
        1. Backpropagation
        2. Gradient descent
        3. Stochastic gradient descent
      5. Playing with TensorFlow playground
      6. Convolutional neural network
        1. Kernel
        2. Max pooling
      7. Recurrent neural networks (RNN)
      8. Long short-term memory (LSTM)
    2. Deep learning for computer vision
      1. Classification
      2. Detection or localization and segmentation
      3. Similarity learning
      4. Image captioning
      5. Generative models
      6. Video analysis
    3. Development environment setup
      1. Hardware and Operating Systems - OS
        1. General Purpose - Graphics Processing Unit (GP-GPU)
          1. Computer Unified Device Architecture - CUDA
          2. CUDA Deep Neural Network - CUDNN
      2. Installing software packages
        1. Python
        2. Open Computer Vision - OpenCV
        3. The TensorFlow library
          1. Installing TensorFlow
          2. TensorFlow example to print Hello, TensorFlow
          3. TensorFlow example for adding two numbers
          4. TensorBoard
          5. The TensorFlow Serving tool
        4. The Keras library
    4. Summary
  26. Image Classification
    1. The bigger deep learning models
      1. The AlexNet model
      2. The VGG-16 model
      3. The Google Inception-V3 model
      4. The Microsoft ResNet-50 model
      5. The SqueezeNet model
      6. Spatial transformer networks
      7. The DenseNet model
    2. Training a model for cats versus dogs
      1. Preparing the data
      2. Benchmarking with simple CNN
      3. Augmenting the dataset
        1. Augmentation techniques 
      4. Transfer learning or fine-tuning of a model
        1. Training on bottleneck features
      5. Fine-tuning several layers in deep learning
    3. Developing real-world applications
      1. Choosing the right model
      2. Tackling the underfitting and overfitting scenarios
      3. Gender and age detection from face
      4. Fine-tuning apparel models 
      5. Brand safety
    4. Summary
  27. Image Retrieval
    1. Understanding visual features
      1. Visualizing activation of deep learning models
      2. Embedding visualization
        1. Guided backpropagation
      3. The DeepDream
      4. Adversarial examples
    2. Model inference
      1. Exporting a model
      2. Serving the trained model 
    3. Content-based image retrieval
      1. Building the retrieval pipeline
        1. Extracting bottleneck features for an image
        2. Computing similarity between query image and target database
      2. Efficient retrieval
        1. Matching faster using approximate nearest neighbour
          1. Advantages of ANNOY
        2. Autoencoders of raw images
      3. Denoising using autoencoders
    4. Summary
  28. Object Detection
    1. Detecting objects in an image
    2. Exploring the datasets
      1. ImageNet dataset
      2. PASCAL VOC challenge
      3. COCO object detection challenge
      4. Evaluating datasets using metrics
        1. Intersection over Union
        2. The mean average precision
    3. Localizing algorithms 
      1. Localizing objects using sliding windows
        1. The scale-space concept
        2. Training a fully connected layer as a convolution layer
        3. Convolution implementation of sliding window
      2. Thinking about localization as a regression problem
        1. Applying regression to other problems
        2. Combining regression with the sliding window
    4. Detecting objects
      1. Regions of the convolutional neural network (R-CNN)
      2. Fast R-CNN
      3. Faster R-CNN
      4. Single shot multi-box detector
    5. Object detection API
      1. Installation and setup
      2. Pre-trained models
      3. Re-training object detection models
        1. Data preparation for the Pet dataset
        2. Object detection training pipeline
        3. Training the model
        4. Monitoring loss and accuracy using TensorBoard
      4. Training a pedestrian detection for a self-driving car
    6. The YOLO object detection algorithm 
    7. Summary
  29. Semantic Segmentation
    1. Predicting pixels
      1. Diagnosing medical images
      2. Understanding the earth from satellite imagery
      3. Enabling robots to see
    2. Datasets
    3. Algorithms for semantic segmentation
      1. The Fully Convolutional Network
      2. The SegNet architecture
        1. Upsampling the layers by pooling
        2. Sampling the layers by convolution
        3. Skipping connections for better training
      3. Dilated convolutions
      4. DeepLab
      5. RefiNet
      6. PSPnet
      7. Large kernel matters
      8. DeepLab v3
    4. Ultra-nerve segmentation
    5. Segmenting satellite images
      1. Modeling FCN for segmentation
    6. Segmenting instances
    7. Summary
  30. Similarity Learning
    1. Algorithms for similarity learning
      1. Siamese networks
        1. Contrastive loss
      2. FaceNet
        1. Triplet loss
      3. The DeepNet model
      4. DeepRank
      5. Visual recommendation systems
    2. Human face analysis
      1. Face detection
      2. Face landmarks and attributes
        1. The Multi-Task Facial Landmark (MTFL) dataset
        2. The Kaggle keypoint dataset
        3. The Multi-Attribute Facial Landmark (MAFL) dataset
        4. Learning the facial key points
      3. Face recognition
        1. The labeled faces in the wild (LFW) dataset
        2. The YouTube faces dataset
        3. The CelebFaces Attributes dataset (CelebA) 
        4. CASIA web face database
        5. The VGGFace2 dataset
        6. Computing the similarity between faces
        7. Finding the optimum threshold
      4. Face clustering 
    3. Summary
  31. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Python: Advanced Guide to Artificial Intelligence
  • Author(s): Giuseppe Bonaccorso, Armando Fandango, Rajalingappaa Shanmugamani
  • Release date: December 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781789957211