O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

TensorFlow for Deep Learning

Book Description

Learn how to solve challenging machine learning problems with TensorFlow, Google’s revolutionary new software library for deep learning. If you have some background in basic linear algebra and calculus, this practical book introduces machine-learning fundamentals by showing you how to design systems capable of detecting objects in images, understanding text, analyzing video, and predicting the properties of potential medicines.

TensorFlow for Deep Learning teaches concepts through practical examples and helps you build knowledge of deep learning foundations from the ground up. It’s ideal for practicing developers with experience designing software systems, and useful for scientists and other professionals familiar with scripting but not necessarily with designing learning algorithms.

  • Learn TensorFlow fundamentals, including how to perform basic computation
  • Build simple learning systems to understand their mathematical foundations
  • Dive into fully connected deep networks used in thousands of applications
  • Turn prototypes into high-quality models with hyperparameter optimization
  • Process images with convolutional neural networks
  • Handle natural language datasets with recurrent neural networks
  • Use reinforcement learning to solve games such as tic-tac-toe
  • Train deep networks with hardware including GPUs and tensor processing units

Table of Contents

  1. Preface
    1. Conventions Used in This Book
    2. Using Code Examples
    3. O’Reilly Safari
    4. How to Contact Us
    5. Acknowledgments
  2. 1. Introduction to Deep Learning
    1. Machine Learning Eats Computer Science
    2. Deep Learning Primitives
      1. Fully Connected Layer
      2. Convolutional Layer
      3. Recurrent Neural Network Layers
      4. Long Short-Term Memory Cells
    3. Deep Learning Architectures
      1. LeNet
      2. AlexNet
      3. ResNet
      4. Neural Captioning Model
      5. Google Neural Machine Translation
      6. One-Shot Models
      7. AlphaGo
      8. Generative Adversarial Networks
      9. Neural Turing Machines
    4. Deep Learning Frameworks
      1. Limitations of TensorFlow
    5. Review
  3. 2. Introduction to TensorFlow Primitives
    1. Introducing Tensors
      1. Scalars, Vectors, and Matrices
      2. Matrix Mathematics
      3. Tensors
      4. Tensors in Physics
      5. Mathematical Asides
    2. Basic Computations in TensorFlow
      1. Installing TensorFlow and Getting Started
      2. Initializing Constant Tensors
      3. Sampling Random Tensors
      4. Tensor Addition and Scaling
      5. Matrix Operations
      6. Tensor Types
      7. Tensor Shape Manipulations
      8. Introduction to Broadcasting
    3. Imperative and Declarative Programming
      1. TensorFlow Graphs
      2. TensorFlow Sessions
      3. TensorFlow Variables
    4. Review
  4. 3. Linear and Logistic Regression with TensorFlow
    1. Mathematical Review
      1. Functions and Differentiability
      2. Loss Functions
      3. Gradient Descent
      4. Automatic Differentiation Systems
    2. Learning with TensorFlow
      1. Creating Toy Datasets
      2. New TensorFlow Concepts
    3. Training Linear and Logistic Models in TensorFlow
      1. Linear Regression in TensorFlow
      2. Logistic Regression in TensorFlow
    4. Review
  5. 4. Fully Connected Deep Networks
    1. What Is a Fully Connected Deep Network?
    2. “Neurons” in Fully Connected Networks
      1. Learning Fully Connected Networks with Backpropagation
      2. Universal Convergence Theorem
      3. Why Deep Networks?
    3. Training Fully Connected Neural Networks
      1. Learnable Representations
      2. Activations
      3. Fully Connected Networks Memorize
      4. Regularization
      5. Training Fully Connected Networks
    4. Implementation in TensorFlow
      1. Installing DeepChem
      2. Tox21 Dataset
      3. Accepting Minibatches of Placeholders
      4. Implementing a Hidden Layer
      5. Adding Dropout to a Hidden Layer
      6. Implementing Minibatching
      7. Evaluating Model Accuracy
      8. Using TensorBoard to Track Model Convergence
    5. Review
  6. 5. Hyperparameter Optimization
    1. Model Evaluation and Hyperparameter Optimization
    2. Metrics, Metrics, Metrics
      1. Binary Classification Metrics
      2. Multiclass Classification Metrics
      3. Regression Metrics
    3. Hyperparameter Optimization Algorithms
      1. Setting Up a Baseline
      2. Graduate Student Descent
      3. Grid Search
      4. Random Hyperparameter Search
      5. Challenge for the Reader
    4. Review
  7. 6. Convolutional Neural Networks
    1. Introduction to Convolutional Architectures
      1. Local Receptive Fields
      2. Convolutional Kernels
      3. Pooling Layers
      4. Constructing Convolutional Networks
      5. Dilated Convolutions
    2. Applications of Convolutional Networks
      1. Object Detection and Localization
      2. Image Segmentation
      3. Graph Convolutions
      4. Generating Images with Variational Autoencoders
    3. Training a Convolutional Network in TensorFlow
      1. The MNIST Dataset
      2. Loading MNIST
      3. TensorFlow Convolutional Primitives
      4. The Convolutional Architecture
      5. Evaluating Trained Models
      6. Challenge for the Reader
    4. Review
  8. 7. Recurrent Neural Networks
    1. Overview of Recurrent Architectures
    2. Recurrent Cells
      1. Long Short-Term Memory (LSTM)
      2. Gated Recurrent Units (GRU)
    3. Applications of Recurrent Models
      1. Sampling from Recurrent Networks
      2. Seq2seq Models
    4. Neural Turing Machines
    5. Working with Recurrent Neural Networks in Practice
    6. Processing the Penn Treebank Corpus
      1. Code for Preprocessing
      2. Loading Data into TensorFlow
      3. The Basic Recurrent Architecture
      4. Challenge for the Reader
    7. Review
  9. 8. Reinforcement Learning
    1. Markov Decision Processes
    2. Reinforcement Learning Algorithms
      1. Q-Learning
      2. Policy Learning
      3. Asynchronous Training
    3. Limits of Reinforcement Learning
    4. Playing Tic-Tac-Toe
      1. Object Orientation
      2. Abstract Environment
      3. Tic-Tac-Toe Environment
      4. The Layer Abstraction
      5. Defining a Graph of Layers
    5. The A3C Algorithm
      1. The A3C Loss Function
      2. Defining Workers
      3. Training the Policy
      4. Challenge for the Reader
    6. Review
  10. 9. Training Large Deep Networks
    1. Custom Hardware for Deep Networks
    2. CPU Training
      1. GPU Training
      2. Tensor Processing Units
      3. Field Programmable Gate Arrays
      4. Neuromorphic Chips
    3. Distributed Deep Network Training
      1. Data Parallelism
      2. Model Parallelism
    4. Data Parallel Training with Multiple GPUs on Cifar10
      1. Downloading and Loading the DATA
      2. Deep Dive on the Architecture
      3. Training on Multiple GPUs
      4. Challenge for the Reader
    5. Review
  11. 10. The Future of Deep Learning
    1. Deep Learning Outside the Tech Industry
      1. Deep Learning in the Pharmaceutical Industry
      2. Deep Learning in Law
      3. Deep Learning for Robotics
      4. Deep Learning in Agriculture
    2. Using Deep Learning Ethically
    3. Is Artificial General Intelligence Imminent?
    4. Where to Go from Here?
  12. Index