You are previewing Building Machine Learning Projects with TensorFlow.
O'Reilly logo
Building Machine Learning Projects with TensorFlow

Book Description

Engaging projects that will teach you how complex data can be exploited to gain the most insight

About This Book

  • Bored of too much theory on TensorFlow? This book is what you need! Thirteen solid projects and four examples teach you how to implement TensorFlow in production.

  • This example-rich guide teaches you how to perform highly accurate and efficient numerical computing with TensorFlow

  • It is a practical and methodically explained guide that allows you to apply Tensorflow’s features from the very beginning.

  • Who This Book Is For

    This book is for data analysts, data scientists, and researchers who want to increase the speed and efficiency of their machine learning activities and results. Anyone looking for a fresh guide to complex numerical computations with TensorFlow will find this an extremely helpful resource. This book is also for developers who want to implement TensorFlow in production in various scenarios. Some experience with C++ and Python is expected.

    What You Will Learn

  • Load, interact, dissect, process, and save complex datasets

  • Solve classification and regression problems using state of the art techniques

  • Predict the outcome of a simple time series using Linear Regression modeling

  • Use a Logistic Regression scheme to predict the future result of a time series

  • Classify images using deep neural network schemes

  • Tag a set of images and detect features using a deep neural network, including a Convolutional Neural Network (CNN) layer

  • Resolve character recognition problems using the Recurrent Neural Network (RNN) model

  • In Detail

    This book of projects highlights how TensorFlow can be used in different scenarios - this includes projects for training models, machine learning, deep learning, and working with various neural networks. Each project provides exciting and insightful exercises that will teach you how to use TensorFlow and show you how layers of data can be explored by working with Tensors. Simply pick a project that is in line with your environment and get stacks of information on how to implement TensorFlow in production.

    Style and approach

    This book is a practical guide to implementing TensorFlow in production. It explores various scenarios in which you could use TensorFlow and shows you how to use it in the context of real world projects. This will not only give you an upper hand in the field, but shows the potential for innovative uses of TensorFlow in your environment. This guide opens the door to second generation machine learning and numerical computation – a must-have for your bookshelf!

    Downloading the example code for this book. You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the code file.

    Table of Contents

    1. Building Machine Learning Projects with TensorFlow
      1. Building Machine Learning Projects with TensorFlow
      2. Credits
      3. About the Author
      4. About the Reviewer
      5. www.PacktPub.com
        1. Why subscribe?
      6. Preface
        1. What this book covers 
        2. What you need for this book 
        3. Who this book is for 
        4. Conventions 
        5. Reader feedback
        6. Customer support
          1. Downloading the example code 
          2. Errata
          3. Piracy
          4. Questions
      7. 1. Exploring and Transforming Data
        1. TensorFlow's main data structure - tensors
          1. Tensor properties - ranks, shapes, and types
            1. Tensor rank
            2. Tensor shape
            3. Tensor data types
          2. Creating new tensors
            1. From numpy to tensors and vice versa
              1. Useful method
          3. Getting things done - interacting with TensorFlow
        2. Handling the computing workflow - TensorFlow's data flow graph
          1. Computation graph building
            1. Useful operation object methods
          2. Feeding
          3. Variables
            1. Variable initialization
          4. Saving data flow graphs
            1. Graph serialization language - protocol buffers
              1. Useful methods
            2. Example graph building
        3. Running our programs - Sessions
        4. Basic tensor methods
          1. Simple matrix operations
            1. Reduction
            2. Tensor segmentation
          2. Sequences
          3. Tensor shape transformations
            1. Tensor slicing and joining
          4. Dataflow structure and results visualization - TensorBoard
            1. Command line use
          5. How TensorBoard works
            1. Adding Summary nodes
            2. Common Summary operations
            3. Special Summary functions
            4. Interacting with TensorBoard's GUI
          6. Reading information from disk
            1. Tabulated formats - CSV
              1. The Iris dataset
            2. Reading image data
            3. Loading and processing the images
            4. Reading from the standard TensorFlow format
        5. Summary
      8. 2. Clustering
        1. Learning from data - unsupervised learning
        2. Clustering
        3. k-means
          1. Mechanics of k-means
            1. Algorithm iteration criterion
          2. k-means algorithm breakdown
            1. Pros and cons of k-means
        4. k-nearest neighbors
          1. Mechanics of k-nearest neighbors
            1. Pros and cons of k-nn
          2. Practical examples for Useful libraries
            1. matplotlib plotting library
              1. Sample synthetic data plotting
            2. scikit-learn dataset module
              1. About the scikit-learn library
            3. Synthetic dataset types
            4. Blobs dataset
              1. Employed method
            5. Circle dataset
              1. Employed method
            6. Moon dataset
        5. Project 1 - k-means clustering on synthetic datasets
          1. Dataset description and loading
            1. Generating the dataset
          2. Model architecture
          3. Loss function description and optimizer loop
          4. Stop condition
          5. Results description
          6. Full source code
          7. k-means on circle synthetic data
        6. Project 2 - nearest neighbor on synthetic datasets
          1. Dataset generation
          2. Model architecture
          3. Loss function description
          4. Stop condition
          5. Results description
          6. Full source code
        7. Summary
      9. 3. Linear Regression
        1. Univariate linear modelling function
          1. Sample data generation
        2. Determination of the cost function
          1. Least squares
        3. Minimizing the cost function
          1. General minima for least squares
          2. Iterative methods - gradient descent
        4. Example section
          1. Optimizer methods in TensorFlow - the train module
            1. The tf.train.Optimizer class
            2. Other Optimizer instance types
        5. Example 1 - univariate linear regression
          1. Dataset description
          2. Model architecture
          3. Cost function description and Optimizer loop
          4. Stop condition
          5. Results description
          6. Reviewing results with TensorBoard
          7. Full source code
        6. Example - multivariate linear regression
          1. Useful libraries and methods
            1. Pandas library
          2. Dataset description
          3. Model architecture
          4. Loss function description and Optimizer loop
          5. Stop condition
          6. Results description
          7. Full source code
        7. Summary
      10. 4. Logistic Regression
        1. Problem description
        2. Logistic function predecessor - the logit functions
          1. Bernoulli distribution
          2. Link function
          3. Logit function
          4. The importance of the logit inverse
        3. The logistic function
          1. Logistic function as a linear modeling generalization
            1. Final estimated regression equation
          2. Properties of the logistic function
            1. Loss function
          3. Multiclass application - softmax regression
            1. Cost function
            2. Data normalization for iterative methods
            3. One hot representation of outputs
        4. Example 1 - univariate logistic regression
          1. Useful libraries and methods
            1. TensorFlow's softmax implementation
          2. Dataset description and loading
            1. The CHDAGE dataset
              1. CHDAGE dataset format
              2. Dataset loading and preprocessing implementation
          3. Model architecture
          4. Loss function description and optimizer loop
          5. Stop condition
          6. Results description
            1. Fitting function representations across epochs
          7. Full source code
            1. Graphical representation
        5. Example 2 - Univariate logistic regression with skflow
          1. Useful libraries and methods
          2. Dataset description
          3. Model architecture
          4. Results description
          5. Full source code
        6. Summary
      11. 5. Simple FeedForward Neural Networks
        1. Preliminary concepts
          1. Artificial neurons
          2. Original example - the Perceptron
            1. Perceptron algorithm
          3. Neural network layers
          4. Neural Network activation functions
          5. Gradients and the back propagation algorithm
          6. Minimizing loss function: Gradient descent
            1. Neural networks problem choice - Classification vs Regression
          7. Useful libraries and methods
            1. TensorFlow activation functions
            2. TensorFlow loss optimization methods
            3. Sklearn preprocessing utilities
        2. First project - Non linear synthetic function regression
          1. Dataset description and loading
          2. Dataset preprocessing
          3. Modeling architecture - Loss Function description
          4. Loss function optimizer
          5. Accuracy and Convergence test
            1. Example code
          6. Results description
        3. Second project - Modeling cars fuel efficiency with non linear regression
          1. Dataset description and loading
          2. Dataset preprocessing
          3. Modeling architecture
          4. Convergency test
          5. Results description
        4. Third project - Learning to classify wines: Multiclass classification
          1. Dataset description and loading
          2. Dataset preprocessing
          3. Modeling architecture
          4. Loss function description
          5. Loss function optimizer
          6. Convergence test
          7. Results description
          8. Full source code
        5. Summary
      12. 6. Convolutional Neural Networks
        1. Origin of convolutional neural networks
          1. Getting started with convolution
            1. Continuous convolution
            2. Discrete convolution
          2. Kernels and convolutions
          3. Interpretation of the convolution operations
          4. Applying convolution in TensorFlow
            1. Other convolutional operations
            2. Sample code - applying convolution to a grayscale image
            3. Sample kernels results
          5. Subsampling operation - pooling
            1. Properties of subsampling layers
            2. Invariance property
            3. Subsampling layers implementation performance.
            4. Applying pool operations in TensorFlow
            5. Other pool operations
            6. Sample code
          6. Improving efficiency - dropout operation
            1. Applying the dropout operation in TensorFlow
              1. Sample code
          7. Convolutional type layer building methods
            1. Convolutional layer
            2. Subsampling layer
        2. Example 1 - MNIST digit classification
          1. Dataset description and loading
          2. Dataset preprocessing
          3. Modelling architecture
          4. Loss function description
          5. Loss function optimizer
          6. Accuracy test
          7. Result description
          8. Full source code
        3. Example 2 - image classification with the CIFAR10 dataset
          1. Dataset description and loading
          2. Dataset preprocessing
          3. Modelling architecture
          4. Loss function description and optimizer
            1. Training and accuracy tests
          5. Results description
            1. Full source code
        4. Summary
      13. 7. Recurrent Neural Networks and LSTM
        1. Recurrent neural networks
          1. Exploding and vanishing gradients
          2. LSTM neural networks
          3. The gate operation - a fundamental component
          4. Operation steps
            1. Part 1 - set values to forget (input gate)
            2. Part 2 - set values to keep, change state
            3. Part 3 - output filtered cell state
          5. Other RNN architectures
          6. TensorFlow LSTM useful classes and methods
            1. class tf.nn.rnn_cell.BasicLSTMCell
            2. class MultiRNNCell(RNNCell)
            3. learn.ops.split_squeeze(dim, num_split, tensor_in)
        2. Example 1 - univariate time series prediction with energy consumption data
          1. Dataset description and loading
          2. Dataset preprocessing
          3. Modelling architecture
          4. Loss function description
          5. Convergency test
          6. Results description
          7. Full source code
        3. Example 2 - writing music "a la" Bach
          1. Character level models
            1. Character sequences and probability representation
            2. Encoding music as characters - the ABC music format
              1. ABC format data organization
            3. Useful libraries and methods
            4. Saving and restoring variables and models
            5. Loading and saving pseudocode
              1. Variable saving
              2. Variable restoring
          2. Dataset description and loading
          3.  Network Training
          4. Dataset preprocessing
            1. Vocabulary definition
            2. Modelling architecture
          5. Loss function description
          6. Stop condition
          7. Results description
          8. Full source code
        4. Summary
      14. 8. Deep Neural Networks
        1. Deep neural network definition
        2. Deep network architectures through time
          1. LeNet 5
        3. Alexnet
          1. Main features
          2. The original inception model
          3. GoogLenet (Inception V1)
          4. Batch normalized inception (V2)
        4. Inception v3
        5. Residual Networks (ResNet)
          1. Other deep neural network architectures
        6. Example - painting with style - VGG style transfer
          1. Useful libraries and methods
          2. Dataset description and loading
          3. Dataset preprocessing
          4. Modeling architecture
          5. Loss functions
            1. Content loss function
          6. Style loss function
            1. Loss optimization loop
          7. Convergency test
          8. Program execution
          9. Full source code
        7. Summary
      15. 9. Running Models at Scale – GPU and Serving
        1. GPU support on TensorFlow
          1. Log device placement and device capabilities
            1. Querying the computing capabilities
          2. Selecting a CPU for computing
            1. Device naming
        2. Example 1 - assigning an operation to the GPU
        3. Example 2 - calculating Pi number in parallel
          1. Solution implementation
          2. Source code
        4. Distributed TensorFlow
          1. Technology components
            1. Jobs
            2. Tasks
            3. Servers
              1. Combined overview
            4. Creating a TensorFlow cluster
              1. ClusterSpec definition format
              2. Creating tf.Train.Server
          2. Cluster operation - sending computing methods to tasks
            1. Sample distributed code structure
        5. Example 3 - distributed Pi calculation
          1. Server script
          2. Client script
          3. Full source code
        6. Example 4 - running a distributed model in a cluster
          1. Sample code
        7. Summary
      16. 10. Library Installation and Additional Tips
        1. Linux installation
          1. Initial requirements
          2. Ubuntu preparation tasks (need to apply before any method)
          3. Pip Linux installation method
            1. CPU version
            2. Testing your installation
            3. GPU support
          4. Virtualenv installation method
            1. Environment test
          5. Docker installation method
            1. Installing Docker
            2. Allowing Docker to run with a normal user
            3. Reboot
            4. Testing the Docker installation
            5. Run the TensorFlow container
          6. Linux installation from source
            1. Installing the Git source code version manager
              1. Git installation in Linux (Ubuntu 16.04)
            2. Installing the Bazel build tool
              1. Adding the Bazel distribution URI as a package source
              2. Updating and installing Bazel
            3. Installing GPU support (optional)
            4. Installing CUDA system packages
              1. Creating alternative locations
              2. Installing cuDNN
            5. Clone TensorFlow source
            6. Configuring TensorFlow build
            7. Building TensorFlow
            8. Testing the installation
        2. Windows installation
          1. Classic Docker toolbox method
            1. Installation steps
              1. Downloading the Docker toolbox installer
              2. Creating the Docker machine
        3. MacOS X installation
          1. Install pip
        4. Summary