O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

GCP: Complete Google Data Engineer and Cloud Architect Guide

Video Description

The Google Cloud for ML with TensorFlow, Big Data with Managed Hadoop

About This Video

  • Certification stuff - Covers pretty much all of the material you ought to need to get past the Google Data Engineer and Cloud Architect certification tests
  • Compute and Storage - AppEngine, Container Enginer (aka Kubernetes) and Compute Engine
  • Big Data and Managed Hadoop - Dataproc, Dataflow, BigTable, BigQuery, Pub/Sub
  • TensorFlow on the Cloud - what neural networks and deep learning really are, how neurons work and how neural networks are trained.
  • DevOps stuff - StackDriver logging, monitoring, cloud deployment manager
  • Security - Identity and Access Management, Identity-Aware proxying, OAuth, API Keys, service accounts
  • Networking - Virtual Private Clouds, shared VPCs, Load balancing at the network, transport and HTTP layer; VPN, Cloud Interconnect and CDN Interconnect
  • Hadoop Foundations: A quick look at the open-source cousins (Hadoop, Spark, Pig, Hive and HBase)

In Detail

This course is a really comprehensive guide to the Google Cloud Platform - it has ~20 hours of content and ~60 demos. The Google Cloud Platform is not currently the most popular cloud offering out there - that's AWS of course - but it is possibly the best cloud offering for high-end machine learning applications. That's because TensorFlow, the super-popular deep learning technology is also from Google.

Table of Contents

  1. Chapter 1 : You, This Course and Us
    1. You, This Course and Us 00:02:01
  2. Chapter 2 : Introduction
    1. Theory, Practice and Tests 00:10:26
    2. Why Cloud? 00:09:43
    3. Hadoop and Distributed Computing 00:09:01
    4. On-premise, Colocation or Cloud? 00:10:05
    5. Introducing the Google Cloud Platform 00:13:20
    6. Lab: Setting Up A GCP Account 00:07:00
    7. Lab: Using The Cloud Shell 00:06:01
  3. Chapter 3 : Compute Choices
    1. Compute Options 00:09:17
    2. Google Compute Engine (GCE) 00:07:39
    3. More GCE 00:08:13
    4. Lab: Creating a VM Instance 00:06:00
    5. Lab: Editing a VM Instance 00:04:45
    6. Lab: Creating a VM Instance Using The Command Line 00:04:44
    7. Lab: Creating And Attaching A Persistent Disk 00:04:00
    8. Google Container Engine - Kubernetes (GKE) 00:10:34
    9. More GKE 00:09:54
    10. Lab: Creating A Kubernetes Cluster And Deploying A Wordpress Container 00:06:55
    11. App Engine 00:06:49
    12. Contrasting App Engine, Compute Engine and Container Engine 00:06:03
    13. Lab: Deploy and Run An App Engine App 00:07:30
  4. Chapter 4 : Storage
    1. Storage Options 00:09:48
    2. Quick Take 00:13:41
    3. Cloud Storage 00:10:38
    4. Lab: Working With Cloud Storage Buckets 00:05:25
    5. Lab: Bucket And Object Permissions 00:03:52
    6. Lab: Life cycle Management On Buckets 00:05:06
    7. Lab: Running a Program On a VM Instance And Storing Results on Cloud Storage 00:07:09
    8. Transfer Service 00:05:07
    9. Lab: Migrating Data Using the Transfer Service 00:05:33
  5. Chapter 5 : Cloud SQL, Cloud Spanner ~ OLTP ~ RDBMS
    1. Cloud SQL 00:07:40
    2. Lab: Creating A Cloud SQL Instance 00:07:55
    3. Lab: Running Commands On Cloud SQL Instance 00:06:31
    4. Lab: Bulk Loading Data Into Cloud SQL Tables 00:09:09
    5. Cloud Spanner 00:07:26
    6. More Cloud Spanner 00:09:18
    7. Lab: Working With Cloud Spanner 00:06:50
  6. Chapter 6 : BigTable ~ HBase = Columnar Store.
    1. BigTable Intro 00:07:57
    2. Columnar Store 00:08:13
    3. Denormalised 00:09:03
    4. Column Families 00:08:10
    5. BigTable Performance 00:13:19
    6. Lab: BigTable demo 00:07:40
  7. Chapter 7 : Datastore ~ Document Database
    1. Datastore 00:14:11
    2. Lab: Datastore demo 00:06:42
  8. Chapter 8 : BigQuery ~ Hive ~ OLAP
    1. BigQuery Intro 00:11:03
    2. BigQuery Advanced 00:10:00
    3. Lab: Loading CSV Data Into Big Query 00:09:04
    4. Lab: Running Queries On Big Query 00:05:27
    5. Lab: Loading JSON Data With Nested Tables 00:07:28
    6. Lab: Public Datasets In Big Query 00:08:16
    7. Lab: Using Big Query Via The Command Line 00:07:45
    8. Lab: Aggregations And Conditionals In Aggregations 00:09:52
    9. Lab: Subqueries And Joins 00:05:45
    10. Lab: Regular Expressions In Legacy SQL 00:05:36
    11. Lab: Using The With Statement For SubQueries 00:10:46
  9. Chapter 9 : Dataflow ~ Apache Beam
    1. Data Flow Intro 00:11:05
    2. Apache Beam 00:03:43
    3. Lab: Running A Python Data flow Program 00:12:57
    4. Lab: Running A Java Data flow Program 00:13:42
    5. Lab: Implementing Word Count In Dataflow Java 00:11:18
    6. Lab: Executing The Word Count Dataflow 00:04:38
    7. Lab: Executing MapReduce In Dataflow In Python 00:09:50
    8. Lab: Executing MapReduce In Dataflow In Java 00:06:08
    9. Lab: Dataflow With Big Query As Source And Side Inputs 00:15:50
    10. Lab: Dataflow With Big Query As Source And Side Inputs 2 00:06:28
  10. Chapter 10 : Dataproc ~ Managed Hadoop
    1. Data Proc 00:08:28
    2. Lab: Creating And Managing A Dataproc Cluster 00:08:11
    3. Lab: Creating A Firewall Rule To Access Dataproc 00:08:25
    4. Lab: Running A PySpark Job OnDataproc 00:07:39
    5. Lab: Running ThePySpark REPL Shell And Pig Scripts On Dataproc 00:08:45
    6. Lab: Submitting A Spark Jar ToDataproc 00:02:11
    7. Lab: Working With Dataproc Using TheGCloud CLI 00:08:19
  11. Chapter 11 : Pub/Sub for Streaming.
    1. Pub Sub 00:08:23
    2. Lab: Working With Pubsub On The Command Line 00:05:35
    3. Lab: Working WithPubSub Using The Web Console 00:04:40
    4. Lab: Setting Up A Pubsub Publisher Using The Python Library 00:05:53
    5. Lab: Setting Up A Pubsub Subscriber Using The Python Library 00:04:09
    6. Lab: Publishing Streaming Data IntoPubsub 00:08:19
    7. Lab: Reading Streaming Data FromPubSub And Writing To BigQuery 00:10:14
    8. Lab: Executing A Pipeline To Read Streaming Data And Write To BigQuery 00:05:55
    9. Lab: Pubsub Source BigQuery Sink 00:10:20
  12. Chapter 12 : Datalab ~ Jupyter
    1. Data Lab 00:03:00
    2. Lab: Creating And Working On A Datalab Instance 00:10:30
    3. Lab: Importing And Exporting Data Using Datalab 00:12:15
    4. Lab: Using the Charting API InDatalab 00:06:43
  13. Chapter 13 : TensorFlow and Machine Learning
    1. Introducing Machine Learning 00:08:04
    2. Representation Learning 00:10:28
    3. NN Introduced 00:07:35
    4. Introducing TF 00:07:17
    5. Lab: Simple Math Operations 00:08:46
    6. Computation Graph 00:10:17
    7. Tensors 00:09:02
    8. Lab: Tensors 00:05:04
    9. Linear Regression Intro 00:09:57
    10. Placeholders and Variables 00:08:45
    11. Lab: Placeholders 00:06:37
    12. Lab: Variables 00:07:49
    13. Lab: Linear Regression with Made-up Data 00:04:52
    14. Image Processing 00:08:06
    15. Images As Tensors 00:08:16
    16. Lab: Reading and Working with Images 00:08:06
    17. Lab: Image Transformations 00:06:38
    18. Introducing MNIST 00:04:13
    19. K-Nearest Neigbors as Unsupervised Learning 00:07:43
    20. One-hot Notation and L1 Distance 00:07:31
    21. Steps in the K-Nearest-Neighbors Implementation 00:09:33
    22. Lab: K-Nearest-Neighbors 00:14:15
    23. Learning Algorithm 00:10:59
    24. Individual Neuron 00:09:52
    25. Learning Regression 00:07:51
    26. Learning XOR 00:10:27
    27. XOR Trained 00:11:12
  14. Chapter 14 : Regression in TensorFlow
    1. Lab: Access Data from Yahoo Finance 00:02:50
    2. Non TensorFlow Regression 00:08:06
    3. Lab: Linear Regression - Setting Up a Baseline 00:11:19
    4. Gradient Descent 00:09:57
    5. Lab: Linear Regression 00:14:42
    6. Lab: Multiple Regression in TensorFlow 00:09:16
    7. Logistic Regression Introduced 00:10:16
    8. Linear Classification 00:05:26
    9. Lab: Logistic Regression - Setting Up a Baseline 00:07:33
    10. Logit 00:08:33
    11. Softmax 00:11:55
    12. Argmax 00:12:13
    13. Lab: Logistic Regression 00:16:57
    14. Estimators 00:04:11
    15. Lab: Linear Regression using Estimators 00:07:49
    16. Lab: Logistic Regression using Estimators 00:04:54
  15. Chapter 15 : Vision, Translate, NLP and Speech: Trained ML APIs
    1. Lab: Taxicab Prediction - Setting up the dataset 00:14:39
    2. Lab: Taxicab Prediction - Training and Running the model 00:11:22
    3. Lab: The Vision, Translate, NLP and Speech API 00:10:54
    4. Lab: The Vision API for Label and Landmark Detection 00:07:00
  16. Chapter 16 : Networking
    1. Virtual Private Clouds 00:07:04
    2. VPC and Firewalls 00:09:26
    3. XPC or Shared VPC 00:07:40
    4. VPN 00:08:49
    5. Types of Load Balancing 00:06:47
    6. Proxy and Pass-through load balancing 00:09:49
    7. Internal load balancing 00:06:02
  17. Chapter 17 : Ops and Security
    1. StackDriver 00:12:08
    2. StackDriver Logging 00:07:39
    3. Cloud Deployment Manager 00:06:06
    4. Cloud Endpoints 00:03:48
    5. Security and Service Accounts 00:07:45
    6. OAuth and End-user accounts 00:08:31
    7. Identity and Access Management 00:08:32
    8. Data Protection 00:12:02
  18. Chapter 18 : Appendix: Hadoop Ecosystem
    1. Introducing the Hadoop Ecosystem 00:01:35
    2. Hadoop 00:09:43
    3. HDFS 00:10:55
    4. MapReduce 00:10:34
    5. Yarn 00:05:30
    6. Hive 00:07:19
    7. Hive vs. RDBMS 00:07:11
    8. HQL vs. SQL 00:07:36
    9. OLAP in Hive 00:07:34
    10. Windowing Hive 00:08:22
    11. Pig 00:08:04
    12. More Pig 00:06:38
    13. Spark 00:08:55
    14. More Spark 00:11:46
    15. Streams Intro 00:07:44
    16. Microbatches 00:05:41
    17. Window Types 00:05:46