O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Data Science Fundamentals Part 1: Learning Basic Concepts, Data Wrangling, and Databases with Python

Video Description

20 Hours of Video Instruction

Data Science Fundamentals LiveLessons teaches you the foundational concepts, theory, and techniques you need to know to become an effective data scientist. The videos present you with applied, example-driven lessons in Python and its associated ecosystem of libraries, where you get your hands dirty with real datasets and see real results.

Description

If nothing else, by the end of this video course you will have analyzed a number of datasets from the wild, built a handful of applications, and applied machine learning algorithms in meaningful ways to get real results. And along the way you learn the best practices and computational techniques used by a professional data scientist. More specifically, you learn how to acquire data that is openly accessible on the Internet by working with APIs. You learn how to parse XML and JSON data to load it into a relational database.

About the Instructor

Jonathan Dinu is an author, researcher, and most importantly, an educator. He is currently pursuing a Ph.D. in Computer Science at Carnegie Mellon's Human Computer Interaction Institute (HCII), where he is working to democratize machine learning and artificial intelligence through interpretable and interactive algorithms. Previously, he founded Zipfian Academy (an immersive data science training program acquired by Galvanize), has taught classes at the University of San Francisco, and has built a Data Visualization MOOC with Udacity. In addition to his professional data science experience, he has run data science trainings for a Fortune 500 company and taught workshops at Strata, PyData, and DataWeek (among others). He first discovered his love of all things data while studying Computer Science and Physics at UC Berkeley, and in a former life he worked for Alpine Data Labs developing distributed machine learning algorithms for predictive analytics on Hadoop.

Jonathan has always had a passion for sharing the things he has learned in the most creative ways he can. When he is not working with students, you can find him blogging about data, visualization, and education at hopelessoptimism.com or rambling on Twitter @jonathandinu.

Skill Level

  • Beginner
What You Will Learn
  • How to get up and running with a Python data science environment
  • The essentials of Python 3, including object-oriented programming
  • The basics of the data science process and what each step entails
  • How to build a simple (yet powerful) recommendation engine for Airbnb listings
  • Where to find quality data sources and how to work with APIs programmatically
  • Strategies for parsing JSON and XML into a structured form
  • The basics of relational databases and how to use an ORM to interface with them in Python
  • Best practices of data validation, including common data quality checks
Who Should Take This Course
  • Aspiring data scientists looking to break into the field and learn the essentials necessary
  • Journalists, consultants, analysts, or anyone else who works with data and looking to take a programmatic approach to exploring data and conducting analyses
  • Quantitative researchers interested in applying theory to real projects and taking a computational approach to modeling.
  • Software engineers interested in building intelligent applications driven by machine learning
  • Practicing data scientists already familiar with another programming environment looking to learn how to do data science with Python
Course Requirements
  • Basic understanding of programming
  • Familiarity with Python and statistics are a plus
Lesson Descriptions

Lesson 1: Introduction to Data Science with Python

Lesson 1 begins with a working definition of data science (as we use it in the course), gives a brief history of the field, and provides motivating examples of data science products and applications. This lesson covers how to get set up with a data science programming environment locally, as well as gives you a crash course in the Python programming language if you are unfamiliar with it or are coming from another language such as R. Finally, it ends with an overview of the concepts and tools that the rest of the lessons cover to hopefully motivate you for and excite you about what's to come!

Lesson 2: The Data Science Process—Building Your First Application

Lesson 2 introduces the data science process by walking through an end-to-end example of building your very first data science application, an AirBnB listing recommender.

You continue to learn how to work with and manipulate data in Python, without any external libraries yet, and leverage the power of the built-in Python standard library. The core application of this lesson covers the basics of building a recommendation engine and shows you how, with simple statistics and a little ingenuity, you can build a compelling recommender, given the right data. And finally, it ends with a formal treatment of the data science process and the individual steps it entails.

Lesson 3: Acquiring Data—Sources and Methods

Lesson 3 begins the treatment of each of the specific stages of the data science process, starting with the first: data acquisition. The lesson covers the basics of finding the appropriate data source for your problem and how to download the datasets you need once you have found them.

Starting with an overview of how the infrastructure behind the Internet works, you learn how to programmatically make HTTP requests in Python to access data through APIs, as well as the basics of two of the most common data formats: JSON and XML. The lesson ends by setting up the dataset we use for the rest of the course: Foursquare Venues.

Working with the Foursquare dataset, you learn how to interact with APIs and do some minor web scraping. You also learn how to find and acquire data from a variety of sources and keep track of its lineage all along the way. You learn to put yourself in the data science mindset and how to see the data (hidden in plain sight) that we interact with every day.

Lesson 4: Adding Structure—Data Parsing and Storage

Lesson 4 picks up with the second stage of what traditionally is referred to as an extract, transform, and load (ETL) pipeline, adding structure through the transformation of raw data.

You see how to work with a variety of data formats, including XML and JSON, by parsing the data we have acquired to eventually load it into an environment better-suited to exploration and analysis: a relational database. But before we load our data into a database, we take a short diversion to talk about how to conceptually model structure in data with code. You get a primer in object-oriented programming and learn how to leverage it to create abstractions and data models that define how you can interface with your data.

Lesson 5: Storing Data: Relational Databases (with SQLite)

Lesson 5 starts with an introduction to one of the most ubiquitous data technologies—the relational database. The lesson serves as an end cap to the ETL pipeline of the previous videos. You learn the ins and outs of the various strategies for storing data and see how to map the abstractions you created in Python to database tables through the use of an object-relational mapper (ORM). By being able to query and manipulate data with Python while persisting data in a database reliably, the interface ORMs provide gives you the best of both worlds.

Lesson 6: Data Validation and Exploration

Lesson 6 starts by showing you how to effectively query your data to understand what it contains, uncover any biases it might contain, and learn the best practices of dealing with missing values. After you have validated the quality of the data, you use descriptive statistics to learn how your data is distributed as well as learn the limits of point statistics (or rather single number estimates) and why it is often necessary to use visual techniques.

About LiveLessons Video Training

The LiveLessons Video Training series publishes hundreds of hands-on, expert-led video tutorials covering a wide selection of technology topics designed to teach you the skills you need to succeed. This professional and personal technology video series features world-leading author instructors published by your trusted technology brands: Addison-Wesley, Cisco Press, IBM Press, Pearson IT Certification, Prentice Hall, Sams, and Que. Topics include: IT Certification, Programming, Web Development, Mobile Development, Home and Office Technologies, Business and Management, and more. View all LiveLessons on InformIT at: http://www.informit.com/livelessons

Table of Contents

  1. Introduction
    1. Data Science Fundamentals Part 1: Introduction 00:07:00
  2. Lesson 1: Introduction to Data Science with Python
    1. Topics 00:01:37
    2. 1.1 Welcome to the Course 00:03:14
    3. 1.2 Why Data Science and Why Now? 00:07:46
    4. 1.3 The Potential of Data Science 00:24:18
    5. 1.4 Getting Set Up with a Data Science Development Environment 00:07:41
    6. 1.5 A Python (3) Primer 00:22:19
    7. 1.6 Python 2 versus Python 3 00:07:49
    8. 1.7 Test Your Knowledge: Wordbuzz 00:27:38
    9. 1.8 Wordbuzz: Putting it all Together 00:12:29
    10. 1.9 Python Review and Resources 00:08:47
    11. 1.10 Python for Data Science 00:14:59
    12. 1.11 What’s to Come 00:10:10
  3. Lesson 2: The Data Science Process: Building Your First Application
    1. Topics 00:01:17
    2. 2.1 Introduction to the Data Science Process 00:07:16
    3. 2.2 Defining Your Problem 00:06:46
    4. 2.3 Acquiring Data 00:21:01
    5. 2.4 Wrangling Data 00:28:05
    6. 2.5 Exploring Data 00:28:47
    7. 2.6 Recommendations through Triangle Closing 00:19:54
    8. 2.7 Python Development Workflow 00:14:00
    9. 2.8 Triadic Closure in Python 00:27:35
    10. 2.9 Challenges of Recommendation Systems 00:11:05
    11. 2.10 Obtaining an Evaluation Baseline 00:18:37
    12. 2.11 Inspecting and Evaluating Results 00:12:04
    13. 2.12 Present and Disseminate 00:15:20
    14. 2.13 The Data Science Process Applied—Cheaper Beds, Better Breakfasts 00:06:27
  4. Lesson 3: Acquiring Data—Sources and Methods
    1. Topics 00:01:41
    2. 3.1 The Data Science Mindset 00:14:57
    3. 3.2 The Data Science Technology Stack 00:15:29
    4. 3.3 Where to Get Data: Sources and Services 00:16:45
    5. 3.4 How the Web Works 00:19:54
    6. 3.5 Making HTTP Requests with Python 00:14:32
    7. 3.6 Adding Context with Open Data 00:09:25
    8. 3.7 Parsing Data with Python—JSON and XML 00:28:12
    9. 3.8 Data and File Formats 00:15:13
    10. 3.9 Working with APIs 00:24:19
    11. 3.10 Parametric API Requests with Python 00:28:11
    12. 3.11 Exploring the Foursquare API 00:13:57
    13. 3.12 Downloading Foursquare Venues 00:22:04
  5. Lesson 4: Adding Structure—Parsing Data and Data Models
    1. Topics 00:01:12
    2. 4.1 Introduction to the ETL Pipeline 00:10:46
    3. 4.2 Data Models—Adding Structure to Data 00:23:56
    4. 4.3 Building Abstractions—Object Oriented Programming 00:11:47
    5. 4.4 Creating Classes in Python 00:19:44
    6. 4.5 Defining Methods and Updating State 00:20:48
    7. 4.6 Magic Methods, Class Attributes, and Introspection 00:25:16
    8. 4.7 Exploring and Structuring the Foursquare Response 00:23:36
    9. 4.8 Data Models Applied—Representing Foursquare Entities with Classes 00:21:58
    10. 4.9 Modeling Behavior with Methods 00:15:16
    11. 4.10 Customizing Model Interfaces with Setter Methods and Virtual Attributes 00:24:28
    12. 4.11 Keeping Things DRY with Inheritance 00:28:51
    13. 4.12 Object-Oriented Programming Use Cases 00:20:45
    14. 4.13 The Case for (and against) OOP 00:13:04
  6. Lesson 5: Storing Data—Persistence with Relational Databases
    1. Topics 00:01:21
    2. 5.1 Introduction to Databases with SQLite 00:26:53
    3. 5.2 Inspecting Databases with the SQLite shell 00:13:12
    4. 5.3 The Database Landscape 00:12:04
    5. 5.4 What's in a Schema? Mapping Data Models to Data Tables 00:22:32
    6. 5.5 Introduction to Object Relational Mappers 00:06:36
    7. 5.6 ORMs in Python with peewee 00:19:26
    8. 5.7 Creating and Querying Records with peewee 00:25:19
    9. 5.8 End-to-end ETL in Python 00:09:20
    10. 5.9 Advantages and Disadvantages of ORMs 00:04:52
    11. 5.10 Extract, Transform, Load—Putting It All Together 00:10:08
  7. Lesson 6: Validating Data—Provenance and Quality Control
    1. Topics 00:01:21
    2. 6.1 Introduction to Exploratory Data Analysis 00:17:15
    3. 6.2 Understanding your Data Quickly with Graphical Tools 00:17:27
    4. 6.3 Inspecting Databases and Building Schemas with peewee 00:20:15
    5. 6.4 Data Quality Checks with peewee 00:18:19
    6. 6.5 Finding Missing Data and Null Values with peewee 00:12:13
    7. 6.6 Dealing with Missing Data 00:10:38
    8. 6.7 EDA for Insight—Describing Data 00:04:39
    9. 6.8 Inspecting Queries and Displaying Results in peewee 00:18:05
    10. 6.9 Groups and Aggregates with peewee 00:14:23
    11. 6.10 Ranking and Sorting Venues 00:20:14
    12. 6.11 SQL Relations and Joins 00:08:25
    13. 6.12 Joins with peewee 00:27:42
    14. 6.13 Querying Across Datasets with Joins 00:26:26
    15. 6.14 Translating peewee to SQL 00:06:58
    16. 6.15 A Visual Introduction to Joins with SQL 00:14:49
  8. Summary
    1. Data Science Fundamentals Part 1: Sunmary 00:03:21