O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

IPython Interactive Computing and Visualization Cookbook - Second Edition

Book Description

Learn to use IPython and Jupyter Notebook for your data analysis and visualization work.

About This Book

  • Leverage the Jupyter Notebook for interactive data science and visualization
  • Become an expert in high-performance computing and visualization for data analysis and scientific modeling
  • A comprehensive coverage of scientific computing through many hands-on, example-driven recipes with detailed, step-by-step explanations

Who This Book Is For

This book is intended for anyone interested in numerical computing and data science: students, researchers, teachers, engineers, analysts, and hobbyists. A basic knowledge of Python/NumPy is recommended. Some skills in mathematics will help you understand the theory behind the computational methods.

What You Will Learn

  • Master all features of the Jupyter Notebook
  • Code better: write high-quality, readable, and well-tested programs; profile and optimize your code; and conduct reproducible interactive computing experiments
  • Visualize data and create interactive plots in the Jupyter Notebook
  • Write blazingly fast Python programs with NumPy, ctypes, Numba, Cython, OpenMP, GPU programming (CUDA), parallel IPython, Dask, and more
  • Analyze data with Bayesian or frequentist statistics (Pandas, PyMC, and R), and learn from actual data through machine learning (scikit-learn)
  • Gain valuable insights into signals, images, and sounds with SciPy, scikit-image, and OpenCV
  • Simulate deterministic and stochastic dynamical systems in Python
  • Familiarize yourself with math in Python using SymPy and Sage: algebra, analysis, logic, graphs, geometry, and probability theory

In Detail

Python is one of the leading open source platforms for data science and numerical computing. IPython and the associated Jupyter Notebook offer efficient interfaces to Python for data analysis and interactive visualization, and they constitute an ideal gateway to the platform.

IPython Interactive Computing and Visualization Cookbook, Second Edition contains many ready-to-use, focused recipes for high-performance scientific computing and data analysis, from the latest IPython/Jupyter features to the most advanced tricks, to help you write better and faster code. You will apply these state-of-the-art methods to various real-world examples, illustrating topics in applied mathematics, scientific modeling, and machine learning.

The first part of the book covers programming techniques: code quality and reproducibility, code optimization, high-performance computing through just-in-time compilation, parallel computing, and graphics card programming. The second part tackles data science, statistics, machine learning, signal and image processing, dynamical systems, and pure and applied mathematics.

Style and approach

IPython Interactive Computing and Visualization Cookbook, Second Edition is a practical, hands-on book that will teach you how to analyze and visualize all kinds of data in the Jupyter Notebook.

Downloading the example code for this book You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Table of Contents

  1. IPython Interactive Computing and Visualization Cookbook Second Edition
    1. Table of Contents
    2. IPython Interactive Computing and Visualization CookbookSecond Edition
      1. Why subscribe?
      2. PacktPub.com
    3. Contributors
      1. About the author
      2. Packt is Searching for Authors Like You
    4. Preface
      1. Who this book is for
      2. What this book covers
          1. Part 1 – Interactive Computing with Jupyter
          2. Part 2 – Standard Methods in Data Science and Applied Mathematics
      3. To get the most out of this book

          1. Installing Python
          2. GitHub repositories
        1. Download the example code files
        2. Download the color images
        3. Conventions used
      4. Sections
        1. Getting ready
        2. How to do it…
        3. How it works…
        4. There's more…
        5. See also
      5. Get in touch
        1. Reviews
    5. 1. A Tour of Interactive Computing with Jupyter and IPython
      1. Introduction
        1. What is Python?
        2. What is IPython?
        3. What is Jupyter?
        4. What is the SciPy ecosystem?
        5. What's new in the SciPy ecosystem?
        6. How to install Python
        7. References
      2. Introducing IPython and the Jupyter Notebook
        1. Getting ready
        2. How to do it...
        3. There's more...
        4. See also
      3. Getting started with exploratory data analysis in the Jupyter Notebook
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      4. Introducing the multidimensional array in NumPy for fast array computations
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      5. Creating an IPython extension with custom magic commands
        1. How to do it...
        2. How it works...
          1. The InteractiveShell class
          2. Loading an extension
        3. There's more...
        4. See also
      6. Mastering IPython's configuration system
        1. How to do it...
        2. How it works...
          1. Configurables
          2. Magics
        3. There's more...
        4. See also
      7. Creating a simple kernel for Jupyter
        1. How to do it...
        2. How it works...
        3. There's more...
    6. 2. Best Practices in Interactive Computing
      1. Introduction
      2. Learning the basics of the Unix shell
        1. Getting ready
        2. How to do it...
        3. There's more...
        4. See also
      3. Using the latest features of Python 3
        1. How to do it...
        2. There's more...
      4. Learning the basics of the distributed version control system Git
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      5. A typical workflow with Git branching
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      6. Efficient interactive computing workflows with IPython
        1. How to do it...
          1. The IPython terminal
          2. IPython and text editor
          3. The Jupyter Notebook
          4. Integrated Development Environments
        2. There's more...
        3. See also
      7. Ten tips for conducting reproducible interactive computing experiments
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      8. Writing high-quality Python code
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      9. Writing unit tests with pytest
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
          1. Test coverage
          2. Workflows with unit testing
          3. Unit testing and continuous integration
      10. Debugging code with IPython
        1. How to do it...
          1. The post-mortem mode
          2. Step-by-step debugging
        2. There's more...
    7. 3. Mastering the Jupyter Notebook
      1. Introduction
        1. The Notebook ecosystem
        2. Architecture of the Jupyter Notebook
          1. Connecting multiple clients to one kernel
          2. JupyterHub
        3. Security in notebooks
        4. References
      2. Teaching programming in the Notebook with IPython Blocks
        1. Getting ready
        2. How to do it...
      3. Converting a Jupyter notebook to other formats with nbconvert
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
      4. Mastering widgets in the Jupyter Notebook
        1. Getting ready
        2. How to do it...
        3. There's more...
        4. See also
      5. Creating custom Jupyter Notebook widgets in Python, HTML, and JavaScript
        1. How to do it...
        2. There's more...
        3. See also
      6. Configuring the Jupyter Notebook
        1. How to do it...
        2. There's more...
        3. See also
      7. Introducing JupyterLab
        1. Getting ready
        2. How to do it...
        3. There's more...
        4. See also
    8. 4. Profiling and Optimization
      1. Introduction
      2. Evaluating the time taken by a command in IPython
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      3. Profiling your code easily with cProfile and IPython
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      4. Profiling your code line-by-line with line_profiler
        1. Getting ready
        2. How do to it...
        3. How it works...
        4. There's more...
        5. See also
      5. Profiling the memory usage of your code with memory_profiler
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      6. Understanding the internals of NumPy to avoid unnecessary array copying
        1. Getting ready
        2. How to do it...
        3. How it works...
          1. Why are NumPy arrays efficient?
          2. What is the difference between in-place and implicit-copy operations?
          3. Why can't some arrays be reshaped without a copy?
          4. What are NumPy broadcasting rules?
        4. There's more...
        5. See also
      7. Using stride tricks with NumPy
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. See also
      8. Implementing an efficient rolling average algorithm with stride tricks
        1. How to do it...
        2. See also
      9. Processing large NumPy arrays with memory mapping
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      10. Manipulating large arrays with HDF5
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
    9. 5. High-Performance Computing
      1. Introduction
        1. CPython and concurrent programming
        2. Compiler-related installation instructions
      2. Using Python to write faster code
        1. How to do it...
        2. There's more...
        3. See also
      3. Accelerating pure Python code with Numba and Just-In-Time compilation
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      4. Accelerating array computations with NumExpr
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. See also
      5. Wrapping a C library in Python with ctypes
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      6. Accelerating Python code with Cython
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      7. Optimizing Cython code by writing less Python and more C
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      8. Releasing the GIL to take advantage of multi-core processors with Cython and OpenMP
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. See also
      9. Writing massively parallel code for NVIDIA graphics cards (GPUs) with CUDA
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      10. Distributing Python code across multiple cores with IPython
        1. Getting started
        2. How to do it...
        3. How it works...
        4. There's more...
          1. References
        5. See also
      11. Interacting with asynchronous parallel tasks in IPython
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      12. Performing out-of-core computations on large arrays with Dask
        1. Getting ready
        2. How to do it...
        3. There's more...
        4. See also
      13. Trying the Julia programming language in the Jupyter Notebook
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
    10. 6. Data Visualization
      1. Introduction
      2. Using Matplotlib styles
        1. How to do it...
        2. There's more...
        3. See also
      3. Creating statistical plots easily with seaborn
        1. How to do it...
        2. There's more...
        3. See also
      4. Creating interactive web visualizations with Bokeh and HoloViews
        1. Getting ready
        2. How to do it...
        3. There's more...
      5. Visualizing a NetworkX graph in the Notebook with D3.js
        1. Getting ready
        2. How to do it...
        3. There's more...
        4. See also
      6. Discovering interactive visualization libraries in the Notebook
        1. Getting started
        2. How to do it...
        3. There's more
      7. Creating plots with Altair and the Vega-Lite specification
        1. Getting started...
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
    11. 7. Statistical Data Analysis
      1. Introduction
        1. What is statistical data analysis?
        2. A bit of vocabulary
          1. Exploration, inference, decision, prediction
          2. Univariate and multivariate methods
          3. Frequentist and Bayesian methods
          4. Parametric and nonparametric inference methods
      2. Exploring a dataset with pandas and Matplotlib
        1. How to do it...
        2. There's more...
      3. Getting started with statistical hypothesis testing — a simple z-test
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      4. Getting started with Bayesian methods
        1. Getting ready
        2. How to do it...
        3. How it works...
          1. Bayes' theorem
          2. Computation of the posterior distribution
          3. Maximum a posteriori estimation
        4. There's more...
          1. Credible interval
          2. Conjugate distributions
          3. Non-informative (objective) prior distributions
        5. See also
      5. Estimating the correlation between two variables with a contingency table and a chi-squared test
        1. How to do it...
        2. How it works...
          1. Pearson's correlation coefficient
          2. Contingency table and chi-squared test
        3. There's more...
        4. See also
      6. Fitting a probability distribution to data with the maximum likelihood method
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      7. Estimating a probability distribution nonparametrically with a kernel density estimation
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. See also
      8. Fitting a Bayesian model by sampling from a posterior distribution with a Markov chain Monte Carlo method
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      9. Analyzing data with the R programming language in the Jupyter Notebook
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
    12. 8. Machine Learning
      1. Introduction
        1. A bit of vocabulary
          1. Learning from data
          2. Supervised learning
          3. Unsupervised learning
          4. Feature selection and feature extraction
          5. Overfitting, underfitting, and the bias-variance tradeoff
          6. Model selection
          7. Machine learning references
      2. Getting started with scikit-learn
        1. Getting ready
        2. How to do it...
        3. How it works...
          1. scikit-learn API
          2. Ordinary Least Squares regression
          3. Polynomial interpolation with linear regression
          4. Ridge regression
          5. Cross-validation and grid search
        4. There's more...
      3. Predicting who will survive on the Titanic with logistic regression
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      4. Learning to recognize handwritten digits with a K-nearest neighbors classifier
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      5. Learning from text – Naive Bayes for Natural Language Processing
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      6. Using support vector machines for classification tasks
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      7. Using a random forest to select important features for regression
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      8. Reducing the dimensionality of a dataset with a principal component analysis
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      9. Detecting hidden structures in a dataset with clustering
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
    13. 9. Numerical Optimization
      1. Introduction
        1. The objective function
        2. Local and global minima
        3. Constrained and unconstrained optimization
        4. Deterministic and stochastic algorithms
        5. References
      2. Finding the root of a mathematical function
        1. How to do it...
        2. How it works...
        3. There's more…
        4. See also
      3. Minimizing a mathematical function
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      4. Fitting a function to data with nonlinear least squares
        1. How to do it...
        2. How it works...
        3. See also
      5. Finding the equilibrium state of a physical system by minimizing its potential energy
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
    14. 10. Signal Processing
      1. Introduction
        1. Analog and digital signals
        2. The Nyquist–Shannon sampling theorem
        3. Compressed sensing
        4. References
      2. Analyzing the frequency components of a signal with a Fast Fourier Transform
        1. How to do it...
        2. How it works...
          1. The discrete Fourier transform
          2. Inverse Fourier transform
        3. There's more...
        4. See also
      3. Applying a linear filter to a digital signal
        1. How to do it...
        2. How it works...
          1. What are linear filters?
          2. Linear filters and convolutions
          3. The FIR and IIR filters
          4. Filters in the frequency domain
          5. The low-, high-, and band-pass filters
        3. There's more...
        4. See also
      4. Computing the autocorrelation of a time series
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
    15. 11. Image and Audio Processing
      1. Introduction
        1. Images
        2. Sounds
        3. References
      2. Manipulating the exposure of an image
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      3. Applying filters on an image
        1. How it works...
        2. How it works...
        3. There's more...
        4. See also
      4. Segmenting an image
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      5. Finding points of interest in an image
        1. How to do it...
        2. How it works...
        3. There's more...
      6. Detecting faces in an image with OpenCV
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
      7. Applying digital filters to speech sounds
        1. Getting ready
        2. How to do it
        3. How it works...
        4. There's more...
        5. See also
      8. Creating a sound synthesizer in the Notebook
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
    16. 12. Deterministic Dynamical Systems
      1. Introduction
        1. Types of dynamical systems
        2. Differential equations
        3. References
      2. Plotting the bifurcation diagram of a chaotic dynamical system
        1. How to do it...
        2. There's more...
        3. See also
      3. Simulating an elementary cellular automaton
        1. How to do it...
        2. How it works...
        3. There's more...
      4. Simulating an ordinary differential equation with SciPy
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      5. Simulating a partial differential equation — reaction-diffusion systems and Turing patterns
        1. How to do it...
        2. How it works...
        3. There's more...
    17. 13. Stochastic Dynamical Systems
      1. Introduction
        1. References
      2. Simulating a discrete-time Markov chain
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      3. Simulating a Poisson process
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      4. Simulating a Brownian motion
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
      5. Simulating a stochastic differential equation
        1. How to do it...
        2. How it works...
        3. There's more...
        4. See also
    18. 14. Graphs, Geometry, and Geographic Information Systems
      1. Introduction
        1. Graphs
          1. Problems in graph theory
          2. Random graphs
          3. Graphs in Python
        2. Geometry in Python
        3. Geographical information systems in Python
        4. References
      2. Manipulating and visualizing graphs with NetworkX
        1. Getting ready
        2. How to do it...
        3. There's more...
        4. See also
      3. Drawing flight routes with NetworkX
        1. Getting ready
        2. How to do it...
        3. See also
      4. Resolving dependencies in a directed acyclic graph with a topological sort
        1. How to do it...
        2. How it works...
        3. There's more...
      5. Computing connected components in an image
        1. How to do it...
        2. How it works...
        3. There's more...
      6. Computing the Voronoi diagram of a set of points
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
        5. See also
      7. Manipulating geospatial data with Cartopy
        1. Getting ready
        2. How to do it...
        3. There's more...
        4. See also
      8. Creating a route planner for a road network
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
    19. 15. Symbolic and Numerical Mathematics
      1. Introduction
        1. LaTeX
      2. Diving into symbolic computing with SymPy
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. See also
      3. Solving equations and inequalities
        1. How to do it...
        2. There's more...
      4. Analyzing real-valued functions
        1. How to do it...
        2. There's more...
      5. Computing exact probabilities and manipulating random variables
        1. How to do it...
        2. How it works...
        3. There's more...
      6. A bit of number theory with SymPy
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
      7. Finding a Boolean propositional formula from a truth table
        1. How to do it...
        2. How it works...
        3. There's more...
      8. Analyzing a nonlinear differential system — Lotka-Volterra (predator-prey) equations
        1. Getting ready
        2. How to do it...
        3. How it works...
        4. There's more...
      9. Getting started with Sage
        1. Getting ready
        2. How to do it...
        3. There's more...
        4. See also
    20. Index