Categorizing a dataset and applying a function to each group, whether an
aggregation or transformation, is often a critical component of a data
analysis workflow. After loading, merging, and preparing a dataset, you may
need to compute group statistics or possibly pivot tables for reporting or
visualization purposes. pandas provides a flexible
groupby interface, enabling you to slice, dice,
and summarize datasets in a natural way.
One reason for the popularity of relational databases and SQL (which stands for “structured query language”) is the ease with which data can be joined, filtered, transformed, and aggregated. However, query languages like SQL are somewhat constrained in the kinds of group operations that can be performed. As you will see, with the expressiveness of Python and pandas, we can perform quite complex group operations by utilizing any function that accepts a pandas object or NumPy array. In this chapter, you will learn how to:
Split a pandas object into pieces using one or more keys (in the form of functions, arrays, or DataFrame column names)
Calculate group summary statistics, like count, mean, or standard deviation, or a user-defined function
Apply within-group transformations or other manipulations, like normalization, linear regression, rank, or subset selection
Compute pivot tables and cross-tabulations
Perform quantile analysis and other statistical group analyses
Aggregation of time series data, a special ...