Spark for Data Science

Inferential statistics

We saw that descriptive statistics were extremely useful in describing and presenting data, but they did not provide a way to use the sample statistics to infer the population parameters or to validate any hypothesis we might have made. So, the techniques of inferential statistics surfaced to address such requirements. Some of the important uses of inferential statistics are:

• Estimation of population parameters
• Hypothesis testing

Please note that a sample can never represent a population perfectly because every time we sample, it naturally incurs sampling errors, hence the need for inferential statistics! Let us spend some time understanding the various types of probability distributions that can help infer the population parameters. ...