O'Reilly logo

Apache Spark for Data Science Cookbook by Padma Priya Chitturi

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Missing value treatment

Missing data in the training dataset can reduce the fitness of a model or can lead to a biased model because we have not analyzed the behavior and relationship with other variables correctly. This could also lead to wrong predictions or classifications. The reasons for the occurrence of the missing values could be that while extracting data from multiple sources, there is a possible chance to have missing data. Hence, using some hashing procedure ensures that the data extraction is correct. The errors that occur at the time of data collection are tougher to correct as the values might miss at random and the missing values might also depend on the unobserved predictors.

Getting ready

To step through this recipe, you will need ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required