O'Reilly logo

Advanced Analytics with Spark by Josh Wills, Sean Owen, Uri Laserson, Sandy Ryza

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 3. Recommending Music and the Audioscrobbler Data Set

De gustibus non est disputandum.

(There’s no accounting for taste.)

When somebody asks what it is I work on for a living, the direct answer of “data science” or “machine learning” sounds impressive but usually draws a blank stare. Fair enough; even actual data scientists seem to struggle to define what these mean—storing lots of data, computing, predicting something? Inevitably, I jump straight to a relatable example:

“OK, you know how Amazon will tell you about books like the ones you bought? Yes? Yes! It’s like that.”

Empirically, the recommender engine seems to be an example of large-scale machine learning that everyone already understands, and most people have seen Amazon’s. It is a common denominator because recommender engines are everywhere, from social networks to video sites to online retailers. We can also directly observe them in action. We’re aware that a computer is picking tracks to play on Spotify, in a way we don’t necessarily notice that Gmail is deciding whether inbound email is spam.

The output of a recommender is more intuitively understandable than other machine learning algorithms. It’s exciting, even. For all that we think that musical taste is so personal and inexplicable, recommenders do a surprisingly good job of identifying tracks we didn’t know we would like.

Finally, for domains like music or movies where recommenders are usually deployed, it’s comparatively easy to reason about ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required