Getting ready

The legality of scraping breaks down into two issues:

  • Ownership of content
  • Denial of service

Fundamentally, anything posted on the web is open for reading.  Every time you load a page, any page, your browser downloads that content from the web server and visually presents it to you. So in a sense, you and your browser are already scraping anything you look at on the web.  And by the nature of the web, because someone is posting content publicly on the web, they are inherently asking you to take that information, but often only for specific purposes.

The big issue comes with creating automated tools that directly look for and make copies of things on the internet, with a thing being either data, images, videos, or music - essentially ...

Get Python Web Scraping Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.