How it works...

We will dive into details of both Requests and Beautiful Soup in the next chapter, but for now let's just summarize a few key points about how this works.  The following important points about Requests:

  • Requests is used to execute HTTP requests.  We used it to make a GET verb request of the URL for the events page.
  • The Requests object holds the results of the request.  This is not only the page content, but also many other items about the result such as HTTP status codes and headers.
  • Requests is used only to get the page, it does not do an parsing.

We use Beautiful Soup to do the parsing of the HTML and also the finding of content within the HTML. 

To understand how this worked, the content of the page has the following ...

Get Python Web Scraping Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.