O'Reilly logo

Getting Started with Beautiful Soup by Vineeth G. Nair

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Using search methods to scrape information from a web page

In Chapter 2, Creating a BeautifulSoup Object, we discussed the fetching of information related to all books published by Packt Publishing available at the link http://www.packtpub.com/books. In the following exercise, we will use Beautiful Soup searching methods to scrape all books from the preceding URL. We will scrape the name of the book, published date, and price from the preceding URL.

The first step involved in this exercise is to analyze the HTML document and to understand its logical structure. This will help us to understand how the required information is stored within the HTML document.

We can use the Google Chrome developer tools to understand the logical structure of the page. ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required