Python Web Scraping Cookbook

How to do it

We can do this using Selenium. We will use two features of Selenium. The first is the ability to click on page elements. The second is the ability to wait until an element with a specific ID is available on the page.

1. First, we get the button and click it. The button's HTML is the following:

<div id='start'>   <button>Start</button></div>

2. When the button is pressed and the load completes, the following HTML is added to the document:

<div id='finish'>   <h4>Hello World!"</h4></div>

3. We will use the Selenium driver to find the Start button, click it, and then wait until a div with an ID of 'finish' is available. Then we get that element and return the text in the enclosed <h4> tag.

You can try this by running 06/03_press_and_wait.py ...