Running the Webbot
Since the output of this webbot contains formatted HTML, it is appropriate to run this webbot within a browser, as shown in Figure 9-2.
Figure 9-2. Running the link-verification webbot
This webbot counts and identifies all the links on the target website. It also indicates the HTTP code and diagnostic message describing the status of the fetch used to download the page and displays the actual amount of time it took the page to load.
Let's take this time to look at some of the libraries used by this webbot.
LIB_http_codes
The following script creates an indexed array of HTTP error codes and their definitions. To use the array, simply ...
Get Webbots, Spiders, and Screen Scrapers now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
