Error Handlers
When a webbot cannot adjust to changes, the only safe thing to do is to stop it. Not stopping your webbot may otherwise result in odd performance and suspicious entries in the target server’s access and error log files. It’s a good idea to write a routine that handles all errors in a prescribed manner. Such an error handler should send you an email that indicates the following:
Which webbot failed
Why it failed
The date and time it failed
A simple script like the one in Example 28-13 works well for this purpose.
Example 28-13. Simple error-reporting script
function webbot_error_handler($failure_mode)
{
# Initialization
$email_address = "your.account@someserver.com"; $email_subject = "Webbot Failure Notification"; # Build the failure message ...Get Webbots, Spiders, and Screen Scrapers, 2nd Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
