Webbots, Spiders, and Screen Scrapers, 2nd Edition

Final Thoughts

Years of experience have taught me a few tricks for emulating forms. While it’s not hard to write a webbot that submits a form, it is often difficult to do it right the first time. Moreover, as you read earlier, there are many reasons to submit a form correctly the first time. I highly suggest reading the later chapters on stealth, fault tolerance, and potential legal issues (Chapter 26, Chapter 28, and Chapter 31) before creating webbots that emulate forms. These chapters provide additional insight into potential problems and perils that you’re likely to encounter when writing webbots that submit data to webservers.

Don’t Blow Your Cover

If you’re using a webbot to create a competitive advantage for a client, you don’t want that ...