Final Thoughts

One of the nice things about webbots is that you can create a large effect without making something difficult for customers to use. In fact, customers don’t even need to know that a webbot is involved. For example, your webbots can deliver services through traditional-looking websites. While you know that you’re doing something radically innovative, the end users don’t realize what’s going on behind the scenes—and they don’t really need to know about the hordes of hidden webbots and spiders combing the Internet for the data and services they need. All they know is that they are getting an improved Internet experience. And in the end, that’s all that matters.

Another thing to remember is that there is always a lag between when people ...

Get Webbots, Spiders, and Screen Scrapers, 2nd Edition now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.