Chapter 23. Web Spiders

 

I got remote control and a color T.V.

I don't change channels so they must change me.

 
 --Close to the Borderline—Billy Joel

When the Web was young—or at least when you were new to the Web—it was interesting to spend hours clicking links and looking at Web pages. Eventually, you got over that, and now you just want the information you want, when you want it, and you no longer want to do the work for yourself. That's where spiders come in.

Spiders are programs that walk the Web for you, following links and grabbing information. They're also ...

Get Apache Administrator's Handbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.