Chapter 16. Webpluck

Ed Hill

The promises of smart little web agents that run around the web and grab things of interest have gone unfulfilled. Like me, you probably have a handful of web pages that you check on a regular basis, and if you had the time, you’d check many more.

Listed below are a few of the bookmarks that I check on a regular basis. Each of these pages has content that changes every day, and it is, of course, the content of these pages that I am interested in—not their layout, nor the advertising that appears on the pages.

Dilbert (of course)
CNN U.S. News
Astronomy Picture of the Day
C|Net’s News.com
The local paper (The Daily Iowan)
ESPNET Sportszone

These pages are great sources of information. My problem is that I don’t have time to check each one every day to see what is there or if the page has been updated. What I want is my own personal newspaper built from the sources listed above.

Similar Tools

This is not an original idea, but after spending many hours searching for a tool to do what I wanted, I gave up. Here are the contenders I considered, and why they didn’t do quite what I wanted.

First there is the “smart” agent, a little gremlin that roams the net trying to guess what you want to see using some AI technique. Firefly was an example; you indicate interest in a particular topic and it points you at a list of sites that others have ranked. When I first looked at Firefly, it suggested that since I was interested in “computers and the internet,” I should check out ...

Get Web, Graphics & Perl/Tk Programming now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.