Webbots, Spiders, and Screen Scrapers, 2nd Edition

Adding Filtering to Your Aggregation Webbot

Your webbots can also modify or filter data received from RSS (or any other source). In this chapter’s news aggregator, you could filter (i.e., not use) any stories that don’t contain specific keywords or key phrases. For example, if you only want news stories that contain the words webbots, web spiders, and spiders, you could create a filter array like the one shown in Example 12-7.

Example 12-7. Creating a filter array

$filter_array[]="webbots";
$filter_array[]="web spiders";
$filter_array[]="spiders";

We can use $filter_array to select articles for viewing by modifying the download_parse_rss() function used in Example 12-4. This modification is shown in Example 12-8.

Example 12-8. Adding filtering to ...