You are previewing Web Client Programming with Perl.
O'Reilly logo
Web Client Programming with Perl

Book Description

On the World Wide Web, people are accustomed to using graphical browsers such as Netscape Navigator or Mosaic as their only interface for visiting remote sites, accessing up-to-date documents, and filling out forms. But graphical browsers can be limiting: the very interactivity that makes them so intuitive to use also makes them clumsy for automating tasks. If you want to get the latest weather report every few hours, track a Federal Express package online, or use a dictionary server repeatedly throughout the day, using your browser to perform the same task over and over can become cumbersome. As with any repetitive task, these applications are best done by writing a script.Web Client Programming with Perl shows you how to extend scripting skills to the Web. This book teaches you the basics of how browsers communicate with servers and how to write your own customized Web clients to automate common tasks. It is intended for those who are motivated to develop software that offers a more flexible and dynamic response than a standard Web browser.Using this book, you'll learn how to:

  • Automate repetitive queries on the Web

  • Detect broken hyperlinks on your site

  • Write simple "robots" that traverse hyperlinks across a site, and across the Web in general

  • This book will be of interest to:

  • Web administrators who need to automate repetitive tasks or reduce maintenance time

  • UNIX shell programmers who want to interface their scripts to the Web

  • Commercial software developers and consultants who need reference material for technical Web specifications and proof-of-concept examples

  • Most of the examples in this book use Perl, a versatile and portable language that is already familiar to many CGI programmers and UNIX power users. The book does not teach Perl, but the techniques used in the book should be easily followed by anyone with some programming background and can be adapted to whatever language you choose.

    Table of Contents

    1. Cover Page
    2. Title Page
    3. Contents
    4. Preface
      1. Who This Book Is For
      2. Is This Book for You?
      3. Organization
      4. Source Code in This Book Is Online
      5. Conventions Used in This Book
      6. Request for Comments
      7. Acknowledgments
    5. Chapter 1. Introduction
      1. Why Write Your Own Clients?
      2. The Web and HTTP
      3. The Programming Interface
      4. A Word of Caution
    6. Chapter 2. Demystifying the Browser
      1. Behind the Scenes of a Simple Document
      2. Retrieving a Document Manually
      3. Behind the Scenes of an HTML Form
      4. Behind the Scenes of Publishing a Document
      5. Structure of HTTP Transactions
    7. Chapter 3. Learning HTTP
      1. Structure of an HTTP Transaction
      2. Client Request Methods
      3. Versions of HTTP
      4. Server Response Codes
      5. HTTP Headers
    8. Chapter 4. The Socket Library
      1. A Typical Conversation over Sockets
      2. Using the Socket Calls
      3. Server Socket Calls
      4. Client Connection Code
      5. Your First Web Client
      6. Parsing a URL
      7. Hypertext UNIX cat
      8. Shell Hypertext cat
      9. Grep out URL References
      10. Client Design Considerations
    9. Chapter 5. The LWP Library
      1. Some Simple Examples
      2. Listing of LWP Modules
      3. Using LWP
    10. Chapter 6. Example LWP Programs
      1. Simple Clients
      2. Periodic Clients
      3. Recursive Clients
    11. Chapter 7. Graphical Examples with Perl/Tk
      1. A Brief Introduction to Tk
      2. A Dictionary Client: xword
      3. Check on Package Delivery: Track
      4. Check if Servers Are up: webping
    12. Appendix A HTTP Headers
      1. General Headers
      2. Client Request Headers
      3. Server Response Headers
      4. Entity Headers
      5. Summary of Support Across HTTP Versions
    13. Appendix B Reference Tables
      1. Media Types
      2. Character Encoding
      3. Languages
      4. Character Sets
    14. Appendix C The Robot Exclusion Standard
      1. Syntax for the /robots.txt File
    15. Index