Programming Web Services with XML-RPC

Chapter 4. XML-RPC and Perl

XML-RPC and the Perl scripting language are a particularly powerful combination for creating flexible web services rapidly. Perl has long been the language of choice to obtain and manipulate data for the Web, and it is moving into the growing field of web services. One of Perl’s guiding philosophies is “Easy things should be easy, and hard things should be possible.” The Perl module for XML-RPC, Frontier::RPC, embodies this.

To show how easy Perl’s XML-RPC library, Frontier::RPC, makes remote procedure calls, consider the following code snippet:

use Frontier::Client;
my $client = Frontier::Client->new 
           ( url => "http://example.com:1080");
print "helloWorld('Bob') returned: ",
           $client->call('helloWorld', 'Bob'), "\n";

Assume that on a machine called example.com, there is an XML-RPC server running on port 1080 that has implemented a procedure named helloWorld( ). Given these assumptions, these three lines of Perl code are all that’s needed to make an XML-RPC call. Although this chapter explores the details of using this library more thoroughly, many XML-RPC Perl clients aren’t any more complicated than this example.

This chapter begins with a discussion of the history, requirements, and architecture of Perl’s XML-RPC library, Frontier::RPC. Then it covers how to create XML-RPC clients and servers using Perl, including instructions for running an XML-RPC server from a web server.

Perl’s Implementation of XML-RPC

As of this writing, there’s only one XML-RPC implementation on the Comprehensive Perl Archive Network (CPAN) (http://www.cpan.org). It’s named Frontier::RPC and consists of several Frontier modules, a number of examples, and Apache::XMLRPC , which embeds an XML-RPC server in the Apache web server using mod_perl.

Why is this implementation tagged “Frontier” and not “XMLRPC”? Because way back in 1998, when Ken MacLeod was first putting Frontier::RPC together, the XML-RPC specification didn’t exist. The protocol was merely “RPC over HTTP via XML,” the one used at Dave Winer’s UserLand site, implemented in the Frontier language. Ken’s work was, in fact, the first third-party implementation of the protocol to be released. Note that this chapter is based on Version 0.07 of Frontier::RPC. There are significant changes from the previous version, including the introduction of a new module, Responder, so be sure to upgrade if you have an earlier version.

Frontier::RPC uses other modules for much of its work: Data::Dumper, MIME::Base64, MD5, HTML::Parser, URI, various Net:: modules from the libnet distribution, various modules from the libwww-perl (LWP) suite, and XML::Parser. These are all available from CPAN.

All but one of these Perl modules should install without difficulty. However, the latest version of the XML::Parser module (a front end to the expat XML parser, written in C) does not include the expat C source. You must download this source from http://sourceforge.net/projects/expat. Fortunately, expat compiles cleanly on most versions of Unix, and Windows users of ActiveState Perl will find that XML::Parser is already installed.

A Perl script that wants to be an XML-RPC client uses the Frontier::Client module. An XML-RPC client script creates a Frontier::Client object and makes remote procedure calls through methods of that object.

A standalone Perl script that wants to be an XML-RPC server uses the Frontier::Daemon module. (“daemon” is the traditional Unix term for a long-running server process.) The script creates a long-lived Frontier::Daemon object that listens for XML-RPC method calls on a specific TCP/IP port. Unfortunately, Frontier::Daemon is not a particularly high-performance server. For web services that require better response time, consider using Frontier::Responder. This module lets a standard CGI process answer XML-RPC client calls. Used in conjunction with Apache and mod_ perl, these kinds of XML-RPC listeners are more responsive than programs written using Frontier::Daemon.

Regardless of whether you’re writing a client or server, you’ll probably use some of the Frontier::RPC2::* modules to consistently translate between XML-RPC data types and Perl. Although the server and client classes can guess how to convert between Perl and XML-RPC data types, these objects provide a way to remove the guesswork.

Data Types

The interesting part of an XML-RPC call is how language-specific data types are turned into XML-RPC tagged data. Each XML-RPC parameter must have one of the data types documented in the XML-RPC specification and introduced in Chapter 2. For example, here’s how the floating-point value 2.718 is encoded as a parameter:

<param><value><double>2.718</double></value></param>

Table 4-1 shows the correspondence (or lack thereof) between XML-RPC data types and Perl data types.

Table 4-1. Data types: XML-RPC versus Perl

XML-RPC data type(XML element)	Perl built-in data type	Frontier::RPC2 module
<int> (or <i4>)	scalar	Frontier::RPC2::Integer
<double>	scalar	Frontier::RPC2::Double
<string>	scalar	Frontier::RPC2::String
<boolean>	scalar	Frontier::RPC2::Boolean
<dateTime.iso8601>	scalar	Frontier::RPC2::DateTime::ISO8601
<base64>	scalar	Frontier::RPC2::Base64
<array>	array	Array
<struct>	hash	Hash

Translating Perl Values to XML-RPC Elements

Both clients and servers translate Perl values to XML elements as follows:

• When a Perl client’s XML-RPC call includes an argument list that contains a set of Perl values, the client translates each value to XML for inclusion in the outgoing <methodCall> message.

• When a Perl server creates a return value, it translates a single Perl value to XML and includes the result in a <methodResponse> message.

Packaging Scalar Values

Although Perl has just the scalar data type, XML-RPC has many types: int, double, string, Boolean, and sometimes even dateTime and base64. You can either let the Frontier libraries make an educated guess as to the appropriate XML-RPC encoding for Perl data types, or you can use Frontier::RPC2::* objects to force explicit representations.

The data types of the parameters to an XML-RPC call are part of the exposed API. If a server expects an integer and you send it a string form of that integer, you’ve done something wrong. Similarly, if a server expects a string and you send it an integer, you’re at fault. Although it may seem clumsy to use objects instead of simple scalars, they have a purpose: they formalize the encoding, and in doing so ensure that your code plays well with others.

If you are a Perl developer, you are used to working in an extremely flexible environment in which there is little worry about which data type applies to a particular variable. Does Perl consider “1” to be a number or a string? Actually, this detail depends on how the value is used. Unfortunately, XML-RPC doesn’t offer the same flexibility.

When you include a scalar value (either as a literal or with a scalar variable) in the argument list of an XML-RPC call, a Frontier::Client object tries to interpret it as a numeric value -- first as an integer, then as a floating-point number -- before simply treating it as a string. To encode “37” as a string and not an integer, then you’d need to create a Frontier::RPC2::String object as a wrapper for the value.

For example, these two calls:

               rpc-call(... , 1776, ...)
rpc-call(... , "1776", ...)

both produce an integer-valued parameter in the outgoing <methodCall> message:

<param><value><int>1776</int></value></param>

To explicitly encode the value as a string, you need to say:

$val = Frontier::RPC2::String->new("1776");
rpc-call(... , $val, ...)

or:

               rpc-call(... , Frontier::RPC2::String->new("1776"), ...)

When writing an XML-RPC server, you have the same choice of either letting the Frontier library implicitly encode return values or explicitly encode them with the Frontier::RPC2 classes.

Preparing Date-Time Data

XML-RPC date-time parameters are passed as strings in ISO 8601 format.^[7] Creating a date-time parameter involves two steps:

• Create a string in ISO 8601 format, specifying a particular date and time. Because this is a very simple format, you may find it practical to create the string yourself. For example, it doesn’t take too much work to write “10/12/00 at 2:57 PM” as the string 20001012T14:57:00. For a bit more automation, you may want to use the strftime( ) function in the standard Perl module POSIX. For example, here’s how to turn the current time, provided by built-in Perl functions, into an ISO 8601 format string:

  use POSIX "strftime";
    ...
  $dt_string = strftime("%Y%m%dT%H:%M:%S", localtime(time(  )));

• Wrap the ISO 8601 format string in an ISO 8601 object:

  $date_obj = Frontier::RPC2::DateTime::ISO8601->new($dt_string);

  Note that the Frontier::RPC2::DateTime::ISO8601 package name has an extra component (ISO 8601).

Including this ISO 8601 object in the argument list of an XML-RPC call produces the appropriate date-time parameter in the outgoing <methodCall>. Thus, the call:

               rpc-call(... , $date_obj, ...)

produces something like this:

<param><value>
<dateTime.iso8601>20001009T17:26:00</dateTime.iso8601>
</value></param>

Likewise, a server can specify an ISO 8601 object as a return value to produce the appropriate date-time parameter in the outgoing <methodResponse>.

Preparing Encoded Binary Data

The strategy for handling string-encoded binary parameters closely parallels that for date-time parameters. XML-RPC binary parameters are passed as strings encoded in the base64 content-transfer-encoding scheme. Creating a binary argument involves two steps:

• Create a base64 string that represents the binary data. You can do this using the encode_base64( ) function in the standard Perl module: MIME::Base64. For example, here’s how to turn the contents of a small binary data file mypicture.jpg into a base64 string:

  use MIME::Base64;
    ...
  open F, "mypicture.jpg" or die "Can't open file: $!";
  read F, $bin_string, 10000;
  $base64_string = encode_base64($bin_string);

• Then wrap the base64 string in a Base64 object. The package name is Frontier::RPC2::Base64:

  $bin_obj = Frontier::RPC2::Base64->new($base64_string);

Including this Base64 object in the argument list of an XML-RPC call produces the appropriate encoded-binary parameter in the outgoing <methodCall>. Thus, the call:

               rpc-call(... , $bin_obj, ...)

produces something like this:

<param><value>
<base64>/9j/4AAQSkZJRgABAQEASABIAAD/2wBDAAUDBAQE
AwUEBAQFBQUGBwwIBwcHBw8LCwkMEQ8SEhEPERETFhwXExQ=</base64>
</value></param>

Likewise, a server can specify a Base64 object as a return value to produce the appropriate encoded-binary parameter in the outgoing <methodResponse>.

Using Helper Methods to Create Objects

Instead of creating a Frontier::RPC2::Integer object directly, you can use your Client object’s int( ) method:

$int_obj = $client->int(1776);

The int( ) “helper method” creates an Integer object and returns it. But first, it performs a very valuable job: it checks that the scalar value to be wrapped in an Integer object really is an integer. The argument to int( ) must consist of digits only, optionally preceded by a plus (+) or a minus sign (-); otherwise, a die error occurs. Invoke the int( ) method in an eval block to deal with the error case:

eval { $int_obj = $client->int($value) };
if ($@) {
    # oops: $value wasn't really an integer
} else {
    $response = $client->call(... , $int_obj, ...);
}

Similarly, helper methods create all other Frontier::RPC objects: double( ) , string( ), boolean( ), date_time( ), and base64( ). At the time of writing, only the int( ), double( ), and boolean( ) methods perform data validation.

Preparing Array and Hash Data

Each argument type described in the preceding sections is an individual value: a single number, a single string, or a single block of binary data. XML-RPC also defines two aggregate (or “collection”) data types: <array> and <struct>. Happily, these XML-RPC types correspond exactly to Perl built-in data types:

• An XML-RPC <array> corresponds to a Perl array. It’s a sequence of values, each of which can be either individual or aggregate (another <array> or <struct>).

• An XML-RPC <struct> corresponds to a Perl hash (associative array). It’s a collection of name/value pairs; each name is a string, and each value can be either individual or aggregate.

To create an XML-RPC <array> or <struct> parameter, create a Perl array or hash with the appropriate values. Then specify a reference to the array or hash (not the aggregate data structure itself) as an argument to an XML-RPC call or as a server’s return value.

For example, suppose you want to create an argument that’s a three-item array: a string (employee name), an integer (employee ID), and a Boolean (is this a full-time employee?). Here’s how you might create the required array reference:

$emp_name = "Mary Johnson";
$emp_id   = "019";
$emp_perm = Frontier::RPC2::Boolean->new(1);

$ary_ref = [$emp_name, $emp_id, $emp_perm];

You can include the array reference $ary_ref in the argument list of any XML-RPC call. This call:

               rpc-call(... , $ary_ref, ...)

produces the following parameter in the outgoing <methodCall> message:

<param><value>
<array><data>
<value><string>Mary Johnson</string></value>
<value><int>019</int></value>
<value><boolean>1</boolean></value>
</data></array>
</value></param>

Frontier::RPC can encode nested hashes and arrays, just as you would in any other Perl script.

Translating XML-RPC Elements to Perl Values

Both clients and servers translate XML elements to Perl values as follows:

• When processing an XML-RPC method call, a Perl server translates the call’s parameters, encoded as XML elements in the <methodCall> message, to Perl values.

• A Perl client translates the server’s response, encoded as an XML element in the server’s <methodResponse> message, to a Perl value.

Certain data types can be translated to either a Perl object or a scalar value. Frontier::RPC defines a use_objects mode for both Client objects, which make XML-RPC method calls, and for Daemon objects, which service those calls. In use_objects mode, the Client or Daemon translates an incoming XML-RPC <int>, <double>, or <string> value to a Frontier::RPC Integer, Double, or String object. When not in use_objects mode, the Client or Daemon translates an XML-RPC <int>, <double>, or <string> value to a Perl scalar. All other XML-RPC values are translated to the corresponding built-in Perl object (array or hash) or Frontier::RPC object.

Let’s consider the viewpoint of a client receiving a response to its method call from the server. (The situation of a server processing the parameters of an XML-RPC method call is entirely similar.) The client code might be:

$response = rpc-call(...);

If the Client is in use_objects mode, $response is guaranteed to be a reference. For example:

• If the XML-RPC server responds with a <string> value, $response gets a reference to a Frontier::RPC2::String object.

• If the server sends a <boolean>, $response gets a reference to a Frontier::RPC2::Boolean object.

• If the server sends a <struct> or <array>, $response gets a reference to a Perl hash or array.

If the Client is not in use_objects mode, $response sometimes gets a regular Perl scalar -- when the server’s response is an <int>, <double>, or <string>. Incoming <int>, <double>, and <string> values embedded in an <array> or <struct> response are converted similarly: they become either Frontier::RPC objects (if in use_objects mode) or Perl scalars (if not). The resulting values are embedded within the array or hash that represents the overall response.

Extracting Values from Objects

In the previous section, we noted that an incoming XML-RPC data item is frequently translated to an object reference. There are a couple of methods for getting to the real data. If the reference is to a Perl array or hash, use the standard Perl arrow operator:

$response->[3]           # item at position 3 in a response array
$response->{"emp_name"}  # item with key emp_name in a response hash

If the reference is to one of the Frontier::RPC objects, use the object’s value( ) method:

$response->value(  )    # value wrapped in a Frontier::RPC object

The value( ) method always returns a Perl scalar. The value extracted from a Frontier::RPC2::Integer object is an integer; the value extracted from a Boolean object is either or 1; the value extracted from a Base64 object is a block of binary data encoded as a Base64 string; and so on. (To turn a Base64 string back into a block of binary data, use the decode_base64( ) function in the standard Perl module MIME::Base64.)

Determining the Type of Object

When you have an object in hand, you may need to be able to find out what kind of object it is. Of course, in most cases, you should know in advance, because the XML-RPC server’s API should be well documented. For example, the documentation may say that the response to a GetExpirationDate method should be an ISO 8601 object, the response to an Expired method should be a Boolean object, and a SetSize method should take a string argument and an integer argument. But maybe the API isn’t well documented, or perhaps you don’t want to trust the server to document its API. Or maybe you want to implement a method that accepts an argument of any data type.

The standard Perl tool for determining the type of a data item is the built-in ref function. If variable $response contains a Perl scalar value, the expression:

ref $response

yields the empty string. If $response contains an object reference returned by the call( ) method, this expression must yield one of the following strings:

ARRAY
HASH
Frontier::RPC2::Integer
Frontier::RPC2::Double
Frontier::RPC2::String
Frontier::RPC2::Boolean
Frontier::RPC2::DateTime::ISO8601
Frontier::RPC2::Base64

The following code skeleton shows how you can handle a “mystery” value:

$response = rpc-call( ... );
$objtype = ref $response;
if (not $objtype) {
    # response is a scalar value
} elsif {$objtype eq "ARRAY") {
    # response is an array
} elsif ($objtype eq "HASH") {
    # response is a hash
} elsif ($objtype eq "Frontier::RPC2::Integer") {
    # response in an Integer object
} elsif ...

XML-RPC Clients

Now that you understand how XML-RPC data types work in Perl, we can get to actually creating a Perl XML-RPC client -- a script that makes calls to an XML-RPC server and gets responses back from it. We’ll start with an overview of the method-call process and then describe how to create a client that does the job.

Client Architecture

To call one or more XML-RPC methods on a particular server, a Perl script creates a Frontier::Client object. This object represents a connection to that server. The script can then invoke the Client object’s call( ) method as many times as desired to make XML-RPC calls. The call( ) method:

• Accepts a user-specified string as the method name

• Converts each user-specified argument from Perl format (scalar or object) to XML format

• Packages Perl data in an XML <methodCall> message and sends it to the server

• Decodes the <methodResponse> message returned by the XML-RPC server into Perl data

Invoking a Remote Procedure

Here are the steps involved in making an XML-RPC call:

1. Create a Frontier::Client object.

2. Call the method, passing arguments.

3. Get the response to the call.

Creating the client object

The only Perl module you need to import explicitly is the one that defines the Frontier::Client package:

use Frontier::Client;

Create an object in the Frontier::Client package in the regular way, using the package’s new( ) method. You must specify a url argument, which sets the web address (URL) of the XML-RPC server host. For example:

$client = Frontier::Client->new(url => "http://www.rpc_wizard.net");

This line specifies that the (fictional) host www.rpc_wizard.net is the XML-RPC server. In specifying the host, keep these points in mind:

• You may need to include a port number in the URL. Some hosts run a regular web server on one port (port 80 is the industry standard) and an XML-RPC server on another port:

  http://www.rpc_wizard.net:8888

• You may need to specify the string RPC2 as extra path information in the URL, to identify it as an XML-RPC call:

  http://www.rpc_wizard.net/RPC2

  (In particular, servers implemented in Perl or Frontier::RPC impose this requirement.)

Each Client object you create is dedicated to a particular XML-RPC server. If you need to switch back and forth among several such servers, just create several clients. In addition to the required URL argument, the Client object-constructor method supports some optional arguments:

proxy

A URL that specifies a proxy host for the outgoing XML-RPC call to be routed through. You may need to use this option if your host is behind an Internet firewall.

debug

A flag (set its value to 1) that turns on display (using the print statement) of both the XML <methodName> message that represents the outgoing XML-RPC call and the <methodResponse> message returned by the server. This is a very valuable “training wheels” feature.

encoding

A string that specifies the character set encoding for the XML document (the <methodName> message) that contains the outgoing XML-RPC call. The string is inserted into the XML document header. For example, the following option:

encoding => "ISO-8859-4"

creates this XML declaration:

<?xml version="1.0" encoding="ISO-8859-4"?>

Be careful with this option. If the XML parser used by the XML-RPC server is unable to process the encoding you specify, an error occurs.

use_objects

A flag (set its value to 1) that enables use_objects mode in this Client. This enabling causes each scalar value in the <methodResponse> message returned by the server to be converted to an object of type Frontier::RPC2::Integer, Frontier::RPC2::Double, or Frontier::RPC2::String. (The use_objects mode is discussed further in Section 4.2.7.)

fault_as_object

A flag (set its value to 1) that changes the way in which the Client executes a die statement if it gets a <fault> response from the server. By default, the Client places a string value in variable $@ as it dies. If this flag is set, $@ gets a reference to a hash created from the <fault> structure.

You can list the argument-name/argument-value pairs in any order in the invocation of the new( ) method. For example, here’s an invocation that specifies several options:

$client = Frontier::Client->new(
          url         => "http://www.rpc_wizard.net:8888",
          use_objects => 1,
          debug       => 0,
          proxy       => "http://mylocalproxy.org");

Calling the XML-RPC method

As the preceding sections have suggested, you make an XML-RPC remote procedure call by invoking the call( ) method of a Client object:

$response = $client->call(method, parameter, ... );

The first -- and only required -- argument specifies the name of the remote procedure. Thus, a minimalist call might look like this (if $client is a Client object):

$client->call("hello_world");

Following the method-name argument, you can specify as many additional arguments as you like. Each such argument must be one of these data types:

• A Perl scalar value (integer, floating-point number, or string)

• A Frontier::RPC-defined object that represents some XML-RPC data type

• An array reference or a hash reference

The call( ) method packages all this data into a <methodCall> XML message and sends it to the XML-RPC server. See Example 4-1 for real Frontier::Client code in action.

Getting the response to the call

The Client object’s call( ) method accepts a response from the XML-RPC server. This response is in the form of an XML <methodResponse> message that contains a single <value> element (or a <fault> element, in the case of an error). In many cases, it’s a simple numeric value (for example, an int such as 47) or string value (such as "10-4 good buddy“). But the response can be any one of the XML-RPC data types, including the aggregate types <array> and <struct>, so that single response value might actually be a complex data hierarchy. Note that the call( ) method converts each XML-RPC data item in the response <value> back into the corresponding Perl value.

The response that comes back from the remote XML-RPC server becomes the return value of the call( ) method. So most of your calls will probably look like this:

$response = $client->call( ... );

Thus, the $response variable might get a scalar value, but it also might get an array reference, a hash reference, or an object.

Handling error responses

Sometimes the XML-RPC server cannot successfully execute a remote procedure call. Maybe you named a non-existent method; maybe you passed a bogus argument (such as a <dateTime.iso8601> value that is not a ISO 8601 format string); or maybe there was a bug in the method you called. When the server detects an error, it sends a special error response back across the wire. At the XML level, it’s a <fault> element containing a <struct> whose members are named faultCode and faultString.

When the Client’s call( ) method gets the error response, it calls die, placing a string that incorporates the faultCode and faultString values in the scalar variable $@. If the Client’s fault_as_object option is enabled, the $@ value is a reference to a hash created from the <struct> response:

$@->{"faultCode"}
$@->{"faultString"}

The previous description applies to error messages generated by the XML-RPC server. You can also experience errors in which a server host is never contacted at all: network unavailable, incorrect host name/address, or bogus port number. In these cases, the call( ) method generates a simple error message (in clear text, not in XML format). For example:

500 Can't connect to www.boomer.com:8889 (Unknown error)

This particular error message comes from the HTTP::Request object embedded in the Client object.

A Client Script

Example 4-1 shows an XML-RPC client to a simplified punch-clock system. The server for this system is defined later in Example 4-2. Often, a manager needs to track the time each workgroup spends on a particular project. The API for this system defines five procedures:

punch_in( $user )

Records that the specified user is beginning work now

punch_out( $user )

Records that the specified user is ending work now

project_total_time( )

Returns the total time all workgroup members spent on this project

emp_total_time( $user )

Returns the total time a given user has spent on this project

add_entry( $user , $start , $end )

Takes the start and end times a user provides and creates a record of that information

In Example 4-1, there are only two users in the workgroup, “jjohn” and “bob.” Although this script shows typical actions that would happen in a real system, it is more likely that these client calls would be made from a CGI script or other GUI that presented a friendlier interface to the punch-clock system.

Example 4-1. Punch-clock client

#!/usr/bin/perl --
# XML-RPC client for the punch-clock application

use strict;
use Frontier::Client;
use Time::Local;
use POSIX;

use constant SAT_MAY_5_2001_9AM => timelocal(0,0,9,5,4,101);
use constant SAT_MAY_5_2001_5PM => timelocal(0,0,17,5,4,101);
use constant XMLRPC_SERVER => 
                 'http://marian.daisypark.net:8080/RPC2';

# create client object
my $client = Frontier::Client->new( url   => XMLRPC_SERVER,
                                    debug => 0,
                                  );

# jjohn starts working on the project
print 
  "Punching in 'jjohn': ", 
  status( $client->call('punch_in', 'jjohn') ),
  "\n";

# bob stops working on the project
print
  "Punching out 'bob': ",
  status( $client->call('punch_out', 'bob') ),
  "\n";

# output the total time spent on the project
printf
  "Total time spent on this project by everyone: %s:%s:%s\n",
  @{$client->call('project_total_time')};

# output bob's time on the project
printf
  "Time 'bob' has spent on this project: %s:%s:%s\n",
  @{$client->call('emp_total_time', 'bob')};
 

# set up date-time values
my $iso8601_start = strftime("%Y%m%dT%H:%M:%S", 
                       localtime(SAT_MAY_5_2001_9AM)); 

my $iso8601_end   = strftime("%Y%m%dT%H:%M:%S", 
                       localtime(SAT_MAY_5_2001_5PM));

my $encoder = Frontier::RPC2->new;

my $start = $encoder->date_time($iso8601_start);
my $end   = $encoder->date_time($iso8601_end);

# log overtime hours for bob 
print 
  "Log weekend work for 'bob': ",
  status( $client->call('add_entry', 'bob', $start, $end) ),
  "\n";

sub status {
  return $_[0] ? 'succeeded' : 'failed';
}

As is typical of XML-RPC clients, a new Frontier::Client object needs to be created before invoking any remote procedures:

my $client = Frontier::Client->new( url   => XMLRPC_SERVER,
                                    debug => 0,
                                  );

By defining constants (such as XMLRPC_SERVER) at the top of the program, it becomes easier to change the URL of the XML-RPC server when neccessary. Including the debug parameter in the object initialization is a good idea, even if it’s not needed right away. Although client-side debugging is turned off in this example, it would be simple to turn it on again, if needed.

When a users come to work, they punch in. In a production environment, this functionality would be wrapped inside a nice GUI, but the XML-RPC call would still look something like this:

print 
  "Punching in 'jjohn': ", 
  status( $client->call('punch_in', 'jjohn') ),
  "\n";

In this case, the user “jjohn” is starting to work on this project. Because punch_in( ) returns a Boolean value indicating success, the call is wrapped in the status( ) subroutine to print out a result more understandable to humans. If a user tries to punch_out( ) without having successfully called punch_in( ), the procedure returns a value of false.

To see how much time all users have spent on this project, the program calls project_total_time( ). This function returns a three-element list that contains hours, minutes, and seconds, respectively. This list can be printed easily with printf:

printf
  "Total time spent on this project by everyone: %s:%s:%s\n",
  @{$client->call('project_total_time')};

It can be a little tricky to deal with date values in XML-RPC. To log Bob’s weekend overtime using the add_entry( ) procedure, the Unix date values for the start and end times need to be converted into ISO 8601 format. Using the Perl module Time::Local, a Unix date can be determined and assigned to a constant:

use constant SAT_MAY_5_2001_9AM => timelocal(0,0,9,5,4,101);

Then, using the POSIX function strftime, that date can be converted into ISO 8601 format:

my $iso8601_start = strftime("%Y%m%dT%H:%M:%S", 
                      localtime(SAT_MAY_5_2001_9AM));

Finally, this string can be used to create a Frontier::RPC2::ISO8601 object, which the server expects to receive. In the code fragment that follows, both $start and $end are Frontier::RPC2::ISO8601 objects:

$client->call('add_entry', 'bob', $start, $end) )

Note that although Example 4-1 does work, it is possible to take a lot more care in checking that each call succeeded and handled problems appropriately.

XML-RPC Servers

Now we can turn to the operation and construction of a Perl XML-RPC server -- a script that receives calls from an XML-RPC client and sends responses back to it. After presenting an overview of server operation, we describe how to create a server.

Server Architecture

To set up a server to handle incoming XML-RPC calls, a Perl script creates a Frontier::Daemon object. This object implements a server process that listens for XML-RPC calls on a particular port. The server dispatches each call to a corresponding Perl subroutine, and then it sends the subroutine’s return value back to the client as the response to the XML-RPC call.

The Frontier::Daemon object gets its knowledge of HTTP communications by being a specialization of the standard Perl HTTP::Daemon object. The Daemon object:

• Uses an XML parser to interpret the incoming XML-RPC <methodCall> message.

• Determines the method name (a string).

• Converts each parameter from XML format to a Perl scalar value or object.

• Invokes a Perl subroutine that corresponds to the specified method name. The Daemon passes the arguments (now in Perl format) to the subroutine in the standard manner, as the contents of the @_ array.

• Packages the subroutine’s return value in an XML <methodResponse> message and sends it back to the client.

Setting Up an XML-RPC Server

The most important part of creating an XML-RPC server is implementing the defined API. With the Frontier::Daemon library, remote procedures are implemented as simple Perl subroutines. As shown later in Example 4-2, there is a mapping between API procedure names and the Perl subroutines that implement them. This is convenient when API names conflict with Perl reserved words. Here is an example of a server that implements a single procedure called helloWorld with an anonymous subroutine:

use Frontier::Daemon;
Frontier::Daemon->new( LocalPort => 8080,
                       method => {
                                 helloWorld => 
                                   sub {return "Hello, $_[0]"}
                                 },
                      );

Frontier::Daemon is a subclass of HTTP::Daemon, which is a subclass of IO::Socket::INET. This means that the object constructor for Frontier::Daemon uses the named parameter LocalPort to determine the port on which it should listen.

The next most important parameter in this object’s constructor is called methods; it points to a hash reference that maps API procedure names to the subroutine references that implement them. (Note that different API procedure names can map to the same Perl subroutine.)

The Perl subroutine that implements an XML-RPC API procedure receives its argument list in the standard @_ array. The data types of these arguments depend on whether the Daemon object was created in use_objects mode. The guidelines are as follows:

• If an argument is a Perl scalar, you can use it directly.

• If an argument is a Frontier::RPC object, invoke its value( ) method to determine its value (which is always a scalar).

Normally, the Perl subroutine that implements an XML-RPC procedure produces a return value. The Daemon object takes this value and it sends back across the wire, in the form of an XML <methodResponse> message, to the client. All subroutines must return at most one scalar value. This can be a simple scalar, like an integer or string; or it can be a reference to a complex value, such as a list of hashes of lists. The value’s complexity is irrelevant. (Veteran Perl hackers shouldn’t expect wantarray( ) to work in XML-RPC servers. Remember, the clients calling these servers may not have any notion of list versus scalar context.)

Sometimes the Perl subroutine that implements an XML-RPC method doesn’t return a value. Perhaps the subroutine encounters a runtime error (e.g., division by zero, file not found, etc.); or maybe there is no subroutine because the client requested a nonexistent method. XML-RPC terms this situation a fault.

When any of these situations occurs, the Daemon automatically responds with a special <struct> value, containing two members:

• A <faultCode> element containing an integer value (<int> parameter).

• A <faultString> element containing a string value (<string> parameter).

A Server Script

Example 4-2 implements the punch-clock system described in Section 4.3.3. Refer to that section for a discussion of the API. This system stores its information in a MySQL database, accessed with standard DBI calls.

Example 4-2. Punch-clock server

#!/usr/bin/perl --
# XML-RPC server for the punch-clock application

use strict;
use Frontier::Daemon;
use DBI;

# create database handle
my $Dbh = DBI->connect('dbi:mysql:punch_clock', 'editor', 'editor') ||
          die "ERROR: Can't connect to the database";

END { $Dbh->disconnect; }

# initialize XML-RPC server
Frontier::Daemon->new(
                       methods   => {
                              punch_in            => \&punch_clock,
                              punch_out           => \&punch_clock,
                              project_total_time  => \&total_time,
                              emp_total_time      => \&total_time,
                              add_entry           => \&add_entry,,
                                     },
                       LocalPort => 8080,
                     );

# punch_clock API function
sub punch_clock {
  my ($user) = @_ or die "ERROR: No user given";

  $Dbh->do("Lock Tables punch_clock WRITE") 
      or die "ERROR: Couldn't lock tables";

  if( is_punched_in($user) ){
    my $sth = $Dbh->prepare(<<EOT);
update punch_clock set end=NOW(  ) where username = ? 
and (end = "" or end is NULL)
EOT
    $sth->execute($user) || die "ERROR: SQL execute failed";

  }else{
    my $sth = $Dbh->prepare(<<EOT);
insert into punch_clock (username, begin) values (?, NOW(  ))
EOT
    $sth->execute($user) || die "ERROR: SQL execute failed";

  }

  $Dbh->do("Unlock Tables");

  my $encoder = Frontier::RPC2->new;
  return $encoder->boolean(1);
}

# total_time API function
sub total_time {
  my ($user) = @_;

  my $sth; 
  if( $user ){
    $sth = $Dbh->prepare(<<EOT);
Select SEC_TO_TIME(SUM(UNIX_TIMESTAMP(end) - UNIX_TIMESTAMP(begin))) 
from punch_clock where username = ?
EOT
  $sth->execute($user) || die "ERROR: SQL execute failed";

  }else{
    $sth = $Dbh->prepare(<<EOT);
Select SEC_TO_TIME(SUM(UNIX_TIMESTAMP(end) - UNIX_TIMESTAMP(begin))) 
from punch_clock
EOT
    $sth->execute || die "ERROR: SQL execute failed";
  }

  my $total = $sth->fetchall_arrayref;
  # return an hour, minute, second list
  if( $total && $total->[0] ){
    return [ split ':', $total->[0]->[0], 3 ];
  }else{ 
    die "ERROR: Couldn't retrieve any data"
  } 
}

# add_entry API function
sub add_entry {
  my ($user, $start, $end) = @_;

  if( !($user && $start && $end) ){
    die "ERROR: Need username and start and end dates";
  }

  my $sth = $Dbh->prepare(<<EOT);
insert into punch_clock (username, begin, end) values (?,?,?)
EOT
 
  unless( $sth->execute($user, 
                  iso2mysql($start->value), 
                  iso2mysql($end->value))   ){
    die "ERROR: SQL execute failed";
  }

  my $encoder = Frontier::RPC2->new;
  return $encoder->boolean(1);
  
}

# helper functions

sub is_punched_in {
  my ($user) = @_;
  
  my $sth = $Dbh->prepare(<<EOT);
select begin from punch_clock where username = ? 
and (end = "" or end is NULL)
EOT

  $sth->execute($user) || die "ERROR: SQL execute failed";
  my $result = $sth->fetchall_arrayref;
  if( $result && $result->[0]){
    if( $result->[0]->[0] ){
      return 1;
    }
  }
  return;
}

sub iso2mysql {
  my ($iso) = @_;
  
  $iso =~ s/T/ /;
  $iso =~ s/^(.{4})(.{2})(.{2})/$1-$2-$3/;
  return $iso;
}

Because this is a single process server, it creates one DBI handle as a file-scoped global variable that is visible to every subroutine that needs to get at the SQL tables. The DBI handle is created as follows:

my $Dbh = DBI->connect('dbi:mysql:punch_clock', 'editor', 'editor') ||
          die "ERROR: Can't connect to the database";

If you are unfamiliar with Perl’s DBI module, look at Programming the Perl DBI, by Alligator Descartes and Tim Bunce (also published by O’Reilly & Associates, 2000). For security reasons, you should certainly choose a better username (“editor”) and password (ahem, “editor”). Because DBI complains to STDERR if DBI handles aren’t explicitly closed, an END subroutine is used to make sure this is done whenever the process is killed:

END { $Dbh->disconnect; }

The object initialization of this Frontier::Daemon server should look familiar by now:

Frontier::Daemon->new(
                       methods   => {
                              punch_in            => \&punch_clock,
                              punch_out           => \&punch_clock,
                              project_total_time  => \&total_time,
                              emp_total_time      => \&total_time,
                              add_entry           => \&add_entry,,
                                     },
                       LocalPort => 8080,
                     );

Again, the mapping of API procedure names to Perl subroutines happens here. Notice that the five API procedures map to only three Perl subroutines.

Here is the SQL needed to define the tables this punch-clock system uses. The first table, users, simply maps usernames to first and last names:

create table users (
  username  varchar(12) auto_increment not null primary key,
  firstname varchar(25),
  lastname  varchar(25)
);

The second table, punch_clock, maps usernames to start and end times:

create table punch_clock (
  username varchar(12) not null,
  begin   datetime not null,
  end     datetime,
  primary key (username, begin)
);

When a user tries to punch in or out, the subroutine punch_clock( ) is called. By looking at the punch_clock table, this routine can figure out if the user needs to punch out (if there’s a row in the table without a defined “end” column) or punch in. To prevent another process from updating the table, the lock tables MySQL directive is issued. This isn’t strictly necessary here, but in a larger system, selecting from a table and then updating it are not atomic actions. In other words, another process could alter the table between the select and update. Upon successful completion, punch_clock( )returns a Boolean object to the client:

my $encoder = Frontier::RPC2->new;
return $encoder->boolean(1);

When either project_total_time( ) or emp_total_time( ) is called, total_time( ) is invoked in the server. Some fancy MySQL-specific code here converts from the MySQL date-time type to a Unix timestamp to figure out the interval between the start and end times:

    $sth = $Dbh->prepare(<<EOT);
Select SEC_TO_TIME(SUM(UNIX_TIMESTAMP(end) - 
                       UNIX_TIMESTAMP(begin))) 
from punch_clock where username = ?
EOT

The SUM( ) function takes all rows that have the given username and adds the difference of every row’s end and start times (calculating the user’s total work time). The result is a number of seconds, which is then converted back into hours, minutes, and seconds by SEC_TO_TIME( ). When fetchall_arrayref fetches the result, it contains only one row with one field, which is a colon-separated string of hours, minutes, and seconds:

return [ split ':', $total->[0]->[0], 3 ];

This can be split into a three-element list easily. Recall that only single values can be returned to XML-RPC clients, so this list needs to be enclosed in an anonymous array.

The last subroutine that implements an API procedure is add_entry( ). While the helper function iso2mysql doesn’t do anything tricky, it does use regular expressions to turn ISO 8601 date values into something MySQL can use to populate its date-time fields.

Integrating XML-RPC into a Web Server

The preceding section describes a standalone server -- a process dedicated to handling XML-RPC method calls that listens for those calls on a dedicated TCP/IP port. But this approach ignores (or, at least, minimizes) one of XML-RPC’s main design points: its use of the HTTP protocol. In many situations, the machine designated to handle XML-RPC method calls is already running a process that accepts HTTP requests -- a standard web server.

So why not let the web server handle all the HTTP-level communications? A browser might retrieve a regular web page at one location on the server (for example, http://MyStore.com/catalog/mittens.html), while an XML-RPC client might make method calls at another location (for example, http://MyStore.com/catalogAPI). Same server, same port -- different web-based information services.^[8]

This section describes how to implement an XML-RPC server as part of a web server, using CGI to have a Perl script handle an incoming XML-RPC method call. Note that your code is invoked only when a method call arrives; the web server itself is the long-lived listener process. Note also that the CGI script runs in a separate OS-level process (or thread) than the web server.

An XML-RPC client just needs to know which URL to specify; it doesn’t need to know whether it’s communicating with a standalone server, a CGI script, or an Apache virtual document.

This discussion assumes that you already have a CGI-enabled web server running. A good resource in this area is CGI Programming with Perl, 2nd Edition by Scott Guelich, Shishir Gundavaram, and Gunther Birznieks (published by O’Reilly, 2000).

Here’s a procedure for taking an existing Perl script that implements a standalone XML-RPC server and turning it into a CGI script:

1. In a directory configured to hold executable CGI programs, create a Perl script file that uses the Frontier::Responder module.

2. Copy the subroutines that implement the XML-RPC API into the new script.

3. Create a mapping of the XML-RPC API procedure names to the subroutines copied from the existing script.

4. Initialize a new Frontier::Responder object with this mapping.

To map the XML-RPC API procedure names to real Perl subroutines, simply create a hash whose keys are the API procedure names and whose values are references to the implementing subroutines. In this instance, that hash looks like this:

$map = (
         punch_in           => \&punch_clock,
         punch_out          => \&punch_clock,
         project_total_time => \&total_time,
         emp_total_time     => \&total_time,
         add_entry          => \&add_entry,
        );

The next step is to create a new Frontier::Responder object that is constructed with this hash. The hash variable, $map, refers to the previous code:

use Frontier::Responder;

# process the call, using Responder object to 
# translate to/from XML
my $response = Frontier::Responder->new( methods => $map );
print $response->answer;

One important difference between standalone servers and CGI servers is how they respond to print statements. In a standalone server, print output goes to the server’s console; but in a CGI script, print output goes into the HTTP response and will most likely garble it. So, don’t use print in a CGI-based XML-RPC server.

Clients that talked to a Frontier::Daemon server need to change the URL parameter in the Frontier::Client object initialization when the server is ported to a CGI environment. Fortunately, this change is small and isolated:

$client = Frontier::Client->new(
          url => "http://somewhere.com/cgi-bin/xmlrpc.pl");

Frontier::Responder can also be used in mod_perl environments. Use of it can help improve the performance of an XML-RPC server. Using mod_perl, code is invoked only when a client call arrives; the web server itself is the long-lived listener process. Unlike traditional CGI scripts, however, your program is cached in the web server and doesn’t run in a separate process. Besides providing a big performance boost, this lets you do things like maintain persistent data, cache connections, and take advantage of Apache features such as authentication and logging. For more information on mod_perl, see the book Writing Apache Moduleswith Perl and C, by Lincoln Stein and Doug MacEachern (published by O’Reilly, 1999).

---

^[7] For a brief discussion of this format, see http://www.cl.cam.ac.uk/~mgk25/iso-time.html.

^[8] As we discussed in Chapter 1, there is a lot of controversy on the subject of using the same TCP/IP port to provide multiple services with different performance requirements, security requirements, etc. We’ll duck those issues here and just tell you how to get the job done.