The term web service refers to a set of standards for exchanging data between two systems. Though the systems may be built with completely different platforms, the web service protocols allow the systems to exchange information. For example, a service built with Perl scripts on a Linux machine could exchange information with a Visual Basic application on a Microsoft computer because both platforms can speak the common web service language.
Tip
Sometimes a web service is referred to as an application programming interface (API), a similar pre-Web concept. The terms are used interchangeably throughout this book.
The phrase web service has also come to describe a specific method of exchanging data using XML files sent over the familiar HTTP protocol. XML is a textual, structured representation of data that both computers and humans can read, and HTTP is the standard protocol for delivering content across the Web. Yahoo! has implemented a straightforward XML over HTTP architecture for its web services.
Yahoo! has chosen a web services standard called REST for delivering most of its data. If you’ve used the Web, you’ll be familiar with how REST works. A specially constructed URL will return the data you’re after—just as the URL for a document on the Web returns that document. Instead of a web page (HTML document), REST requests return an XML document. The key to using Yahoo! Web Services is learning how to construct the proper request URLs.
Before creating your own request URLs though, it’d be good to know exactly what you can get your hands on through the API.
Not every piece of data available at Yahoo! is available via web services. For example, movie showtimes [Hack #42] and television schedules [Hack #44] are still limited to the Web. But you can get access to Yahoo! Search results and a few services that aren’t available via the Web.
The list of web services that Yahoo! provides is continually evolving, and you should browse to the Yahoo! Developer Network (http://developer.yahoo.net) to keep up with additions and changes. At the time of this writing, the services available include:
- Search
As you’d expect, you can access search results across the Web, Image, Local, News, Video, Audio, Shopping, and Contextual [Hack #97] Searches.
- Text Analysis
In addition to the search results, you can access some features of the Yahoo! Search pages including Related Queries and Spelling Suggestions. You can also access the Content Analysis feature that extracts keywords from text.
- My Web
You can access information from Yahoo!’s My Web [Hack #7] , giving you the ability to integrate your saved pages into other applications.
- Maps
By specifying the URL of a geo-encoded RSS file, you can plot your own points [Hack #91] on an interactive Yahoo! Map.
- Flickr
Purchased by Yahoo! in 2005, Flickr [Hack #67] joined the Yahoo! family with a fully realized web services offering of its own. You can use the Flickr API to programmatically access every facet of the photo-sharing service from adding photos to browsing information about Flickr members.
- RSS
While RSS is not technically a web service, it’s important to keep in mind that Yahoo! offers RSS feeds across their sites, and the feeds provide a simple way to integrate data with your web site.
Along with offering web services, Yahoo! has started a conversation with developers at the Yahoo! Developer Network. Browse to the Community Resources page (http://developer.yahoo.net/community) to see how other developers are using Yahoo! data in their applications.
Yahoo! has made this data available for free, but there are a few rules you have to play by to keep your access to the data. The key rule to keep in mind is that Yahoo! data can’t be used in commercial applications. That means you can’t use the API in an application you plan to sell or on a web site you charge people to access. Yahoo! also asks that you add attribution somewhere in your application. Adding the phrase “Powered by Yahoo! Search” to a site that uses Yahoo! data will fulfill the requirement.
Tip
If you’re a lawyer (or think like one) you can read the complete Yahoo! APIs Terms of Use at http://developer.yahoo.net/terms.
If the API is free, what’s in it for Yahoo!? Yahoo! Search Evangelist Jeremy Zawadony said, “By exposing interesting pieces of Yahoo! to the larger developer community, we think they’ll build applications that benefit both us and our users.” In addition to showing you some basic code to access Yahoo! in various programming languages, this chapter will show you some of the applications that outside developers are building with Yahoo! data.
Yahoo! imposes some limits on the number of requests that can be made per day. The request limit can vary by service, but at the time of this writing, you’re limited to 5,000 queries per day for most services. Yahoo! tracks usage by IP address (a numeric address for every machine connected to the Internet), so a single machine can’t make more than 5,000 requests in a single day.
If you ever reach the limit, you’ll receive an HTTP 403 “Forbidden” error that indicates you’re over your limit for the day.
Yahoo! also requires that for every application you build, you include an ID unique to the application with every request the application makes. You can request a unique ID at any time by browsing to http://api.search.yahoo.com/webservices/register_application. You’ll need to be logged in with your Yahoo! ID to request an application ID.
Keep in mind that Yahoo! doesn’t use application IDs to monitor request limits; the ID is simply used to track your application usage. In turn, Yahoo! provides the usage data for your application IDs at http://api.search.yahoo.com/webservices/register_application. Figure 4-1 shows the Application ID report available at the Yahoo! Developer Network.
Now that you know the rules, you can start making Yahoo! Web Service requests by assembling URLs for the different services.
The Search, Text, and My Web services all use the same structure for request URLs. The URL starts with the base service URL, http://api.search.yahoo.com. From there, you add a service name, version number, and specific function to the URL. Calling the Web Search (
webSearch) function of the Web Search service (WebSearchService) looks like this:
http://api.search.yahoo.com/WebSearchService/V1/webSearch
At the time of this writing, all Yahoo! Search services are in Version 1, so the V1 parameter will be consistent throughout. As Yahoo! upgrades the service in the future, this parameter might change.
The final step to assembling the proper URL is adding parameters to the query. The only parameters required for a web search are
appid and query, so a Web Search request could be as simple as:
http://api.search.yahoo.com/WebSearchService/V1/webSearch?appid=insert app
Once you have the query built, you can even plug the URL in your browser’s address bar to bring up the response XML and see exactly which information you’ll receive. Figure 4-2 shows a Yahoo! Search response for the term hacks in a browser.
Every Yahoo! service contains a different set of potential request parameters you can use to modify your requests. For example, the Yahoo! Web Search service allows you to filter searches by language (language), country (country), adult content (
adult_ok), document type (type), or Creative Commons license (license). Each service also has parameters for controlling the page of the response. Just as the Web Search lists responses across a number of pages, Yahoo! Search Web Services also breaks the responses into pages. You can adjust the response by specifying the number of results (results) up to 50 per page and specifying the result position (start) the current page should start with.
You can read the full documentation and find all of the parameters available for each of the services at the developer site (http://developer.yahoo.net).
To create your own map with custom points, you’ll need an application ID and the URL of a specially prepared RSS file on your own server. You can assemble these pieces into a URL like this:
http://api.maps.yahoo.com/Maps/V1/AnnotatedMaps?appid=insert app
The Maps API is a bit different because you’re receiving not XML, but a fully realized HTML document. You’ll find a custom map example [Hack #91] in Chapter 5.
At the time of this writing, the Flickr API operates under different rules from other Yahoo! Search APIs. You’ll need to request a Flickr API Key at http://www.flickr.com/services/api/key.gne to make requests, and you have the option of using a REST, XML-RPC, or SOAP interface for the requests. You can find a complete list of API methods available for Flickr at http://www.flickr.com/services/api.
You’ll find Yahoo! RSS documents throughout Yahoo! sites, and you don’t need anything extra to use them. Simply copy the RSS URL and paste it into your favorite RSS newsreader, or integrate the files with your own web sites. Yahoo! doesn’t limit access or require an application ID to use RSS, and the feeds aren’t covered by the Yahoo! Web Services terms of service.
The way to parse XML responses will be unique to your development environment, and this chapter provides basic examples in nine different languages. Typically, you’ll need two pieces of software that aren’t always built into your environment of choice: code to handle HTTP requests and code to parse XML responses.
Yahoo! responses are in a proprietary XML format that’s unique to the Yahoo! APIs. The top-level tag in Yahoo! Search API responses is <ResultSet>, with a number of <Result> tags holding information about each individual result.
Each result detail will vary by service, but typically you’ll receive all of the data necessary to duplicate what you find on the corresponding Yahoo! site. For example, each Web Search result includes the following XML tags: <Title>, <Summary>,< Url>, <ClickUrl>, <MimeType>, <ModificationDate>, and <Cache>. As you can tell from the tag names, the data you’ll receive is almost identical to the data on a Yahoo! Web Search results page outlined in the introduction to Chapter 1.
The key to understanding what’s available and how you can use it is simply playing with data and building some sample applications. The hacks in this chapter should give you a head start on integrating Yahoo! data into your own programs.
Get Yahoo! Hacks now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
