O'Reilly logo

Data Mashups in R by Xiao-Yi Li, Jeremy Leipzig

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 1. Mapping Foreclosures

Messy Address Parsing

To illustrate how to combine data from disparate sources for statistical analysis and visualization, let’s focus on one of the messiest sources of data around: web pages.

The Philadelphia sheriff’s office posts foreclosure auctions on its website each month. How do we collect this data, massage it into a reasonable form, and work with it? First, create a new folder (for example, ~/Rmashup) to contain our project files. It is helpful to change the R working directory to your newly created folder.

  #In Unix/MacOS
  > setwd("~/Documents/Rmashup/")
  #In Windows
  > setwd("C:/~/Rmashup/")

We can download this foreclosure listings web page from within R (or you may instead choose to save the raw HTML from your web browser):

  > download.file(url="http://www.phillysheriff.com/properties.html",
    destfile="properties.html") 

Here is some of this web page’s source HTML, with addresses highlighted:

    6321 Farnsworth St.
           62nd                                            Ward
    1,379.88 sq. ft.  BRT# 621533500  Improvements: Residential Property
    <br><b>
    HOMER SIMPSON
    &nbsp;   &nbsp;
    </b>   C.P. January Term, 2006  No. 002619 &nbsp; &nbsp; $27,537.87
    &nbsp;  &nbsp;  Phelan Hallinan & Schmieg, L.L.P.
     <hr />
    <center><b>      243-467           </b></center>
    1402 E. Mt. Pleasant Ave.
    &nbsp;   &nbsp;  50th                                            Ward
    approximately 1,416 sq. ft. more or less  BRT# 502440300 ...

The sheriff’s raw HTML listings are inconsistently formatted, but with the right regular expression we can identify street addresses: notice how they ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required