Posted on by & filed under Content - Highlights and Reviews, Programming & Development, Tech.

code Tom Barker is a software engineer, an engineering manager, a professor and an author. Currently he is Director of Software Engineering and Development at Comcast, and an Adjunct Professor at Philadelphia University. He has authored Pro JavaScript Performance: Monitoring and Visualization, Pro Data Visualization with R and JavaScript, and Technical Management: A Primer, and can be reached at @tomjbarker.

Tracking web performance is a key to maintaining your web site. One of the key tools that I use to keep track of the web performance of my sites is WebPagetest.

WebPagetest was originally created by AOL and open sourced for public consumption and contribution in 2008. It is available as a public web site, as an open source project, or for download to run as a private instance. The code repository is found here. The public web site is located here. The public site is maintained and run by Pat Meenan, via the WPO Foundation.

WebPagetest is a web application that takes a URL, and a set of configuration parameters as input and runs performance tests on that URL. The amount of parameters that you can configure for WebPagetest is extraordinarily robust.

If you want to run tests on web sites that are not publicly available – like a QA or development environment, or if you can only have your test results stored on your own servers because of legal or other reasons, then installing your own private instance of WebPagetest is the way to go.

Otherwise, there is no reason not to use the public instance.

You can choose from a set of locations from around the world where your tests can be run. Each location comes with one or more browsers that can be used for the test at that location. You can also specify the connection speed and the number of tests to run.

In the advanced panel you can have the test stop running at document complete. That will tell you when the document.onload event is fired, instead of when all assets on the page are loaded. This is useful because XHR communications that may happen after page load could register as new activity and skew the test results.

You can also have the test ignore SSL certification errors that would otherwise block the test because an interaction with the end user would be needed to either allow the transaction to proceed, view the certificate, or cancel the transaction.

There are a number of other options in the advanced tab, like having the test capture the packet trace and network log, or have the test keep the user agent string of the browser running the test instead of appending a string to identify the visit as a WebPagetest test.

In the auth tab you can specify credentials to use if your web site uses HTTP authentication for access.

Sometimes you may need to test very specific conditions. Maybe you are running a multivariate test on a certain feature set where you are only serving specific features on specific client configurations, like iPhone specific features. Or perhaps you are targeting certain features for users that are grouped by inferred usage habits. You’d want to run performance tests on these features that are only triggered by special events.

The script tab allows you to do just that. You can run more complex tests that involve multiple steps including navigate to multiple URLs, send Click and Key events to the DOM, submit form data, execute ad hoc JavaScript, and update the DOM. You can even alter the HTTP request settings to do things like set specific cookies, set the host IP, or change the user agent.

For example, if you want to make a client appear to be an iphone, you can simply add the following script in the script tag:

The setUserAgent command spoofs the client user agent and the navigate command points the test to the specified URL. You can read more about the syntax and some of the great things you can do with scripting WebPagetest here:

The block tab allows youo to block content coming in your request. This is useful to compare results with and without ads, with or without JavaScript, and with or without images. Instead of using the block tab, you could just incorporate a blocking command as part of the script in the script tab. If you wanted to script out blocking all PNGs in a site, it would look like this:

And finally, the Video tab allows you to capture screen shots of your page as it loads and view them as a video. This is useful for being able to see what your page looks like as it loads, which is most useful for when you have content loaded in asynchronously. This allows you to see at what point in the process your page looks to be usable.

So, once you set all of the configuration choices you can run your test. You can see the results screen below.


You first have the summary screen, which aggregates all of the vital relevant information for you. At the top right you have a summary of the Page Speed results for your page. This is a high level representation of the same information you would be presented with if you had run a test in Page Speed, but shown in YSlow-esque letter grading format.

Sitting in a table above the waterfall charts and screen shots you have page level metrics, numbers for the load time of the full page, how long the first byte took to load, and how long until the first piece of content was drawn to the stage. You can also access here how many DOM elements are on the page, the time it took for the document.onload event to fire, the time it took for all elements on the page to load, and how many HTTP requests were needed to draw the page.

Make note of this. This data contains the fundamental information that make up the quantitative metrics that you will use to chart your web performance. This is the true essence of web performance.

Below this table you’ll find two columns. On the left you see a waterfall chart, and on the right you see the screen shots for both the first time view and the cached repeat view. You should be aware of how useful waterfall charts are.

Below these are two pie charts. The chart on the left shows the breakdown of percent of requests by content type. The chart on the right shows the percent of bytes by content type. This is useful for identifying the largest areas that can be optimized. If your JavaScript is only 5% of your overall payload, but your images are 70%, you would be better served optimizing images first.

This summary page aggregates at a high level all of the data that you can find in the pages laid out in the sub navigation. Click on the Details, Performance Review, Page Speed, Content Breakdown, Domains, and Screen Shot links in the sub navigation for a deeper dive into each. The Content Breakdown section can be seen below.



As you can see, WebPagetest provides a wealth of information about the web performance of a site, but best of all it’s fully programmable. It provides an API that you can call to provide yourself with all of this information. In the next post we explore the API and construct our own application for tracking and reporting out web performance.

For more details about web performance, see the resources below from Safari Books Online.

Not a subscriber? Sign up for a free trial.

Safari Books Online has the content you need

Web Performance Tuning, Second Edition is about getting the best possible performance from the Web. This book isn’t just about tuning web server software; it’s also about streamlining web content, getting optimal performance from a browser, tuning both client and server hardware, and maximizing the capacity of the network itself.
Web Performance Daybook Volume 2 is a guide that contains 32 leading web performance experts that offer practical tips, techniques, and advice for optimizing your site’s user experience. This collection of articles will inspire you to squeeze every ounce of performance from your site—whether you’re a web developer, mobile developer, or web designer.
Pro JavaScript Performance: Monitoring and Visualization by Tom Barker, gives you the tools to observe and track the performance of your web applications over time from multiple perspectives, so that you are always aware of, and can fix, all aspects of your performance.

Tags: AOL, tests, web performance, webpagetest,

Comments are closed.