Posted on by & filed under django, programming, testing.

See Part 1 and Part 2 for the rest of this series.

Now that we’ve figured out how to get Selenium tests to run in various browsers on different operating systems, we should probably get some actual test cases written. The Django documentation gives an example like this:

def test_login(self):
    self.selenium.get('%s%s' % (self.live_server_url, '/login/'))
    username_input = self.selenium.find_element_by_name("username")
    username_input.send_keys('myuser')
    password_input = self.selenium.find_element_by_name("password")
    password_input.send_keys('secret')
    self.selenium.find_element_by_xpath('//input[@value="Log in"]').click()

This is a good start, and is pretty cool in its own right, but there are a few potential problems:

  • Selenium sometimes returns to your code after a GET before the page has fully rendered in the browser, and often before any initial JavaScript has finished running.  It’s possible that the elements you want to interact with aren’t ready by the time your code starts to send the browser commands regarding them.
  • XPath, while having some minor advantages over CSS selectors, is generally much slower.  While it’s not a huge difference for any single execution, if you have a large test suite you should probably prefer fetching elements by ID or CSS selector whenever possible; this is both faster and more consistent with how you’re probably writing CSS and JavaScript anyway.
  • Quite a bit of code is repeated here, and this is just a single small test case.

I’ll address this last point first, because it informs work on the other ones: for any non-trivial test suite, it’s a good idea to write your own API to wrap the most common Selenium operations.  The Selenium webdriver API was optimized for ease of implementation by browsers, not for ease of writing tests.  Since we have a common base class for all of our Django/Selenium tests already, we can just build our custom API into that.  Let’s start with a simple example, abbreviating that GET operation:

def get(self, relative_url):
    self.sel.get('%s%s' % (self.live_server_url, relative_url))
    self.screenshot()

This is a minor win in conciseness, we can abbreviate the first command in the test case to self.get('/login/'). But the other handy thing is that second part of the method, which is implemented like this:

SCREENSHOT_DIR = os.path.dirname(__file__) + '/../../log/selenium_screenshots'

def screenshot(self):
    if hasattr(self, 'sauce_user_name'):
        # Sauce Labs is taking screenshots for us
        return
    name = "%s_%d.png" % (self._testMethodName, self._screenshot_number)
    path = os.path.join(SCREENSHOT_DIR, name)
    self.sel.get_screenshot_as_file(path)
    self._screenshot_number += 1

(You’ll also need to add self._screenshot_number = 1 to __init__().) That’s pretty useful; even when not using Sauce Labs, we can now generate screenshots for each page we load.  And we don’t need to specify when to do it in the tests, it just happens automatically.  Now let’s try a slightly trickier one, that duplicated code for text entry:

def enter_text(self, name, value):
    field = self.wait_for_element_by_name(name)
    field.send_keys(value)
    self.screenshot()
    return field

def wait_for_element_by_name(self, name):
    element_is_present = lambda driver: driver.find_element_by_name(name)
    msg = "An element named '%s' should be on the page" % name
    element = Wait(self.sel).until(element_is_present, msg)
    self.screenshot()
    return element

Ok, we’re going a little screenshot-happy, but it’s only disk space and they could be useful when debugging a failing test. Note that we’re being careful by explicitly waiting for the input field to be present before we try adding text to it. It’s a better idea to wait until a specific condition is met like this than to wait for a fixed duration, since different systems take different amounts of time to do stuff and you don’t want your test to take forever because it’s always waiting long enough for the slowest possible computer to finish each operation. Selenium provides a WebDriverWait class for handling cases like this, but I like to expand on it a little:


# Default operation timeout in seconds
TIMEOUT = 10

# Default operation retry frequency
POLL_FREQUENCY = 0.5

class Wait(WebDriverWait):
    """ Subclass of WebDriverWait with predetermined timeout and poll
    frequency.  Also deals with a wider variety of exceptions. """
    def __init__(self, driver):
        """ Constructor """
        super(Wait, self).__init__(driver, TIMEOUT, POLL_FREQUENCY)

    def until(self, method, message=''):
        """Calls the method provided with the driver as an argument until the 
        return value is not False."""
        end_time = time.time() + self._timeout
        while(True):
            try:
                value = method(self._driver)
                if value:
                    return value
            except NoSuchElementException:
                pass
            except StaleElementReferenceException:
                pass
            time.sleep(self._poll)
            if(time.time() > end_time):
                break
        raise TimeoutException(message)

    def until_not(self, method, message=''):
        """Calls the method provided with the driver as an argument until the
        return value is False."""
        end_time = time.time() + self._timeout
        while(True):
            try:
                value = method(self._driver)
                if not value:
                    return value
            except NoSuchElementException:
                return True
            except StaleElementReferenceException:
                pass
            time.sleep(self._poll)
            if(time.time() > end_time):
                break
        raise TimeoutException(message)

So basically we try to get the element, and keep trying every half-second until we either get it or (after 10 seconds total) throw an exception because we figure it’s not going to show up when we thought it should. The expanded implementations of until() and until_not() don’t gain us a lot in this particular case, but can be very useful when we’re doing something a little more complicated:

from selenium.webdriver.support.color import Color

def wait_for_background_color(self, selector, color_string):
    color = Color.from_string(color_string)
    correct_color = lambda driver: Color.from_string(driver.find_element_by_css_selector(selector).value_of_css_property("background-color")) == color
    msg = "The color of '%s' should be %s" % (selector, color_string)
    Wait(self.sel).until(correct_color, msg)
    self.screenshot()

In cases like this, I often find the normal WebDriverWait class failing a test because the element I’m inspecting was replaced by the browser (either due to JavaScript activity or browser implementation details) between obtaining it and trying to read its properties. The Wait subclass catches this StaleElementReferenceException and just tries again, up to the timeout limit.

Finally, let’s automate that submit button click a little better also:

def click(self, selector):
    element = self.wait_until_visible(selector)
    element.click()
    return element

def wait_until_visible(self, selector):
    """ Wait until the element matching the selector is visible """
    element_is_visible = lambda driver: self.sel.find_element_by_css_selector(selector).is_displayed()
    msg = "The element matching '%s' should be visible" % selector
    Wait(self.sel).until(element_is_visible, msg)
    self.screenshot()
    return element

So the sample test case given in the Django docs could now be abbreviated as follows:

def test_login(self):
    self.get('/login/')
    self.enter_text('username', 'myuser')
    self.enter_text('password', 'secret')
    self.click('input[value="Log in"]')

In addition to saving typing for new test cases, it’s also now easier to see at a glance what’s going on. We also get screenshots and appropriate wait durations for free. We’ve written a lot more support code than the number of characters saved here, but that’ll all be useful in other test cases as well. You did actually plan to write tests, and not just play around with Selenium because it was interesting, right?

Tags:

8 Responses to “Writing a Selenium Test Framework for a Django Site (Part 3)”

  1. ugetfollowers.com

    Excellent post. I was checking continuously this blog and I am impressed!
    Very useful info particularly the last part :
    ) I care for such info a lot. I was looking for this certain information for
    a very long time. Thank you and good luck.

  2. http://news-max.net

    Nice post. I was checking constantly this blog and I am impressed!

    Extremely helpful information specially the last part :)
    I care for such information a lot. I was looking for this certain information for a long time.
    Thank you and best of luck.

  3. http://doesbraineticswork.wordpress.com/

    My programmer is trying to persuade me to move to .

    net from PHP. I have always disliked the idea because of the costs.

    But he’s tryiong none the less. I’ve been using WordPress on a variety of
    websites for about a year and am nervous about switching to another platform.
    I have heard fantastic things about blogengine.net. Is there a way I can transfer all my wordpress
    posts into it? Any kind of help would be really appreciated!

  4. sumit kher

    I am comparatively new to selenium and played a little with selenium IDE, and trying to do with selenium rc. I am baffled to use the scripting language there. Can you tell me what is the most used language in selenium across industries? And also I came across this course http://www.wiziq.com/course/12451-selenium-automated-web-browser-testing-for-web-applications os selenium automated web browser testing is this good? If someone does in Java and he joins a company where everyone does in ruby, then it’ll be a pain to learn ruby again. And also it would be great if you address any comparison about the available languages like (perl, python, ruby, java etc.) or tell me any other guidance would really appreciate help and also i would like to thank for all the information you are providing.

  5. Jet Table Saw

    Hey There. I found your blog using msn. This is a
    really well written article. I will be sure to bookmark it and come back to read more of your useful information.

    Thanks for the post. I’ll definitely return.

Trackbacks/Pingbacks

  1.  A Smattering of Selenium #130 « Official Selenium Blog
  2.  Optimizing JavaScript in a Django Project: django-require « Safari Books Online: Publishing & Technology