Test-Driven Development with Python

Chapter 21. The Token Social Bit, the Page Pattern, and an Exercise for the Reader

Are jokes about how “everything has to be social now” slightly old hat? Everything has to be all A/B tested big data get-more-clicks lists of 10 Things This Inspiring Teacher Said That Will Make You Change Your Mind About Blah Blah now … anyway. Lists, be they Inspirational or otherwise, are often better shared. Let’s allow our users to collaborate on their lists with other users.

Along the way we’ll improve our FTs by starting to implement the interact/wait Selenium pattern that we learned in the last chapter. We’ll also experiment with something called the Page Object pattern.

Then, rather than showing you explicitly what to do, I’m going to let you write your unit tests and application code by yourself. Don’t worry, you won’t be totally on your own! I’ll give an outline of the steps to take, as well as some hints and tips.

An FT with Multiple Users, and addCleanup

Let’s get started—we’ll need two users for this FT:

functional_tests/test_sharing.py. 

from selenium import webdriver
from .base import FunctionalTest

def quit_if_possible(browser):
    try: browser.quit()
    except: pass


class SharingTest(FunctionalTest):

    def test_logged_in_users_lists_are_saved_as_my_lists(self):
        # Edith is a logged-in user
        self.create_pre_authenticated_session('[e-mail]')
        edith_browser = self.browser
        self.addCleanup(lambda: quit_if_possible(edith_browser))

        # Her friend Oniciferous is also hanging out on the lists site
        oni_browser = webdriver.Firefox()
        self.addCleanup(lambda: quit_if_possible(oni_browser))
        self.browser = oni_browser
        self.create_pre_authenticated_session('[e-mail]')

        # Edith goes to the home page and starts a list
        self.browser = edith_browser
        self.browser.get(self.server_url)
        self.get_item_input_box().send_keys('Get help\n')

        # She notices a "Share this list" option
        share_box = self.browser.find_element_by_css_selector('input[name=email]')
        self.assertEqual(
            share_box.get_attribute('placeholder'),
            '[e-mail]'
        )

The interesting feature to note about this section is the addCleanup function, whose documentation you can find here. It can be used as an alternative to the tearDown function as a way of cleaning up resources used during the test. It’s most useful when the resource is only allocated halfway through a test, so you don’t have to spend time in tearDown figuring out what does or doesn’t need cleaning up.

addCleanup is run after tearDown, which is why we need that try/except formulation for quit_if_possible—whichever of edith_browser and oni_browser is also assigned to self.browser at the point at which the test ends will already have been quit by the tearDown function.

We’ll also need to move create_pre_authenticated_session from test_my_lists.py into base.py.

OK, let’s see if that all works:

$ python3 manage.py test functional_tests.test_sharing
[...]
Traceback (most recent call last):
  File "/workspace/superlists/functional_tests/test_sharing.py", line 29, in
test_logged_in_users_lists_are_saved_as_my_lists
    share_box = self.browser.find_element_by_css_selector('input[name=email]')
[...]
selenium.common.exceptions.NoSuchElementException: Message: 'Unable to locate
element: {"method":"css selector","selector":"input[name=email]"}' ;

Great! It seems to have got through creating the two user sessions, and it gets onto an expected failure—there is no input for an email address of a person to share a list with on the page.

Let’s do a commit at this point, because we’ve got at least a placeholder for our FT, we’ve got a useful modification of the create_pre_authenticated_session function, and we’re about to embark on a bit of an FT refactor:

$ git add functional_tests
$ git commit -m "New FT for sharing, move session creation stuff to base"

Implementing the Selenium Interact/Wait Pattern

Before we continue, let’s take a closer look at the interactions with the site which we have in our FT so far:

functional_tests/test_sharing.py. 

    # Edith goes to the home page and starts a list
    self.browser = edith_browser
    self.browser.get(self.server_url)
    self.get_item_input_box().send_keys('Get help\n')  #

    # She notices a "Share this list" option
    share_box = self.browser.find_element_by_css_selector('input[name=email]')  #
    self.assertEqual(
        share_box.get_attribute('placeholder'),
        '[e-mail]'
    )

Interaction with site

Assumption about updated state of page

We learned in the last chapter that it’s dangerous to assume too much about the state of the browser after we do an interaction (like send_keys). In theory, implicitly_wait will make sure that, if the call to find_element_by_css_selector doesn’t find our input[name=email] at first, it will silently retry a few times. But it can also go wrong—imagine if there was an input on the previous page, with the same name=email, but a different placeholder text? We’d get a strange failure, because Selenium could theoretically pick up the element from the previous page while the new page is loading. That tends to raise a StaleElementException.

Tip

Unexpected StaleElementException errors from Selenium often mean you have some kind of race condition. You should probably specify an explicit interaction/wait pattern.

Instead, it’s always prudent to follow the “wait-for” pattern whenever we want to check on the effects of an interaction that we’ve just triggered. Something like this:

functional_tests/test_sharing.py. 

    self.get_item_input_box().send_keys('Get help\n')

    # She notices a "Share this list" option
    self.wait_for(
        lambda:  self.assertEqual(
            self.browser.find_element_by_css_selector(
                'input[name=email]'
            ).get_attribute('placeholder'),
            '[e-mail]'
        )
    )

The Page Pattern

But do you know what would be even better? This is an occasion for a “three strikes and refactor”. This test, and many others, all begin off with the user starting a new list. What if we had a helper function called “start new list” that would do the wait_for as well as the list item input?

We’ve already seen how to use helper methods on the base FunctionalTest class, but if we continue using too many of them, it’s going to get very crowded. I’ve worked on a base FT class that was over 1,500 lines long, and that got pretty unwieldy.

One accepted pattern for splitting up your FT helper code is called the Page pattern, and it involves having objects to represent the different pages on your site, and to be the single place to store information about how to interact with them.

Let’s see how we would create Page objects for the home and lists pages. Here’s one for the home page:

functional_tests/home_and_list_pages.py. 

class HomePage(object):

    def __init__(self, test):
        self.test = test  #


    def go_to_home_page(self):  #
        self.test.browser.get(self.test.server_url)
        self.test.wait_for(self.get_item_input)
        return self  #


    def get_item_input(self):
        return self.test.browser.find_element_by_id('id_text')


    def start_new_list(self, item_text):  #
        self.go_to_home_page()
        inputbox = self.get_item_input()
        inputbox.send_keys(item_text + '\n')
        list_page = ListPage(self.test)  #
        list_page.wait_for_new_item_in_list(item_text, 1)  #
        return list_page  #

It’s initialised with an object that represents the current test. That gives us the ability to make assertions, access the browser instance via self.test.browser, and use the wait_for function.

Most Page objects have a “go to this page” function. Notice that it implements the interaction/wait pattern—first we get the page URL, then we wait for an element that we know is on the home page.

Returning self is just a convenience. It enables method chaining.

Here’s our function that starts a new list. It goes to the home page, finds the input box, and sends the new item text to it, as well as a carriage return. Then it uses a wait to check that the interaction has completed, but as you can see that wait is actually on a different Page object:

The ListPage, which we’ll see the code for shortly. It’s initialised just like a HomePage.

We use the ListPage to wait_for_new_item_in_list. We specify the expected text of the item, and its expected position in the list.

Finally, we return the list_page object to the caller, because they will probably find it useful (as we’ll see shortly).

Here’s how ListPage looks:

functional_tests/home_and_list_pages.py (ch21l006). 

[...]

class ListPage(object):

    def __init__(self, test):
        self.test = test

    def get_list_table_rows(self):
        return self.test.browser.find_elements_by_css_selector(
            '#id_list_table tr'
        )

    def wait_for_new_item_in_list(self, item_text, position):
        expected_row = '{}: {}'.format(position, item_text)
        self.test.wait_for(lambda: self.test.assertIn(
            expected_row,
            [row.text for row in self.get_list_table_rows()]
        ))

Note

It’s usually best to have a separate file for each Page object. In this case, HomePage and ListPage are so closely related it’s easier to keep them together.

Let’s see how to use it in our test:

functional_tests/test_sharing.py (ch21l007). 

from .home_and_list_pages import HomePage
[...]

        # Edith goes to the home page and starts a list
        self.browser = edith_browser
        list_page = HomePage(self).start_new_list('Get help')

Let’s continue rewriting our test, using the Page object whenever we want to access elements from the lists page:

functional_tests/test_sharing.py (ch21l008). 

        # She notices a "Share this list" option
        share_box = list_page.get_share_box()
        self.assertEqual(
            share_box.get_attribute('placeholder'),
            '[e-mail]'
        )

        # She shares her list.
        # The page updates to say that it's shared with Oniciferous:
        list_page.share_list_with('[e-mail]')

We add the following three functions to our ListPage:

functional_tests/home_and_list_pages.py (ch21l009). 

    def get_share_box(self):
        return self.test.browser.find_element_by_css_selector(
            'input[name=email]'
        )


    def get_shared_with_list(self):
        return self.test.browser.find_elements_by_css_selector(
            '.list-sharee'
        )


    def share_list_with(self, email):
        self.get_share_box().send_keys(email + '\n')
        self.test.wait_for(lambda: self.test.assertIn(
            email,
            [item.text for item in self.get_shared_with_list()]
        ))

The idea behind the Page pattern is that it should capture all the information about a particular page in your site, so that if, later, you want to go and make changes to that page—even just simple tweaks to its HTML layout for example—you have a single place to go and look for to adjust your functional tests, rather than having to dig through dozens of FTs.

The next step would be to pursue the FT refactor through our other tests. I’m not going to show that here, but it’s something you could do, for practice, to get a feel for what the trade-offs between D.R.Y. and test readability are like…

Extend the FT to a Second User, and the “My Lists” Page

Let’s spec out just a little more detail of what we want our sharing user story to be. Edith has seen on her list page that the list is now “shared with” Oniciferous, and then we can have Oni log in and see the list on his “My Lists” page, maybe in a section called “lists shared with me”:

functional_tests/test_sharing.py (ch21l010). 

    [...]
    list_page.share_list_with('[e-mail]')

    # Oniciferous now goes to the lists page with his browser
    self.browser = oni_browser
    HomePage(self).go_to_home_page().go_to_my_lists_page()

    # He sees Edith's list in there!
    self.browser.find_element_by_link_text('Get help').click()

That means another function in our HomePage class:

functional_tests/home_and_list_pages.py (ch21l011). 

class HomePage(object):

    [...]

    def go_to_my_lists_page(self):
        self.test.browser.find_element_by_link_text('My lists').click()
        self.test.wait_for(lambda: self.test.assertEqual(
            self.test.browser.find_element_by_tag_name('h1').text,
            'My Lists'
        ))

Once again, this is a function that would be good to carry across into test_my_lists.py, along with maybe a MyListsPage object. Exercise for the reader!

In the meantime, Oniciferous can also add things to the list:

functional_tests/test_sharing.py (ch21l012). 

    # On the list page, Oniciferous can see says that it's Edith's list
    self.wait_for(lambda: self.assertEqual(
        list_page.get_list_owner(),
        '[e-mail]'
    ))

    # He adds an item to the list
    list_page.add_new_item('Hi Edith!')

    # When Edith refreshes the page, she sees Oniciferous's addition
    self.browser = edith_browser
    self.browser.refresh()
    list_page.wait_for_new_item_in_list('Hi Edith!', 2)

That’s a couple more additions to our Page object:

functional_tests/home_and_list_pages.py (ch21l013). 

ITEM_INPUT_ID = 'id_text'
[...]

class HomePage(object):
    [...]

    def get_item_input(self):
        return self.test.browser.find_element_by_id(ITEM_INPUT_ID)


class ListPage(object):
    [...]

    def get_item_input(self):
        return self.test.browser.find_element_by_id(ITEM_INPUT_ID)


    def add_new_item(self, item_text):
        current_pos = len(self.get_list_table_rows())
        self.get_item_input().send_keys(item_text + '\n')
        self.wait_for_new_item_in_list(item_text, current_pos + 1)


    def get_list_owner(self):
        return self.test.browser.find_element_by_id('id_list_owner').text

It’s long past time to run the FT and check if all of this works!

$ python3 manage.py test functional_tests.test_sharing

    share_box = list_page.get_share_box()
    [...]
selenium.common.exceptions.NoSuchElementException: Message: 'Unable to locate
element: {"method":"css selector","selector":"input[name=email]"}' ;

That’s the expected failure; we don’t have an input for email addresses of people to share with. Let’s do a commit:

$ git add functional_tests
$ git commit -m "Create Page objects for Home and List pages, use in sharing FT"

An Exercise for the Reader

Here’s an outline of the steps to take to get this new feature implemented:

1. We’ll need a new section in list.html, with, at first, a form with an input box for an email address. That should get the FT one step further.
2. Next, we’ll need a view for the form to submit to. Start by defining the URL in the template, maybe something like lists/<list_id>/share.
3. Then, our first unit test. It can be just enough to get a placeholder view in. We want the view to respond to POST requests, and it should respond with a redirect back to the list page, so the test could be called something like ShareListTest.test_post_redirects_to_lists_page.
4. We build out our placeholder view, as just a two-liner that finds a list and redirects to it.
5. We can then write a new unit test which creates a user and a list, does a POST with their email address, and checks the user is added to list_.shared_with.all() (a similar ORM usage to “My Lists”). That shared_with attribute won’t exist yet, we’re going outside-in.
6. So before we can get this test to pass, we have to move down to the model layer. The next test, in test_models.py, can check that a list has a shared_with.add method, which can be called with a user’s email address and then check the lists’ shared_with.all() queryset, which will subsequently contain that user.
7. You’ll then need a ManyToManyField. You’ll probably see an error message about a clashing related_name, which you’ll find a solution to if you look around the Django docs.
8. It will need a database migration.
9. That should get the model tests passing. Pop back up to fix the view test.
10. You may find the redirect view test fails, because it’s not sending a valid POST request. You can either choose to ignore invalid inputs, or adjust the test to send a valid POST.
11. Then back up to the template level; on the “My Lists” page we’ll want a <ul> with a for loop of the lists shared with the user. On the lists page, we also want to show who the list is shared with, as well as mention of who the list owner is. Look back at the FT for the correct classes and IDs to use. You could have brief unit tests for each of these if you like, as well.
12. You might find that spinning up the site with runserver will help you iron out any bugs, as well as fine-tune the layout and aesthetics. If you use a private browser session, you’ll be able to log multiple users in.

By the end, you might end up with something that looks like Figure 21-1.

Figure 21-1. Sharing lists

The Page Pattern, and the Real Exercise for the Reader

Apply DRY to your functional tests

Once your FT suite starts to grow, you’ll find that different tests will inevitably find themselves using similar parts of the UI. Try to avoid having constants, like the HTML IDs or classes of particular UI elements duplicated between your FTs.

The Page pattern

Moving helper methods into a base FunctionalTest class can become unwieldy. Consider using individual Page objects to hold all the logic for dealing with particular parts of your site.

An exercise for the reader

I hope you’ve actually tried this out! Try to follow the “Outside-In” method, and occasionally try things out manually if you get stuck. The real exercise for the reader, of course, is to apply TDD to your next project. I hope you’ll enjoy it!

In the next chapter, we’ll wrap up with a discussion of testing “best practices”.