Testing¶
Tests are central to our development process and hence some understanding of how testing is done will also help you to understand Reahl – and the rest of this tutorial.
Here’s how we like to think of tests: if you had to explain a system to a newcomer from scratch, you’d explain a number of facts – some facts also build upon on other facts that have already been explained. We try to identify these explanatory facts, and write a test per fact. The code of the test itself provides an example and/or more detail about that fact. Tests are thus a specification of the requirements of the system, and can be used to explain the system to someone.
In order to facilitate these ideas, some testing tools grew with Reahl. This section gives a brief introduction.
Fixtures¶
In order to test something, you usually need to create a number of related objects first for use in the test itself. Creating these objects can be tedious – especially if they depend on each other.
Such a collection of objects used together by a test is called a test Fixture. Reahl allows one to write a Fixture as a separate class. Defining a Fixture separately from the test that is using it allows for the Fixture to be re-used by different tests, and even by other Fixture classes. Separating the Fixture also helps keep the test itself to read more like a requirements specification and less like code.
Here’s how Fixtures work: For each object that will form part of the Fixture, a programmer writes a method on the Fixture class which will create the object when called. The name of the method is “new_” prefixed to the name of the object. (Assuming an object named my_object, you would add a method to the fixture named new_my_object.) In order to use a Fixture with a specific test, decorate the test using the @test() decorator, passing the Fixture class to @test(). The test method or function should also expect a single argument in its signature: the Fixture instance.
The first time a program references .my_object on the Fixture instance, the corresponding new_ method will be called behind the scenes to create the object. Subsequent accesses of .my_object will always bring back the same instance which was created on the first access.
Here is a simple test which illustrates how Fixtures work:
from __future__ import unicode_literals
from __future__ import print_function
from reahl.tofu import test, Fixture
class SimpleFixture(Fixture):
def new_name(self):
return 'John'
@test(SimpleFixture)
def fixture_singletons(fixture):
"""Accessing an attribute on the Fixture always brings back the same
object, as created by a similarly named new_ method on the fixture."""
assert fixture.name == 'John'
assert fixture.name is fixture.name # ie, the same object is always returned
Fixtures are all about objects that depend on one another. The following example shows a Fixture with two objects: a .name, and a .user. The User object is initialised using the .name which accompanies it on the Fixture. For simplicity, the actual code of User is just included in the test. Note also how SimpleFixture is reused here, by deriving from it:
class User(object):
def __init__(self, name):
self.name = name
class InterestingFixture(SimpleFixture):
def new_user(self):
return User(self.name)
@test(InterestingFixture)
def dependent_setup_objects(fixture):
"""Different attributes on a Fixture can reference one another."""
assert fixture.user.name is fixture.name
Things can get more interesting though. A useful convention is to create new_* methods with keyword arguments. This way one can use the new_* method to create slightly modified versions of the default object available on the Fixture:
class MoreInterestingFixture(SimpleFixture):
def new_user(self, name=None):
return User(name or self.name)
@test(MoreInterestingFixture)
def bypassing_the_singleton(fixture):
"""new_ methods can be supplied with kwargs in order to create test objects that differ from the default."""
jane = fixture.new_user(name='Jane')
assert jane.name == 'Jane'
assert fixture.user is not jane
other_jane = fixture.new_user(name='Jane')
assert jane is not other_jane
Set_up, tear_down, and run fixtures¶
Since Fixtures mostly consist of a bunch of new_ methods, they usually do not need traditional methods for settting up a fixture, or tearing the fixture down afterwards. In some rare cases this is still needed though. One may want to start a web server before a test, and stop it afterwards, for example. Any method on a Fixture can be run before or after the test it is used with. Just annotate the method with @set_up or @tear_down respectively.
Sometimes such setup can take a long time and would slow down tests if it happens for each and every test. When testing web applications, for example, you may want to fire up a browser before the test – something that takes quite a long time. For this reason Reahl provides an extension to nosetests to which you can specify a “run Fixture”. A run Fixture is used for all tests that are run together. It is set up before all the tests are run, and torn down at the end of all test runs. Normal Fixtures that are attached to individual tests also have access to the current run Fixture.
To specify which run Fixture should be used for a test run, use the --with-run-fixture (or -F) argument to nosetests.
Testing without a real browser¶
Enough talk. Its time for a first test.
In the previous section an application was developed which lets one add, view and edit Addresses on different Views. The second version of it used Buttons to navigate to the edit Views of individual Addresses. This application is starting to get cumbersome to test by hand each time a change is made to it. It needs a test. A Test for it will also make it possible to set breakpoints and investigate its internals while running.
Here is a first stab at a test for it. Sometimes it is useful to write a test per explained fact; other times it is useful to write a test for a little scenario illustrating a “user story”. This test is an example of the latter:
from __future__ import unicode_literals
from __future__ import print_function
from reahl.tofu import test
from reahl.web_dev.fixtures import WebFixture
from reahl.webdev.tools import Browser, XPath
from reahl.doc.examples.tutorial.parameterised2.parameterised2 import AddressBookUI
@test(WebFixture)
def adding_an_address(fixture):
"""To add a new address, a user clicks on "Add Address" link on the menu, then supplies the
information for the new address and clicks the Save button. Upon successful addition of the
address, the user is returned to the home page where the new address is now listed."""
browser = Browser(fixture.new_wsgi_app(site_root=AddressBookUI))
browser.open('/')
browser.click(XPath.link_with_text('Add an address'))
actual_title = browser.title
assert actual_title == 'Add an address', 'Expected to be on the Add an address page'
browser.type(XPath.input_labelled('Name'), 'John Doe')
browser.type(XPath.input_labelled('Email'), '[email protected]')
browser.click(XPath.button_labelled('Save'))
actual_title = browser.title
assert actual_title == 'Addresses', 'Expected to be back on the home page after editing'
assert browser.is_element_present(XPath.paragraph_containing('John Doe: [email protected]')), \
'Expected the newly added address to be listed on the home page'
The test should be run with nosetests, using --with-run-fixture=reahl.webdev.fixtures:BrowserSetup.
There are a number of Fixtures available as part of Reahl to help you test applications. Explaining all of them is outside of the scope of this introductory section. For this test, it is useful to know some of the things that BrowserSetup does: Before the tests start to run, it creates an empty database with the complete schema of your application. The Elixir/Sqlalchemy classes used by your application are also initialised properly (no need to create_all/setup_all, etc).
The test is run with a WebFixture. WebFixture depends on the presence of a BrowserSetup run Fixture – that’s why BrowserSetup should be specified to nosetests when run.
The only obvious feature of WebFixture used in this little test is the creation of a WSGI application. WebFixture.new_wsgi_app() can create a WSGI application in various different ways. This example shows how to create a ReahlWSGIApplication from a supplied UserInterface. By default javascript is not turned on (although, as you will see later, this can be specified as well).
Behind the scenes, WebFixture also has the job of starting a new database transaction before each test, and rolling it back after each test. Hence no real need to tear down anything...
Two other important tools introduced in this test are: Browser and XPath. Browser is not a real browser – it is our thin wrapper on top of what is available from WebTest. Browser has the interface of a Browser though. It has methods like .open() (to open an url), .click() (to click on anything), and .type() (to type something into an element of the web page).
We like this interface, because it makes the tests more readable to a non-programmer.
When the browser is instructed to click on something or type into something, an XPath expression is used to specify how to find that element on the web page. XPath has handy methods for constructing XPath expressions while keeping the code readable. Compare, for example the following two lines to appreciate what XPath does. Isn’t the one using XPath much more explicit and readable?
browser.click('//a[href="/news"]')
browser.click(XPath.link_labelled('News'))
Readable tests¶
Often in tests, there are small bits of code which would be more readable if it was extracted into properly named methods (as done with XPath above). If you create a Fixture specially for testing the AddressBookUI, such a fixture is an ideal home for such methods. In the expanded version of our test below, AddressAppFixture was added. AddressAppFixture now contains the nasty implementation of a couple of things we’d like to assert about the application, as well as some objects used by the tests. (Compare this implementation of .adding_an_address() to the previous implementation.)
from __future__ import unicode_literals
from __future__ import print_function
from reahl.tofu import test
from reahl.web_dev.fixtures import WebFixture
from reahl.webdev.tools import Browser, XPath
from reahl.doc.examples.tutorial.parameterised2.parameterised2 import AddressBookUI, Address
class AddressAppFixture(WebFixture):
def new_browser(self):
return Browser(self.new_wsgi_app(site_root=AddressBookUI))
def new_existing_address(self):
address = Address(name='John Doe', email_address='[email protected]')
address.save()
return address
def is_on_home_page(self):
return self.browser.title == 'Addresses'
def is_on_add_page(self):
return self.browser.title == 'Add an address'
def is_on_edit_page_for(self, address):
return self.browser.title == 'Edit %s' % address.name
def address_is_listed_as(self, name, email_address):
return self.browser.is_element_present(XPath.paragraph_containing('%s: %s' % (name, email_address)))
@test(AddressAppFixture)
def adding_an_address(fixture):
"""To add a new address, a user clicks on "Add Address" link on the menu, then supplies the
information for the new address and clicks the Save button. Upon success addition of the
address, the user is returned to the home page where the new address is now listed."""
browser = fixture.browser
browser.open('/')
browser.click(XPath.link_with_text('Add an address'))
assert fixture.is_on_add_page()
browser.type(XPath.input_labelled('Name'), 'John Doe')
browser.type(XPath.input_labelled('Email'), '[email protected]')
browser.click(XPath.button_labelled('Save'))
assert fixture.is_on_home_page()
assert fixture.address_is_listed_as('John Doe', '[email protected]')
@test(AddressAppFixture)
def editing_an_address(fixture):
"""To edit an existing address, a user clicks on the "Edit" button next to the chosen Address
on the "Addresses" page. The user is then taken to an "Edit" View for the chosen Address and
can change the name or email address. Upon clicking the "Update" Button, the user is sent back
to the "Addresses" page where the changes are visible."""
browser = fixture.browser
existing_address = fixture.existing_address
browser.open('/')
browser.click(XPath.button_labelled('Edit'))
assert fixture.is_on_edit_page_for(existing_address)
browser.type(XPath.input_labelled('Name'), 'John Doe-changed')
browser.type(XPath.input_labelled('Email'), '[email protected]')
browser.click(XPath.button_labelled('Update'))
assert fixture.is_on_home_page()
assert fixture.address_is_listed_as('John Doe-changed', '[email protected]')
Testing JavaScript¶
The Browser class used above cannot be used for all tests, since it cannot execute javascript. If you want to test something which makes use of javascript, you’d need the tests (or part of them) to execute in something like an actual browser. Doing this requires Selenium, and the use of the web server started by BrowserSetup for exactly this eventuality.
DriverBrowser is a class which provides a thin layer over Selenium’s WebDriver interface. It gives a programmer a similar set of methods to those provided by the Browser, as used above. An instance of it is available on the WebFixture via its .driver_browser attribute.
The standard Fixtures that form part of Reahl use Chromium by default in order to run tests, and they expect the executable for chromium to be in ‘/usr/lib/chromium-browser/chromium-browser’.
You can change which browser is used by creating a new run Fixture that inherits from BrowserSetup, and overriding its .web_driver() method. Some details regarding how the browser is configured (such as the binary location of the browser) can similarly be changed by overriding the relevant .new_ method of that class.
When the following is executed, a browser will be fired up, and Selenium used to test that the validation provided by EmailField works in javascript as expected. Note also how javascript is enabled for the web application upon creation:
from __future__ import unicode_literals
from __future__ import print_function
from reahl.tofu import test
from reahl.web_dev.fixtures import WebFixture
from reahl.webdev.tools import XPath
from reahl.doc.examples.tutorial.parameterised2.parameterised2 import AddressBookUI, Address
class AddressAppErrorFixture(WebFixture):
def new_wsgi_app(self):
return super(AddressAppErrorFixture, self).new_wsgi_app(site_root=AddressBookUI, enable_js=True)
def new_existing_address(self):
address = Address(name='John Doe', email_address='[email protected]')
address.save()
return address
def error_is_displayed(self, text):
return self.driver_browser.is_element_present(XPath.error_label_containing(text))
@test(AddressAppErrorFixture)
def edit_errors(fixture):
"""Email addresses on the Edit an address page have to be valid email addresses."""
fixture.reahl_server.set_app(fixture.wsgi_app)
browser = fixture.driver_browser
existing_address = fixture.existing_address # Doing this just creates the Address in the database
browser.open('/')
browser.click(XPath.button_labelled('Edit'))
browser.type(XPath.input_labelled('Email'), 'invalid email address')
assert fixture.error_is_displayed('Email should be a valid email address')