Stubble – A collection of tools for writing stubs in unit tests (reahl.stubble)¶

Introduction¶

Stubble is a set of tools which can be used to simplify Python unit testing code by means of the responsible use of so-called “stub” classes. Stubble includes several special classes that can be used to stub out (and restore) sys.stdout or to mock methods on a class temporarily.

Stubble allows you to write arbitrary classes for use as stubs instead of real classes while testing. This can be dangerous in a test, since your ad-hoc stub classes may implement interfaces that are not representative of the real classes they stand in for.

However, Stubble lets you link a stub class loosely to the real class which it is a stub for. This information is then used to ensure that tests will break if there is a discrepancy between the interface supported by your stub class and that of the real class it stands in for.

Stubble also includes some support for stubbing a Python egg and its associated meta information when you are using setuptools.

(More information is provided in the Overview below.)

Stub classes¶

What stub classes are¶

A stub class is a class you use in a test as a substitute for another class. Here are a few reasons for wanting to use such stub classes:

You may need to provide an instance of, say, MyBulkyRealClass in a test fixture. But constructing such an instance may be difficult, cumbersome, or outright impossible. For example: your test code may only need to call a single method on the provided MyBulkyRealClass instance - but merely in order to construct the needed instance, you have to construct a forest of other instances. This is unnecessary. Another example is where you need to construct an instance of a GUI library object, such as a Window…but, the GUI toolkit you are using may not allow instantiation of such objects outside of its control.

In both these examples, it would have been much easier to just create a simple class containing the single method which would be called by your testing code - possibly even with an implementation hard-coded for this particular scenario.
You may want to decouple the unit tests of different modules: you don’t really want to have the unit tests of one module fail because of errors in another module. By using stub classes for real classes in other modules, the unit tests of the module being tested are not dependent on the actual other module, just on its expected interface.
You may want to monitor certain things in a test, such as the values of parameters that were passed to a particular method call. Or, you may want to hard-code a special implementation for a particular method specifically for your test scenario.

The careful use of stub classes can greatly simplify and speed up the writing of unit test code.

Prerequisites for examples¶

This section shows by example how the stub classes in Stubble can be used. The examples all assume the following real class:

class RealClass(object):
    b = 123

    def foo(self, a):
        print('i am the real foo')

    def bar(self):
        print('i am the real bar')

Basic functionality¶

Any class can be a Stubble-enabled stub. To create a stub class, you just prepend the class with a descriptor like this:

from reahl.stubble import stubclass

@stubclass(RealClass)
class Stub(object):
    def foo(self, a):
        'i am a fake foo'

The class Stub is now declared as being a stub of RealClass. When Python encounters the declaration, it will check each method declared on this class and make sure that:

There is a similarly named method on the real class; and

the corresponding method on the real class has the exact same signature as declared by the stub.

If any of these conditions are not met, an exception is raised.

The simplest solution is usually to derive stub classes from object. But, sometimes it is useful to derive from the real class which is being stubbed.

The main reason for not deriving from the real class is usually that parameters (or other side-effects) needed for the __init__ of the real class are cumbersome to provide (or unwanted).

But, sometimes, you want to inherit from the real class, to inherit some its real behaviour, but also override some of it:

from reahl.stubble import stubclass

@stubclass(RealClass)
class Stub(RealClass):
    def foo(self, a):
        self.foo_called = True

Exempt methods¶

Not all methods on a stub class need to be strictly checked against those in the real class - utility methods you use in the test, for instance. To mark a method as being exempt from the checking, you can use a decorator on that method:

from reahl.stubble import stubclass, exempt

@stubclass(RealClass)
class Stub(object):
    @exempt
    def my_own_method(self):
        print('i am my own method')

Attributes¶

At first it was thought important to let Stubble do strict checking on any class attribute similarly to what it does for methods. In practice we found that it is a bit bothersome to do that. But if you really want to, you can do the following:

from reahl.stubble import stubclass

@stubclass(RealClass, check_attributes_also=True)
class Stub(object):
    b = 'a value'

This would additionally check that RealClass has an attribute “b” and fail if it does not have one.

EasterEggs¶

Setuptools provide (amongst others) support for Python eggs. Python eggs are somewhat similar to OSGI bundles: they are components which consist (mainly) of Python code, but also have metadata associated with them. Amongst other things, this allows components to publish their interfaces, and it allows components to specify how they can be extended by 3rd parties. Setuptools also includes a method (the ResourceManager API) by which packages can request the contents of “files” regardless of how these files have been packaged or where they are physically located.

To help here, Stubble provides the EasterEgg. EasterEggs are not real Python eggs. A correctly initialised global EasterEgg instance is constructed as reahl.stubble.easter_egg. It should suffice for most purposes, but you can construct additional instances if needed.

Just be sure to always add your EasterEggs to pkg_resources.working_set (or similar), else they won’t have any effect. Also, each added EasterEgg should be named uniquely (or it won’t be added).

Stub entry points¶

If you work with setuptools you may be testing code to which you want to supply stub objects via the setuptools entry point mechanism.

EasterEgg has two methods for adding stub classes as entry points, exemplified here:

reahl.stubble.easter_egg.add_entry_point_from_line(group_name,
                      'test1 = examples.setuptools:StubClass1')
reahl.stubble.easter_egg.add_entry_point(group_name, 'test2', StubClass2)

Actual code under test would now probably do something like this (and be oblivious to the fact that the provided classes are stubs):

#  (we just print out each class it finds...)
for i in pkg_resources.iter_entry_points(group_name):
    print(i.load())

Where you tear down test fixtures after a test run, you should clear the fake entry points registered with the easter_egg:

reahl.stubble.easter_egg.clear()

Stubbed resources¶

The EasterEgg can also be used for testing code that makes use of the ResourceManager API in setuptools. You can put the real files that the ResourceManager API should provide in a directory somewhere - and then specify that location as the EasterEgg’s module_path:

reahl.stubble.easter_egg.location = '/some/where'

(By default, this path is os.getcwd())

Intercepting calls¶

Stubble includes a number of classes that can be used as context managers to temporarily intercept calls to code that may not even be under your control. This is done by swapping certain methods out temporarily for special ones that are restored after a particular block of code finished executing.

SystemOutStub¶

class reahl.stubble.intercept.SystemOutStub¶

The SystemOutStub can be used as context manager to test output that some code sends to sys.stdout.

For example, the following code does not output anything to the console:

with SystemOutStub() as monitor:
    print('hello')

assert monitor.captured_output == 'hello\n'

captured_output¶: The output captured during the time the SystemOutStub was active.

CallMonitor¶

class reahl.stubble.intercept.CallMonitor(method)¶

The CallMonitor is a context manager which records calls to a single method of an object or class. The calls are recorded, but the original method is also executed.

For example:

class SomeClass:
    def foo(self, arg):
        return 'something'

s = SomeClass()

with CallMonitor(s.foo) as monitor:
    assert s.foo('a value') == 'something'

assert monitor.times_called == 1
assert monitor.calls[0].args == ('a value',)
assert monitor.calls[0].kwargs == {}
assert monitor.calls[0].return_value == 'something'

property times_called¶: The number of calls that were made during the time the CallMonitor was active.

MonitoredCall¶

class reahl.stubble.intercept.MonitoredCall(args, kwargs, return_value)¶

The record of one call that was made to a method. This class is not intended to be instantiated by a programmer. Programmers can query instances of MonitoredCall returned by a reahl.stubble.intercept.CallMonitor or reahl.stubble.intercept.InitMonitor.

return_value¶: The value returned by the call

args¶: The tuple with positional arguments passed during the call

kwargs¶: The dictionary with keyword arguments passed during the call

InitMonitor¶

class reahl.stubble.intercept.InitMonitor(monitored_class)¶

An InitMonitor is like a reahl.stubble.intercept.CallMonitor, except it is used to intercept calls to the __init__ of a class. (See reahl.stubble.intercept.CallMonitor for attributes.)

property times_called¶: The number of calls that were made during the time the CallMonitor was active.

replaced¶

reahl.stubble.intercept.replaced(method, replacement, on=None)¶

A context manager which replaces the method passed in as method with the callable in replacement for the duration of the managed context.

class SomethingElse:
    def foo(self, n, y='yyy'):
        assert None, 'This should never be reached in this test'

s = SomethingElse()

def replacement(n, y=None):
    return y

with replaced(s.foo, replacement):
    assert s.foo(1, y='a') == 'a'
    assert s.foo(2) == None

assert s.foo(2) == 'yyy'

Experimental features¶

The simple functionality of Stubble explained so far is really what it is all about. But, having started off as an experiment, Stubble provides several interesting experimental features. However interesting some of these may sound, they are not really used at all in practice… sometimes because they’re just theoretically nice ideas with little use in practice; sometimes because they’re nice ideas that proved too difficult to implement transparently.

They’re provided as part of Stubble for interest’s sake. Maybe a skilled Python programmer out there feels like the challenge…

Impostors¶

Passing a stub class instead of the real class to code that is being tested works well in most cases. The notable exception is when the code actually checks the type of the class itself, such as with isinstance or issubclass, etc.

Ideally speaking, you’d want a stub that pretends in all respects to be the real thing to the code being tested. Impostors are an attempt at such stub classes.

To make your stub class an Impostor, you have to let it derive from reahl.stubble.Impostor:

from reahl.stubble import Impostor, stubclass

@stubclass(RealClass)
class Stub(Impostor):
    pass

With such a declaration, you gain the dubious benefit expressed in the code:

assert not issubclass(Stub, RealClass)  #issubclass catches Impostors out
s = Stub()
assert isinstance(s, RealClass)         #but the foolery works well here

The value derived from this is debatable… most often you get more benefit by deriving your stub class from the real class it is a stub for.

Delegation¶

Delegation is actually a more useful idea than Impostors. It arose from the problem sometimes encounters where the programmer does not have control over the creation of the instance that has to be stubbed.

For example, a GUI framework may create a bunch of instances for you, and you just want to replace one node in this object forest with a special kind of stub instance. Also, you actually would only want to override a single method, but have the rest of the behaviour delegated to the real instance “as usual”.

Delegation (in this context) is the strategy of creating a stub class whose instance is ‘superimposed’ upon an instance of a real class.

A delegate is declared like this:

from reahl.stubble import stubclass, Delegate

@stubclass(RealClass)
class Stub(Delegate):

    shadowed = ['foo', 'aa']

    def foo(self, a):
        print('i am a fake foo')

And instance of it is then created like this:

real_instance = RealClass()  # first we need an instance to delegate to
s = Stub(real_instance)      # the stub instance

Our stub instance how has the following interesting behaviour:

assert isinstance(s, RealClass)  # Delegates act like Impostors do
s.foo(1)                         # calls the fake
s.bar()                          # calls the real

try:
    s.aa                         # breaks regardless of
                                 # whether or not aa is on real_instance
except:
    pass

s.aa = 123                              # `aa` is set on the fake
assert s.aa == 123                      # `aa` is read from the fake
assert not hasattr(real_instance, 'aa')  # see, it was not set there

The unsolved problem with Delegation¶

At present, although such Delegates would have been nice to have, their implementation (which is a bit tricky) has a serious flaw which will make Delegates behave contrary to what you’d expect in certain circumstances.

The problem is when one shadows attributes or methods that are accessed by the real class itself from within code delegated to. An example illustrates best:

from reahl.stubble import stubclass, Delegate

class AnotherRealClass(object):
    def foo(self):
        self.aa = 123

real_instance = AnotherRealClass()

@stubclass(AnotherRealClass)
class Stub(Delegate):
  shadowed = ['aa']

s = Stub(real_instance)

The behaviour we would expect here is that, upon calling s.foo, the real foo is called, which sets ‘aa’ on the stub, even though the setting of ‘aa’ happens in real code:

s.foo()

try:
    #this expected behaviour would have been tested
    # like this:
    assert s.aa == 123
    assert not hasattr(real_instance, 'aa')
except:
    pass

But alas, we cannot do that… The variable is in fact set on the real class.

The same problem manifests itself if you call a method which is delegated to the real class, and the real class in turn tries to call a method which is being shadowed on the stub.

This problem severely limits the use of Delegates and can cause bugs in tests that are very difficult to find – hence Delegates are not used in practice.

Up for a challenge?¶

Anyone interested in giving it a bash: the one solution is to change reahl.stubble.Delegate.__init__, so it changes the __class__ of ‘real’ to a substitute with a __getattribute__ which can do the necessary voodoo.

That’s the solution in theory only. In practice, object layout differences prevent this particular solution… But, you may know better.

Dealing with Instance variables¶

In Python, it is very difficult to check anything when it comes to instance variables… simply because the class does not have any information about which instance variables it will have.

Some people use __slots__ as a way to specify instance variables for this reason. However, that is not what __slots__ is intended for. __slots__ is an optimisation feature which, if used for other reasons will behave contrary to your expectations.

Such as in this example originally posted on comp.lang.python by Blair Hall on Apr 10 2003 (I modified it a bit, though…):

class A(object):
    def __init__(self):
        pass

class B(A):
    __slots__ = ('z',)
    def __init__(self):
        super(B, self).__init__()

# now, if you used __slots__ thinking that, since 'c' is not
# in B.__slots__ and that the code above would complain,
# you're in for a surprise...
b = B()
b.c = 3     # passes

Using __slots__ also interferes with a number of other pythonic flexibilities, so its use is not really recommended unless really necessary.

If you are, however, interested in checking instance variables too, other conventions are possible. For example:

Always initialise all your instance variables in __init__ (even if they are == None), so you can expect your class invariant to include ‘instances of this class has all those attributes’)

Always put class variables in the class for each variable instances of it would have (the class variable values also serve as handy default values for unset instance variables).

Using the latter approach, stubble could be used to check attributes by your either specifying a default value for the attribute in the stub class, or by using reahl.stubble.checkedinstance:

from reahl.stubble import stubclass, checkedinstance

class RealClassFollowingConventions(object):
    a = None
    b = None

@stubclass(RealClassFollowingConventions, check_attributes_also=True)
class Stub(object):
    a = 'asd'
    b = checkedinstance()

(remove either a or b in the real class to see it complain)
(PS: the difference between a and b below is that for b we do not
give a default value, we just state that it should be in both)

For the stubborn, who insist on using __slots__, and who even insist on using it as a checked list of allowed attributes, stubble deals in the following drug (remove ‘aa’ from __slots__ to see it complain):

from reahl.stubble import stubclass, slotconstrained

class StubbornRealClass(object):
    __slots__ = ('aa')

@stubclass(StubbornRealClass)
class Stub(object):
    aa = slotconstrained()

s = Stub()
try:
    s.aa                  #even though declared as class attributes,
                          # these behave like instance attributes,
                          # so its not there if not set
except AttributeError, e:
    pass

s.aa = 123                #as usual
assert s.aa == 123        #as usual

reahl

Stubble – A collection of tools for writing stubs in unit tests (reahl.stubble)¶

Introduction¶

Stub classes¶

What stub classes are¶

Prerequisites for examples¶

Basic functionality¶

Exempt methods¶

Attributes¶

EasterEggs¶

Stub entry points¶

Stubbed resources¶

Intercepting calls¶

SystemOutStub¶

CallMonitor¶

MonitoredCall¶

InitMonitor¶

replaced¶

Experimental features¶

Impostors¶

Delegation¶

The unsolved problem with Delegation¶

Up for a challenge?¶

Dealing with Instance variables¶

Table of Contents

Previous topic

Next topic

Navigation

Stubble – A collection of tools for writing stubs in unit tests (reahl.stubble)¶

Introduction¶

Stub classes¶

What stub classes are¶

Prerequisites for examples¶

Basic functionality¶

Exempt methods¶

Attributes¶

EasterEggs¶

Stub entry points¶

Stubbed resources¶

Intercepting calls¶

SystemOutStub¶

CallMonitor¶

MonitoredCall¶

InitMonitor¶

replaced¶

Experimental features¶

Impostors¶

Delegation¶

The unsolved problem with Delegation¶

Up for a challenge?¶

Dealing with Instance variables¶

Table of Contents

Previous topic

Next topic

Quick search

Navigation