Weboob » Development »

Automated tests

Summary

Weboob is a wide project which has several backends and applications, and changes can impact a lot of subsystems. To be sure that everything works fine after an important change, it's necessary to have automated tests on each subsystems.

How it works

You need nose installed.

To run the automated tests, use this script:

$ tools/run_tests.sh

It looks for every files named test.py, and find classes derivated from TestCase of BackendTest (in weboob.tools.test).

Then, it run every method which name starts with test_.

Write a test case

Normal test

Use the class weboob.tools.test.TestCase to derivate it into your new test case. Then, write methods which name starts with test_.

A test fails when an assertion error is raised. Also, when an other kind of exception is raised, this is an error.

You can use assert to check something, or the base methods assertTrue, assertIf, failUnless, etc. Read the unittest documentation to know more.

Backend test

Create a class derivated from weboob.tools.test.BackendTest, and set the BACKEND class attribute to the name of the backend to test.

Then, in your test methods, the backend attribute will contain the loaded backend. When the class is instancied, it loads every configured backends of the right type, and randomly choose one.
If no one is found, the tests are skipped.

Example:

 1 from weboob.tools.test import BackendTest
 2 
 3 class YoutubeTest(BackendTest):
 4     BACKEND = 'youtube'
 5 
 6     def test_youtube(self):
 7         l = [v for v in self.backend.iter_search_results('lol')]
 8         self.assertTrue(len(l) > 0)
 9         v = l[0]
10         self.backend.fillobj(v, ('url',))
11         self.assertTrue(v.url and v.url.startswith('http://'), 'URL for video "%s" not found: %s' % (v.id, v.url))

Note: BackendTest inherits TestCase, so the checks work exactly the same, and you can use the same base methods.

Buildbot

The buildbot system starts checks each times a set of commits is pushed on the official repository.

The checks are as follow:
  • Build of sources;
  • Run pyflakes to detect trivial errors;
  • Run automated tests.

When an error occures, buildbot sends an email to every authors of the commit set impacted.

Each developer has his own buildbot slave on his system, to multiplicate the checks on several backends and configurations. If you want to help us by hosting a slave, read this page.

To see the results of checks by buildbot, see the waterfall page.

Also available in: HTML TXT