Running Tests

gevent has an extensive regression test suite, implemented using the standard unittest module. It uses a custom testrunner that provides enhanced test isolation (important for monkey-patching), runs tests in parallel, and takes care of other gevent-specific quirks.

Note

The gevent test process runs Python standard library tests with gevent’s monkey-patches applied to ensure that gevent behaves correctly (matches the standard library). The standard library test must be available in order to do this.

This is usually the case automatically, but some distributions remove this module. Notably, on Debian, you will probably need libpythonX.Y-testsuite installed to run all the tests.

The test runner has a number of options:

$ python -mgevent.tests --help
usage: __main__.py [-h] [--ignore IGNORE] [--discover] [--config CONFIG]
                   [--coverage] [--quiet] [--verbose] [--debug]
                   [--package PACKAGE] [--processes PROCESSES] [--no-combine]
                   [-u RES1,RES2,...] [--travis-fold MSG]
                   [--second-chance | --failfast]
                   [tests ...]

positional arguments:
  tests

options:
  -h, --help            show this help message and exit
  --ignore IGNORE
  --discover            Only print the tests found.
  --config CONFIG       The path to the config file containing FAILING_TESTS,
                        IGNORED_TESTS and RUN_ALONE. Defaults to
                        known_failures.py.
  --coverage            Enable coverage recording with coverage.py.
  --quiet               Be quiet. Defaults to True. Also the GEVENTTEST_QUIET
                        environment variable.
  --verbose
  --debug               Enable debug settings. If the GEVENT_DEBUG environment
                        variable is not set, this sets it to 'debug'. This can
                        also enable PYTHONTRACEMALLOC and the debug
                        PYTHONMALLOC allocators, if not already set. Defaults
                        to False.
  --package PACKAGE     Load tests from the given package. Defaults to
                        gevent.tests.
  --processes PROCESSES, -j PROCESSES
                        Use up to the given number of parallel processes to
                        execute tests. Defaults to 4.
  --no-combine          Do not combine tests into process groups.
  -u RES1,RES2,..., --use RES1,RES2,...
                        specify which special resource intensive tests to run.
                        "all" is the default; "none" may also be used. Disable
                        individual resources with a leading -.For example,
                        "-u-network". GEVENTTEST_USE_RESOURCES is used if no
                        argument is given. To only use one resources, specify
                        "-unone,resource".
  --travis-fold MSG     Emit Travis CI log fold markers around the output.
  --second-chance       Give failed tests a second chance.
  --failfast, -x        Stop running after the first failure.

The simplest way to run all the tests is just to invoke the test runner, typically from the root of the source checkout:

(gevent-env) $ python -mgevent.tests
Running tests in parallel with concurrency 7
...
Ran 3107 tests (skipped=333) in 132 files in 01:52

You can also run an individual gevent test file using the test runner:

(gevent-env) $ python -m gevent.tests test__util.py
Running tests in parallel with concurrency 1
+ /.../python -u -mgevent.tests.test__util
- /.../python -u -mgevent.tests.test__util [Ran 9 tests in 1.1s]

Longest-running tests:
1.1 seconds: /.../python -u -mgevent.tests.test__util

Ran 9 tests in 1 files in 1.1s

Or you can run a monkey-patched standard library test:

(gevent-env) $ python -m gevent.tests.test___monkey_patching test_socket.py
Running tests in parallel with concurrency 1
+ /.../python -u -W ignore -m gevent.testing.monkey_test test_socket.py
Running with patch_all(Event=False): test_socket.py
Added imports 1
Skipped testEmptyFileSend (1)
...
Ran 555 tests in 23.042s

OK (skipped=172)
- /.../python -u -W ignore -m gevent.testing.monkey_test test_socket.py [took 26.7s]

Longest-running tests:
26.7 seconds: /.../python -u -W ignore -m gevent.testing.monkey_test test_socket.py

Ran 0 tests in 1 files in 00:27

Environment Variables

Some testrunner options have equivalent environment variables. Notably, --quiet is GEVENTTEST_QUIET and -u is GEVENTTEST_USE_RESOURCES.

Using tox

Before submitting a pull request, it’s a good idea to run the tests across all supported versions of Python, and to check the code quality using prospector/pylint. This is what is done on CI. Locally it can be done using tox:

pip install tox
tox

Measuring Code Coverage

This is done on CI so it’s not often necessary to do locally.

The testrunner accepts a --coverage argument to enable code coverage metrics through the coverage.py package. That would go something like this:

python -m gevent.tests --coverage
coverage combine
coverage html -i
<open htmlcov/index.html>

Limiting Resource Usage

gevent supports the standard library test suite’s resources. All resources are enabled by default. Disabling resources disables the tests that use those resources. For example, to disable tests that access the external network (the Internet), disable the network resource. There’s an option for this:

$ python -m gevent.tests -u-network

And an environment variable:

$ GEVENTTEST_USE_RESOURCES=-network python -m gevent.tests