Runners

behave provides an extension point for builtin and user-defined test runners.

The following runners are currently supported:

Name

Runner Class

Description

default

behave.runner:Runner

The default test runner provided by behave.

help

—

Shows which runners are currently available in your context.

You specify a runner by using the -r <RUNNER> or --runner=<RUNNER> command-line option. A <RUNNER> option value can be:

  • a runner alias, defined in [behave.runners] section in the behave.ini config-file

  • a scoped class name, like: <SCOPED_MODULE_NAME>:<RUNNER_CLASS_NAME>

Attention

If you provide an own test runner, you are in the inner parts of behave:

  • You should know what you are doing.

  • Inner parts of behave may change without notice.

  • While it is not intended to break your test runner implementation, changes in the inner parts of behave may occur, that will break parts of your implementation.

User-Defined Runners

behave allows you to provide your own test runner (class):

SHELL
# USING COMMAND-LINE OPTION: -r/--runner=<RUNNER>
$ behave --runner=behave.runner:Runner ...

The usage of a user-defined runner can be simplified by providing an alias name for it in the configuration file:

FILE: behave.ini
# ALIAS SUPPORTS: behave -r default ...
[behave.runners]
default = behave.runner:Runner

Use behave --runner=help to:

  • Inspect which runners are currently defined/supported in your workspace

  • Check if the runner definitions have a problem, like: ModuleNotFoundError

SHELL
$ behave --runner=help
AVAILABLE RUNNERS:
  default  = behave.runner:Runner

DESIGN CONSTRAINTS:

  • A runner class must implement the behave.api.runner.ITestRunner interface

Tip

See also features/runner.use_runner_class.feature for more information.

Parallel Test Runners

The default test runner of behave runs the features sequentially. Parallel test runners are separately installable community runners that use the runner extension point. You select such a runner like any other user-defined runner and use the --jobs option (or the jobs : positive_number config-file parameter) to specify the number of concurrent jobs:

SHELL
$ behave --runner=<SCOPED_MODULE_NAME>:<RUNNER_CLASS_NAME> --jobs=4 ...

The default test runner ignores the --jobs option.

Hint

For implementors of a test runner with worker processes

A Configuration object cannot be sent to another process. But it remembers how it was built, so that a worker process can build an equivalent one:

config2 = Configuration(config.command_args,
                        load_config=config.command_load_config,
                        **config.command_kwargs)

Failure Syndromes with User-Defined Runners

Exception

Failure Kinde

Description

ModuleNotFoundError

User Error

Python package with runner is probably not installed yet.

ClassNotFoundError

User or Devel Error

Python package is installed but class is not found (maybe: misspelled).

InvalidClassError

Developer Error

Runner class is not valid (for one of several reaons).

There are a number of reasons why the InvalidClassError exception occurs, like:

  • The ITestRunner interface is not implemented.

  • The ITestRunner interface contract is broken.

  • The ITestRunner interface is only partially.

Tip

See also features/runner.use_runner_class.feature for more information and the different failure syndromes that may occur.