cfme.utils.testgen module

Test generation helpers

Intended to functionalize common tasks when working with the pytest_generate_tests hook.

When running a test, it is quite often the case that multiple parameters need to be passed to a single test. An example of this would be the need to run a Provider Add test against multiple providers. We will assume that the providers are stored in the yaml under a common structure like so:

providers:
    prov_1:
        name: test
        ip: 10.0.0.1
        test_vm: abc1
    prov_2:
        name: test2
        ip: 10.0.0.2
        test_vm: abc2

Our test requires that we have a Provider Object and as an example, the ‘test_vm’ field of the object. Let’s assume a test prototype like so:

test_provider_add(provider_obj, test_vm):

In this case we require the test to be run twice, once for prov_1 and then again for prov_2. We are going to use the generate function to help us provide parameters to pass to pytest_generate_tests(). pytest_generate_tests() requires three pieces of information, argnames, argvalues and an idlist. argnames turns into the names we use for fixtures. In this case, provider_obj and provider_mgmt_sys. argvalues becomes the place where the provider_obj and provider_mgmt_sys items are stored. Each element of argvalues is a list containing a value for both provider_obj and provider_mgmt_sys. Thus, taking an element from argvalues gives us the values to unpack to make up one test. An example is below, where we assume that a provider object is obtained via the Provider class, and the mgmt_sys object is obtained via a Wrapanapi class.

~ provider_obj test_vm
prov1 Provider(prov1) abc1
prov2 Provider(prov2) abc2

This is analogous to the following layout:

~ argnames[0] argnames[1]
idlist[0] argvalues[0][0] argvalues[0][1]
idlist[1] argvalues[1][0] argvalues[1][1]

This could be generated like so:

def gen_providers:

    argnames = ['provider_obj', 'test_vm']
    argvalues = []
    idlist = []

    for provider in yaml['providers']:
        idlist.append(provider)
        argvalues.append([
            Provider(yaml['providers'][provider]['name']),
            yaml['providers'][provider]['test_vm'])
        ])

    return argnames, argvalues, idlist

This is then used with pytest_generate_tests like so:

pytest_generate_tests(gen_providers)

Additionally, py.test joins the values of idlist with dashes to generate a unique id for this test, falling back to joining argnames with dashes if idlist is not set. This is the value seen in square brackets in a test report on parametrized tests.

More information on parametrize can be found in pytest’s documentation:

cfme.utils.testgen.all_providers(metafunc, **options)[source]

Returns providers of all types

cfme.utils.testgen.auth_groups(metafunc, auth_mode)[source]

Provides two test params based on the ‘auth_modes’ and ‘group_roles’ in cfme_data:

group_name:
expected group name in provided by the backend specified in auth_mode
group_data:
list of nav destinations that should be visible as a member of group_name
Parameters:auth_mode – One of the auth_modes specified in cfme_data.get('auth_modes', {})
cfme.utils.testgen.config_managers(metafunc)[source]

Provides config managers

cfme.utils.testgen.generate(*args, **kwargs)[source]

Functional handler for inline pytest_generate_tests definition

Parameters:
  • gen_func – Test generator function, expected to return argnames, argvalues, and an idlist suitable for use with pytest’s parametrize method in pytest_generate_tests hooks
  • indirect – Optional keyword argument. If seen, it will be removed from the kwargs passed to gen_func and used in the wrapped pytest parametrize call
  • scope – Optional keyword argument. If seen, it will be removed from the kwargs passed to gen_func and used in the wrapped pytest parametrize call
  • filter_unused – Optional keyword argument. If True (the default), parametrized tests will be inspected, and only argnames matching fixturenames will be used to parametrize the test. If seen, it will be removed from the kwargs passed to gen_func.
  • *args – Additional positional arguments which will be passed to gen_func
  • **kwargs – Additional keyword arguments whill be passed to gen_func

Usage:

# Abstract example:
pytest_generate_tests = testgen.generate(arg1, arg2, kwarg1='a')

# Concrete example using all infrastructure providers and module scope
pytest_generate_tests = testgen.generate([InfraProvider], scope="module")

# Another concrete example using only VMware and SCVMM providers with 'retire' flag
pf = ProviderFilter(
    classes=[WMwareProvider, SCVMMProvider]), required_flags=['retire'])
pytest_generate_tests = testgen.generate(
    gen_func=testgen.providers, filters=[pf], scope="module")

Note

filter_unused is helpful, in that you don’t have to accept all of the args in argnames in every test in the module. However, if all tests don’t share one common parametrized argname, py.test may not have enough information to properly organize tests beyond the ‘function’ scope. Thus, when parametrizing in the module scope, it’s a good idea to include at least one common argname in every test signature to give pytest a clue in sorting tests.

cfme.utils.testgen.parametrize(metafunc, argnames, argvalues, *args, **kwargs)[source]

parametrize wrapper that calls _param_check(), and only parametrizes when needed

This can be used in any place where conditional parametrization is used.

cfme.utils.testgen.providers(metafunc, filters=None)[source]

Gets providers based on given (+ global) filters

Note

Using the default ‘function’ scope, each test will be run individually for each provider before moving on to the next test. To group all tests related to single provider together, parametrize tests in the ‘module’ scope.

Note

testgen for providers now requires the usage of test_flags for collection to work. Please visit http://cfme-tests.readthedocs.org/guides/documenting.html#documenting-tests for more details.

cfme.utils.testgen.providers_by_class(metafunc, classes, required_fields=None)[source]

Gets providers by their class

Parameters:
  • metafunc – Passed in by pytest
  • classes – List of classes to fetch
  • required_fields – See cfme.utils.provider.ProviderFilter

Usage:

# In the function itself
def pytest_generate_tests(metafunc):
    argnames, argvalues, idlist = testgen.providers_by_class(
        [GCEProvider, AzureProvider], required_fields=['provisioning']
    )
metafunc.parametrize(argnames, argvalues, ids=idlist, scope='module')

# Using the parametrize wrapper
pytest_generate_tests = testgen.parametrize([GCEProvider], scope='module')
cfme.utils.testgen.pxe_servers(metafunc)[source]

Provides pxe data based on the server_type

Parameters:server_name – One of the server names to filter by, or ‘all’.