cfme.utils.testgen module¶
Test generation helpers
Intended to functionalize common tasks when working with the pytest_generate_tests hook.
When running a test, it is quite often the case that multiple parameters need to be passed to a single test. An example of this would be the need to run a Provider Add test against multiple providers. We will assume that the providers are stored in the yaml under a common structure like so:
providers:
prov_1:
name: test
ip: 10.0.0.1
test_vm: abc1
prov_2:
name: test2
ip: 10.0.0.2
test_vm: abc2
Our test requires that we have a Provider Object and as an example, the ‘test_vm’ field of the object. Let’s assume a test prototype like so:
test_provider_add(provider_obj, test_vm):
In this case we require the test to be run twice, once for prov_1 and then again for prov_2.
We are going to use the generate function to help us provide parameters to pass to
pytest_generate_tests()
. pytest_generate_tests()
requires three pieces of
information, argnames
, argvalues
and an idlist
. argnames
turns into the
names we use for fixtures. In this case, provider_obj
and provider_mgmt_sys
.
argvalues
becomes the place where the provider_obj
and provider_mgmt_sys
items are stored. Each element of argvalues
is a list containing a value for both
provider_obj
and provider_mgmt_sys
. Thus, taking an element from argvalues
gives us the values to unpack to make up one test. An example is below, where we assume
that a provider object is obtained via the Provider
class, and the mgmt_sys object
is obtained via a Wrapanapi
class.
~ | provider_obj | test_vm |
---|---|---|
prov1 | Provider(prov1) | abc1 |
prov2 | Provider(prov2) | abc2 |
This is analogous to the following layout:
~ | argnames[0] | argnames[1] |
---|---|---|
idlist[0] | argvalues[0][0] | argvalues[0][1] |
idlist[1] | argvalues[1][0] | argvalues[1][1] |
This could be generated like so:
def gen_providers:
argnames = ['provider_obj', 'test_vm']
argvalues = []
idlist = []
for provider in yaml['providers']:
idlist.append(provider)
argvalues.append([
Provider(yaml['providers'][provider]['name']),
yaml['providers'][provider]['test_vm'])
])
return argnames, argvalues, idlist
This is then used with pytest_generate_tests like so:
pytest_generate_tests(gen_providers)
Additionally, py.test joins the values of idlist
with dashes to generate a unique id for this
test, falling back to joining argnames
with dashes if idlist
is not set. This is the value
seen in square brackets in a test report on parametrized tests.
More information on parametrize
can be found in pytest’s documentation:
-
cfme.utils.testgen.
auth_groups
(metafunc, auth_mode)[source]¶ Provides two test params based on the ‘auth_modes’ and ‘group_roles’ in cfme_data:
group_name
:- expected group name in provided by the backend specified in
auth_mode
group_data
:- list of nav destinations that should be visible as a member of
group_name
Parameters: auth_mode – One of the auth_modes specified in cfme_data.get('auth_modes', {})
-
cfme.utils.testgen.
generate
(*args, **kwargs)[source]¶ Functional handler for inline pytest_generate_tests definition
Parameters: - gen_func – Test generator function, expected to return argnames, argvalues, and an idlist suitable for use with pytest’s parametrize method in pytest_generate_tests hooks
- indirect – Optional keyword argument. If seen, it will be removed from the kwargs passed to gen_func and used in the wrapped pytest parametrize call
- scope – Optional keyword argument. If seen, it will be removed from the kwargs passed to gen_func and used in the wrapped pytest parametrize call
- filter_unused – Optional keyword argument. If True (the default), parametrized tests will be inspected, and only argnames matching fixturenames will be used to parametrize the test. If seen, it will be removed from the kwargs passed to gen_func.
- *args – Additional positional arguments which will be passed to
gen_func
- **kwargs – Additional keyword arguments whill be passed to
gen_func
Usage:
# Abstract example: pytest_generate_tests = testgen.generate(arg1, arg2, kwarg1='a') # Concrete example using all infrastructure providers and module scope pytest_generate_tests = testgen.generate([InfraProvider], scope="module") # Another concrete example using only VMware and SCVMM providers with 'retire' flag pf = ProviderFilter( classes=[WMwareProvider, SCVMMProvider]), required_flags=['retire']) pytest_generate_tests = testgen.generate( gen_func=testgen.providers, filters=[pf], scope="module")
Note
filter_unused
is helpful, in that you don’t have to accept all of the args in argnames in every test in the module. However, if all tests don’t share one common parametrized argname, py.test may not have enough information to properly organize tests beyond the ‘function’ scope. Thus, when parametrizing in the module scope, it’s a good idea to include at least one common argname in every test signature to give pytest a clue in sorting tests.
-
cfme.utils.testgen.
parametrize
(metafunc, argnames, argvalues, *args, **kwargs)[source]¶ parametrize wrapper that calls
_param_check()
, and only parametrizes when neededThis can be used in any place where conditional parametrization is used.
-
cfme.utils.testgen.
providers
(metafunc, filters=None)[source]¶ Gets providers based on given (+ global) filters
Note
Using the default ‘function’ scope, each test will be run individually for each provider before moving on to the next test. To group all tests related to single provider together, parametrize tests in the ‘module’ scope.
Note
testgen for providers now requires the usage of test_flags for collection to work. Please visit http://cfme-tests.readthedocs.org/guides/documenting.html#documenting-tests for more details.
-
cfme.utils.testgen.
providers_by_class
(metafunc, classes, required_fields=None)[source]¶ Gets providers by their class
Parameters: - metafunc – Passed in by pytest
- classes – List of classes to fetch
- required_fields – See
cfme.utils.provider.ProviderFilter
Usage:
# In the function itself def pytest_generate_tests(metafunc): argnames, argvalues, idlist = testgen.providers_by_class( [GCEProvider, AzureProvider], required_fields=['provisioning'] ) metafunc.parametrize(argnames, argvalues, ids=idlist, scope='module') # Using the parametrize wrapper pytest_generate_tests = testgen.parametrize([GCEProvider], scope='module')