Development Tips and Tricks


This document is intended to explain some of the extra bits of the framework that are there to make your life easier. Not everything is included here and we encourage people to add new tricks as they are developed and rediscovered.

Version Picking

Dealing with multiple releases, it’s obvious that some things change from version to version. A lot of the time, these changes are simple, such as a string change. So that we can continue using the same codebase for any version, we define the idea of version picking. Version picking essentially returns an object depending on the version of an appliance. It’s particularly useful for things like locator changes because most of the element handling routines are version picking away. This means if they receive a dict as an argument, they will automatically try to resolve it using the version picking tool. To use version picking is easy:

from cfme.utils import version

version.pick({'5.4': "Houses",
              '5.3': "House",
              version.LOWEST: "Boat"})

In this example, if the version is below 5.3, the Boat will be returned. Anything between 5.3 and 5.4 will return House and anything over 5.4 will return Houses. There is also a version.LATEST which points to upstream appliances. Another important point to remember is that one shouldn’t verspick at import time. The best practise is to use it inside locators without using verpick excpliticly. The syntax is pretty simple:

    'properties_form': {
        version.LOWEST: Input('House'),
        '5.6': Input('Houses'),

Defining blockers

Sometimes we know a test fails due to a bug in the codebase. In order to make sure the test isn’t run and attributing an extra fail that doesn’t need to be investigated, we mark it with a meta marker. The meta marker is incredibly useful and integrates with our Bugzilla implementation to ensure that if a bug is still on DEV, or hasn’t even been assigned yet, that the test won’t run. The syntax is really easy:

@pytest.mark.meta(blockers=[12345, 12346])
def test_my_feature():
    # Test the new feature

Note the two bug numbers 12345 and 12346. More information can be found in the fixtures.blockers fixture.

Using blockers in tests

On the odd occasion, you don’t want to disable an entire test, but just a part of it, until a bug is fixed. To do this, we can specify a bug object and ask the framework to skip if a certain bug exists and is not closed. The syntax is pretty simple:

def my_test(provider, bug):
    ui_bug = bug(12234)
    if not ui_bug:
        # Do something unless the bug is still present in which case, it will be skipped

Uncollecting tests

There are times when conditions dictate that we don’t need to run a test if a certain condition is true. Imagine you don’t want to run a test if the appliance version is below a certain value. In these instances, you can use uncollectif which is a pytest marker:

@pytest.mark.uncollectif(lambda appliance: appliance.version < '5.9')
def test_my_feature():
    # Test the new feature

Now if the version of the appliance is less than 5.3. Then the test will not be skipped, it will never even try to be run. This is ONLY to be used when a certain test is not valid for a certain reason. it is NOT to be used if there is a bug in the code. See the Defining blockers section above for skipping because of a bug.

Running commands on another appliance


Though this still works, the stack will be removed in due course. Objects now are linked to an appliance and it is expected that this appliance will be what is used. As this is now the case, it is unlikely that the context manager will be needed for much longer.

We implement a small appliance stack in the framework. When a test first starts it loads up the first appliance in the stack. From then on, all the browsing operations, database operations and ssh commands are run on the top appliance in the stack. From time to time it becomes necessary to run commands on another appliance. Let’s say you were trying to get two appliances to talk to each other, in this case, you would use the context manager for appliances.

By default, even if you add a new appliance onto the stack, the browser operations will keep happening on the last appliance that was used, however, there is a simple way to steal the browsers focus, and this is detailed in the example below:

appl1.ipapp.browser_steal = True
with appl1.ipapp:

In the example we have already created a new utils.appliance.Appliance object and called it appl1. Then we have set it to steal the browser focus. After this, we enter the context manager appl1.ipapp and are able to run operations like provider creates.

This is also why you should use ssh_client and db access from the store.current_appliance and not from the modules directly. If someone else uses your code and is inside an appliance context manager, the commands could be run against the wrong appliance.

Logging in as another user

In a similar way to the Running commands on another appliance section above, we implement a context manager for user operations. This allows the test developer to execute a section of code as a different user and then return to the original user once complete.

A major advantage of this, is that the User object used for the CM operations is the same as the cfme.configure.access_control object. This means that you can create a new user using the cfme.configure.access_control.User object and straight after use it as the context manager object:

cred = Credential(principal='uid', secret='redhat')
user = User(name='user' + fauxfactory.gen_alphanumeric(),
with user:
    navigate_to(current_appliance.server, 'Dashboard')

The User object stores the previous User object in a cache inside itself and on exiting the context, returns this to the pytest store as the current user so that future operations are performed with the original user.

Invalidating cached data

In order to speed things up, we cache certain items of data, such as the appliances version and configuration details. When these get changed, the cache becomes invalid and we must invalidate the cache somehow. You need to call an appropriate method on the appliance object like utils.appliance.IPAppliance.server_details_changed() which invalidates the data.

pytest store

The pytest store provides access to common pytest data structures and instances that may not be readily available elsewhere. It can be found in fixtures.pytest_store, and during a test run is exposed on the pytest module in the store namespace as

Test generation (testgen)

We try to consolidate common test generation functions in the utils.testgen module. When parametrizing tests with the pytest_generate_tests hook, check the testgen module to see if there are functions available that already parametrize on the axis you want (usually by provider, but there are some other helpers in there).

Working with file paths

For any path in the project root, there are several helper functions that can be used. Look at the utils.path module for the complete list of pre-configured directories and available functions.

Expecting Errors

When working with the UI, we can actually run a process and expect to have a certain flash error message. This is built into a context manager so that all you need to do is supply the operation you want to try, and the emssage you expect to get. This means as a test developer, you don’t need to worrk about how to get the flash message, or how to handle the resulting error from the operation failing:

provider.credentials['default'] = get_credentials_from_config('bad_credentials')
with error.expected('Login failed due to a bad username or password.'):

Marking your tests with associated product requirements

Test requirements mapping

This module contains predefined pytest markers for CFME product requirements.

Please import the module instead of elements:

from cfme import test_requirements

pytestmark = [test_requirements.alert]

def test_quota_alert():