cfme.fixtures.ui_coverage module

UI Coverage for a CFME/MIQ Appliance

Usage

py.test --ui-coverage

General Notes

simplecov can merge test results, but doesn’t appear to like working in a multi-process environment. Specifically, it clobbers its own results when running simultaneously in multiple processes. To solve this, each process records its output to its own directory (configured in coverage_hook). You end up with a directory structure like this:

coverage-             |-$ip1-             .      |-$pid1-             .      .       |-.resultset.json (coverage statistics)
         .      .       |-.last_run.json  (overall coverage percentage)
         .      .
         .      |-$pidN
         .
         |-$ipN

Note the .resultset.json format is documented in the ruby Coverage libraries docs:

All of the individual process’ results are then manually merged (coverage_merger) into one big json result, and handed back to simplecov which generates the compiled html (for humans) report.

Workflow Overview

Pre-testing (pytest_configure hook):

  1. Add Gemfile.dev.rb to the rails root, then run bundler to install simplecov and its dependencies.
  2. Patch application with manageiq-17302 patch so that coverage_hook will be loaded by the application. Eventually this will be in CFME and we won’t have to do this.
  3. Install coverage hook (copy coverage_hook to config/).
  4. Restart EVM to start running coverage on the appliance processes.

Post-testing (pytest_unconfigure hook):

  1. Stop EVM, but nicely this time so the coverage atexit hooks run: systemctl stop evmserverd
  2. Pull the coverage dir back for parsing and archiving

Post-testing (e.g. ci environment): * This is changing *

  1. Use the generated rcov report with the ruby stats plugin to get a coverage graph
  2. Zip up and archive the entire coverage dir for review
class cfme.fixtures.ui_coverage.CoverageManager(ipappliance)[source]

Bases: object

collect()[source]
collection_appliance
install()[source]
merge()[source]
print_message(message)[source]
class cfme.fixtures.ui_coverage.UiCoveragePlugin[source]

Bases: object

pytest_collection_finish()[source]
pytest_configure(config)[source]
pytest_sessionfinish(exitstatus)[source]
pytest_sessionstart(session)[source]
cfme.fixtures.ui_coverage.appliance_coverage_root = local('/var/www/miq/vmdb/coverage')

coverage root, should match what’s in the coverage hook and merger scripts

cfme.fixtures.ui_coverage.clean_coverage_dir()[source]
cfme.fixtures.ui_coverage.manager()[source]
cfme.fixtures.ui_coverage.pytest_addoption(parser)[source]
cfme.fixtures.ui_coverage.pytest_cmdline_main(config)[source]
cfme.fixtures.ui_coverage.rails_root = local('/var/www/miq/vmdb')

Corresponds to Rails.root in the rails env