cfme.fixtures.ui_coverage module

UI Coverage for a CFME/MIQ Appliance


py.test --ui-coverage

General Notes

simplecov can merge test results, but doesn’t appear to like working in a multi-process environment. Specifically, it clobbers its own results when running simultaneously in multiple processes. To solve this, each process records its output to its own directory (configured in coverage_hook). You end up with a directory structure like this:

coverage-             |-$ip1-             .      |-$pid1-             .      .       |-.resultset.json (coverage statistics)
         .      .       |-.last_run.json  (overall coverage percentage)
         .      .
         .      |-$pidN

Note the .resultset.json format is documented in the ruby Coverage libraries docs:

All of the individual process’ results are then manually merged (coverage_merger) into one big json result, and handed back to simplecov which generates the compiled html (for humans) report.

Workflow Overview

Pre-testing (pytest_configure hook):

  1. Add to the rails root, then run bundler to install simplecov and its dependencies.

  2. Patch application with manageiq-17302 patch so that coverage_hook will be loaded by the application. Eventually this will be in CFME and we won’t have to do this.

  3. Install coverage hook (copy coverage_hook to config/).

  4. Restart EVM to start running coverage on the appliance processes.

Post-testing (pytest_unconfigure hook):

  1. Stop EVM, but nicely this time so the coverage atexit hooks run: systemctl stop evmserverd

  2. Pull the coverage dir back for parsing and archiving

Post-testing (e.g. ci environment): * This is changing *

  1. Use the generated rcov report with the ruby stats plugin to get a coverage graph

  2. Zip up and archive the entire coverage dir for review

class cfme.fixtures.ui_coverage.CoverageManager(ipappliance)[source]

Bases: object

property collection_appliance
class cfme.fixtures.ui_coverage.UiCoveragePlugin[source]

Bases: object

cfme.fixtures.ui_coverage.appliance_coverage_root = local('/var/www/miq/vmdb/coverage')

coverage root, should match what’s in the coverage hook and merger scripts

cfme.fixtures.ui_coverage.rails_root = local('/var/www/miq/vmdb')

Corresponds to Rails.root in the rails env