fixtures.ui_coverage module

UI Coverage for a CFME/MIQ Appliance


py.test --ui-coverage

General Notes

simplecov can merge test results, but doesn’t appear to like working in a multi-process environment. Specifically, it clobbers its own results when running simultaneously in multiple processes. To solve this, each process records its output to its own directory (configured in coverage_hook). You end up with a directory structure like this:

coverage-             |-$ip1-             .      |-$pid1-             .      .       |-.resultset.json (coverage statistics)
         .      .       |-.last_run.json  (overall coverage percentage)
         .      .
         .      |-$pidN

Note the .resultset.json format is documented in the ruby Coverage libraries docs:

All of the individual process’ results are then manually merged (coverage_merger) into one big json result, and handed back to simplecov which generates the compiled html (for humans) and rcov (for jenkins) reports.

thing_toucher makes a best-effort pass at requiring all of the ruby files in the rails root, as well as any external MIQ libs/utils outside of the rails root (../lib and ../lib/util). This makes sure files that are never required still show up in the coverage report.

Workflow Overview

Pre-testing (pytest_configure hook):

  1. Add to the rails root, then run bundler to install simplecov and its dependencies.
  2. Install and require the coverage hook (copy coverage_hook to config/, add require line to the end of config/boot.rb)
  3. Restart EVM (Rudely) to start running coverage on the appliance processes: killall -9 ruby; sysemctl start evmserverd
  4. TOUCH ALL THE THINGS (run thing_toucher.rb with the rails runner). Fork this process off and come back to it later

Post-testing (pytest_unconfigure hook):

  1. Poll thing_toucher to make sure it completed; block if needed.
  2. Stop EVM, but nicely this time so the coverage atexit hooks run: systemctl stop evmserverd
  3. Pull the coverage dir back for parsing and archiving

Post-testing (e.g. ci environment): * This is changing *

  1. Use the generated rcov report with the ruby stats plugin to get a coverage graph
  2. Zip up and archive the entire coverage dir for review
class fixtures.ui_coverage.CoverageManager(ipappliance)[source]

Bases: object

class fixtures.ui_coverage.UiCoveragePlugin[source]

Bases: object

fixtures.ui_coverage.appliance_coverage_root = local('/var/www/miq/vmdb/coverage')

coverage root, should match what’s in the coverage hook and merger scripts

fixtures.ui_coverage.rails_root = local('/var/www/miq/vmdb')

Corresponds to Rails.root in the rails env