fixtures.ui_coverage module

UI Coverage for a CFME/MIQ Appliance

Usage

py.test --ui-coverage

General Notes

simplecov can merge test results, but doesn’t appear to like working in a multi-process environment. Specifically, it clobbers its own results when running simultaneously in multiple processes. To solve this, each process records its output to its own directory (configured in coverage_hook). All of the individual process’ results are then manually merged (coverage_merger) into one big json result, and handed back to simplecov which generates the compiled html (for humans) and rcov (for jenkins) reports.

thing_toucher makes a best-effort pass at requiring all of the ruby files in the rails root, as well as any external MIQ libs/utils outside of the rails root (../lib and ../lib/util). This makes sure files that are never required still show up in the coverage report.

Workflow Overview

Pre-testing (pytest_configure hook):

1. Add Gemfile.dev.rb to the rails root, then run bundler to install simplecov and its dependencies.
2. Install and require the coverage hook (copy coverage_hook to config/, add require line to the end of config/boot.rb)
3. Restart EVM (Rudely) to start running coverage on the appliance processes: killall -9 ruby; sysemctl start evmserverd
4. TOUCH ALL THE THINGS (run thing_toucher.rb with the rails runner). Fork this process off and come back to it later

Post-testing (pytest_unconfigure hook):

1. Poll thing_toucher to make sure it completed; block if needed.
2. Stop EVM, but nicely this time so the coverage atexit hooks run: systemctl stop evmserverd
3. Run coverage_merger.rb with the rails runner, which compiles all the individual process reports and runs coverage again, additionally creating an rcov report
4. Pull the coverage dir back for parsing and archiving
5. For fun: Read the results from coverage/.last_run.json and print it to the test terminal/log

Post-testing (e.g. ci environment): 1. Use the generated rcov report with the ruby stats plugin to get a coverage graph 2. Zip up and archive the entire coverage dir for review

class fixtures.ui_coverage.CoverageManager(ipappliance)

Bases: object

collect()

collection_appliance

install()

merge()

print_message(message)

class fixtures.ui_coverage.UiCoveragePlugin

Bases: object

pytest_collection_finish()

pytest_configure(config)

pytest_sessionfinish(exitstatus)

pytest_sessionstart(session)

fixtures.ui_coverage.appliance_coverage_root = local('/var/www/miq/vmdb/coverage')

coverage root, should match what’s in the coverage hook and merger scripts

fixtures.ui_coverage.clean_coverage_dir()

fixtures.ui_coverage.manager()

fixtures.ui_coverage.pytest_addoption(parser)

fixtures.ui_coverage.pytest_cmdline_main(config)

fixtures.ui_coverage.rails_root = local('/var/www/miq/vmdb')

Corresponds to Rails.root in the rails env