fixtures.ui_coverage module¶
UI Coverage for a CFME/MIQ Appliance
Usage¶
py.test --ui-coverage
General Notes¶
simplecov can merge test results, but doesn’t appear to like working in a multi-process environment. Specifically, it clobbers its own results when running simultaneously in multiple processes. To solve this, each process records its output to its own directory (configured in coverage_hook). You end up with a directory structure like this:
coverage- |-$ip1- . |-$pid1- . . |-.resultset.json (coverage statistics)
. . |-.last_run.json (overall coverage percentage)
. .
. |-$pidN
.
|-$ipN
Note the .resultset.json format is documented in the ruby Coverage libraries docs:
All of the individual process’ results are then manually merged (coverage_merger) into one big json result, and handed back to simplecov which generates the compiled html (for humans) and rcov (for jenkins) reports.
thing_toucher makes a best-effort pass at requiring all of the ruby files in the rails root, as well as any external MIQ libs/utils outside of the rails root (../lib and ../lib/util). This makes sure files that are never required still show up in the coverage report.
Workflow Overview¶
Pre-testing (pytest_configure
hook):
- Add
Gemfile.dev.rb
to the rails root, then run bundler to install simplecov and its dependencies. - Install and require the coverage hook (copy
coverage_hook
to config/, add require line to the end ofconfig/boot.rb
) - Restart EVM (Rudely) to start running coverage on the appliance processes:
killall -9 ruby; sysemctl start evmserverd
- TOUCH ALL THE THINGS (run
thing_toucher.rb
with the rails runner). Fork this process off and come back to it later
Post-testing (pytest_unconfigure
hook):
- Poll
thing_toucher
to make sure it completed; block if needed. - Stop EVM, but nicely this time so the coverage atexit hooks run:
systemctl stop evmserverd
- Pull the coverage dir back for parsing and archiving
Post-testing (e.g. ci environment): * This is changing *
- Use the generated rcov report with the ruby stats plugin to get a coverage graph
- Zip up and archive the entire coverage dir for review
-
class
fixtures.ui_coverage.
CoverageManager
(ipappliance)[source]¶ Bases:
object
-
collection_appliance
¶
-
-
fixtures.ui_coverage.
appliance_coverage_root
= local('/var/www/miq/vmdb/coverage')¶ coverage root, should match what’s in the coverage hook and merger scripts
-
fixtures.ui_coverage.
rails_root
= local('/var/www/miq/vmdb')¶ Corresponds to Rails.root in the rails env