Commit graph

339 commits

Author SHA1 Message Date
Jordan Borean
faaa669764
PowerShell - Added coverage collector (#59009)
* Added coverage collection for PowerShell - ci_complete ci_coverage

* uncomment out coverage uploader call

* Generate XML for PowerShell coverage

* Use whitelist to exclude coverage run on non content plugins

* Remove uneeded ignore entry

* Try to reduce diff in cover.py

* Fix up coverage report package - ci_complete ci_coverage
2019-08-28 07:03:23 +10:00
Matt Clay
4d424d0830 Only run package-data sanity once in CI. 2019-08-20 14:09:40 -07:00
Jordan Borean
3a3727d200
Remove Python 2.6 from Windows tests (#60235) 2019-08-08 07:32:09 +10:00
Matt Clay
b446ca100f
Prepare ansible-test code for relocation. (#60110)
* Fix imports in cli.py.

* Fix imports in executor.py.

* Remove old test/runner/ansible-test entry point.

Use the official bin/ansible-test entry point instead, which has been around since Ansible 2.5.

* Use bin/ansible-test on Shippable.
2019-08-05 14:56:38 -07:00
Matt Clay
31c7a61f9d
Clean up old ansible-test references and code. (#60103)
* Remove obsolete ansible-test retry.py.
* Use ansible-test from bin/ intead of test/runner/.
2019-08-05 13:29:50 -07:00
Matt Clay
d3da8e4a5b
Replace old shippable.py with new check_matrix.py. (#60022)
This new script does not depend on ansible-test and provides much more robust job matrix testing.

It is also run on every job in the matrix now, to detect issues with jobs being re-run after matrix changes are made.
2019-08-05 11:27:37 -07:00
Matt Clay
f944bd2358 Update ansible-test to properly skip unit tests.
Unit tests will no longer run on "remote only" Python versions (2.6) for tests which are not "remote" (modules and module_utils).
2019-07-31 00:27:07 -07:00
Matt Martz
95f4282c42
Verify package data in setup.py installs all files (#59537)
* Add sanity test to ensure all non-py files are installed

* Fix mode and regex

* Fix role skel inventory package_data

* Add docs

* Update package_data for inventory files

* Address pylint concerns

* Another tweak to package_data

* Address review feedback

* Change index to 1

* add to ansible-only.txt
2019-07-24 13:58:13 -05:00
Matt Clay
13bbfacb04 Show ansible-test --explain errors in CI. 2019-07-17 15:34:00 -07:00
Matt Clay
3f2b766d10 Add future and metaclass boilerplate to test code.
Continue to ignore:

- test/integration/
- test/legacy/
- test/units/
2019-07-12 09:18:24 -07:00
Matt Clay
79685a4a2d
Use Python 3.7 to run ansible-test coverage in CI. (#57555)
Generating XML for coverage reports on Python 3.7 uses far less
memory than on Python 2.7. This change should avoid the MemoryError
we've been seeing in the nightly coverage runs.
2019-06-07 15:48:17 -07:00
Matt Clay
b1e6ad9754
Limit Python 2.6 units to modules & module_utils. (#57377) 2019-06-04 15:15:36 -07:00
Matt Clay
85ae8f5258 Add Hetzner Cloud to CI integration tests. 2019-04-09 15:41:50 -07:00
Matt Clay
c846501829 Increase unit test timeout for --coverage. 2019-04-05 09:43:42 -07:00
Matt Clay
2a8252ceb9 Slightly increase non-coverage unit test timeout. 2019-03-21 09:30:43 -07:00
Matt Clay
8fa294a33a
Fix ansible-test coverage stub memory usage. (#53652)
Split coverage stub files into multiple parts based on the number
of lines in files included in each stub. This will keep memory usage
down to avoid hanging on Shippable.
2019-03-11 15:14:09 -07:00
Matt Clay
887ab35656
Add support for ansible-test --coverage-check. (#53573) 2019-03-11 15:13:09 -07:00
Matt Clay
9546761407 Fix coverage stub generation in CI. 2019-03-08 16:29:14 -08:00
Matt Clay
a8d829d9c3
Add timeout support to ansible-test. (#53302)
* Add timeout support to ansible-test.
* Fix ansible-test tar filename filter bug.
* Update timeouts used on Shippable.
* Kill subprocesses when parent process terminates.
* Require explicit use of env --show option.
2019-03-05 11:58:13 -08:00
Matt Clay
b2f354ce29 Ignore errors during container cleanup in CI. 2019-02-21 21:20:48 -08:00
Matt Clay
a1fec259c1
Reduce console noise at start of CI session. (#52387)
* Reduce console noise at start of CI session.
* Use allowed shebang.
2019-02-18 13:41:45 -08:00
Matt Clay
6a82827ddd Add Python 3.8 to ansible-test and CI. 2019-02-14 23:24:24 -08:00
Matt Clay
d2a9b16f7e Make Shippable timing script more resilient.
This will permit the script to pass through content which could
not be previously decoded or encoded. This could occur when running
some tests on macOS using file paths with non-ASCII characters.
2019-01-17 00:03:38 -08:00
John R Barker
721a28202e
Revert Debug: Size of coverage file #50380 (#50945) 2019-01-15 20:36:12 +00:00
Matt Clay
31c1bdd6a8 Fix run.py --branch argument.
The API docs state that both `branch` and `branchName` are valid,
but only `branchName` appears to work.
2019-01-07 13:56:35 -08:00
John R Barker
e6ced9c73f
Debug: Size of coverage file (#50380) 2018-12-29 13:01:35 +00:00
John R Barker
7ec82cdfc9
Revert TEMP HACK: Upload all to codecov (#50279)
Codecov.io have made some improvements to results processing.
Upload all the coverage data to see is codecov can process this.

Reverts https://github.com/ansible/ansible/pull/47515
2018-12-23 09:42:29 +00:00
Matt Clay
01833b6fb1
Add env command to ansible-test and run in CI. (#50176)
* Add `env` command to ansible-test and run in CI.
* Avoid unnecessary docker pull.
2018-12-20 22:08:57 -08:00
Matt Clay
9979a32e5c Fix passing of env vars to Shippable. 2018-11-14 10:50:31 -08:00
John R Barker
fb2e838a42
TEMP HACK: Avoid overloading codecov.io (#47515)
Without this patch we are overloading codecov.io by uploading 90+ tests.

As a workaround limit uploading to only "Group 1"

Will be removed/updated based on codecov.io's support team
2018-10-23 17:32:34 +01:00
Matt Clay
6a51b5ed26 Use 4 nodes for sanity tests in CI. 2018-10-16 09:55:26 -07:00
Matt Clay
2761fe8272 Revert "Use 4 nodes for sanity tests in CI."
This reverts commit b887333029.
2018-10-15 12:11:17 -07:00
Matt Clay
b887333029 Use 4 nodes for sanity tests in CI. 2018-10-15 12:03:43 -07:00
Jordan Borean
25c627d256
shippable: fix py3 issues with downloader (#46522) 2018-10-05 18:21:59 +10:00
Matt Clay
e53390b3b1
Fix ansible-test multi-group smoke test handling. (#46363)
* Fix ansible-test smoke tests across groups.
* Fix ansible-test list arg defaults.
* Fix ansible-test require and exclude delegation.
* Fix detection of Windows specific changes.
* Add minimal Windows testing for Python 3.7.
2018-10-02 12:26:14 -07:00
Matt Clay
ac492476e5
Bug fixes and cleanup for ansible-test. (#45991)
* Remove unused imports.
* Clean up ConfigParser usage in ansible-test.
* Fix bare except statements in ansible-test.
* Miscellaneous cleanup from PyCharm inspections.
* Enable pylint no-self-use for ansible-test.
* Remove obsolete pylint ignores for Python 3.7.
* Fix shellcheck issuers under newer shellcheck.
* Use newer path for ansible-test.
* Fix issues in code-smell tests.
2018-09-21 11:38:22 -07:00
Matt Clay
771768966f Flush output from timing script. 2018-08-31 07:58:12 -07:00
Matt Clay
a96a51b0c6 List quay.io images during Shippable run. 2018-08-29 18:04:22 -07:00
Matt Clay
78bd020429 Remove obsolete test/utils/docker/ directory. 2018-08-29 18:04:10 -07:00
Matt Clay
5461bfbf31 Split sanity tests into 3 groups. 2018-08-27 13:43:11 -07:00
Matt Clay
31a5b874a1 Use runtime instead of datetime for test output. 2018-08-23 12:05:10 -07:00
Matt Clay
4e489d1be8
Update Shippable integration test groups. (#43118)
* Update Shippable integration test groups.
* Update integration test group aliases.
* Rebalance AWS and Azure tests with extra group.
* Rebalance Windows tests with another group.
2018-07-23 20:46:22 -07:00
Matt Clay
528de5b82f Remove unwanted containers at CI job start. 2018-07-09 21:48:11 -07:00
Matt Clay
a7d7df1450 Make docs-build sanity test disabled by default. 2018-05-09 17:55:00 -07:00
Matt Clay
ee02d0c120 Remove obsolete test containers.
- opensuse42.1 - EOL
- opensuse42.2 - EOL
- ubuntu1204 - EOL
- cloudstack-simulator - Migrated to https://github.com/ansible/cloudstack-test-container
2018-04-25 11:26:42 -07:00
Matt Clay
26312b6557 Remove migrated httptester files.
The httptester has been migrated to:

https://github.com/ansible/http-test-container
2018-04-17 15:03:51 -07:00
Matt Clay
27def6c5dc Remove vcenter-simulator container files.
The simulator has moved to its own repository:

https://github.com/ansible/vcenter-test-container
2018-04-16 16:47:32 -07:00
Matt Clay
8a223009ca
Improve handling of integration test aliases. (#38698)
* Include change classification data in metadata.
* Add support for disabled tests.
* Add support for unstable tests.
* Add support for unsupported tests.
* Overhaul integration aliases sanity test.
* Update Shippable scripts to handle unstable tests.
* Mark unstable Azure tests.
* Mark unstable Windows tests.
* Mark disabled tests.
2018-04-12 16:15:28 -07:00
Martin Krizek
cb5e594c13 Fix building from ubuntu1404 Dockerfile 2018-03-28 14:17:31 -07:00
Matt Martz
7c311ad615 Update httptester for httpbin==0.6.2 2018-03-21 11:21:39 -07:00