Commit graph

229 commits

Author SHA1 Message Date
Matt Davis
4c5ce5a1a9
module compat for py3.8+ controller ()
* module compat for py3.8+ controller

* replaced internal usages of selinux bindings with internal ctypes binding (allows basic selinux operations from any Python interpreter), plus tests

* added new respawn_module API to allow modules to import Python packages that are only available under a well-known interpreter, plus tests

* added respawn logic to modules that need Python libs from a specific system interpreter (apt, apt_repository, dnf, yum)

minimize internal HAVE_SELINUX usage

spurious junk

pep8

* pylint fixes

* add RHEL8 Python 3.8 testing

* more pylint

* import sanity

* unit tests

* changelog update

* fix a bunch of stuff

* tweak changelog

* fix setup_rpm_repo on EL8

* misc sanity/test fixes

* misc feedback tweaks

* fix import fallback in test module

* fix selinux MU test

* fix dnf tests to avoid python-dependent test packages

* add trailing LFs to aliases

* fix yum tests to avoid test package with Python deps

* hack create_repo for EL6 to create noarch package
2021-02-10 21:32:59 -08:00
Matt Clay
3a633b278d [devel] Azure Pipelines configuration. 2020-12-02 13:40:01 -08:00
Sam Doran
6984081111
Test changes to allow using "macos" test image ()
* Add link to remote.sh from macos.sh
* Add skip/macos to tests that have skip/osx
2020-09-22 09:10:36 -04:00
Rick Elrod
4f54fb8d43
Remove incidental azure tests ()
Change:
- All incidental coverage from these tests has moved to intentional
  coverage.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-08-08 00:56:53 -05:00
Matt Martz
fa1fb2d13b
Test ansible-galaxy against pulp/pulp_ansible ()
* Test galaxy cli against pulp

* linting fix

* Renames and small fixes

* Better handling for resetting pulp

* Clean up some things, add a comment

* I can't spell

* Bump fallaxy, use alternate pulp image

* Only reset pulp when we're are executing against pulp

* Update for updated pulp container

* Update some comments with correct URLs and typos

* Linting fix

* Pin pulp-fedora31 to a digest

* Address review comments for documentation
2020-06-17 16:06:07 -05:00
Matt Clay
0faec62354 Remove temporary migration hack from CI scripts. 2020-06-16 11:25:39 -07:00
Felix Fontein
a114da80ee
Use antsibull-changelog instead of packaged changelog generator ()
Replace the ansible-base changelog linting and generation tool with antsibull-changelog and make it available for linting collections. Previously changelog linting was limited to ansible-base.
2020-06-11 11:21:21 -07:00
Jordan Borean
f5f3ba7ab5
ansible-test - fix up relative util import for powershell validate-modules ()
* ansible-test - fix up relative util import for powershell validate-modules

* Use different tactic for generic group

* Use python 2 and 3
2020-05-29 14:31:59 +10:00
Matt Clay
5fc01c0a1d
Use a custom build container on Shippable. ()
* Use a custom build container on Shippable.

This supports key generation before git_sync, to avoid issues with pre-migration PRs.

If the node pool is switched to another version, a matching build container should be built and used.

* Don't kill our own custom container.
2020-05-26 14:05:46 -07:00
Yanis Guenane
46d82179d8
Testing: Add support for CentOS Linux On Power platform ()
* Testing: Add CentOS Linux On Power platform

* Add arch designation to remotes.

This avoids overloading the provider with the arch.

Also add a changelog entry.

Co-authored-by: Matt Clay <matt@mystile.com>
2020-04-15 16:22:17 -07:00
Matt Clay
41f6c73be0 Update ansible-test network test provisioning. ()
* Update network provisioning defaults.

* Update network test requirements.
2020-03-23 11:14:21 -05:00
Matt Clay
27ce6dbc1f Remove test support option from ansible-test.
This option was added to facilitate migration and is no longer needed.
2020-03-23 11:14:21 -05:00
Matt Clay
fb3e6d6b00 Remove unused Shippable scripts. 2020-03-23 11:14:21 -05:00
Matt Clay
4cecacb915 Update Shippable test matrix. ()
* Update Shippable test matrix.

* Remove unit test groups.

* Remove extra windows groups.
2020-03-23 11:14:21 -05:00
Matt Clay
a51266ba85 Skip removed windows CI groups after migration. 2020-03-04 12:08:05 -08:00
Matt Clay
3508e1197e
Update test matrix to prepare for migration. ()
* Split fallaxy tests into their own group.

This keeps expected pass/fail tests separate during migration prep.

* Move network tests down in matrix.

This will help keep pass/fail tests grouped together during migration prep.

* Group all incidental tests together.

This will keep pass/fail tests separate leading up to migration.
2020-03-03 16:23:48 -08:00
Matt Clay
f4b923a29b Automatically enable test plugins after migration. 2020-03-03 16:23:27 -08:00
Matt Clay
64782dd815
Split unit tests into 3 groups. () 2020-03-01 22:57:48 -08:00
Matt Clay
42eba3ce25 Increase unit test timeout. 2020-03-01 19:22:56 -08:00
Matt Clay
bcddf2bda9 Increase unit test coverage timeout. 2020-03-01 13:59:09 -08:00
Matt Clay
7c8b046b5f
Fourth batch of incidental integration tests. ()
* Copy in incidental posix tests.

* Update incidental test aliases.

* Update target names.

* Add support plugins.

* Fix paths.

* Update ignores.

* Update integration-aliases sanity test.

* Add incidental tests to CI.
2020-02-28 20:37:50 -08:00
Matt Clay
f735fd672a
Third batch of incidental integration tests. ()
* Copy in incidental windows tests.

* Update incidental test aliases.

* Add support plugins.

* Update target references.

* Update sanity ignores.

* Update integration-aliases test.

* Add to CI.
2020-02-27 16:05:47 -08:00
Matt Clay
e3591223a0
Second batch of incidental integration tests. ()
* Update incidental test aliases.

* Rewrite target references for renamed targets.

* Add incidental tests to CI.

* Update sanity tests for incidental cloud tests.

* Initial copy of incidental tests.

* Copy contrib files into test.

* Update paths in test.

* Add support plugins.

* Update plugin to work around missing deps.

* Update sanity ignores.

* Fix matrix entries.

* Remove debug echo.
2020-02-25 23:18:50 -08:00
Matt Clay
4e571248a9
First batch of incidental integration tests. ()
* Initial copy of incidental network tests.

* Update incidental test aliases.

* Add incidental tests to CI.

* Rewrite module references in tests.

This should not be necessary once module redirection is supported.

* Rewrite target references for renamed targets.

* Add support collections for incidental tests.

* Add ignores for test support code.

* Remove echo used for debugging.
2020-02-24 17:31:14 -08:00
Matt Clay
2dadc259d3
Split hcloud tests into two CI groups. ()
* Split hcloud tests into two CI groups.

This will hopefully resolve CI failures when running with code coverage.

* Fix smoketest logic.
2020-02-23 21:17:03 -08:00
Matt Clay
4a885072af Split vyos tests to avoid timeouts with coverage. 2020-02-23 20:43:05 -08:00
Matt Clay
a64b562071
Fix up network testing on Shippable. ()
This prepares for code coverage collection for initial setup of incidental test coverage.
2020-02-23 18:37:26 -08:00
Matt Clay
19f6fc68d4
Relocate Shippable tools. ()
* Move Shippable tools to hacking directory.

These limits the `test/utils/shippable/` directory to scripts required for CI.

* Fix `test/utils/shippable/` file classification.

* Update package-data sanity test.
2020-02-19 09:55:34 -08:00
Matt Clay
80561ccef0 Improve sanity test handling in CI after migration. 2020-02-07 21:29:39 -08:00
Matt Clay
a0242b2173 Improve unit test handling in CI after migration. ()
* Improve unit test handling in CI after migration.

* Fix undefined var in bash 4.3 and earlier.

See https://git.savannah.gnu.org/cgit/bash.git/tree/CHANGES?id=3ba697465bc74fab513a26dea700cc82e9f4724e#n878
2020-02-07 21:28:46 -08:00
Matt Clay
5e68bb3d93
Add code coverage target analysis to ansible-test. ()
* Refactor coverage file enumeration.
* Relocate sanitize_filename function.
* Support sets when writing JSON files.
* Generalize setting of info_stderr mode.
* Split out coverage path checking.
* Split out collection regex logic.
* Improve sanitize_filename type hints and docs.
* Clean up coverage erase command.
* Fix docs and type hints for initialize_coverage.
* Update type hints on CoverageConfig.
* Split out logic for finding modules.
* Split out arc enumeration.
* Split out powershell coverage enumeration.
* Raise verbosity level of empty coverage warnings.
* Add code coverage target analysis to ansible-test.
2020-02-05 22:16:15 -08:00
Yanis Guenane
4fd2dce7f3
Testing: Add support for AIX platform () 2020-01-29 15:56:51 -08:00
Matt Clay
0971a0e7bc Consolidate remote Shippable scripts. 2020-01-28 17:48:54 -08:00
Matt Clay
5c1fe78685
Move CI coverage stub to sanity/5. () 2020-01-27 16:19:49 -08:00
Matt Clay
452ade007e Clean up Shippable scripts in tools directory. 2020-01-07 14:38:15 -08:00
Matt Clay
78e476eb20 Increase unit test timeout in CI. 2019-11-01 17:16:07 -07:00
Matt Clay
d2562cde1c Temporarily increase CI timeout. 2019-11-01 10:10:02 -07:00
Matt Clay
ada02f1966 Limit Shippable matrix check to ansible repo. 2019-10-01 09:53:19 -07:00
Gonéri Le Bouder
cc758ea574 Revert "shippable: temporary bump the timeout for Azure jobs"
The Azure timeout should not accure anymore since
173d47d1f4.

This reverts commit aaa8835311.
2019-09-09 18:17:25 -07:00
Sam Doran
c9e7a2122d
Increase unit test timeout to account for long image download times () 2019-09-03 12:14:55 -04:00
Gonéri Le Bouder
aaa8835311 shippable: temporary bump the timeout for Azure jobs 2019-08-30 18:47:51 -04:00
Matt Clay
e4e5005640
Fix ansible-test coverage path handling. ()
* Fix ansible-test coverage path handling.
* Split CI unit tests into two groups.
2019-08-30 00:18:49 -07:00
Sam Doran
e5b24a68cc Add additional group for sanity tests
Put validate-modules in its own group
2019-08-29 00:21:07 -07:00
Matt Clay
20d9c34862 Fix CI test result placement. 2019-08-28 09:58:00 -07:00
Matt Clay
f5d829392a
Overhaul ansible-test test path handling. ()
* Remove .keep files from test/results/ dirs.

* Remove classification of test/results/ dir.

* Add results_relative to data context.

* Use variables in delegation paths.

* Standardize file writing and results paths.

* Fix issues reported by PyCharm.

* Clean up invocation of coverage command.

It now runs through the injector.

* Hack to allow intercept_command in cover.py.

* Simplify git ignore for test results.

* Use test result tmp dir instead of cache dir.

* Remove old .pytest_cache reference.

* Fix unit test docker delegation.

* Show HTML report link.

* Clean up more results references.

* Move import sanity test output to .tmp dir.

* Exclude test results dir from coverage.

* Fix import sanity test lib paths.

* Fix hard-coded import test paths.

* Fix most hard-coded integration test paths.

* Fix PyCharm warnings.

* Fix import placement.

* Fix integration test dir path.

* Fix Shippable scripts.

* Fix Shippable matrix check.

* Overhaul key pair management.
2019-08-27 23:40:06 -07:00
Jordan Borean
faaa669764
PowerShell - Added coverage collector ()
* Added coverage collection for PowerShell - ci_complete ci_coverage

* uncomment out coverage uploader call

* Generate XML for PowerShell coverage

* Use whitelist to exclude coverage run on non content plugins

* Remove uneeded ignore entry

* Try to reduce diff in cover.py

* Fix up coverage report package - ci_complete ci_coverage
2019-08-28 07:03:23 +10:00
Matt Clay
4d424d0830 Only run package-data sanity once in CI. 2019-08-20 14:09:40 -07:00
Jordan Borean
3a3727d200
Remove Python 2.6 from Windows tests () 2019-08-08 07:32:09 +10:00
Matt Clay
b446ca100f
Prepare ansible-test code for relocation. ()
* Fix imports in cli.py.

* Fix imports in executor.py.

* Remove old test/runner/ansible-test entry point.

Use the official bin/ansible-test entry point instead, which has been around since Ansible 2.5.

* Use bin/ansible-test on Shippable.
2019-08-05 14:56:38 -07:00
Matt Clay
d3da8e4a5b
Replace old shippable.py with new check_matrix.py. ()
This new script does not depend on ansible-test and provides much more robust job matrix testing.

It is also run on every job in the matrix now, to detect issues with jobs being re-run after matrix changes are made.
2019-08-05 11:27:37 -07:00