Commit graph

639 commits

Author SHA1 Message Date
Matt Clay
d3da8e4a5b
Replace old shippable.py with new check_matrix.py. (#60022)
This new script does not depend on ansible-test and provides much more robust job matrix testing.

It is also run on every job in the matrix now, to detect issues with jobs being re-run after matrix changes are made.
2019-08-05 11:27:37 -07:00
Matt Clay
4df97c20b6 Handle quoted paths in ansible-test diff parsing. 2019-08-05 09:31:43 -07:00
Sam Doran
ea209e1c2f Wait for Ansible Tower instance to be ready 2019-08-01 11:32:45 -07:00
Gonéri Le Bouder
c4bb38d2b1 vcenter: disable ConfigParser interpolation (#59851)
Password can come with the '%' character. If we keep ConfigParser
interpolation enabled, it will try to modify the value.
Typical error looks like this one:

```
configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%Z,sq'
```
See: https://docs.python.org/3/library/configparser.html#interpolation-of-values
2019-07-31 09:53:11 -07:00
Matt Clay
93931810c8
Run pytest with dedicated pytest.ini. (#59842)
Also disable the pytest cache plugin.
2019-07-31 00:28:39 -07:00
Matt Clay
f944bd2358 Update ansible-test to properly skip unit tests.
Unit tests will no longer run on "remote only" Python versions (2.6) for tests which are not "remote" (modules and module_utils).
2019-07-31 00:27:07 -07:00
Matt Clay
2198ecb1d2
Improve checking for unusable sanity test ignores. (#59833)
* Check sanity ignore paths against test paths.

This prevents ignores from being added for paths which will never be tested by the test being ignored or skipped.

* Fix sanity ignore handling for no/all targets.

This allows checking of ignores for tests which do not use a target list.

It also allows checking of the full ignore list on every test run for tests that always use all targets.
2019-07-30 18:58:37 -07:00
Matt Clay
f61b044bf0 Support directories in sanity tests. 2019-07-30 18:22:09 -07:00
Matt Clay
e07c4f41d7 Standardize ansible-test sanity test filters. 2019-07-30 15:31:33 -07:00
Matt Clay
1351962eee Move "code smell" sanity test config loading. 2019-07-30 12:07:33 -07:00
Matt Clay
e1c2891188 Add paths_to_dirs function to ansible-test. 2019-07-30 10:46:39 -07:00
Matt Clay
53af57b08b Fix type discrepancy in ansible-test. 2019-07-30 10:46:05 -07:00
Lukas Kämmerling
5bddd2f04a Add hcloud_network module (#59366)
* Add hcloud_network module

* Apply suggestions from code review

Co-Authored-By: Matt Clay <matt@mystile.com>

* Add notice about dependency

* Adjust tests

* Apply suggestions from code review

Co-Authored-By: Felix Fontein <felix@fontein.de>

* Allow changing of ip range

* Add Setup step
2019-07-30 14:00:33 +02:00
Matt Clay
5d7cc993dd
Clean up sanity test ignore handling. (#59749)
* Allow sanity tests to easily ignore themselves.

Useful for simple regex based sanity tests that error on their own script due to the regex or error message generated.

* Simplify no-smart-quotes sanity test.

Pruning of unversioned directories is handled by ansible-test.

* Remove directory pruning from boilerplate tests.

Files not passing these tests should be ignored instead.

* Add missing sanity ignore entries.
2019-07-29 14:37:44 -07:00
Adam Miller
e9d7156f53 podman-docker docker compat for ansible-test (#59539)
* podman-docker docker compat for ansible-test

Signed-off-by: Adam Miller <admiller@redhat.com>

* remove reprs, use ex.stderr instead

Signed-off-by: Adam Miller <admiller@redhat.com>

* remove u''s ... not needed

Signed-off-by: Adam Miller <admiller@redhat.com>

* Update test/runner/lib/docker_util.py

Co-Authored-By: Matt Clay <matt@mystile.com>

* Update test/runner/lib/docker_util.py

Co-Authored-By: Matt Clay <matt@mystile.com>

* make sanity tests happy

Signed-off-by: Adam Miller <admiller@redhat.com>
2019-07-29 17:34:45 -04:00
Sam Doran
ed4a729fd6
Update constraints for coverage on Python 3.8 (#59733)
This fixes the unit tests hanging when run with Python 3.8 with coverage enabled
2019-07-29 16:05:29 -04:00
Matt Clay
66654475e1 Update ansible-test Python version handling.
Minimum version requirements for sanity tests have been standardized:

- All single version sanity tests now require Python 3.5 or later.
- All multiple version sanity tests continue to use all supported Python versions.
- All version neutral sanity tests continue to work on any supported Python version.

Previously some tests required 3.5 or later with most of the remaining tests requiring 2.7 or later.

When using the `--python` option to specify a Python version:

- Tests which do not support the specified Python version will be skipped with a warning.
- If the specified Python version is not available, any test attempting to use it will generate an error.

When not using the `--python` option to specify a Python version:

- Multiple version tests will attempt to run on all supported versions.
- Single version tests will use the current version if supported and available, or if no supported version is available.
- Single version tests will use the lowest available and supported version if the current version is not supported.
- Any versions which are not available or supported will be skipped with a warning.

Unit tests automatically skip unavailable Python versions unless `--python` was used to specify a version.
2019-07-29 11:54:03 -07:00
Matt Clay
2f6de3aea4 Add missing decorator to ansible-test sanity.
This fixes sanity test errors so they show the proper error code instead of "bound method".
2019-07-26 09:04:37 -07:00
Matt Clay
cf9999692a Fix ansible-test encoding issues.
This fixes `ansible-test sanity --list-tests` which was failing when `LC_ALL=C` was set.
2019-07-25 23:29:42 -07:00
Matt Clay
db7e9aa051
Use consolidated ignore.txt in ansible-test. (#59618) 2019-07-25 23:29:23 -07:00
Matt Clay
ba90263c22 Fix ansible-test cloud testing for collections. 2019-07-24 00:17:11 -07:00
Matt Clay
3c68a2e5f0 Ignore deleted when querying git in ansible-test. 2019-07-23 23:36:08 -07:00
Matt Clay
51083537fa Fix incorrect collections coverage warning. 2019-07-23 22:06:17 -07:00
Matt Clay
7c728118e6
Fix import sanity test for collections. (#59484) 2019-07-23 18:10:24 -07:00
Matt Clay
d65d8d6e13 Fix default pylint config for collections. 2019-07-23 17:14:10 -07:00
Matt Clay
44a262714c
Add common sanity ignore & skip to ansible-test. (#59416)
* Add common sanity ignore & skip to ansible-test.
* Switch ansible-doc sanity test to single version.
* Remove duplicate ignore entry.
2019-07-23 13:31:54 -07:00
Matt Clay
79eca9c8fb
Initial ansible-test support for collections. (#59197)
* Initial ansible-test support for collections.
* Include cloud config in delegation payload.
* Add missing types import and fix `t` shadowing.
* Fix plugin traceback when config_path not set.
* Fix encoding issues.
* Remove unused imports.
* More encoding fixes.
* Handle delegation outside exception handler.
* Inject ssh keys only if not already in place.
* More defensive approach to getting remote pwd.
* Add missing string format var.
* Correct PowerShell require regex.
* Rename `is_install` and `INSTALL_ROOT`.
2019-07-22 19:24:48 -07:00
Matt Clay
52ad0f3217 Update ansible-test sanity requirements.
Skip installation of packages on Python 2.x which are only used on 3.x.
2019-07-22 10:14:33 -07:00
Gonéri Le Bouder
e9b1e36d76 vcenter provider: avoid a TypeError exception
Avoid `TypeError: option values must be strings` with `ConfigParser`.
Default values must be string, not `None`.

The error happens when `test/integration/cloud-config-vcenter.ini` does
not have the `vmware_proxy_host` or `vmware_proxy_port` keys defined.
2019-07-22 11:19:39 +02:00
Matt Clay
b378e885ed Make ansible-test TestMessage hashable. 2019-07-19 21:34:04 -07:00
Matt Clay
4f1d0f1b39 Skip using paths for ansible-doc sanity test. 2019-07-19 16:17:56 -07:00
Yunge Zhu
cb201bfba1 add xmltodict to azure requirements file (#55748)
* add xmltodict

* fix lint
2019-07-18 01:32:51 -04:00
Matt Clay
cb0dfc9793 Prepare pylint test for collections support. 2019-07-16 11:24:16 -07:00
Matt Clay
2ed78b650f Update to ansible-test to prepare for collections.
Changes to simplify merging of collections support.
2019-07-16 08:04:07 -07:00
Matt Clay
4b9391f32a Clean up ansible-test target handling. 2019-07-15 20:33:23 -07:00
Matt Clay
4255ee8e5a Prepare ansible-test for collections support.
Another round of changes to prepare ansible-test for supporting collections to help keep later PRs a manageable size.
2019-07-15 17:46:29 -07:00
Matt Clay
f81238012b Clean up pylint issues in test infra. 2019-07-15 14:45:30 -07:00
Matt Clay
d910c971b4
Minor ansible-test code cleanup. (#59052)
* Minor ansible-test code cleanup.
* Fix type annotations.
2019-07-12 21:58:19 -07:00
Matt Clay
6645054329
Code cleanup for test infrastructure. (#59046)
* Remove useless object inheritance in test code.
* Remove unnecessary pass statements from test code.
* Comment on why certain pylint rules are ignored.
* Add more pylint test contexts.
* Fix import order.
* Remove unused variables.
* Remove unnecessary pass statement.
* Fix bad continuations.
* Remove unnecessary elif.
* Allow legitimate broad except statements.
* Allow legitimate protected access.
* Clean up names to make pylint pass.
2019-07-12 13:17:20 -07:00
Matt Clay
3f2b766d10 Add future and metaclass boilerplate to test code.
Continue to ignore:

- test/integration/
- test/legacy/
- test/units/
2019-07-12 09:18:24 -07:00
Gonéri Le Bouder
31baab85f9 ansible-test: vcenter through a HTTP proxy
Give the ansible-test user the ability to target a lab behind a HTTP proxy.
2019-07-12 10:56:36 +02:00
Matt Clay
6327a6114b Add ansible-test units --num-workers option. 2019-07-11 17:15:16 -07:00
Matt Clay
43c3b9fb0e Simplify ansible-test cwd setup. 2019-07-11 17:15:16 -07:00
Matt Clay
bc3a599678 Fix invalid type hints in ansible-test. 2019-07-11 14:25:57 -07:00
Matt Clay
86ad4c99ba Reorganize util code in ansible-test.
Code in util.py that depends on CommonConfig is now in util_common.py.
2019-07-11 08:54:13 -07:00
Toshio Kuratomi
33d2728879 Rename python files in hacking/ directory to have .py suffix
ansible-test only passes files which have the .py suffix for sanity
tests on python files.  This change will allow sanity tests to run on
the Python files in hacking/

* Rename test-module to test-module.py
* Symlink test-module for backwards compat since end users may be using
  test-module
* Fix test-module sanity errors that are now triggered
* Rename ansible_profile to ansible-profile.py
* Rename build-ansible
2019-07-10 22:17:35 -07:00
Matt Clay
c79afb1cba Move ansible-test unit coverage code into plugin. 2019-07-10 10:09:46 -07:00
Matt Clay
520af5cb47 Sanity test fixes for future collections support. 2019-07-10 07:03:48 -07:00
Matt Clay
1e1463401d
Prepare ansible-test for supporting collections. (#58886)
This is a small but incomplete set of the initial changes for supporting testing of collections with ansible-test.
2019-07-09 17:31:04 -07:00
Sam Doran
b9dafdbade
Move integration tests to using RHEL 8.0 GA (#57772) 2019-07-09 17:45:06 -04:00