Commit graph

427 commits

Author SHA1 Message Date
Matt Clay
e07c4f41d7 Standardize ansible-test sanity test filters. 2019-07-30 15:31:33 -07:00
Matt Clay
1351962eee Move "code smell" sanity test config loading. 2019-07-30 12:07:33 -07:00
Matt Clay
e1c2891188 Add paths_to_dirs function to ansible-test. 2019-07-30 10:46:39 -07:00
Matt Clay
53af57b08b Fix type discrepancy in ansible-test. 2019-07-30 10:46:05 -07:00
Matt Clay
5d7cc993dd
Clean up sanity test ignore handling. (#59749)
* Allow sanity tests to easily ignore themselves.

Useful for simple regex based sanity tests that error on their own script due to the regex or error message generated.

* Simplify no-smart-quotes sanity test.

Pruning of unversioned directories is handled by ansible-test.

* Remove directory pruning from boilerplate tests.

Files not passing these tests should be ignored instead.

* Add missing sanity ignore entries.
2019-07-29 14:37:44 -07:00
Adam Miller
e9d7156f53 podman-docker docker compat for ansible-test (#59539)
* podman-docker docker compat for ansible-test

Signed-off-by: Adam Miller <admiller@redhat.com>

* remove reprs, use ex.stderr instead

Signed-off-by: Adam Miller <admiller@redhat.com>

* remove u''s ... not needed

Signed-off-by: Adam Miller <admiller@redhat.com>

* Update test/runner/lib/docker_util.py

Co-Authored-By: Matt Clay <matt@mystile.com>

* Update test/runner/lib/docker_util.py

Co-Authored-By: Matt Clay <matt@mystile.com>

* make sanity tests happy

Signed-off-by: Adam Miller <admiller@redhat.com>
2019-07-29 17:34:45 -04:00
Matt Clay
66654475e1 Update ansible-test Python version handling.
Minimum version requirements for sanity tests have been standardized:

- All single version sanity tests now require Python 3.5 or later.
- All multiple version sanity tests continue to use all supported Python versions.
- All version neutral sanity tests continue to work on any supported Python version.

Previously some tests required 3.5 or later with most of the remaining tests requiring 2.7 or later.

When using the `--python` option to specify a Python version:

- Tests which do not support the specified Python version will be skipped with a warning.
- If the specified Python version is not available, any test attempting to use it will generate an error.

When not using the `--python` option to specify a Python version:

- Multiple version tests will attempt to run on all supported versions.
- Single version tests will use the current version if supported and available, or if no supported version is available.
- Single version tests will use the lowest available and supported version if the current version is not supported.
- Any versions which are not available or supported will be skipped with a warning.

Unit tests automatically skip unavailable Python versions unless `--python` was used to specify a version.
2019-07-29 11:54:03 -07:00
Matt Clay
2f6de3aea4 Add missing decorator to ansible-test sanity.
This fixes sanity test errors so they show the proper error code instead of "bound method".
2019-07-26 09:04:37 -07:00
Matt Clay
cf9999692a Fix ansible-test encoding issues.
This fixes `ansible-test sanity --list-tests` which was failing when `LC_ALL=C` was set.
2019-07-25 23:29:42 -07:00
Matt Clay
db7e9aa051
Use consolidated ignore.txt in ansible-test. (#59618) 2019-07-25 23:29:23 -07:00
Matt Clay
ba90263c22 Fix ansible-test cloud testing for collections. 2019-07-24 00:17:11 -07:00
Matt Clay
3c68a2e5f0 Ignore deleted when querying git in ansible-test. 2019-07-23 23:36:08 -07:00
Matt Clay
51083537fa Fix incorrect collections coverage warning. 2019-07-23 22:06:17 -07:00
Matt Clay
7c728118e6
Fix import sanity test for collections. (#59484) 2019-07-23 18:10:24 -07:00
Matt Clay
d65d8d6e13 Fix default pylint config for collections. 2019-07-23 17:14:10 -07:00
Matt Clay
44a262714c
Add common sanity ignore & skip to ansible-test. (#59416)
* Add common sanity ignore & skip to ansible-test.
* Switch ansible-doc sanity test to single version.
* Remove duplicate ignore entry.
2019-07-23 13:31:54 -07:00
Matt Clay
79eca9c8fb
Initial ansible-test support for collections. (#59197)
* Initial ansible-test support for collections.
* Include cloud config in delegation payload.
* Add missing types import and fix `t` shadowing.
* Fix plugin traceback when config_path not set.
* Fix encoding issues.
* Remove unused imports.
* More encoding fixes.
* Handle delegation outside exception handler.
* Inject ssh keys only if not already in place.
* More defensive approach to getting remote pwd.
* Add missing string format var.
* Correct PowerShell require regex.
* Rename `is_install` and `INSTALL_ROOT`.
2019-07-22 19:24:48 -07:00
Gonéri Le Bouder
e9b1e36d76 vcenter provider: avoid a TypeError exception
Avoid `TypeError: option values must be strings` with `ConfigParser`.
Default values must be string, not `None`.

The error happens when `test/integration/cloud-config-vcenter.ini` does
not have the `vmware_proxy_host` or `vmware_proxy_port` keys defined.
2019-07-22 11:19:39 +02:00
Matt Clay
b378e885ed Make ansible-test TestMessage hashable. 2019-07-19 21:34:04 -07:00
Matt Clay
4f1d0f1b39 Skip using paths for ansible-doc sanity test. 2019-07-19 16:17:56 -07:00
Matt Clay
cb0dfc9793 Prepare pylint test for collections support. 2019-07-16 11:24:16 -07:00
Matt Clay
2ed78b650f Update to ansible-test to prepare for collections.
Changes to simplify merging of collections support.
2019-07-16 08:04:07 -07:00
Matt Clay
4b9391f32a Clean up ansible-test target handling. 2019-07-15 20:33:23 -07:00
Matt Clay
4255ee8e5a Prepare ansible-test for collections support.
Another round of changes to prepare ansible-test for supporting collections to help keep later PRs a manageable size.
2019-07-15 17:46:29 -07:00
Matt Clay
f81238012b Clean up pylint issues in test infra. 2019-07-15 14:45:30 -07:00
Matt Clay
d910c971b4
Minor ansible-test code cleanup. (#59052)
* Minor ansible-test code cleanup.
* Fix type annotations.
2019-07-12 21:58:19 -07:00
Matt Clay
6645054329
Code cleanup for test infrastructure. (#59046)
* Remove useless object inheritance in test code.
* Remove unnecessary pass statements from test code.
* Comment on why certain pylint rules are ignored.
* Add more pylint test contexts.
* Fix import order.
* Remove unused variables.
* Remove unnecessary pass statement.
* Fix bad continuations.
* Remove unnecessary elif.
* Allow legitimate broad except statements.
* Allow legitimate protected access.
* Clean up names to make pylint pass.
2019-07-12 13:17:20 -07:00
Matt Clay
3f2b766d10 Add future and metaclass boilerplate to test code.
Continue to ignore:

- test/integration/
- test/legacy/
- test/units/
2019-07-12 09:18:24 -07:00
Gonéri Le Bouder
31baab85f9 ansible-test: vcenter through a HTTP proxy
Give the ansible-test user the ability to target a lab behind a HTTP proxy.
2019-07-12 10:56:36 +02:00
Matt Clay
6327a6114b Add ansible-test units --num-workers option. 2019-07-11 17:15:16 -07:00
Matt Clay
43c3b9fb0e Simplify ansible-test cwd setup. 2019-07-11 17:15:16 -07:00
Matt Clay
bc3a599678 Fix invalid type hints in ansible-test. 2019-07-11 14:25:57 -07:00
Matt Clay
86ad4c99ba Reorganize util code in ansible-test.
Code in util.py that depends on CommonConfig is now in util_common.py.
2019-07-11 08:54:13 -07:00
Matt Clay
c79afb1cba Move ansible-test unit coverage code into plugin. 2019-07-10 10:09:46 -07:00
Matt Clay
520af5cb47 Sanity test fixes for future collections support. 2019-07-10 07:03:48 -07:00
Matt Clay
1e1463401d
Prepare ansible-test for supporting collections. (#58886)
This is a small but incomplete set of the initial changes for supporting testing of collections with ansible-test.
2019-07-09 17:31:04 -07:00
Andrea Tartaglia
da1a621bd9 Removed nested role for ec2_instance tests
Also moved environment setup and environment cleanup to separate task
files

Added CI group

Updated c4.large instances to t3.nano, use ENA enabled AMI

Updated AMIs with Amazon Linux AMI

Updated us-east-1 AMI id

Specify EBS size

typo, volume_size

Removed 'state: running', updated AMI with amz ami

Also added several wait: false to speed things up

Check for instance state, correctly

Accept pending as valid state

tags_and_vpc_settings do not wait for instance to complete

wait for termination protected instance

Updated IAM role name to match with policy

Skip env cleanup when in CI

Do not wait instance in external_resource_attach

ENI remove is done in env_cleanup/CI

wait some time for instance_profile instnace to be up

Updated ebs_optimized_images

Corrected task name

Added aws_cleanup var in cloud-config-aws

typo in runme

default aws_cleanup to true
2019-07-02 17:48:07 -04:00
Gonéri Le Bouder
2401154fc4 VMware: rename VMWare to VMware (#58535)
Follow the nomenclature convention decided in https://github.com/ansible/ansible/pull/56814
2019-06-30 19:30:42 +05:30
Sam Doran
8cfd0ba039 Allow for test groups with double digits (#58532) 2019-06-28 16:24:14 -07:00
Gonéri Le Bouder
e28bc46353 vcenter provider: set default values
Set default value for the following optional parameters:

- vcenter_port
- vmware_proxy_host
- vmware_proxy_port

This way we won't get an error if they are not define in the
configuration file.
2019-06-28 13:13:30 -04:00
Matt Clay
4403f866e3 Simplify Git class in ansible-test. 2019-06-27 18:01:02 -07:00
Matt Clay
b06f5634c5 Simplify ansible-test target expansion.
Targets are always expanded to full lists now instead of optimizing
for shorter lists by collapsing directories.

This change only affects unit tests and the ansible-doc sanity test,
as they were the only remaining tests using the old behavior.
2019-06-27 08:27:48 -07:00
Matt Clay
535e646a66 Add test for invalid ignore entries. 2019-06-24 13:13:32 -07:00
Felix Fontein
b78d23d4ad Use newer test container. 2019-06-24 12:32:57 -07:00
Matt Clay
37ea47c848 Fix python injection when using a virtualenv. 2019-06-14 14:27:07 -07:00
Jill R
7ce85ee9e8
vcenter test provider handle config vars handling (#57813)
Make vcenter test provider handle config file vars the same regardless of
whether it comes from static config or from worldstream provisioning
2019-06-13 16:48:35 -07:00
Felix Fontein
c50a75a41e
acme_*: new test container, tests for IP certificates (#57530)
* Bump container version.

* Use new URLs.

* Add IP certificate tests.

* Disable IPv6 IP from IP certificate.

Background: in CI, the ACME test container apparently has no IPv6 support.
Without IPv6 support, Pebble can't connect to validate, and thus validation
fails.
2019-06-08 14:02:44 +02:00
Matt Clay
817ec5ab48 Add PyYAML check for libyaml support. 2019-06-05 15:20:47 -07:00
Matt Clay
d758d5bab1 Disable code-smell tests on Python 2.6. 2019-06-05 15:20:47 -07:00
Matt Clay
490edbdc48 Add python version checking when using --tox. 2019-06-05 15:20:47 -07:00