Commit graph

121 commits

Author SHA1 Message Date
Matt Clay
4f51b7d323 Remove ansible-test --tox delegation option.
Use the --venv option instead.

This option was only available when running from source to test the ansible/ansible repository.

This will have no effect on testing collections or running from an installed version of Ansible.

Also update docs to reference the --venv option instead of the --tox option.
2019-11-14 17:18:00 -08:00
Matt Martz
b93d92ef9a
Set alter_sys=True instead of False to address backwards incompat (#64670)
* Set alter_sys=True instead of False to address backwards incompat

* ci_complete

* Add integration test

* ci_complete

* sanity

* ci_complete

* Changelog fragment

* Update import test and validate-modules to match
2019-11-11 16:47:35 -06:00
Matt Clay
75646037dc Switch from RHEL 8.0 & 8.1b to 8.1 in CI. 2019-11-07 13:17:49 -08:00
Matt Clay
adcf9458f1
Fix ansible-test collections sanity import test. (#64467)
* Fix ansible-test collections sanity import test.

Resolves https://github.com/ansible/ansible/issues/64466

* Fix get_source implementation also.

* Fix is_package function.
2019-11-05 16:06:57 -08:00
Felix Fontein
f4ff45dcd4 ansible-test: bumping ACME test container version (#64424)
* Bumping ACME test container version.

* Add changelog.
2019-11-05 10:51:22 -05:00
Matt Clay
e9f8a34dce
Fixes for validate-modules import handling. (#63932)
* Fix validate-modules support for collections.

- Relative imports now work correctly.
- The collection loader is now used.
- Modules are invoked as `__main__`.

* Remove obsolete validate-modules code ignores.

* Handle sys.exit in validate-modules.

* Add check for AnsibleModule initialization.

* Remove `missing-module-utils-import` check.

This check does not support relative imports or collections.

Instead of trying to overhaul the test, we can rely on the `ansible-module-not-initialized` test instead.

* Fix badly named error codes with `c#` in the name.

The `#` conflicts with comments in the sanity test ignore files.

* Add changelog entries.
2019-10-30 09:48:21 -07:00
Matt Clay
92ccdeac31 Fix various import sanity test issues.
- Relative imports are now properly recognized.
- Correct script invocation of Ansible modules is used.
- Warnings are now consistently reported as errors.
- Errors are now consistently reported with the file tested.

Resolves https://github.com/ansible/ansible/issues/62723
Resolves https://github.com/ansible/ansible/issues/61884
2019-10-22 23:00:35 -07:00
Felix Fontein
04252cf90d Improve validation of module return values (#63411)
* Add contains: validation for return values.

* Only require returned: on top level.

* Fix various return value problems.

* Update ignore.txt.

* Two more.
2019-10-18 16:01:52 +01:00
Matt Clay
92b387aa27 Fix hcloud requirements.
Avoid installing hcloud on Python 3.9 since it is not supported.
2019-10-17 17:40:57 -07:00
Matt Clay
6be4741f72 Fix ansible-test env var mixing running commands. 2019-10-17 17:40:34 -07:00
Matt Clay
437e9b7063 Fix ansible-test sanity requirements install.
This fixes ansible-test so it no longer tries to install sanity test dependencies on unsupported Python versions.
2019-10-17 17:40:04 -07:00
Matt Clay
7448084858 Update default-test-container to 1.10.1.
This brings in the final Python 3.8.0 release instead of a release candidate.
2019-10-16 09:07:09 -07:00
Matt Clay
32b57d57a0 Add constraint for pathspec.
This fixes installation of pathspec on Python 2.6.
2019-10-15 14:39:29 -07:00
Felix Fontein
054285c34c crypto modules: improve return value list documentation (#62929)
* Improve return value documentation by allowing entry for return values.
* Add docs formatting, adjust styling.
* Fix sample return value. (Taken from https://tools.ietf.org/html/rfc7517#appendix-A.1.)
* Work around abuse of .
2019-10-11 13:59:15 -05:00
Gonéri Le Bouder
73febd4ea6 prepare_vmware_tests: use module_defaults (#63209) 2019-10-11 19:46:54 +05:30
Matt Martz
03b98f6351 Fix plain format output functionality (#63107) 2019-10-10 15:46:36 -04:00
Matt Davis
6d52bdf4db
fix default collection resolution in adhoc (#63298)
* fix default collection resolution in adhoc

* if an adhoc command is run with a playbook-dir under a configured collection, default collection resolution is used to resolve unqualified module/action names

* Set ANSIBLE_PLAYBOOK_DIR in integration tests.

* Fix config conflict in ansible integration test.

* add adhoc default collection test

* text-ify warning string
2019-10-10 12:42:03 -07:00
Matt Clay
811127d64d Add RHEL 8.1b to the Shippable test matrix. 2019-10-08 13:06:28 -07:00
Pedro Magalhães
67d9cc45bd maven_artifact.py - add support for version ranges by using spec (#54309) (#61813) 2019-10-08 15:24:50 +01:00
Matt Clay
022335669c
Work around ssh-keygen issue in ansible-test. (#63211)
Newer versions of ssh-keygen create PEM keys that are not recognized by Paramiko.

Now ansible-test compensates for this by updating they keys it generates so Paramiko will recognize them.
2019-10-07 14:36:05 -07:00
Matt Clay
4c79f1ec4d Fix ansible-test integration temp dir location.
Previously the temporary directory used to run integration tests resided under the user's home directory. This prevented ansible-playbook from detecting the default collection when running tests.

Now the temporary directory is created within the collection to facilitate default collection detection.
2019-10-07 13:12:59 -07:00
Matt Clay
32979430d0 Add missing var for format string in ansible-test. 2019-10-07 13:10:42 -07:00
Matt Clay
b91f452f4f Fix ansible-test virtualenv real python search.
ansible-test now properly searches for `pythonX.Y` instead of `python` when looking for the real python that created a `virtualenv`.
2019-10-03 17:12:33 -07:00
Gonéri Le Bouder
2f5886f4ef vmware/test: add vsphere-automation-sdk dep
The following modules depend on `vSphere Automation SDK`:

- `vmware_rest_client`
- `vmware_guest_info`
- `vmware_tag_manager`
- `vmware_vm_inventory`

The associated test cannot be run with `govcsim`. But the situation is
changing since we will soon run them on a regular lab, and so, we
need to install the dependency.

Bumping the default-test-container version to 1.9.3 to get a fresh version
of pip and requests.

Depends-On: https://github.com/ansible/ansible/pull/62412
2019-10-02 14:57:23 -04:00
Matt Clay
3f2380ccce Exclude tmp dir from ansible-test results copy.
This fixes test errors related to failures copying temporary test results files from a remote system back to the local system.

It also speeds up processing of test results and reduces network utilization by avoiding the temporary files.
2019-09-30 18:18:37 -04:00
Matt Clay
831e1bf2e0 Fix ansible-test PYTHONPATH handling.
Running from an installed version of ansible-test now results in tests using a dedicated directory for PYTHONPATH instead of using the site-packages directory where ansible is installed.

This provides consistency with tests running from source, which already used a dedicated directory.

Resolves https://github.com/ansible/ansible/issues/62716
2019-09-26 14:02:19 -04:00
Gonéri Le Bouder
5ecbe9cbbb vcenter provider: Only rely on VMWARE_TEST_PLATFORM
Until now, the vcenter provider was switching between `static` and
`govcsim` depending on the presence of the following configuration file:
`test/integration/cloud-config-vcenter.ini`.

This was not consistent with Worldstream, which we enable with the
`VMWARE_TEST_PLATFORM` environment variable.

We now only rely on `VMWARE_TEST_PLATFORM` to know which platform should be
used. `govcsim` is still the default, this to preserve the original
behaviour.

This commit also rename the following variables to be consistent with the rest
of the code base. It also ensures they are alway defined, even with `govcsim`:

- `VCENTER_HOSTNAME`
- `VCENTER_USERNAME`
- `VCENTER_PASSWORD`
2019-09-20 10:33:44 -04:00
Lukas Kämmerling
fd7e156ccd Add hcloud_floating_ip module (#62409) 2019-09-18 11:26:53 +02:00
Gonéri Le Bouder
b68f5b406a test: bump default-test-container
VMware VSphere SDK needs an up to date version of `pip` for the
installation step. With the current image, we face the following error:

```
(...)
02:27 Collecting git+https://github.com/vmware/vsphere-automation-sdk-python.git (from -r /root/ansible/test/lib/ansible_test/_data/requirements/integration.cloud.vcenter.txt (line 2))
02:27   Cloning https://github.com/vmware/vsphere-automation-sdk-python.git to /tmp/pip-req-build-pm27t16b
02:33 Requirement already satisfied: pyvmomi in /usr/local/lib/python3.6/dist-packages (from -r /root/ansible/test/lib/ansible_test/_data/requirements/integration.cloud.vcenter.txt (line 1)) (6.7.1.2018.12)
02:33 Requirement already satisfied: lxml>=4.3.0 in /usr/local/lib/python3.6/dist-packages (from vSphere-Automation-SDK==1.4.0->-r /root/ansible/test/lib/ansible_test/_data/requirements/integration.cloud.vcenter.txt (line 2)) (4.4.0)
02:33 Processing ./\\localhost/tmp/pip-req-build-pm27t16b/lib/vapi-runtime/vapi_runtime-2.12.0-py2.py3-none-any.whl
02:33 Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: '/root/ansible/\\\\localhost/tmp/pip-req-build-pm27t16b/lib/vapi-runtime/vapi_runtime-2.12.0-py2.py3-none-any.whl'
```

Bump default-test-container to 1.9.3 to get an up to date release of
`pip` (was 19.0.2, is now 19.2.3).
2019-09-17 17:35:57 -04:00
Matt Clay
9f7b124a6f Mark ansible-test cloud credentials as sensitive. 2019-09-17 13:08:14 -07:00
Matt Clay
0631e057e9 Redact sensitive values by default in ansible-test 2019-09-17 08:09:17 -07:00
Felix Fontein
d8d3790d6a Bump acme-test-container version. (#62381) 2019-09-17 13:06:33 +02:00
Matt Clay
b73e7721df Mark AWS credentials in ansible-test as sensitive.
This avoids displaying the credentials in CI when retrying tests at maximum verbosity.
2019-09-16 20:21:23 -07:00
Matt Clay
32d965e2c6 Fix display of sanity test doc links.
The documentation links are now displayed when running from an install.

Previously the links were only displayed when running from source.
This was due to ansible-test checking for the presence of documentation files locally, which are only present when running from source.
The check is no longer necessary since there is a sanity test in place to enforce the presence of documentation for all sanity tests.
2019-09-13 22:24:14 -07:00
flowerysong
d933278f3d Add version 2.14 to the removed_in list (#62298)
devel is now 2.10, so this needs to be bumped as well.
2019-09-13 14:32:11 -05:00
Sam Doran
08279c7d54
Update default test container with Python 3.8b4 (#62100) 2019-09-13 12:38:47 -04:00
Evgeni Golov
48a39f5616 typo: missing-subption-docs → missing-suboption-docs (#62180)
updates the name of the test and all ignore.txt entries
2019-09-12 14:51:20 -05:00
Matt Clay
cd4882e229 Fix ansible-test vcenter test filtering.
The default behavior of the ansible-test vcenter plugin is to use the govcsim container to run tests.

However, unless the govcsim mode was specified using the VMWARE_TEST_PLATFORM environment variable, the filter code would skip the tests unless the tests ran on Shippable or the user had an ansible-core-ci key.

Now the filter correctly recognizes that govcsim is the default.
2019-09-12 02:13:57 -07:00
Matt Clay
cdc4926340
Fix ansible-test collections requirements installation. (#62181)
* Fix location of unit test requirements.

* Preserve ansible-test unit test requirements.

* Remove redundant unit test requirements.

* Fix location of network test requirements.

* Preserve ansible-test network test requirements.

* Remove redundant network test requirements.

* Add missing ordereddict requirements.

* Load collection requirements correctly.

* Add changelog fragment.
2019-09-12 02:00:33 -07:00
Matt Clay
aaa6d2ecb0
Fix ansible-test pytest plugin loading. (#62119)
* Avoid assertion rewriting in pytest plugins.

Adding PYTEST_DONT_REWRITE to the ansible-test pytest plugin docstrings disables assertion rewriting in pytest for those plugins.

This avoids warnings during test execution if the plugins are loaded multiple times (such as being imported within tests).

* Run ansible-test pytest plugins early.

The ansible-test pytest plugins need to load and run earlier than conftest modules.

To facilitate this, the pytest_configure function is run during loading, which works since they are loaded (but not always run) before conftest modules are loaded.

A check has also been added to the pytest_configure functions to prevent them from running multiple times in the same process.

* Load pytest plugins using an env var.

The -p command line option loads plugins before conftest, but only during collection.
The PYTEST_PLUGINS environment variable loads plugins before confest, both during collection and test execution.
2019-09-10 23:27:05 -07:00
Jordan Borean
6fb1d56fdc
Fix ansible-test coverage --all (#62115) 2019-09-11 15:12:38 +10:00
Matt Clay
a7bc11ce67
Work around virtualenv/venv issue in ansible-test. (#62111)
Creating a virtual environment using `venv` when running in a virtual environment created by `virtualenv` results in a copy of the original virtual environment instead of creation of a new one.

To work around this, `ansible-test` now identifies when it is running in a `virtualenv` created virtual environment and uses the real Python interpreter to create the `venv` virtual environment.
2019-09-10 19:27:12 -07:00
Matt Clay
983f0ad5a0 Properly exclude tests/output/ from code coverage. 2019-09-10 15:50:22 -07:00
Matt Clay
f110abb806
Fix ansible-test to ignore tests/output/. (#62084)
The `test/results/` directory for Ansible test output was already ignored when not using git.

When Ansible Collections were switched to `tests/output/` the ignore entry was previously overlooked.
2019-09-10 15:49:40 -07:00
Gonéri Le Bouder
0f52b18f3f
bump hcloud version to 1.4.1 (#62097)
* bump hcloud version to 1.4.1

`hcloud`<=1.4.0 has requirement `requests==2.20.0`. This prevents the
installation of the Vcenter Automation SDK which depends on `requests>=2.22.0`.

`hcloud` 1.4.1 does not have the problem: 8bff356efb
Bumping the dependency will resolve the issue.
2019-09-10 16:49:23 -04:00
Matt Clay
064e8e1ef4
Fix ansible-doc traceback and sanity test. (#62040)
* Fix ansible-doc traceback for removed modules.

This avoids tracebacks with errors like the following when a module has been removed:

module module_name missing documentation (or could not parse documentation): 'NoneType' object does not support item assignment

* Fix ansible-doc sanity test warning handling.

Warnings about removed modules/plugins on stderr are now properly ignored.

Previously an ansible-doc error could result in unrelated errors going undetected because tests were stopped early and the underlying error was ignored.
2019-09-10 00:29:34 -07:00
Matt Clay
c77ab11051
Fix issues with ansible-test --venv option. (#62033)
* Fix ansible-test venv activation.

When using the ansible-test --venv option, an execv wrapper for each python interpreter is now used instead of a symbolic link.

* Fix ansible-test execv wrapper generation.

Use the currently running Python interpreter for the shebang in the execv wrapper instead of the selected interpreter.

This allows the wrapper to work when the selected interpreter is a script instead of a binary.

* Fix ansible-test sanity requirements install.

When running sanity tests on multiple Python versions, install requirements for all versions used instead of only the default version.

* Fix ansible-test --venv when installed.

When running ansible-test from an install, the --venv delegation option needs to make sure the ansible-test code is available in the created virtual environment.

Exposing system site packages does not work because the virtual environment may be for a different Python version than the one on which ansible-test is installed.
2019-09-09 18:32:29 -07:00
Gonéri Le Bouder
8e22a60f97 ansible-test: vcenter+Worldstream: hide password
Hide the temporary password when `ansible-test` is called with the
`--redact` parameter.
2019-09-03 10:34:34 -04:00
Sam Doran
3d78dad84b Update Fedora 29 test image 2019-08-30 19:49:40 -04:00
Sam Doran
e218c9814c ansible-test - Continue if the git command returns an error (#61605)
* ansible-test - Contiune if the git command returns an error

* Just return stdout

* Use to_text() when displaying exception

* Add a message property to SubprocessError
2019-08-30 12:03:39 -07:00