Commit graph

424 commits

Author SHA1 Message Date
Brian Coca
2296da95a2
fix module schema (#74295)
* fix module schema

  not out of date hardcoded list anymore, uses 'current + 4' to
  set valid deprecation targets

Co-authored-by: Matt Clay <matt@mystile.com>
2021-04-20 10:02:39 -04:00
Matt Clay
7786dbbdc1 Split out module/module_utils unit test execution.
- Unit tests for `modules` and `module_utils` are now limited to importing only `ansible.module_utils` from the `ansible` module.
- Unit tests other than `modules` and `module_utils` are now run only on Python versions supported by the controller (Python 3.8+).
- Unit tests are now run in separate contexts (`controller`, `modules`, `module_utils`), each using separate invocations of `pytest`.
2021-04-19 15:44:58 -07:00
Matt Clay
277a06f641
Clean up ansible-test cloud plugins. (#74322)
- Improve code reuse.
- Add missing type hints, fix existing ones and convert them to PEP 484 style.
- Add missing imports and clean up existing ones.
- Add missing docstrings and clean up existing ones.
2021-04-16 18:49:22 -07:00
Brian Coca
07939b04f3
[2.12] 'attributes' (#73707)
* wip 'attributes'

* added version added tests

* syntzx

* not bile

* correztlys merges

* moved desc to frag

* simpler as dict

* unused

* clog

* Update lib/ansible/utils/plugin_docs.py

Co-authored-by: Jacob Floyd <cognifloyd@gmail.com>

* unnoted

* added action plugins

* longer list

* add sttri schema

* huh?

* itsdict

* dictit

* yolo

* gnore for now

* moar attribs

* allow extras

* positive

* added loop, documented 'imports'

* support is now none/partial/full

* import_playbook is outside host loop

Co-authored-by: Jacob Floyd <cognifloyd@gmail.com>
2021-04-16 18:17:00 +02:00
Brian Coca
0a7670d1f7
deprecated include (#71262)
* deprecated include

  Update lib/ansible/modules/_include.py
  updated version numbers in schema check (real fix in separate PR)

Co-authored-by: flowerysong <junk+github@flowerysong.com>
Co-authored-by: Matt Clay <matt@mystile.com>
2021-04-15 14:19:04 -04:00
Rick Elrod
f38d03f6fb [ansible-test] nuke virtualenv-isolated.sh
Change:
- This isn't used in our tests, so the suggestion was to just remove
  it.

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-04-12 15:09:18 -07:00
Matt Clay
b752d07163 Overhaul ansible-test container management.
This brings ansible-test closer to being able to support split controller/remote testing.
2021-04-12 12:40:36 -07:00
Matt Clay
becf941673 Add PyPI proxy container for tests on Python 2.6. 2021-04-08 15:47:23 -07:00
Sam Doran
4df9ea86c2 ansible-test - change diff mathching regexp so prefixes are optional
The `a/` and `b/` prefixes can be disabled in the `git diff` output by setting
`diff.noprefix` to `true`. The output is still a valid diff, but `ansible-test` would
raise an exception since without the prefixes, it thought the diff line was invalid.
2021-04-07 12:50:39 -07:00
Sam Doran
7b79b69369
ansible-test - ensure trailing separator is added for connection unit test target (#74176)
For connection units tests, if the `__init__.py` file is modified, the trailing separator was
not added, making the target invalid.
2021-04-07 13:27:09 -04:00
Matt Clay
459ea5a4a4
Update test containers in ansible-test. (#74166)
* Update distro test containers to version 2.0.2.

* Update the default test containers.
2021-04-06 15:52:56 -07:00
Matt Clay
f436058fad Add ansible-test constraint for decorator. 2021-04-05 11:56:38 -07:00
Matt Clay
4e2da0f894 Add ansible-test six constraint for Python 2.6. 2021-04-01 14:02:46 -07:00
Matt Clay
b75844af45 Fix ansible-test coverage exporting.
ci_complete
ci_coverage
2021-03-30 14:30:15 -07:00
Baptiste Mille-Mathias
3e63eb8418 fix typo 2021-03-18 12:50:06 -07:00
Paul Belanger
9ec4e08534
Don't match passive_interface for validate-modules (#73880)
This is a follow up to:

  https://github.com/ansible/ansible/pull/73508

To avoid adding no_log statements to passive_interface args.

Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2021-03-13 08:28:57 -06:00
Felix Fontein
7c0af58449
validate-modules no_log test: extend search range, add more known false-positives (#73882)
* validate-modules no_log test: extend search range, add more known false-positives.

* Mark false-positives.
2021-03-13 08:28:33 -06:00
Mark Chappell
ed18fcac3b
Ensure unit test paths for connection and inventory plugins are based on the context (#73877) 2021-03-12 12:46:40 -08:00
Felix Fontein
b2015c98e2
validate-modules: make sure that options that potentially contain secret data have no_log set (#73508)
* Catch more potential errors (and increase false-positive rate).
* Flag some false-positives in lib/ansible/modules/ with no_log=False.

Co-authored-by: John Barker <john@johnrbarker.com>
2021-03-11 12:57:11 -05:00
Matt Clay
15064c7a42 Update resource_prefix syntax for ansible-test. 2021-03-10 09:00:12 -08:00
Matt Clay
f40dde44f4 Remove old ansible-test windows completion entries. 2021-03-02 10:48:03 -08:00
Matt Clay
b6add939ff Remove outdated --docker and --remote entries. 2021-03-02 10:05:39 -08:00
Alexei Znamensky
920b68f5f2
Fixed/improved regular expresssion for collection names (#73577)
* added changelog fragment
* added a couple of tests to coll name validation
2021-03-01 10:40:38 -05:00
Felix Fontein
b6566d26df
pylint sanity test: stop ignoring 'used-before-assignment' (#73639)
* Remove pylint ignore 'used-before-assignment'.

* Adjust changelog fragment.
2021-02-18 12:36:28 -08:00
Felix Fontein
29aef842d7
ansible-test: make sure tests are also run for the new plugin types (#73599)
* Make sure tests are also run for the new plugin types.
* New plugin types are not documentable.
2021-02-13 14:05:16 -08:00
Matt Clay
d092356fc5 Fix ansible-test handling of egg-info.
Resolves https://github.com/ansible/ansible/issues/67990
2021-02-13 01:05:00 -08:00
Felix Fontein
1f3a90270b
Import sanity test for plugins (#72497) 2021-02-12 14:09:50 -08:00
Matt Clay
30a4ef4414 Update ansible-test default test containers. 2021-02-11 19:43:02 -08:00
Gonéri Le Bouder
0a8d5c0983 ansible-test: yamllint, check the assigment
Ensure `yamllint`'s `check_assignment()` correctly ignore the
attribute assignment. Those don't have any `.id` attribute and will
trigger an `AttributeError` exception.

See: https://github.com/ansible/ansible/pull/73322
2021-02-11 19:42:27 -08:00
Matt Clay
f533d46572
Fix ansible-test constraints issues. (#73578)
* Silence Python 3.5 EOL notice in pip.
* Skip import test on compat/selinux.py utils.
* Improve Python version skip warning.
* Use Python 3.6 as minimum Python for sanity tests.
* Improve min Python handling for code-smell tests.
* Overhaul test-constraints sanity test.
* Merge sanity test constraints with requirements.
* Remove legacy content specific constraints.
* Add changelog fragment.
2021-02-11 16:45:53 -08:00
Matt Clay
8a175f59c9
Upgrade pylint used by ansible-test. (#70155)
* Upgrade pylint and deps in ansible-test.
* Enable pylint on Python 3.9.
* Update pylint config.
* Add ignore for vendored six.
* Add ignores for support plugins.
* Fix issue reported by pylint.
2021-02-10 15:52:41 -08:00
Matt Clay
1a2da990a4
Temporary fix for cryptography issues. (#73530) 2021-02-08 14:35:00 -08:00
Felix Fontein
82cdd7e735 Bump antsibull-changelog version for changelog sanity test. 2021-02-04 16:42:26 -08:00
Florian Heiderich
8f887b7adb
add self-signed https endpoint for ansible-test (#73463)
* introduce self-signed.ansible.http.tests

* forwarding of port 444

* forward port 8444 to port 444 on http test container

* Fix port forwarding for Windows under docker

* add changelog fragment

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2021-02-05 05:12:22 +10:00
Gonéri Le Bouder
7cf80f50d1
validate_modules: fails with .id attribute not found (#73322)
* validate_modules: fails with .id attribute not found

This patch addresses a problem in the `found_try_except_import` test.

This module tries to identify lines like:

`HAS_FOO = True`

In this case, the target (`HAS_FOO`) is of type `ast.Name` and has a
`id` attribute which provide the name.

In my case, I've a line that set a module attribute`. In this case, the
target (`module.var`) has the type `ast.Attribute` and no `id`
attribute. The code trigger an `AttributeError` exception.

This patch ensures we compare a `ast.Name`.

* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py
2021-02-02 13:29:36 -08:00
Matt Clay
a9b5bebab3
Overhaul ansible-test SSH key management. (#73451)
* Pass remote.sh to shell over stdin.
* Pass docker.sh to shell over stdin.
* Standardize SSH key management.
* Update docker containers.
2021-02-02 08:43:54 -08:00
Rick Elrod
bce01655cc
[ansible-test] Bump CentOS 6 image (yum repos) (#73446)
Change:
- Bump centos6 image version to one which includes multiple fallbacks
  for vault.centos.org content.

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-01 16:24:21 -06:00
Matt Clay
81cd8e46f2 Update ansible-test distro containers to 1.29.0. 2021-02-01 09:24:28 -08:00
Matt Clay
c7cb944315 Always use python exec wrapper in ansible-test. 2021-01-28 15:59:55 -08:00
Matt Clay
902c402b6f Update ansible-test default containers. 2021-01-28 09:58:54 -08:00
Matt Clay
ded2d0b90e Use sh for ansible-test raw remote shell. 2021-01-28 09:00:55 -08:00
Matt Clay
b1344a1847 Avoid sphinx usage in changelog sanity test. 2021-01-27 23:52:13 -08:00
Matt Clay
418b9b781d No longer warn about missing pip/python pair.
This warning was intended to help improve test environment configuration.
However, it has ended up producing mostly non-actionable warning noise instead.

In most situations a missing pip or python should result in test failures.
Reporting a missing pip also implies that it should be used by tests, which is not the case.
Tests should be invoking pip as a python module with the appropriate python interpreter instead.
2021-01-27 23:51:48 -08:00
Matt Clay
672941fb0d
Add test support for Python 3.8 on RHEL and FreeBSD. (#73393)
* Support Python 3.8 on RHEL 8.2+ in ansible-test.

* Support Python 3.8 on FreeBSD in ansible-test.

* Use libyaml with PyYAML on FreeBSD.
2021-01-27 21:35:29 -08:00
Matt Clay
1b157ef42f
Limit rstcheck sanity test to ansible-core. (#73391) 2021-01-27 21:06:00 -08:00
Sviatoslav Sydorenko
595413d113
Replace the inhouse collection dependency resolver with resolvelib
PR #72591

This change:

  * Adds an artifacts manager that abstracts away extracting the
    metadata from artifacts, downloading and caching them in a
    temporary location.

  * Adds `resolvelib` to direct ansible-core dependencies[0].

  * Implements a `resolvelib`-based dependency resolver for
    `collection` subcommands that replaces the legacy
    in-house code.

    This is a dependency resolution library that pip 20.3+ uses
    by default. It's now integrated for use for the collection
    dependency resolution in ansible-galaxy CLI.

  * Refactors of the `ansible-galaxy collection` CLI.
    In particular, it:

      - reimplements most of the `download`, `install`, `list` and
        `verify` subcommands from scratch;

      - reuses helper bits previously moved out into external modules;

      - replaces the old in-house resolver with a more clear
        implementation based on the resolvelib library[0][1][2].

  * Adds a multi Galaxy API proxy layer that abstracts accessing the
    version and dependencies via API or local artifacts manager.

  * Makes `GalaxyAPI` instances sortable.

  * Adds string representation methods to `GalaxyAPI`.

  * Adds dev representation to `GalaxyAPI`.

  * Removes unnecessary integration and unit tests.

  * Aligns the tests with the new expectations.

  * Adds more tests, integration ones in particular.

[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing

Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>

Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
2021-01-27 22:23:22 +01:00
Matt Clay
13d08d232c Rename ansible-test functions to match constants. 2021-01-26 14:24:16 -08:00
Matt Clay
73fadc5e97
Clean up code in ansible-test. (#73379)
* Relocate code to fix type dependencies.

* Fix missing and unused imports.

* Fix type hints.

* Suppress PyCharm false positives.

* Avoid shadowing `file` built-in.

* Use json.JSONEncoder directly instead of super().

This matches the recommended usage and avoids a PyCharm warning.

* Remove redundant regex escape.

* Remove redundant find_python call.

* Use tarfile.open directly.

* Add changelog fragment.
2021-01-26 14:02:08 -08:00
Matt Clay
fc590aeb21 Use a versioned pip bootstrapper in ansible-test. 2021-01-25 12:31:11 -08:00
Matt Clay
484e4af4d0 Handle get-pip.py breaking change on Python 2.7. 2021-01-23 11:45:53 -08:00