Commit graph

304 commits

Author SHA1 Message Date
Matt Clay
356ea0b8d0
[stable-2.10] Fix ansible-test coverage traceback. (#71446) (#71578)
* [stable-2.10] Fix ansible-test coverage traceback. (#71446)

* Add integration test for ansible-test coverage.

* Fix ansible-test coverage traceback.

* Fix coverage reporting on Python 2.6.
(cherry picked from commit f5b6df14ab)

Co-authored-by: Matt Clay <mclay@redhat.com>

* Add empty ignore.txt file for tests.
2020-09-01 17:05:19 -07:00
Matt Clay
d699d38dd5 [stable-2.10] Update ansible-test remote endpoint handling. (#71413)
* Request ansible-core-ci resources by provider.
* Remove obsolete us-east-2 CI endpoint.
* Add new --remote-endpoint option.
* Add warning for --remote-aws-region option.
* Update service endpoints.
* Allow non-standard remote stages.
* Add changelog fragment.
(cherry picked from commit d099591964)

Co-authored-by: Matt Clay <mclay@redhat.com>
2020-09-01 14:54:17 -07:00
Felix Fontein
e33fff922a
ansible-test: bump acme test container version to 2.0.0 (#71097) (#71165)
(cherry picked from commit 050841324c)
2020-08-27 12:58:06 -05:00
Toshio Kuratomi
7662568bd8
Antsibull build format backport 71080 (#71285)
* Updates to docs build for new antsibull CLI

* --ansible-base-cache renamed to --ansible-base-source
* _acd_version in the .deps file has been renamed to _ansible_version

(cherry picked from commit 22d2c97b81)

* Fix expr regex for MacOSX compat

MacOSX seems to want bare `+` whereas GNU expr wants escaped `+` (`\+`)
to mean match one or more.  Use `\{1,\}` instead which will match one or
more on both MaxOSX and GNU-using systems.

Fixes #71053

(cherry picked from commit 99cac0b135)

* Force an upgrade to a newer version of antsibull

(cherry picked from commit d816a5966e)
2020-08-27 12:53:29 -05:00
Felix Fontein
1252323e34
Make changelog tool be more strict about suffixes (#70909) (#71135)
* Bump antsibull-changelog version.

* Flag all dotfiles, except .keep and .gitkeep.

* Enable ignoring other fragment extensions.

(cherry picked from commit fbfc0f99eb)
2020-08-21 09:33:08 -07:00
Sandra McCann
8775001671
Add the ansible_2_10 intersphinx reference. (#71028) (#71384)
There are links from the scenario guides to collections but collections
docs aren't built in testing or locally when we're on the devel branch.
Due to that we need to make sure those references resolve to the
production docsite.  We can use intersphinx to make sure that happens.

* The test for binary files wasn't reading enough of the file.

Checking for null bytes in the first 1024 bytes failed to diagnose the
ansible_2_10.inv file as binary

(cherry picked from commit e10902d744)

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
2020-08-20 16:13:45 -05:00
Matt Clay
fa18566c3e [stable-2.10] Fix ansible-test self-test change classification.
Changes to sanity and unit tests now trigger the ansible-test self-test integration tests.

No changelog entry since this only affects tests for ansible itself and not collections.
(cherry picked from commit b53e7f8720)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-08-17 11:16:56 -07:00
Matt Davis
3b9ee86381 [stable-2.10] fix collection package root location under pytest >=6.0.0 (#70963)
(cherry picked from commit 86b6c4bbb6)

Co-authored-by: nitzmahone <nitzmahone@users.noreply.github.com>
2020-08-17 11:16:08 -07:00
Sandra McCann
b9a406f890
Docs [2.10] Backportapalooza 6 (#71129)
* Misc typo fixes (#71089)

(cherry picked from commit 504ef607f3)

* Add some documentation for the format of meta/runtime.yml (#71035)

* Document the format of meta/runtime.yml

* Document multiple Ansible versions

Clarify difference between deprecation and tombstone fields

* add note

(cherry picked from commit a9eb8b0488)

* add note to uninstall older versions of ansible for pip (#71023)

* add note to uninstall older versions of ansible for pip

* combine with the other PR

(cherry picked from commit 72d3d44163)

* VMware: Inventory scenario guide for hostnames (#71055)

Added a scenario guide for ``hostnames`` parameter
for vmware_vm_inventory.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit 0055673c70)

* Document string tests a bit more (#71049)

- Explain how `regex` differs from `match` and `search`.
- Document `multiline` and `ignorecase`.

Signed-off-by: Rick Elrod <rick@elrod.me>
(cherry picked from commit 701c638757)

* docs: Add a note about package requirements for fact gathering (#70796)

Fixes: #26148

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
(cherry picked from commit a6725d6e2a)

* added note about fakeroot (#71018)

see #70895

(cherry picked from commit 11a31e99e6)

* Update documentation of httpapi's handle_httperror method for clarity (#70991)

(cherry picked from commit a0523e5b8a)

* DOCS: add 2.10 collections roadmap (#70975)

* draft of 2.10 collections roadmap

* incorporates feedback from felixfontein

* gundalow and samccann feedback, fix link

Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
(cherry picked from commit 9879da8e23)

* updates changelog types; some updates for easier translation (#71027)

Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
(cherry picked from commit 4f4436c124)

* Document common return values with examples (#71046)

* adding return value examples
* shift to console code blocks
* cleaning up whitespace and shortening invocation example
* reordering diff section

(cherry picked from commit 864573a38d)

* Update intro_getting_started.rst (#71039)

Added two additional learning resources in the See also: section- forgot closing backticks

(cherry picked from commit 9850915bd6)

* Guide users to use ansible-runner (#71063)

Update the docs to guide users to use `ansible-runner` instead of using Python API directly. In many use cases, executing Ansible playbooks are sufficient. In those use cases, `ansible-runner` is easier and much stable to use comparing with Python API, but there is no mention of it.

(cherry picked from commit 0c855dc70b)

* Porting guides for ansible-base 2.10 and ansible 2.10 (#70891)

* Fix changelog link title.

* Rename Ansible 2.10 and 2.11 porting guides to Ansible-base porting guides.

* Add stub for automatically generated 2.10 porting guide.

* Move things that should not be in the ansible-base porting guide to the ansible porting guide.

* Apply changes to base porting guides.

* Add remark that ansible-base is mainly for developers.

* Ansible Base -> Ansible-base

* Fix link in base porting guide.

* Add generated porting guide.

* Use same header signs as antsibull-changelog's RST builder.

* Update generated porting guide.

(cherry picked from commit 61b36c6f30)

* Update network platform guides with FQCN (#70699)

* fqcn all the docs things!

(cherry picked from commit 54bee7152b)

* Document how to upgrade to ansible with pip (#70768)

Fixes #70348

(cherry picked from commit 5019335660)

* document how to migrate between collections (#70243)

* document how to migrate between collections
* Apply suggestions from code review

Co-authored-by: John R Barker <john@johnrbarker.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
(cherry picked from commit 58145dff9c)

* remove github link from plugins (#70951)

(cherry picked from commit e28b20d729)

* Add latest rc from ansible-base (#70974)

* Add latest rc from ansible-base

(cherry picked from commit d62dffafb3)

* Document to_json will convert to ASCII strings by default (#70954)

... as reported in issue #68702

(cherry picked from commit 8c48366f1c)

* Update the porting guide for ansible-2.10.0a8 (#71141)

(cherry picked from commit 0a9638ce4b)

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Nathaniel Case <ncase@redhat.com>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Co-authored-by: Terciero <terciero@users.noreply.github.com>
Co-authored-by: Brendon O'Sullivan <49501251+bjosullivan@users.noreply.github.com>
Co-authored-by: EthanHur <ethan0311@gmail.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Baptiste Mille-Mathias <baptiste.millemathias@gmail.com>
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
2020-08-07 16:04:19 -05:00
Matt Clay
b764d381f0
[stable-2.10] Fix ansible-test relative import analysis. (#70993)
(cherry picked from commit 2e0097ada3)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-07-30 13:36:49 -07:00
Felix Fontein
c39e536d84
validate-modules: fix version_added validation for top-level, fix error codes (#70869) (#70947)
* Also validate top-level version_added.

* Fix error code.

* Produce same version_added validation error in schema than in code (and stop returning it twice).

* Return correct error codes for invalid version_added for options and return values.

* Add changelog.

* Fix forgotten closing braket.

* Accept 'historical' for some top-level version_added.

(cherry picked from commit 7e2cc7db12)
2020-07-29 13:54:45 -07:00
Felix Fontein
155041d8ee
Fix ansible-test error in community.aws (#70507) (#70873)
* Fix ansible-test error in community.aws

* Add changelog entry for fix

* Change check from None to string_types

* Update changelogs/fragments/70507-validate-null-author.yaml

clarify wording "or a list of strings"

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/schema.py

clarify wording - single string or not specified valid

Co-authored-by: Felix Fontein <felix@fontein.de>

* Do not fail but return None when given outside list

Co-authored-by: Felix Fontein <felix@fontein.de>
(cherry picked from commit b0d9deeae3)

Co-authored-by: Alan Rominger <arominge@redhat.com>
2020-07-29 13:51:47 -07:00
Matt Clay
4280efccc4 [stable-2.10] Cap pytest version to avoid relative import issue.
(cherry picked from commit 3a8ac62596)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-07-28 17:14:30 -07:00
Rick Elrod
c53a68bf16
Update ansible-base test container to 1.5.0. (#70856) (#70857)
Co-authored-by: Matt Clay <mclay@redhat.com>
2020-07-23 21:36:41 -05:00
Matt Clay
c2b69fd30e [stable-2.10] Fix ansible-test virtualenv management.
(cherry picked from commit 1e02a201a6)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-07-14 02:08:17 -07:00
Matt Clay
ddc78622a4 [stable-2.10] Update ansible-test change detection for plugins.
(cherry picked from commit 09f02980a4)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-07-13 23:55:15 -07:00
Matt Clay
a707466c0b [stable-2.10] Fix yamllint sanity test line numbers.
Parse errors from libyaml now compensate for the offset of the documentation within a module.
(cherry picked from commit 1a0d8a51cd)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-07-13 18:28:30 -07:00
Matt Clay
73d9054b32 [stable-2.10] Test netconf plugins in ansible-doc sanity test.
(cherry picked from commit 8d92df4537)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-07-13 18:14:42 -07:00
Jordan Borean
70da336de9 ansible-test - do not validate blacklisted ps modules (#70376)
* ansible-test - do not validate blacklisted ps modules

* Update changelogs/fragments/validate-modules-ps-doc-blacklist.yaml

Co-authored-by: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 40ce448657)
2020-06-30 15:15:13 -07:00
Matt Clay
73ee85145b [stable-2.10] Pin ansible-test requirements for RHEL.
The `packaging` and `pyparsing` packages are now installed by `ansible-test` during provisioning of RHEL instances to match the downstream vendored versions.
(cherry picked from commit 70c59423fc)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-06-26 11:32:16 -07:00
Matt Clay
8152d8bc1a Disabled inconsistent pylint checks. 2020-06-17 10:10:17 -07:00
Monty Taylor
0133757d44 Add an author exception for OpenStack Ansible SIG
The current author line wants to match a github author id. But
some people, including the OpenStack project, do not use github,
and additionally do not claim individual ownership but instead
group ownership.

Since there are already a couple of hard-coded examples in the
regex, just add one more. Alternately we could come up with some
mechanism to indicate that the author is purposely not listing
a github id, but that seems a bit heavywight.
2020-06-17 08:21:14 -07:00
Matt Clay
598786e16d Update ansible-test default test containers.
The main change is the upgrade to Python 3.9.0b3.
2020-06-16 16:50:04 -07:00
Matt Clay
f69b1846f9 Add pyparsing constraint to ansible-test.
The upcoming pyparsing 3 release will require Python 3.5 or later, see:

https://github.com/pypa/packaging/issues/313

Unfortunately pip 8.x and earlier versions do not support python version requirements, which is why this constraint is needed.
2020-06-16 10:38:16 -07:00
Matt Clay
7641d32f8e
Fix ansible-test import sanity test issues. (#70084) 2020-06-15 15:09:15 -07:00
Felix Fontein
80410f292b Fix missing allow_date=False. 2020-06-15 11:18:51 -07:00
Felix Fontein
0617514176
Boolean error: fix deprecation date sanity check. (#70047) 2020-06-13 09:05:40 -04:00
Matt Clay
5b3719805e Update ansible-test test containers. 2020-06-12 09:19:18 -07:00
Felix Fontein
2fa65eb5e1
validate-modules: allow YAML dates in module documentation and meta/runtime.yml. (#70025) 2020-06-12 15:36:38 +01:00
John R Barker
37785255f2
validate ansible-base's and collections runtime.yml (#69742)
* Validate ansible-base & collection's runtime.yml

Add new test `runtime-metadata`

* Schema validation of file
* Error if a a legacy meta/routing.yml exist in a collection
* removal_date OR removal_version

* Add tombstone validation.

* Allow both ISO 8601 date strings and datetime.date objects (from YAML dates).

* Address review comments.

* Add metadata to test collection.

* Add requirements file.

Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Matt Clay <matt@mystile.com>
2020-06-12 15:34:34 +01:00
Matt Clay
9f49db1f99 Avoid use of deprecated junit-xml method. 2020-06-12 00:47:07 -07:00
Matt Clay
26f318d277 Ignore return code from antsibull_changelog lint. 2020-06-11 19:27:40 -07:00
Matt Clay
8ffaed00f8 Add Azure Pipelines support to ansible-test. 2020-06-11 14:57:42 -07:00
Matt Martz
4a4a11d282
Add mccabe complexity testing (#64623)
* Add mccabe complexity testing

* Make mccabe complexity an optional error

* Add mccabe to new sanity pylint requirements

* Add a changelog fragment.

Co-authored-by: Matt Clay <matt@mystile.com>
2020-06-11 13:02:06 -07:00
Matt Clay
2251b239a3 Fix changelog sanity test config detection. 2020-06-11 11:57:58 -07:00
James Cassell
47d14a33bd
config: singular ANSIBLE_COLLECTIONS_PATH (#70007)
* config: singular ANSIBLE_COLLECTIONS_PATH

Every other *_PATH setting in ansible is singular, and the traditional
$PATH variable is also singular despite containing a list of
directories.  Let's be consistent both internally and with POSIX
tradition.

* update all ANSIBLE_COLLECTIONS_PATHS env references to be singular

* deprecate plural ANSIBLE_COLLECTIONS_PATHS setting
2020-06-11 11:40:13 -07:00
Felix Fontein
a114da80ee
Use antsibull-changelog instead of packaged changelog generator (#69313)
Replace the ansible-base changelog linting and generation tool with antsibull-changelog and make it available for linting collections. Previously changelog linting was limited to ansible-base.
2020-06-11 11:21:21 -07:00
Gonéri Le Bouder
4869874337
ansible-test: vcenter initialize group/vmware (#70011)
Ensure the vcenter provider initialize the `module_defaults` of all
the vmware modules, not just `vmware_guest`.
The VMware CI relies on this for the authentication of the different
VMware modules.

The commit adjust `incidental_vmware_prepare_tests/tasks/init_vcsim.yml`.
The test-suite uses a copy of `vmware_guest` that is not in the
`group/vmware` group. As a result, we need to manually pass the
authentification parameter.
2020-06-11 12:21:44 -04:00
Matt Clay
6c2be04b6f Intercept code-smell tests using subprocess.
Some code-smell sanity tests for ansible-base use subprocess to invoke ansible commands.
Intercept these commands to make sure the correct script and python version are used.
2020-06-10 16:07:47 -07:00
Jordan Borean
a58fcde3a0
Use common ps sanity requirements file (#69992)
* Use common pssanity requirements file

* Fix up sanity ignore
2020-06-11 06:38:17 +10:00
Matt Clay
a5679d4dd1
Additional ansible-test sanity adjustments. (#69974)
* Fix ansible-test check_pyyaml usage.
* Avoid unnecessary pip commands.
* Avoid pointless pip check.
* Only install ansible-test requirements as needed.
2020-06-10 09:39:25 -07:00
Matt Clay
dc9c1d202b Fix ansible-test pip filter on Fedora 32.
The original version works on CentOS 8 but not Fedora 32.
2020-06-10 09:37:01 -07:00
Felix Fontein
f1ab7cf0c6
Validate-modules: adjust test to fixes in #69977 (#69978) 2020-06-10 18:02:01 +10:00
Jordan Borean
15f07e7a6c
Fix up schema for collection deprecation (#69977)
* Fix up schema for collection deprecation

* Fix up modules using wrong name

* Another fix
2020-06-10 16:33:08 +10:00
Matt Clay
a81dd4f06a
Update ansible-test default containers. (#69819)
* Move ansible-test completion code.

* Fix a few type hints.

* Change docker completion based on context.

Collections now use version 2.0 of the default-test-container.
This is an updated version of the earlier 1.x default-test-container with ansible-base and cloud specific requirements removed.

Testing of ansible-base now uses version 1.0 of a new ansible-base-test-container.
This container is similar to the earlier 1.x default-test-container, but with unnecessary duplication of requirements across Python versions removed.

Collections which have tests that depend on requirements no longer present in the default test container should specify them in their test requirements files:

* tests/integration/requirements.txt
* tests/unit/requirements.txt

* Bump test container versions

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2020-06-09 15:40:56 -07:00
Sloane Hertel
51f6d129cb
support hard coded module_defaults.yml groups for collections (#69919)
* Only allow groups which were hardcoded in module_defaults.yml

only load action groups from the collection if module_defaults contains a potential group for the action

* Fix tests using modules that override those whitelisted in lib/ansible/config/module_defaults.yml

Third party modules should not be using group/ - use the action name instead

* add externalized module_defaults tests

add the missing group and collections

ci_complete

Co-authored-by: Matt Davis <mrd@redhat.com>

* changelog

ci_complete

* Fix import in tests

ci_complete

* Update with requested changes

ci_complete

* don't traceback since we don't validate the contents of module_defaults

ci_complete

Co-authored-by: Matt Davis <mrd@redhat.com>
2020-06-09 15:38:57 -07:00
Felix Fontein
a862ff2d43
Deprecation revisited (#69926)
* Allow to specify collection_name separately for deprecation.

* Use new functionality in Ansible.

* Use new functionality in tests.

* Update tagging/untagging functions.

* Update pylint deprecated sanity test.

* Update validate-modules. Missing are basic checks for version_added (validate semantic version format for collections).

* Improve version validation. Re-add version_added validation.

* Make sure collection names are added to return docs before schema validation.

* Extra checks to avoid crashes on bad data.

* Make C# module utils code work, and update/extend tests.

* Add changelog fragment.

* Stop extracting collection name from potentially tagged versions/dates.

* Simplify C# code.

* Update Windows modules docs.

* Forgot semicolons.
2020-06-09 15:21:19 -07:00
Matt Clay
7bff3d312f
Split out and install sanity test requirements. (#69971)
* Split out sanity test requirements.

* Run each --venv test separately.

This provides verification that the requirements for each test are properly specified.

* Use a separate requirements file per sanity test.

* Skip setuptools/cryptography setup for sanity.

* Eliminate pyyaml missing warning.

* Eliminate more pip noise.

* Fix conflicting generate_pip_install commands.

* Add changelog fragment.
2020-06-09 13:38:36 -07:00
Sam Doran
c41a160951
Update test images to version 1.16.0 (#66771) 2020-06-09 10:08:31 -04:00
Felix Fontein
3e566768c1 Fix broken validate-modules. 2020-06-06 11:00:10 -07:00