Commit graph

2784 commits

Author SHA1 Message Date
Matt Clay
fb0d5609cb
Code cleanup in ansible-test. (#74529)
* Fix get_available_python_versions calls.
* Make run_playbook vars optional.
* Use ansible_pipelining in inventory.
* Fix type hint.
* Fix order of conditional evaluation.
* Remove unused ibmi platform.
* Add changelog fragment.
2021-04-30 18:50:00 -07:00
Maxim Masiutin
daecb30ac9
The apt_key module did not properly handle GnuPG errors for certain actions (#74478)
* Not all GnuPG return codes were analyzed (rc != 0) and not all relevant GnuPG error information was returned by the 'ansible.builtin.apt_key' module (https://github.com/ansible/ansible/issues/74477)

* Update changelogs/fragments/74478-apt_key-gpg-error-check.yaml

Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Sam Doran <sdoran@redhat.com>
2021-04-30 16:53:41 -04:00
Harm Geerts
9bd70045c9
Fix BaseFileCacheModule#keys to respect prefix (#72789)
* Fix BaseFileCacheModule#keys to respect prefix

Change:
- Previously BaseFileCacheModule#keys would return keys with the cache
  prefix. These keys are impossible to retrieve from the cache without
  removing the prefix or using the cache without a prefix.
  Now it removes the prefix from the key and only returns keys that
  share the same prefix as the cache.

Test Plan:
- New unit tests

* Add explicit BaseFileCacheModule#keys test

Test that files that do not match the cache prefix are ignored.
Test that the prefix is removed from the cache key.
2021-04-30 15:28:28 -04:00
Rick Elrod
d8c20a73a4
[ansible-test] Add Fedora 34, update Fedora 33 (#74506)
Change:
- Added Fedora 34 container image to docker.txt
- Dropped Fedora 32 from CI
- Added Fedora 34 to CI
- Updated Fedora 32 and 33 containers for ssh-related fixes
- Move cron fix from 698eae3f3d into
  cron_setup and make it more generic; it can affect modern distros too.

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-04-30 11:35:12 -05:00
Maxim Masiutin
6ea81b84ed
apt_key - correct error message about the failure to import a 'gpg' (#74476) 2021-04-30 12:06:38 -04:00
Matt Clay
a30c55f68a Remove deprecated common.removed module_util.
Tests have been updated to reflect its removal as well.
2021-04-29 16:07:56 -07:00
David Shrewsbury
f1a5c411d9
Fix sequence lookup message and add tests (#74472)
* add test for bad kv arg value
* add simple form parsing tests and make error messages the same
* add changelog
2021-04-29 15:12:23 -04:00
Sam Doran
74b2add460
paramiko_ssh - mark connection as connected when successful (#74459)
* Remove unused import from test
2021-04-29 15:11:02 -04:00
Brian Coca
8a2fc854f4
Add plugin config lists (#49627)
* add plugin config lists
* and dump config for plugins
*  also list configs under PLUGINS for 'all' list
2021-04-29 15:01:05 -04:00
Brian Coca
3cff54d69b
Fix missing delegate display (#74370)
* dont rely on vars, task already gives us info
* ensure we always display delegation in host label
* also added parens with ansible_host to show  target host vs resolved host
* delegating to self is not delegating
* delegated vars restoration for backwards compat
* tests need mock task with delegate_to
2021-04-28 13:24:38 -04:00
Martin Krizek
57d661e96f
Templar: remove _fail_on_{lookup,filter}_errors (#73785)
Both were added in 62d79568be and never
used, always being set to True.
2021-04-28 17:49:16 +02:00
Maxim Masiutin
3bc2e7783c
Binary GnuPG keys downloaded via URLs by the 'ansible.builtin.apt_key' module were corrupted so 'gpg' could not import them (https://github.com/ansible/ansible/issues/74424) (#74474) 2021-04-28 10:41:14 -04:00
Abhijeet Kasurde
9e61952cdc
command - add command name to warning (#74475) 2021-04-28 10:22:15 -04:00
Anatoly Pugachev
04e57d28e3
[facts] add hpux fc info, refactor aix and solaris code (#57406)
* correctly parse device from string
* check for command presence before running them
* check for command presence and return code for solaris and aix as well
* add changelog
2021-04-28 10:17:19 -04:00
Logistic Bot
7099657dd7
Basic: don't call lstat when check_mode (#64279)
While mode is specified in check_mode, don't call lstat.
Since file may not present.

Fixes: #61185
2021-04-28 13:47:03 +05:30
Matt Clay
877e5b3dfe Fix ansible-test AZP build lookup. 2021-04-27 18:32:46 -07:00
Matt Clay
c4e76a7f80 Add collection config support to ansible-test. 2021-04-27 12:09:34 -07:00
Matt Clay
8223dfccbe ansible-test - Improve __main__ checks. 2021-04-27 09:00:26 -07:00
Rick Elrod
fcbf5c3185
[module_utils] make to_bits actually return (#74409)
Change:
- Instead of returning the `str` type, return the value that was
  calculated.

Test Plan:
- New unit tests.

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-04-27 17:15:24 +02:00
Matt Clay
cb7f4f1971 Fix pypi-test-container port conflict. 2021-04-26 14:41:02 -07:00
Brian Coca
c1879a5011
use versioned doclink for url references (#74245)
* use versioned doclink for url references

Co-authored-by: Felix Fontein <felix@fontein.de>
2021-04-26 13:07:38 -04:00
Brian Coca
85e7108d52
force version on deprecation (#74338) 2021-04-26 13:02:00 -04:00
Brian Coca
d8fa2b50d5
Fix world readable setting (#74324)
* Fix world readable setting

   now uses shell option in all cases, the old constant still exists
    as 'documentation' but it not settable.
   also fix the docsite link in warnings/errors

* fix typose#

* fix comment#

* added deprecated to config

* missing :

* import cleanup
2021-04-26 12:49:59 -04:00
Alexander Sowitzki
68bdfd0052 Let unarchive handle huge files
Allow extracting archives that contain files which size exceeds free system memory.

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-04-26 14:06:10 +02:00
Matt Clay
ecc5a53288
Update default containers to 3.4.0. (#74415)
* Update default containers to 3.4.0.

The 3.4.0 containers use Python 3.6 (the system Python) for `/usr/bin/python3`.

Python 3.9 continues to be the default Python version selected by `ansible-test` for these containers.

* Fix shebang on build-ansible.py.

Using `python` instead of `python3` allows `ansible-test` python interception and requirements install to function.
2021-04-25 21:25:32 -07:00
Alexandre Garnier
8698855ffd
Case-insensitive set theory filters (#74256)
Fixes #74255

* Fix call to 'unique(case_sensitive=False)' triggering error when falling back to Ansible's version which **is** case-sensitive
* Test multiple situations of 'unique' filter errors with fallback not handling specific parameters

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
2021-04-23 12:44:43 -05:00
Fernando Correia
7fd989d008
Detect Homebrew on Mac M1 (Apple Silicon) (#74378)
Homebrew's default install location for macOS on ARM is /opt/homebrew.
Source: https://docs.brew.sh/FAQ

On a Mac M1 (Apple Silicon), homebrew will be installed at
/opt/homebrew/bin/brew.
2021-04-23 11:01:52 -04:00
Matt Clay
6cfe36fa1f
Set ansible-test min controller Python to 3.8. (#74395)
- Version neutral sanity tests now require Python 3.8 or later.
- Unit tests for controller-only code now require Python 3.8 or later.
2021-04-23 00:26:19 -07:00
Matt Clay
f541c0af23
Update ansible-test default test container. (#74393)
* Update ansible-test default test container.
* Switch generic Python tests from 3.6 to 3.9.
2021-04-22 23:23:16 -07:00
Paul Arthur
7b5dad2321 validate-modules: don't error on valid Ansible YAML in EXAMPLES 2021-04-22 12:42:47 -07:00
Matt Clay
14ff5e213c Fix ansible-test docker container IP detection. 2021-04-22 11:54:54 -07:00
Matt Martz
bf7d4ce260
Abstract away libyaml details into yaml utils (#72607)
* Add yaml utils file and use it

* Linting fix and missing import

* Abstract a few more details

* Parser imports

* Don't use CSafeDumper in AnsibleDumper

* Move and use convert_yaml_objects_to_native when libyaml is present

* yaml_load and yaml_dump, instead of safe_*

* re-use HAS_LIBYAML from utils.yaml

* add changelog fragment

* Address recent changes

* Use representer instead of recursive type converter

* Restore needed import

* move yaml utils to module_utils

* Properly guard imports

* Update from_yaml(_all)? to handle text wrappers with CSafeLoader

* Use yaml utils for legacy_collection_loader

* Add HAS_YAML, and ignore pylint issue

* oops

* GPL->BSD
2021-04-21 16:47:03 -05:00
Brian Coca
6418f368e3
Fix pb subs collapse (#74301)
* fix subdir parsing for plays

  fixes #74283

  Co-authored-by: Nikolaos Kakouros <nkak@kth.se>
2021-04-21 17:27:41 -04:00
Rick Elrod
738132da95
[ansible-test] add freebsd/13.0 remote (#74328)
Change:
- FreeBSD 13.0 was released on the 13th, add it to ansible-test.

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-04-21 15:13:41 -05:00
Sam Doran
7625c8f3d3
arg_spec - remove unused imports (#74354) 2021-04-21 11:38:23 -04:00
elara-leitstellentechnik
437a08eb6d
Map Debian 8 to Python 2 (#74152)
* Map Debian 8 to Python 2

If Python 3 is installed on Debian 8 Ansible cannot run, as the version
is too old (3.4)

* Add integration test for python interpreter discovery on Debian 8

* fix test issue on Debian 9, add changelog

* un"fix" not broken test :D

Co-authored-by: Fabian Klemp <fabian.klemp@elara-gmbh.de>
Co-authored-by: Matt Davis <mrd@redhat.com>
2021-04-20 15:28:24 -07:00
Matt Clay
e6af2d6827 Fix ansible-test coverage traceback when no data.
If no coverage directory exists, ansible-test coverage would traceback.

Now it silently continues just as if the directory was present but empty.
2021-04-20 14:33:05 -07:00
Amin Vakil
49d4442378
Add umask option to user module (#73821)
* Add umask option to user module

* Fail on setting both umask and local: True

* Add integration test

* Add changelog

* Run integration tests only if HOME_MODE is not set

* Run integration tests only on Linux

Co-authored-by: Matt Clay <matt@mystile.com>
2021-04-20 12:01:56 -04:00
Brian Coca
4494ef3a9d
defend against bad or missing crypt (#74304)
* defend against bad or missing crypt

  fixes #74279
2021-04-20 11:47:34 -04:00
Brian Coca
2296da95a2
fix module schema (#74295)
* fix module schema

  not out of date hardcoded list anymore, uses 'current + 4' to
  set valid deprecation targets

Co-authored-by: Matt Clay <matt@mystile.com>
2021-04-20 10:02:39 -04:00
Rick Elrod
9f1513d4c0 [become] Add Solaris chmod ACL fallback
Change:
- Newer Solaris drops setfacl. Add a fallback for its chmod ACL syntax.

Test Plan:
- New units

Tickets:
- Fixes #74282

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-04-19 20:28:57 -05:00
Rick Elrod
7d64aebdd3 [become] Fix solaris permissions regression
Change:
- Regression introduced in #70785
- When macOS chmod ACL syntax is used, Solaris-derived chmods return
  with a status of 5. This is also used for our sshpass handling,
  because sshpass will return 5 on auth failure. This means on Solaris,
  we incorrectly assume auth failure when we reach this branch of logic
  and try to run chmod with macOS syntax.
- We now wrap this specific use of chmod in an exception handler that
  looks for AnsibleAuthenticationFailure and skips over it. This adds
  another authentication attempt (something we normally avoid to prevent
  account lockout), but seems better than the regression of not allowing
  other fallbacks to be used.
- Without this patch, if setfacl fails on Solaris (and sshpass is used),
  we do not try common_remote_group or world-readable tmpdir fallbacks.

Test Plan:
- New unit

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-04-19 20:28:57 -05:00
Matt Clay
7786dbbdc1 Split out module/module_utils unit test execution.
- Unit tests for `modules` and `module_utils` are now limited to importing only `ansible.module_utils` from the `ansible` module.
- Unit tests other than `modules` and `module_utils` are now run only on Python versions supported by the controller (Python 3.8+).
- Unit tests are now run in separate contexts (`controller`, `modules`, `module_utils`), each using separate invocations of `pytest`.
2021-04-19 15:44:58 -07:00
Matt Martz
3740d7b028
Only check if play.hosts is a template when the play hasn't been finalized (#73941)
* Extend finalized logic to strategy
* Add changelog fragment
2021-04-19 13:03:40 -04:00
Ganesh Nalawade
d8bf4206e4
netconf - handle import error when running in FIPS mode (#73992)
* Handle netconf plugin ncclient import error when running in FIPS mode

*  While running in FIPS mode importing ncclient result in
   InternalError raised by cryptography
*  Refer https://github.com/ansible/ansible/pull/65477
2021-04-19 12:41:55 -04:00
Matt Clay
277a06f641
Clean up ansible-test cloud plugins. (#74322)
- Improve code reuse.
- Add missing type hints, fix existing ones and convert them to PEP 484 style.
- Add missing imports and clean up existing ones.
- Add missing docstrings and clean up existing ones.
2021-04-16 18:49:22 -07:00
Matt Martz
17237c1d88
Deprecate Py2.6 support for target nodes (#74165) 2021-04-16 11:42:29 -05:00
Brian Coca
07939b04f3
[2.12] 'attributes' (#73707)
* wip 'attributes'

* added version added tests

* syntzx

* not bile

* correztlys merges

* moved desc to frag

* simpler as dict

* unused

* clog

* Update lib/ansible/utils/plugin_docs.py

Co-authored-by: Jacob Floyd <cognifloyd@gmail.com>

* unnoted

* added action plugins

* longer list

* add sttri schema

* huh?

* itsdict

* dictit

* yolo

* gnore for now

* moar attribs

* allow extras

* positive

* added loop, documented 'imports'

* support is now none/partial/full

* import_playbook is outside host loop

Co-authored-by: Jacob Floyd <cognifloyd@gmail.com>
2021-04-16 18:17:00 +02:00
Alex Willmer
185d410316
Factor out host_label() in default stdout callback plugin (#73814)
This simplifies rendering the hostname (or hostname+delegated host) in
the default callback module, and reduces code duplication

I've chosen not move where in each handler the host label is rendered,
in case subsequent operations has side effects. However I'm happy to
change that if considered safe.

I've chosen not to change the formatting operator used (%), to avoid
changes in rendering that might result.

Signed-off-by: Alex Willmer <alex@moreati.org.uk>
2021-04-16 11:12:25 -04:00
Amin Vakil
a7c8093ce4
[2.12] Add accept_newhostkey option to git module (#73819)
* Add integration test for accept_newhostkey option

* Add changelog
2021-04-16 11:10:35 -04:00