Commit graph

2767 commits

Author SHA1 Message Date
Brian Coca
37cb550185 fix su localization optoin
fixes 
2021-03-11 11:53:28 -08:00
Brian Coca
8ef54759ec
moved vs deprecated world readable tmp setting () 2021-03-11 11:43:27 -08:00
Felix Fontein
b2015c98e2
validate-modules: make sure that options that potentially contain secret data have no_log set ()
* Catch more potential errors (and increase false-positive rate).
* Flag some false-positives in lib/ansible/modules/ with no_log=False.

Co-authored-by: John Barker <john@johnrbarker.com>
2021-03-11 12:57:11 -05:00
Matt Davis
3e1f6484d7
add optional module_utils import support ()
* add optional module_utils import support

Treat core and collections module_utils imports nested within any Python block statement (eg, `try`, `if`) as optional. This allows Ansible modules to implement runtime fallback behavior for missing module_utils (eg from a newer version of ansible-core), where previously, the module payload builder would always fail when unable to locate a module_util (regardless of any runtime behavior the module may implement).

* sanity test fixes

ci_complete
2021-03-10 11:08:13 -08:00
Matt Clay
15064c7a42 Update resource_prefix syntax for ansible-test. 2021-03-10 09:00:12 -08:00
Brian Coca
9db557e431
Nonfatal facts ()
continue with local facts vs at script error
 actually capture execution errors
 better error messages in general
 add more local facts tests

 fixes 
2021-03-08 16:20:37 -05:00
Abhijeet Kasurde
fb66b4ffbc
distribution: Add Amazon Linux distribution facts ()
* Update Amazon Linux Distribution facts gathering logic
* Update tests

Fixes: 

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-03-05 09:16:10 -05:00
David Shrewsbury
78d3810fdf
Auto cleanup of async cache file ()
* Auto cleanup of async cache file

* Add changelog
2021-03-04 11:06:27 -08:00
Brian Coca
42c3c51665
Set fact fixes ()
* fixes to set_fact

  correctly give error messages that were previously ignored
  corrected and expanded docs
2021-03-03 16:23:56 -05:00
Brian Coca
ce1de28061
Fix notify on import_tasks ()
* Fix notify on import_tasks

  also able to apply to blocks now.
2021-03-03 16:11:09 -05:00
Brian Coca
935528e22e
finish migrating ssh plugin to config system ()
* finish migrating ssh plugin to config system

  fixes 
  fixes 

* fix connection detection in reset
* correct options for connection meta reset

Co-authored-by: David Shrewsbury <Shrews@users.noreply.github.com>
2021-03-03 15:25:16 -05:00
Brian Coca
43300e2279
module output is only json objects ()
* module output is only json objects

 remove json lists as they are not valid from modules
 fixes 
2021-03-03 15:11:18 -05:00
Brian Coca
527bff6b79
Don't fail for mixed typed keys ()
* Don't fail for mixed typed keys

  but warn that content cound not be sorted because of this

* added tests
2021-03-03 14:24:50 -05:00
Brian Coca
2e0835b312
minor detection improvement on j2plugin loader ()
* minor  improvement on j2plugin loader, comments and var names clarified
2021-03-03 11:31:29 -05:00
Rick Elrod
24d41180ea
New release v2.11.0b1 () 2021-03-02 15:16:50 -06:00
Matt Clay
f40dde44f4 Remove old ansible-test windows completion entries. 2021-03-02 10:48:03 -08:00
Matt Clay
b6add939ff Remove outdated --docker and --remote entries. 2021-03-02 10:05:39 -08:00
Matt Martz
ba3f84883f
Reduce complexity of Templar._lookup slightly ()
* A little more complexity reduction
* restore logic
* Readability
* Add ran check back
* Add clog
2021-03-01 16:11:54 -05:00
Abhijeet Kasurde
6cb324bb0e
selinux: return selinux_getpolicytype facts ()
* selinux: return selinux_getpolicytype facts

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

* add basic selinux facts tests

* fix selinux facts test when selinux missing

Co-authored-by: Matt Davis <mrd@redhat.com>
2021-03-01 11:11:09 -08:00
ylmrx
e7e3c12ad2
Fixes for ansible-console usability ()
- Fix Ctrl+C/D behavior ()
 - correct few typos and missing infos
 - Fixes doc ()
2021-03-01 14:04:59 -05:00
Matt Davis
d0e991e892
Clarify CLI version number as core version ()
* clarify CLI version number as core version

* reduce confusion with `ansible` PyPI package >= 2.10 drifting from core version

* fix units
2021-03-01 11:14:56 -06:00
Alexei Znamensky
920b68f5f2
Fixed/improved regular expresssion for collection names ()
* added changelog fragment
* added a couple of tests to coll name validation
2021-03-01 10:40:38 -05:00
Alexander Sowitzki
e804fccf1c Let file module not change link to absolute when src not given ()
The file module changes existing sym links from relative to absolute
if the src is not stated in the tasks since it uses `os.path.realpath`
to fetch the link source and not `os.readlink`. Changed that.
2021-03-01 15:14:03 +01:00
Rick Elrod
17fabd8b45
[changelog config] Ansible Base -> ansible-core ()
Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-27 21:51:11 -06:00
Brian Coca
9690512069
Pipe it to connections ()
* pipelining tweaks
  added 'defaults'  entry for ini pipelining from ssh plugin
2021-02-25 15:08:11 -05:00
Sloane Hertel
8628c12f30
find module - stop traversing directories with os.walk when depth is already exceeded () 2021-02-26 05:32:49 +10:00
Matt Martz
950ab74758
Normalize ConfigParser between Python2 and Python3 ()
* Normalize config parser between py2 and py3

* Add tests and changelog

* Use different config entry, since we supply certain env vars
2021-02-25 11:03:03 -06:00
manas-init
eb72c36a71
galaxy: Handle ignored directory names in role skeleton ()
* galaxy: restore left hand slicing in assignment

Fix 'ansible-galaxy role init --role-skeleton=role-skeleton' when the role skeleton
contains an ignored directory.

The issue was because the 'dirs' variable was changed to reference a different list,
but needs to be mutated instead to stop os.walk from traversing ignored directories.

Fixes: 

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-02-24 18:52:24 -05:00
Brian Coca
dabfee4d5c
commented function () 2021-02-24 01:20:47 -05:00
Matt Martz
11f1177e6c
Operate pexpect with bytes to limit encoding issues ()
* Operate pexpect with bytes to limit encoding issues

* Update tests to ensure no pepxect encoding issues

* Add changelog fragment

* Add multiline note

* Use rst formatting directly
2021-02-23 18:57:25 +01:00
Rick Elrod
fa046d302c
[InventoryManager] Fix two unhandled exceptions ()
Change:
- Fix regression: unhandled exception when given inventory directory
  is empty or contains empty subdirectories.
- Fix unhandled exception when limit file is actually a directory
  instead of a file.
- Fix inventory tests which previously could never fail due to missing
  `set -e`. Fixed up tests that failed after `set -e` was added. Added
  several tests.

Test Plan:
- New tests
- Fixed existing tests which previously could never fail

Tickets:
- Fixes 

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-22 16:18:23 -05:00
David Shrewsbury
c8ee186e11
Remove warning about role validation with deps ()
* Remove warning about role validation with deps

* add changelog

* update cl
2021-02-18 17:16:03 -06:00
Felix Fontein
b6566d26df
pylint sanity test: stop ignoring 'used-before-assignment' ()
* Remove pylint ignore 'used-before-assignment'.

* Adjust changelog fragment.
2021-02-18 12:36:28 -08:00
Aaron Guise
ac0dedda72
hostname - add Almalinux support () 2021-02-18 10:24:36 -05:00
Sloane Hertel
00bd0b893d
ansible-galaxy - set the cache file after getting all collection versions ()
* Manage the in-memory cache in _call_galaxy but let the caller set the file cache after getting paginated results

* Add a test for caching successful and not caching unsuccessful paginated results

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-02-15 09:45:01 -05:00
Felix Fontein
29aef842d7
ansible-test: make sure tests are also run for the new plugin types ()
* Make sure tests are also run for the new plugin types.
* New plugin types are not documentable.
2021-02-13 14:05:16 -08:00
Matt Clay
d092356fc5 Fix ansible-test handling of egg-info.
Resolves https://github.com/ansible/ansible/issues/67990
2021-02-13 01:05:00 -08:00
Sam Doran
18f7282ccf
ansible-doc - account for empty meta/main.yml ()
* ansible-doc - account for empty meta/main.yml

 from_yaml() will return None when encrounting an empty file.
2021-02-12 17:37:00 -05:00
David Shrewsbury
d1d9406066
Add rolespec_validate to import/include_role ()
* Add rolespec_validate to import/include_role

* Add changelog

* fix sanity, not private
2021-02-12 16:26:33 -06:00
Felix Fontein
1f3a90270b
Import sanity test for plugins () 2021-02-12 14:09:50 -08:00
Felix Fontein
ff0edca8cc Catch more potential errors when setting up curses. 2021-02-12 13:09:02 -08:00
David Shrewsbury
f0ec10dbc3
Role arg spec validation implementation ()
* Initial import of modified version of alikins' code
* Add unit testing for new Role methods
* Fix validate_arg_spec module for sanity test. Add test_include_role_fails.yml integration test from orig PR.
* Add testing of suboptions
* Use new ArgumentSpecValidator class instead of AnsibleModule
* fix for roles with no tasks, use FQ name of new plugin
* Add role dep warning
2021-02-12 16:00:17 -05:00
Brian Coca
ea2f37d253
allow constructed to use vars plugin ()
Allow constructed to optionally use vars plugin data

* mostly for those looking to leverage group_vars/ and host_vars/
* limited to already processed sources
2021-02-12 11:14:50 -05:00
Matt Clay
30a4ef4414 Update ansible-test default test containers. 2021-02-11 19:43:02 -08:00
Gonéri Le Bouder
0a8d5c0983 ansible-test: yamllint, check the assigment
Ensure `yamllint`'s `check_assignment()` correctly ignore the
attribute assignment. Those don't have any `.id` attribute and will
trigger an `AttributeError` exception.

See: https://github.com/ansible/ansible/pull/73322
2021-02-11 19:42:27 -08:00
Matt Clay
f533d46572
Fix ansible-test constraints issues. ()
* Silence Python 3.5 EOL notice in pip.
* Skip import test on compat/selinux.py utils.
* Improve Python version skip warning.
* Use Python 3.6 as minimum Python for sanity tests.
* Improve min Python handling for code-smell tests.
* Overhaul test-constraints sanity test.
* Merge sanity test constraints with requirements.
* Remove legacy content specific constraints.
* Add changelog fragment.
2021-02-11 16:45:53 -08:00
Sam Doran
b6811dfb61
Add argument spec validator ()
Add argument spec validator class
2021-02-11 19:17:14 -05:00
Alexander Sowitzki
d0fda3e901
Let vault lookup output unicode string. ()
Until now, the lookup plugin returned a byte string.
Changed this to output a unicode string instead.
2021-02-11 14:27:47 -05:00
Brian Coca
ec8a556538
only add data when there is data to add ()
Only add data when there is data to add

  also avoid clobbering existing data with empty file
  fixes 

* remove redundant code, update comments
* fix mock dataloader, original does not return None
* added test
2021-02-11 10:07:20 -05:00
Matt Davis
4c5ce5a1a9
module compat for py3.8+ controller ()
* module compat for py3.8+ controller

* replaced internal usages of selinux bindings with internal ctypes binding (allows basic selinux operations from any Python interpreter), plus tests

* added new respawn_module API to allow modules to import Python packages that are only available under a well-known interpreter, plus tests

* added respawn logic to modules that need Python libs from a specific system interpreter (apt, apt_repository, dnf, yum)

minimize internal HAVE_SELINUX usage

spurious junk

pep8

* pylint fixes

* add RHEL8 Python 3.8 testing

* more pylint

* import sanity

* unit tests

* changelog update

* fix a bunch of stuff

* tweak changelog

* fix setup_rpm_repo on EL8

* misc sanity/test fixes

* misc feedback tweaks

* fix import fallback in test module

* fix selinux MU test

* fix dnf tests to avoid python-dependent test packages

* add trailing LFs to aliases

* fix yum tests to avoid test package with Python deps

* hack create_repo for EL6 to create noarch package
2021-02-10 21:32:59 -08:00
Matt Clay
8a175f59c9
Upgrade pylint used by ansible-test. ()
* Upgrade pylint and deps in ansible-test.
* Enable pylint on Python 3.9.
* Update pylint config.
* Add ignore for vendored six.
* Add ignores for support plugins.
* Fix issue reported by pylint.
2021-02-10 15:52:41 -08:00
Jeff Squyres
6fd0a74601
unarchive: add support for .tar.zst (zstd compression) () 2021-02-09 23:14:41 -06:00
pushkarkumar15
4344607d7d
user - add password expiration parameters ()
* Add integration test

Co-authored-by: Sam Doran <sdoran@redhat.com>
2021-02-09 16:41:15 -05:00
Sam Doran
2f5c83dfb1
Add AlmaLinux to the family of Red Hat-like operating systems ()
* Add changelog and fixtures for AlmaLinux support

Co-authored-by: Christoph Schug <com+github@schug.net>
2021-02-09 13:04:39 -05:00
Sam Doran
7d18ea5e93
default callback - add "show_task_path_on_failure" option ()
When running in verbosity <2, display the file and line number for tasks that fail. This provides
useful information without having to run at increased verbosity.

* Move _print_task_path to CallbackBase class
* Add integration tests
* Add color parameter to _print_task_path()

* Keep color output consistent for now
  Currently the path is display with COLOR_DEBUG formatting with verbosity >= 2.
  Instead of the color of the path changing based on verbosity level, just keep it at the
  currently behavior of COLOR_DEBUG. Having the color of the same information change
  based on verbosity level seems incorrect and makes visual parsing of the information
  more difficult.

Co-authored-by: tahar.jegham <jeghamseifeddine@gmail.com>
2021-02-09 12:43:59 -05:00
Felix Fontein
ca448f7c35
Update collection routing ()
* Docker content from community.general migrated to community.docker.

* Hetzner Robot content from community.general migrated to community.hrobot.

* RouterOS content from community.network migrated to community.routeros.

* PostgreSQL content from community.general migrated to community.postgresql.

* OC content from community.general migrated to community.okd.

* Cisco NSO content from community.network migrated to cisco.nso.

* Add changelog fragment.

* Google content from community.general migrated to community.google.

* Hashi Vault content from community.general migrated to community.hashi_vault.

* KubeVirt content from community.general migrated to community.kubevirt.

* FortiOS content from community.network migrated to community.fortios.

* Update changelog fragment.

* DellEMC content from community.general migrated to dellemc.openmanage.
2021-02-09 09:31:35 -08:00
Eugene Vilensky
65fc8c9e35
INTERPRETER_PYTHON_DISTRO_MAP: Treat oracle same as rhel/centos ()
* treat oracle same as rhel/centos

* fix misunderstood use of yaml anchors

* add changelog fragment

* Update changelogs/fragments/73498-INTERPRETER_PYTHON_DISTRO_MAP-Treat-oracle-same-as-rhel-centos.yml
2021-02-09 10:40:39 -05:00
Matt Martz
d3441bc313
Add new split filter ()
* Add version added
2021-02-09 10:37:15 -05:00
Matt Clay
1a2da990a4
Temporary fix for cryptography issues. () 2021-02-08 14:35:00 -08:00
Shane McDonald
67f5bb39c7
galaxy - add format options for collection list ()
* Include all collections in single json object / yaml document
* Add tests
* For galaxy list yaml/json output, use dictionary of dictionaries instead of list
* Add tests for listing single collection in yaml / output format
* --output -> --format
* Add explicit test for listing collection in human format
* Fix bug where empty json object was emitted + add test
2021-02-05 13:24:59 -05:00
Matt Martz
d3f3784b86
Don't treat host_pinned as lockstep ()
* Don't treat host_pinned as lockstep. Fixes 

* Add intg tests
2021-02-05 09:06:19 -06:00
Florian Heiderich
8d6136eab9
uri - add ca_path parameter ()
* add changelog fragment for  (ca_path for uri)
* add integration tests for ca_path in the uri module
* return path of ca cert instead of its content
* connect to port 444 on self_signed_host
  and use quay.io/ansible/http-test-container:1.3.0
* state that the certificate in ca_path is used for validation
2021-02-05 09:40:52 -05:00
Felix Fontein
82cdd7e735 Bump antsibull-changelog version for changelog sanity test. 2021-02-04 16:42:26 -08:00
jakegatsby
f271d02a9f
bugfix: use to_text(stdout) in psrp.Connection.put_file method ()
Co-authored-by: jakegatsby <jakegatsby@example.com>
2021-02-05 10:01:07 +10:00
Jordan Borean
0cdc410dce
no_log mask suboption fallback values and defaults CVE-2021-20228 ()
* no_log mask suboption fallback values and defaults

* Added changelog

* Remove lambda expression
2021-02-05 09:11:55 +10:00
Michael Shnit
4315e18807
Fix regression introduced in b77abd0491 causing bug in inventory modu… ()
* Fix regression introduced in b77abd0491 causing bug in inventory modules which break functionality in user setting  use_contrib_script_compatible_sanitization  parameter.

* Add changelog

Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
2021-02-04 17:25:08 -05:00
Florian Heiderich
8f887b7adb
add self-signed https endpoint for ansible-test ()
* introduce self-signed.ansible.http.tests

* forwarding of port 444

* forward port 8444 to port 444 on http test container

* Fix port forwarding for Windows under docker

* add changelog fragment

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2021-02-05 05:12:22 +10:00
Sloane Hertel
9759e0ca49
add option to upgrade collections ()
* Add a flag to ansible-galaxy to update collections


Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-02-04 07:08:02 +10:00
Gonéri Le Bouder
7cf80f50d1
validate_modules: fails with .id attribute not found ()
* validate_modules: fails with .id attribute not found

This patch addresses a problem in the `found_try_except_import` test.

This module tries to identify lines like:

`HAS_FOO = True`

In this case, the target (`HAS_FOO`) is of type `ast.Name` and has a
`id` attribute which provide the name.

In my case, I've a line that set a module attribute`. In this case, the
target (`module.var`) has the type `ast.Attribute` and no `id`
attribute. The code trigger an `AttributeError` exception.

This patch ensures we compare a `ast.Name`.

* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py
2021-02-02 13:29:36 -08:00
Jose Angel Munoz
69631da889
lineinfile - add search_string parameter for non-regexp searching ()
* Add tests for search_string
* Improve examples
* Add changelog
2021-02-02 15:37:06 -05:00
Rick Elrod
9a9272305a
Correct pkg_mgr for Fedora-derived OSTree distros ()
Change:
- Remove check that states that only Fedora can be an OSTree
  distribution.
- This allows us to correctly return "atomic_container" as the pkg_mgr
  fact for RHEL for Edge, Fedora/RHEL/CentOS Atomic Host, etc.

Test Plan:
- Created local RHEL for Edge image and tested against it.
- Tested against regular RHEL 8 and still got `dnf` as expected.
- Tested against RHEL 7 Atomic Host and got `atomic_container` now.
- New unit tests.

Tickets:
- Fixes 

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-02 15:09:30 -05:00
Alexander Sowitzki
1c83672532
Allow $ & ~ inside paths in galaxy roles ()
ansible-galaxy currently behaves bad then a role to be installed
contains ~ or $ at any place in the path of a file in that role.
It extracts the parent directory of the offending path level as an
empty file. This explodes if that directory contains anything else.

Change this behaviour. `~` is now allowed allowed when it is
not a full level (Yes: `some~thing/`, no: `~/`). The code should
get refactoring in an other PR.
2021-02-02 18:10:05 +01:00
Matt Clay
a9b5bebab3
Overhaul ansible-test SSH key management. ()
* Pass remote.sh to shell over stdin.
* Pass docker.sh to shell over stdin.
* Standardize SSH key management.
* Update docker containers.
2021-02-02 08:43:54 -08:00
Alexander Sowitzki
218f5c3648
user - Prevent user fact lookup failure if LOGNAME is set () ()
The `UserFactCollector` queries the user login name via
`getpass.getuser()` and looks up the corresponding entry
in the password database.
The login name may differ from the actual user name,
eg. if the `LOGNAME` env variable is set. The lookup
fails in this case. Added a fallback in this case that
tries to get the entry via the user ID.
2021-02-02 10:36:02 -05:00
Rick Elrod
bce01655cc
[ansible-test] Bump CentOS 6 image (yum repos) ()
Change:
- Bump centos6 image version to one which includes multiple fallbacks
  for vault.centos.org content.

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-01 16:24:21 -06:00
Sloane Hertel
840bdc1e10
Fix warning for nonexistent inventory cache ()
* Fix inventory cache warning by checking if the key exists before loading it
* changelog
2021-02-01 14:29:36 -05:00
Matt Clay
81cd8e46f2 Update ansible-test distro containers to 1.29.0. 2021-02-01 09:24:28 -08:00
Matt Clay
c7cb944315 Always use python exec wrapper in ansible-test. 2021-01-28 15:59:55 -08:00
Matt Martz
c986cbb996
Add Python 3.8 and Python 3.9 to the fallback list () 2021-01-28 11:14:45 -08:00
Matt Clay
902c402b6f Update ansible-test default containers. 2021-01-28 09:58:54 -08:00
Matt Clay
ded2d0b90e Use sh for ansible-test raw remote shell. 2021-01-28 09:00:55 -08:00
Matt Clay
b1344a1847 Avoid sphinx usage in changelog sanity test. 2021-01-27 23:52:13 -08:00
Matt Clay
418b9b781d No longer warn about missing pip/python pair.
This warning was intended to help improve test environment configuration.
However, it has ended up producing mostly non-actionable warning noise instead.

In most situations a missing pip or python should result in test failures.
Reporting a missing pip also implies that it should be used by tests, which is not the case.
Tests should be invoking pip as a python module with the appropriate python interpreter instead.
2021-01-27 23:51:48 -08:00
Matt Clay
672941fb0d
Add test support for Python 3.8 on RHEL and FreeBSD. ()
* Support Python 3.8 on RHEL 8.2+ in ansible-test.

* Support Python 3.8 on FreeBSD in ansible-test.

* Use libyaml with PyYAML on FreeBSD.
2021-01-27 21:35:29 -08:00
Matt Clay
1b157ef42f
Limit rstcheck sanity test to ansible-core. () 2021-01-27 21:06:00 -08:00
Brian Coca
5aa4295d74
change detection and check mode fixes for apt_key ()
Change detection and check mode fixes for apt_key

* allow apt-key to use proxies
* add note about deprecation of apt-key itself
* expanded error msgs
* show all keys
* fix short_format parsing
* added more return info and documented it
2021-01-27 18:40:58 -05:00
Sviatoslav Sydorenko
595413d113
Replace the inhouse collection dependency resolver with resolvelib
PR 

This change:

  * Adds an artifacts manager that abstracts away extracting the
    metadata from artifacts, downloading and caching them in a
    temporary location.

  * Adds `resolvelib` to direct ansible-core dependencies[0].

  * Implements a `resolvelib`-based dependency resolver for
    `collection` subcommands that replaces the legacy
    in-house code.

    This is a dependency resolution library that pip 20.3+ uses
    by default. It's now integrated for use for the collection
    dependency resolution in ansible-galaxy CLI.

  * Refactors of the `ansible-galaxy collection` CLI.
    In particular, it:

      - reimplements most of the `download`, `install`, `list` and
        `verify` subcommands from scratch;

      - reuses helper bits previously moved out into external modules;

      - replaces the old in-house resolver with a more clear
        implementation based on the resolvelib library[0][1][2].

  * Adds a multi Galaxy API proxy layer that abstracts accessing the
    version and dependencies via API or local artifacts manager.

  * Makes `GalaxyAPI` instances sortable.

  * Adds string representation methods to `GalaxyAPI`.

  * Adds dev representation to `GalaxyAPI`.

  * Removes unnecessary integration and unit tests.

  * Aligns the tests with the new expectations.

  * Adds more tests, integration ones in particular.

[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing

Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>

Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
2021-01-27 22:23:22 +01:00
Abhijeet Kasurde
82b74f7fd7
iptables: Fixed get_chain_policy API ()
While getting policy name in get_chain_policy API,
module does not require any additional parameters except chain
Enabling flag in get_chain_policy API call fixes this.

Fixes: 

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-01-27 15:24:53 -05:00
Brian Coca
c7d4acc12f
allow env to override unspecified unsafe_writes ()
* allow env var for fallback value for unspecified unsafe_writes
2021-01-27 14:16:10 -05:00
Brian Coca
2b0cd2c13f
use correct executable and options in all cases ()
Use correct ssh executable and options in all cases on connection plugin

  * Also nicer naming/comments

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-01-26 20:21:24 -05:00
Matt Clay
13d08d232c Rename ansible-test functions to match constants. 2021-01-26 14:24:16 -08:00
Matt Clay
73fadc5e97
Clean up code in ansible-test. ()
* Relocate code to fix type dependencies.

* Fix missing and unused imports.

* Fix type hints.

* Suppress PyCharm false positives.

* Avoid shadowing `file` built-in.

* Use json.JSONEncoder directly instead of super().

This matches the recommended usage and avoids a PyCharm warning.

* Remove redundant regex escape.

* Remove redundant find_python call.

* Use tarfile.open directly.

* Add changelog fragment.
2021-01-26 14:02:08 -08:00
Martin Krizek
a2af8432f3
Local vars should have highest precedence in AnsibleJ2Vars ()
Ability to add local variables into AnsibleJ2Vars was added in
18a9eff11f to fix . Local variables
are added using ``AnsibleJ2Vars.add_locals()`` method when creating a
new context - typically when including/importing a template with
context. For that use case local template variables created using
``set`` should override variables from higher contexts - either from the
play or any parent template, or both; Jinja behaves the same way.

Also removes AnsibleJ2Vars.extras instance variable which is not used.

Also adds missing test for .

Fixes 
Fixes 

ci_complete
2021-01-26 11:37:56 +01:00
Matt Clay
fc590aeb21 Use a versioned pip bootstrapper in ansible-test. 2021-01-25 12:31:11 -08:00
Matt Clay
484e4af4d0 Handle get-pip.py breaking change on Python 2.7. 2021-01-23 11:45:53 -08:00
Brian Coca
e0c9f285ff
undeprecated hash_merge setting ()
added updates from sloan's PR
 feedback from others

Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>

Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>
2021-01-22 15:00:19 -05:00
Alexander Sowitzki
e6da544310 Let command always return stdout & stderr ()
The `command` module does not return stdout & sterr when calling
a non existing executable or an unknown exception arises. This fix
lets the module return empty byte strings in those cases.
2021-01-22 08:40:53 +01:00
Rick Elrod
1934ca9a55
Add Ubuntu 20.04 to CI and ansible-test ()
Change:
- Add Ubuntu 20.04 to CI now that venv is default instead of virtualenv in ansible-test.

Test Plan:
- CI

Tickets:
- Fixes 

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-21 22:58:31 -06:00
Rick Elrod
4357a78130
[ansible-test] First attempt at freebsd/11.4 ()
Change:
- Try CI against freebsd 11.4

Test Plan:
- ci_complete

Tickets:
- Fixes 

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-21 17:12:18 -06:00
Alexander Sowitzki
fe792fdcd2
Vanquish rare container bootstrap failure ()
The init script for the test container writes additional lines to
the .bashrc of the user. This was done via a `cat` multiline
instruction, which is implemented internally by writing a
temporary file to TMPDIR (/tmp in this case) first. Docker fails
to provide /tmp just after creation, which results in a race
condition that rarely makes the init fail. Changed the `cat`
statement to multiple `echo`s.

Co-authored-by: Matt Clay <matt@mystile.com>
2021-01-21 12:22:58 -05:00
Joshua Bayfield
823c72bcb5
Shadow input for encrypt_string by default unless asked (fixes ) ()
* Shadow input for encrypt_string by default unless asked (fixes )
2021-01-20 15:50:24 -05:00
Jordan Borean
bc60d8ccda
Galaxy - make versions list consistent across versions ()
* Galaxy - make versions list consistent across versions

* Fix up unit tests
2021-01-20 21:19:29 +01:00
Oldřich Jedlička
ec55a221f6
Do not pretend expression is filename in compile() Python built-in call ()
When calling compile(), the filename argument should be either a real file
name or a string. According to Python docs, suggested one is '<string>'.
 Keep the current behaviour (encapsulate the actual expression), 
but enclose it into angle brackets.

Signed-off-by: Oldřich Jedlička <oldium.pro@gmail.com>
2021-01-19 12:20:26 -05:00
Sam Doran
e8d4b62b41
Fix YAML error message when error is at the end of the file ()
* Fix YAML error message when error is at the end of the file

If a YAML file fails to load due to a syntax error in a file, or there is an error in the last line of a
file, PyYAML reports the last line number of the file as the index where the error occurred.

When reading the file lines, we use that index to the get the relevant line.  If the index value is out
of range, the relevant line is lost for error reporting.

Subtract one from the index value to avoid the IndexError in this specific scenario. It is possible
to still get an IndexError, which will be handled as it is currently.

* Update existing tests and add new tests
2021-01-19 10:07:36 -05:00
Brian Coca
30d93995dd
Use a pty for local connections ()
* Use a pty for local connections

Fixes 

Co-authored-by: James Cammarata <jimi@sngx.net>
2021-01-18 16:02:04 -05:00
Brian Coca
1e27d4052a
fix inventory source parse error handling ()
fixes 

added test cases
2021-01-18 09:33:51 -05:00
Sandra McCann
9d71bbd13e
add changelog fragment for 73176 () 2021-01-15 16:26:53 -06:00
Brian Coca
1202dd000f
Allow restricting config values to enumerated list ()
* Allow restricting config values to enumerated list
* dont document internal entries
* also ignore private defs for ansible-config
* remove invalid value from tests
* added porting entry
2021-01-14 15:11:30 -05:00
Sam Doran
e2308ed6e3
ansible-galaxy - correct collections-path command line arg ()
It should be plural to match other options and environment variables.
2021-01-14 14:13:01 -05:00
Felix Fontein
df9cf368c0
Always mention the name of the deprecated plugin in routing deprecation messages () 2021-01-14 17:45:02 +01:00
Sam Doran
df451636e7
facts - properly report virtualization facts for Linux guests on bhyve () 2021-01-14 10:53:03 -05:00
Sam Doran
0e6c334115
pause - adjust warning when run in background ()
When the pause module is run in the background and seconds parameter is provided,
do not warn.

* Add tests
* Fix existing tests
  The test wasn't failing when it should have.
2021-01-14 09:35:39 -05:00
Rick Elrod
7f0eb7ad79
[facts] Differentiate CentOS vs CentOS Stream ()
Change:
- On CentOS Stream, make distribution_release be "Stream"
- On CentOS Core, it continues to be "Core"
- Implement custom distribution file parser for CentOS, so we can look
  for "CentOS Linux" and "CentOS Stream"
- Two new fixtures introduced (CentOS Linux 8.1 and CentOS Stream 8)
- Removed two dicts from `Distribution` class that were seemingly not
  used anywhere.

Test Plan:
- ci_complete
- New test fixtures

Tickets:
- Fixes 

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-13 17:54:04 -05:00
Matt Clay
4164cb26f0 Remove obsolete tower plugin from ansible-test. 2021-01-13 12:37:21 -08:00
Matt Clay
21f1811ddf
Cleanup provisioning code in ansible-test. ()
* Remove unused code in ansible-test.

* Remove obsolete endpoint logic from ansible-test.

* Remove obsolete region selection in ansible-test.

* Remove obsolete port logic in ansible-test.

* Clean up ansible-test remote providers.
2021-01-12 16:18:28 -08:00
Brian Coca
d6670da1d7
Added clog missing for issue 70722 () 2021-01-12 12:09:59 -05:00
Martin Zimmermann
be59caa59b
iptables: Added set module/match_set support () 2021-01-11 16:32:53 -05:00
Sam Doran
a7e834071c
Add macOS 11 to CI ()
* Fix connection_paramiko_ssh test for macOS 11
* Update Azure Pipelines config
* Add changelog
2021-01-11 12:20:47 -05:00
Sam Doran
264e08f21a
user - properly handle password and password lock when used together ()
Do the right thing on Linux when password lock and a password hash are provided by writing
out the password hash prepended by the appropriate lock string rather than using -U and -L.
This is the correct way to set and lock the account in one command.

On BSD, run separate commands as appropriate since locking and setting the password cannot 
be done in a single action.

FreeBSD requires running several commands to get the account in the desired state. As a result,
the rc, output, and error from all commands need to be combined and evaluated so an accurate
and complete summary can be given at the end of module execution.

* Improve integration tests to cover this scenario.
* Break up user integration tests into smaller files
* Properly lock account when creating a new account and password is supplied

* Simplify rc collection in FreeBSD class
  Since the _handle_lock() method was added, the rc would be set to None, which could make
  task change reporting incorrect. My first attempt to solve this used a set and was a bit too
  complicated. Simplify it my comparing the rc from _handle_lock() and the current value of rc.

* Improve the Linux password hash and locking behavior
  If password lock and hash are provided, set the hash and lock the account by using a password
  hash since -L cannot be used with -p.

* Ensure -U and -L are not combined with -p since they are mutually exclusive to usermod.

* Clarify password_lock behavior.
2021-01-11 11:42:35 -05:00
Matt Clay
05ba49cbba ansible-test - No virtualenv install on Python 3
Tests should use the Python built-in ``venv`` module on Python 3 instead of the standalone ``virtualenv`` module.

On Python 2 the ``virtualenv`` module continues to be the only option.
The version installed is either the OS packaged version or the last release to support Python 2, which is version 16.7.10.
2021-01-07 20:12:39 -08:00
Brian Coca
b1ee1a285a
Added NO_COLOR as config option () 2021-01-07 15:00:31 -05:00
Rick Elrod
44ee04bd1f
[dnf] Make "remove" filtering closer to dnf CLI ()
Change:
- Internally, use dnf.subject.Subject#get_best_query for state: absent
- Add a bunch of tests for removing packages, given a bunch of different
  pkg specs (nv, nvr, nvra, wildcard, etc.)

Test Plan:
- New tests
- Local experiments with DNF API via PDB.

Tickets:
- Fixes 

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-07 11:32:06 -06:00
Matt Clay
77942acefc Refactor ansible-test cryptography install code. 2021-01-07 09:04:33 -08:00
David Shrewsbury
c8666f07d7
Add support for argument_specs attribute to RoleMetadata class () 2021-01-07 11:41:46 -05:00
Abhijeet Kasurde
25792d2388
galaxy: handle plus sign in user token appearing in role url ()
Fixes: 

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-01-06 10:50:44 -05:00
Sviatoslav Sydorenko
5b4f0b3370 Declare support for Python 3.9 in the metadata
Co-authored-by: Rick Elrod <rick@elrod.me>
2021-01-05 20:48:08 -08:00
Rick Elrod
b96732fa6b [ansible-test] Add Fedora 33 support and drop F31
Change:
- Changes to make F33 work with current tests
- Add F33 to docker.txt
- Add F33 to AZP config

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-05 20:48:08 -08:00
Brian Coca
de3844cba9
discourage hexdigits in password lookup ()
* actually add new docs

* Update lib/ansible/plugins/lookup/password.py

Co-authored-by: Sandra McCann <samccann@redhat.com>
2021-01-05 15:03:27 -05:00
Lloyd Parkes
e0c75a6756
Replace the use of the function string.replace with the method str.replace ()
* Replace the use of the function string.replace() with the method str.replace()
because that's what works for both Python 2 and 3.

* Cleanup the unused string import.

Added a changelog fragment.

* The documentation for os.write() seems a bit iffy, but in Python 3 we
definitely cannot pass it a string and we need to encode it into
bytes.

The Python documentation at
https://docs.python.org/3/howto/pyporting.html#text-versus-binary-data
says that this code will work in Python 2 as well.

Co-authored-by: Lloyd Parkes <lloyd@must-have-coffee.gen.nz>
2021-01-05 11:20:44 -05:00
Rick Elrod
20509b6507
[facts] fix version facts on FreeBSD RC/PRERELEASE ()
Change:
- The FreeBSD release can contain -RC or -PRERELEASE in addition to
  -RELEASE, -STABLE, or -CURRENT.

Test Plan:
- Added new fixed from an RC version of TrueNAS which uses a -PRERELEASE
  version of FreeBSD.

Tickets:
- Fixes 

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-05 10:16:59 -05:00
Jean-Pierre Matsumoto
3074ede8a3
ExecutionPolicy default value is RemoteSigned ()
* ExecutionPolicy default value is RemoteSigned

##### SUMMARY
In Windows setup steps, settings `ExecutionPolicy` is restored to value `Restricted`. But actually default value for Windows server is not `Restricted` but `RemoteSigned` ([source](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_execution_policies?view=powershell-7.1#default)).

I have got issue with `win_dsc` because `ExecutionPolicy` was `Restricted`.

##### ISSUE TYPE
- Docs Pull Request

+label: docsite_pr

* Add changelog fragment

* Apply suggestions from code review

Co-authored-by: Sandra McCann <samccann@redhat.com>
2021-01-04 15:41:54 -05:00
pva
42bc03f0f5
[get_url] skip checksum during --check ()
Fix get_url failure in check mode (--check) when using the checksum
format <algorithm>:<url>.

Regression introduced in ()

Fixes: 
2020-12-22 21:34:42 +05:30
Abhijeet Kasurde
aa56a2ff6a
galaxy: Handle token as dict while loading using yaml ()
Malformed galaxy_token file creates stacktrace.

Fixes: 

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-12-21 14:53:00 -05:00
Sam Doooran
5ecfb19cad
arg_spec - move validator lookup method to a function ()
* arg_spec - move type checking lookup method to a function

* Change get_wanted_type name and behavior

Change the name to get_validator to bette describe what it is doing.

Change the interface to always return a value. This lines up with the behavior of get_*
functions always returning something or None and check_* functions raising an
Exception if something went wrong during the check.

* Add param to check_type_str()

Not meant to be a long term fix, but gets tests passing. More work is needed to figure
out how to solve this cleanly.

* Remove private attribute mapping types to validator

Since the function that needs it has moved to parameters.py, there is no need to have it as
a attribute of AnsibleModule.

Update tests that were referencing the private attribute.

* Use private method for 'str' type

To avoid having to put the string conversion warning behavior in the check_type_str() method,
use the private _check_type_str() method for 'str' type.

Import CHECK_ARGUMENT_TYPES_DISPATCHER for backwards compalitibility and store it as
a private attribute.

Revert changes to support plugins that are referencing serf._CHECK_ARGUMENT_TYPES_DISPATCHER.

* Add changelog

* Change function name to better reflect its... function

* Change dict name to better reflect its contents

CHECK_ARGUMENT_TYPES_DISPATCHER --> DEFAULT_TYPE_VALIDATORS

* Fix changelog
2020-12-17 17:15:00 -05:00
Sam Doooran
48803604cd
systemd - do not overwrite unit name when searching ()
* systemd - do not overwrite unit name when searching

PR  introduced a bug that changed the unit name when splitting it up for the purpose
of searching for the unit. This only happens on unit file templates on systems that have a 5.8
or newer kernel and a version of systemd that does not contain a bugfix that causes systmed
to fail to parse dbus.

* Use facts rather than a manual probe to determine if systmed is present

* Remov unnecessary block

* Use vars files instead of set_fact

* Add tests for using a templated unit file

* Update changelog fragment

* Use template to get correct path to sleep binary
2020-12-17 15:29:22 -05:00
Sam Doooran
8e022ef00a
import_playbook - change additional params to deprecation ()
* import_playbook - change additional params to deprecation

I incorrectly recommended this be set as a warning when it should have been a deprecation.

* Fix deprecation sanity test to not required a collection name when not inside a collection
2020-12-17 14:14:58 -05:00
Sam Doooran
850a77f639
ansible-test - prefer venv over virtualenv on Python 3 ()
Also pin virtualenv to 16.7.10 for older Mac OS X systems. This was the version being installed
anway with the previous constraint (<20).

On systems with Python 3, now prefer venv over virtualenv. Test to see if venv is functional since
some systems have a non-functional venv installation (such as Debian).
2020-12-17 10:51:42 -05:00
Sloane Hertel
5e03e322de
Pass the top level dictionaries to combine_vars ()
* Pass the top level dictionaries to combine_vars

combine_vars uses dict.update() to replace keys
2020-12-16 11:23:23 -05:00
Matt Clay
37d09f2488
Update ansible-test pylint Python support. ()
* Update ansible-test pylint Python support.

* Python 3.8 is now officially supported.
* Python 3.9 is now skipped with a warning.
2020-12-15 10:27:32 -08:00
Felix Fontein
0ba96d2be8
Compare removal and deprecation dates and versions in runtime metadata against current version and today ()
* Check deprecation and removal versions w.r.t. current version.

* Check removal dates from tombstones w.r.t. today.

* Add changelog.
2020-12-15 09:05:38 -05:00
Brian Coca
d22804c4fb
saner path dir management ()
* saner path dir management

   fixes 

   ensure we always store paths w/o a_c

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
2020-12-14 19:30:13 -05:00
Martin Zimmermann
83fb24b923
iptables: Adding multiport module support () 2020-12-14 10:23:54 -05:00
Adam Miller
0044091a05
Update yum/dnf module docs to include version comp ()
* Update yum/dnf module docs to include version comp

Fixes 

Signed-off-by: Adam Miller <admiller@redhat.com>

* fix up docs syntax to sanity tests are happy

Signed-off-by: Adam Miller <admiller@redhat.com>
2020-12-11 10:31:19 -05:00
Toshio Kuratomi
e7dee73774
Collection list site packages ()
* ansible-galaxy collection list and verify now utilize collections in site-packages.

This is a short term fix for .  The long term fix needs to handle
install (but that discussion is also bound up in how upgrade is going to
work and where things can get installed so it's deferred for 2.11.)

* Add test for ansible-galaxy collection list with site-packages

* Fix sanity issue

Co-authored-by: David Moreau Simard <moi@dmsimard.com>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
2020-12-11 09:59:33 +10:00
Matt Martz
a3b6485073
Fix reset_connection paramiko, winrm, psrp ()
* Ensure we only reset the connection when one has been previously established. Fixes 

* Ensure psrp doesn't trace

* winrm too

* Indentation fix
2020-12-09 12:57:56 -06:00
Matt Clay
36ab3d1189
Fix cryptography constraints in ansible-test. () 2020-12-09 10:46:05 -08:00
Matt Martz
83764ad506
Fix async interpreter parsing ()
* Fix async interpreter parsing. Fixes 

* Target localhost instead of remote host

* Don't forget inventory

* Address shellcheck issue
2020-12-09 11:41:45 -06:00
Martin Krizek
1b70260d5a
Add implicit role_complete block instead of role._eor ()
Co-authored-by: Matt Martz <matt@sivel.net>

Fixes 
2020-12-09 09:53:01 +01:00
Martin Krizek
252685092c
Use _wrap_native_text only for builtin STRING_TYPE_FILTERS () 2020-12-09 09:48:32 +01:00
Sven Wegener
08cc6edc64
vault: Read stdin data as binary on python3 ()
On python3 sys.stdin is an encoded file object that does not support
reading raw binary data. Use the supplied buffer object to do so.

Signed-off-by: Sven Wegener <sven.wegener@inovex.de>

Co-authored-by: Sven Wegener <sven.wegener@inovex.de>
2020-12-09 09:46:32 +01:00