Commit graph

2565 commits

Author SHA1 Message Date
Matt Clay
13d08d232c Rename ansible-test functions to match constants. 2021-01-26 14:24:16 -08:00
Matt Clay
73fadc5e97
Clean up code in ansible-test. (#73379)
* Relocate code to fix type dependencies.

* Fix missing and unused imports.

* Fix type hints.

* Suppress PyCharm false positives.

* Avoid shadowing `file` built-in.

* Use json.JSONEncoder directly instead of super().

This matches the recommended usage and avoids a PyCharm warning.

* Remove redundant regex escape.

* Remove redundant find_python call.

* Use tarfile.open directly.

* Add changelog fragment.
2021-01-26 14:02:08 -08:00
Martin Krizek
a2af8432f3
Local vars should have highest precedence in AnsibleJ2Vars (#72830)
Ability to add local variables into AnsibleJ2Vars was added in
18a9eff11f to fix #6653. Local variables
are added using ``AnsibleJ2Vars.add_locals()`` method when creating a
new context - typically when including/importing a template with
context. For that use case local template variables created using
``set`` should override variables from higher contexts - either from the
play or any parent template, or both; Jinja behaves the same way.

Also removes AnsibleJ2Vars.extras instance variable which is not used.

Also adds missing test for #6653.

Fixes #72262
Fixes #72615

ci_complete
2021-01-26 11:37:56 +01:00
Matt Clay
fc590aeb21 Use a versioned pip bootstrapper in ansible-test. 2021-01-25 12:31:11 -08:00
Matt Clay
484e4af4d0 Handle get-pip.py breaking change on Python 2.7. 2021-01-23 11:45:53 -08:00
Brian Coca
e0c9f285ff
undeprecated hash_merge setting (#73328)
added updates from sloan's PR
 feedback from others

Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>

Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>
2021-01-22 15:00:19 -05:00
Alexander Sowitzki
e6da544310 Let command always return stdout & stderr (#73004)
The `command` module does not return stdout & sterr when calling
a non existing executable or an unknown exception arises. This fix
lets the module return empty byte strings in those cases.
2021-01-22 08:40:53 +01:00
Rick Elrod
1934ca9a55
Add Ubuntu 20.04 to CI and ansible-test (#69161)
Change:
- Add Ubuntu 20.04 to CI now that venv is default instead of virtualenv in ansible-test.

Test Plan:
- CI

Tickets:
- Fixes #69203

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-21 22:58:31 -06:00
Rick Elrod
4357a78130
[ansible-test] First attempt at freebsd/11.4 (#72655)
Change:
- Try CI against freebsd 11.4

Test Plan:
- ci_complete

Tickets:
- Fixes #48782

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-21 17:12:18 -06:00
Alexander Sowitzki
fe792fdcd2
Vanquish rare container bootstrap failure (#73288)
The init script for the test container writes additional lines to
the .bashrc of the user. This was done via a `cat` multiline
instruction, which is implemented internally by writing a
temporary file to TMPDIR (/tmp in this case) first. Docker fails
to provide /tmp just after creation, which results in a race
condition that rarely makes the init fail. Changed the `cat`
statement to multiple `echo`s.

Co-authored-by: Matt Clay <matt@mystile.com>
2021-01-21 12:22:58 -05:00
Joshua Bayfield
823c72bcb5
Shadow input for encrypt_string by default unless asked (fixes #71618) (#73263)
* Shadow input for encrypt_string by default unless asked (fixes #71618)
2021-01-20 15:50:24 -05:00
Jordan Borean
bc60d8ccda
Galaxy - make versions list consistent across versions (#72932)
* Galaxy - make versions list consistent across versions

* Fix up unit tests
2021-01-20 21:19:29 +01:00
Oldřich Jedlička
ec55a221f6
Do not pretend expression is filename in compile() Python built-in call (#73113)
When calling compile(), the filename argument should be either a real file
name or a string. According to Python docs, suggested one is '<string>'.
 Keep the current behaviour (encapsulate the actual expression), 
but enclose it into angle brackets.

Signed-off-by: Oldřich Jedlička <oldium.pro@gmail.com>
2021-01-19 12:20:26 -05:00
Sam Doran
e8d4b62b41
Fix YAML error message when error is at the end of the file (#73241)
* Fix YAML error message when error is at the end of the file

If a YAML file fails to load due to a syntax error in a file, or there is an error in the last line of a
file, PyYAML reports the last line number of the file as the index where the error occurred.

When reading the file lines, we use that index to the get the relevant line.  If the index value is out
of range, the relevant line is lost for error reporting.

Subtract one from the index value to avoid the IndexError in this specific scenario. It is possible
to still get an IndexError, which will be handled as it is currently.

* Update existing tests and add new tests
2021-01-19 10:07:36 -05:00
Brian Coca
30d93995dd
Use a pty for local connections (#73023)
* Use a pty for local connections

Fixes #38696

Co-authored-by: James Cammarata <jimi@sngx.net>
2021-01-18 16:02:04 -05:00
Brian Coca
1e27d4052a
fix inventory source parse error handling (#73160)
fixes #51025

added test cases
2021-01-18 09:33:51 -05:00
Sandra McCann
9d71bbd13e
add changelog fragment for 73176 (#73259) 2021-01-15 16:26:53 -06:00
Brian Coca
1202dd000f
Allow restricting config values to enumerated list (#73162)
* Allow restricting config values to enumerated list
* dont document internal entries
* also ignore private defs for ansible-config
* remove invalid value from tests
* added porting entry
2021-01-14 15:11:30 -05:00
Sam Doran
e2308ed6e3
ansible-galaxy - correct collections-path command line arg (#73193)
It should be plural to match other options and environment variables.
2021-01-14 14:13:01 -05:00
Felix Fontein
df9cf368c0
Always mention the name of the deprecated plugin in routing deprecation messages (#73059) 2021-01-14 17:45:02 +01:00
Sam Doran
df451636e7
facts - properly report virtualization facts for Linux guests on bhyve (#73204) 2021-01-14 10:53:03 -05:00
Sam Doran
0e6c334115
pause - adjust warning when run in background (#73182)
When the pause module is run in the background and seconds parameter is provided,
do not warn.

* Add tests
* Fix existing tests
  The test wasn't failing when it should have.
2021-01-14 09:35:39 -05:00
Rick Elrod
7f0eb7ad79
[facts] Differentiate CentOS vs CentOS Stream (#73034)
Change:
- On CentOS Stream, make distribution_release be "Stream"
- On CentOS Core, it continues to be "Core"
- Implement custom distribution file parser for CentOS, so we can look
  for "CentOS Linux" and "CentOS Stream"
- Two new fixtures introduced (CentOS Linux 8.1 and CentOS Stream 8)
- Removed two dicts from `Distribution` class that were seemingly not
  used anywhere.

Test Plan:
- ci_complete
- New test fixtures

Tickets:
- Fixes #73027

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-13 17:54:04 -05:00
Matt Clay
4164cb26f0 Remove obsolete tower plugin from ansible-test. 2021-01-13 12:37:21 -08:00
Matt Clay
21f1811ddf
Cleanup provisioning code in ansible-test. (#73207)
* Remove unused code in ansible-test.

* Remove obsolete endpoint logic from ansible-test.

* Remove obsolete region selection in ansible-test.

* Remove obsolete port logic in ansible-test.

* Clean up ansible-test remote providers.
2021-01-12 16:18:28 -08:00
Brian Coca
d6670da1d7
Added clog missing for issue 70722 (#73175) 2021-01-12 12:09:59 -05:00
Martin Zimmermann
be59caa59b
iptables: Added set module/match_set support (#72984) 2021-01-11 16:32:53 -05:00
Sam Doran
a7e834071c
Add macOS 11 to CI (#72622)
* Fix connection_paramiko_ssh test for macOS 11
* Update Azure Pipelines config
* Add changelog
2021-01-11 12:20:47 -05:00
Sam Doran
264e08f21a
user - properly handle password and password lock when used together (#73016)
Do the right thing on Linux when password lock and a password hash are provided by writing
out the password hash prepended by the appropriate lock string rather than using -U and -L.
This is the correct way to set and lock the account in one command.

On BSD, run separate commands as appropriate since locking and setting the password cannot 
be done in a single action.

FreeBSD requires running several commands to get the account in the desired state. As a result,
the rc, output, and error from all commands need to be combined and evaluated so an accurate
and complete summary can be given at the end of module execution.

* Improve integration tests to cover this scenario.
* Break up user integration tests into smaller files
* Properly lock account when creating a new account and password is supplied

* Simplify rc collection in FreeBSD class
  Since the _handle_lock() method was added, the rc would be set to None, which could make
  task change reporting incorrect. My first attempt to solve this used a set and was a bit too
  complicated. Simplify it my comparing the rc from _handle_lock() and the current value of rc.

* Improve the Linux password hash and locking behavior
  If password lock and hash are provided, set the hash and lock the account by using a password
  hash since -L cannot be used with -p.

* Ensure -U and -L are not combined with -p since they are mutually exclusive to usermod.

* Clarify password_lock behavior.
2021-01-11 11:42:35 -05:00
Matt Clay
05ba49cbba ansible-test - No virtualenv install on Python 3
Tests should use the Python built-in ``venv`` module on Python 3 instead of the standalone ``virtualenv`` module.

On Python 2 the ``virtualenv`` module continues to be the only option.
The version installed is either the OS packaged version or the last release to support Python 2, which is version 16.7.10.
2021-01-07 20:12:39 -08:00
Brian Coca
b1ee1a285a
Added NO_COLOR as config option (#73105) 2021-01-07 15:00:31 -05:00
Rick Elrod
44ee04bd1f
[dnf] Make "remove" filtering closer to dnf CLI (#73033)
Change:
- Internally, use dnf.subject.Subject#get_best_query for state: absent
- Add a bunch of tests for removing packages, given a bunch of different
  pkg specs (nv, nvr, nvra, wildcard, etc.)

Test Plan:
- New tests
- Local experiments with DNF API via PDB.

Tickets:
- Fixes #72809

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-07 11:32:06 -06:00
Matt Clay
77942acefc Refactor ansible-test cryptography install code. 2021-01-07 09:04:33 -08:00
David Shrewsbury
c8666f07d7
Add support for argument_specs attribute to RoleMetadata class (#73120) 2021-01-07 11:41:46 -05:00
Abhijeet Kasurde
25792d2388
galaxy: handle plus sign in user token appearing in role url (#73057)
Fixes: #45475

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-01-06 10:50:44 -05:00
Sviatoslav Sydorenko
5b4f0b3370 Declare support for Python 3.9 in the metadata
Co-authored-by: Rick Elrod <rick@elrod.me>
2021-01-05 20:48:08 -08:00
Rick Elrod
b96732fa6b [ansible-test] Add Fedora 33 support and drop F31
Change:
- Changes to make F33 work with current tests
- Add F33 to docker.txt
- Add F33 to AZP config

Test Plan:
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-05 20:48:08 -08:00
Brian Coca
de3844cba9
discourage hexdigits in password lookup (#73106)
* actually add new docs

* Update lib/ansible/plugins/lookup/password.py

Co-authored-by: Sandra McCann <samccann@redhat.com>
2021-01-05 15:03:27 -05:00
Lloyd Parkes
e0c75a6756
Replace the use of the function string.replace with the method str.replace (#68793)
* Replace the use of the function string.replace() with the method str.replace()
because that's what works for both Python 2 and 3.

* Cleanup the unused string import.

Added a changelog fragment.

* The documentation for os.write() seems a bit iffy, but in Python 3 we
definitely cannot pass it a string and we need to encode it into
bytes.

The Python documentation at
https://docs.python.org/3/howto/pyporting.html#text-versus-binary-data
says that this code will work in Python 2 as well.

Co-authored-by: Lloyd Parkes <lloyd@must-have-coffee.gen.nz>
2021-01-05 11:20:44 -05:00
Rick Elrod
20509b6507
[facts] fix version facts on FreeBSD RC/PRERELEASE (#73020)
Change:
- The FreeBSD release can contain -RC or -PRERELEASE in addition to
  -RELEASE, -STABLE, or -CURRENT.

Test Plan:
- Added new fixed from an RC version of TrueNAS which uses a -PRERELEASE
  version of FreeBSD.

Tickets:
- Fixes #72331

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-05 10:16:59 -05:00
Jean-Pierre Matsumoto
3074ede8a3
ExecutionPolicy default value is RemoteSigned (#72993)
* ExecutionPolicy default value is RemoteSigned

##### SUMMARY
In Windows setup steps, settings `ExecutionPolicy` is restored to value `Restricted`. But actually default value for Windows server is not `Restricted` but `RemoteSigned` ([source](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_execution_policies?view=powershell-7.1#default)).

I have got issue with `win_dsc` because `ExecutionPolicy` was `Restricted`.

##### ISSUE TYPE
- Docs Pull Request

+label: docsite_pr

* Add changelog fragment

* Apply suggestions from code review

Co-authored-by: Sandra McCann <samccann@redhat.com>
2021-01-04 15:41:54 -05:00
pva
42bc03f0f5
[get_url] skip checksum during --check (#66700)
Fix get_url failure in check mode (--check) when using the checksum
format <algorithm>:<url>.

Regression introduced in (#20532)

Fixes: #61369
2020-12-22 21:34:42 +05:30
Abhijeet Kasurde
aa56a2ff6a
galaxy: Handle token as dict while loading using yaml (#70911)
Malformed galaxy_token file creates stacktrace.

Fixes: #70887

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-12-21 14:53:00 -05:00
Sam Doooran
5ecfb19cad
arg_spec - move validator lookup method to a function (#72667)
* arg_spec - move type checking lookup method to a function

* Change get_wanted_type name and behavior

Change the name to get_validator to bette describe what it is doing.

Change the interface to always return a value. This lines up with the behavior of get_*
functions always returning something or None and check_* functions raising an
Exception if something went wrong during the check.

* Add param to check_type_str()

Not meant to be a long term fix, but gets tests passing. More work is needed to figure
out how to solve this cleanly.

* Remove private attribute mapping types to validator

Since the function that needs it has moved to parameters.py, there is no need to have it as
a attribute of AnsibleModule.

Update tests that were referencing the private attribute.

* Use private method for 'str' type

To avoid having to put the string conversion warning behavior in the check_type_str() method,
use the private _check_type_str() method for 'str' type.

Import CHECK_ARGUMENT_TYPES_DISPATCHER for backwards compalitibility and store it as
a private attribute.

Revert changes to support plugins that are referencing serf._CHECK_ARGUMENT_TYPES_DISPATCHER.

* Add changelog

* Change function name to better reflect its... function

* Change dict name to better reflect its contents

CHECK_ARGUMENT_TYPES_DISPATCHER --> DEFAULT_TYPE_VALIDATORS

* Fix changelog
2020-12-17 17:15:00 -05:00
Sam Doooran
48803604cd
systemd - do not overwrite unit name when searching (#72985)
* systemd - do not overwrite unit name when searching

PR #72702 introduced a bug that changed the unit name when splitting it up for the purpose
of searching for the unit. This only happens on unit file templates on systems that have a 5.8
or newer kernel and a version of systemd that does not contain a bugfix that causes systmed
to fail to parse dbus.

* Use facts rather than a manual probe to determine if systmed is present

* Remov unnecessary block

* Use vars files instead of set_fact

* Add tests for using a templated unit file

* Update changelog fragment

* Use template to get correct path to sleep binary
2020-12-17 15:29:22 -05:00
Sam Doooran
8e022ef00a
import_playbook - change additional params to deprecation (#72987)
* import_playbook - change additional params to deprecation

I incorrectly recommended this be set as a warning when it should have been a deprecation.

* Fix deprecation sanity test to not required a collection name when not inside a collection
2020-12-17 14:14:58 -05:00
Sam Doooran
850a77f639
ansible-test - prefer venv over virtualenv on Python 3 (#73000)
Also pin virtualenv to 16.7.10 for older Mac OS X systems. This was the version being installed
anway with the previous constraint (<20).

On systems with Python 3, now prefer venv over virtualenv. Test to see if venv is functional since
some systems have a non-functional venv installation (such as Debian).
2020-12-17 10:51:42 -05:00
Sloane Hertel
5e03e322de
Pass the top level dictionaries to combine_vars (#72979)
* Pass the top level dictionaries to combine_vars

combine_vars uses dict.update() to replace keys
2020-12-16 11:23:23 -05:00
Matt Clay
37d09f2488
Update ansible-test pylint Python support. (#72972)
* Update ansible-test pylint Python support.

* Python 3.8 is now officially supported.
* Python 3.9 is now skipped with a warning.
2020-12-15 10:27:32 -08:00
Felix Fontein
0ba96d2be8
Compare removal and deprecation dates and versions in runtime metadata against current version and today (#72625)
* Check deprecation and removal versions w.r.t. current version.

* Check removal dates from tombstones w.r.t. today.

* Add changelog.
2020-12-15 09:05:38 -05:00