Commit graph

3978 commits

Author SHA1 Message Date
Brian Coca
22330dd322
Correctly set path and fullpath for template vars (#73924)
* Correctly set path and fullpath for template vars

 don't expect path to always be full path
 also added exception/tb on action fail
2021-03-26 16:33:25 -04:00
Sam Doran
abacf6a108
Use ArgumentSpecValidator in AnsibleModule (#73703)
* Begin using ArgumentSpecValidator in AnsibleModule

* Add check parameters to ArgumentSpecValidator

Add additional parameters for specifying required and mutually exclusive parameters.
Add code to the .validate() method that runs these additional checks.

* Make errors related to unsupported parameters match existing behavior

Update the punctuation in the message slightly to make it more readable.
Add a property to ArgumentSpecValidator to hold valid parameter names.

* Set default values after performining checks

* FIx sanity test failure

* Use correct parameters when checking sub options

* Use a dict when iterating over check functions

Referencing by key names makes things a bit more readable IMO.

* Fix bug in comparison for sub options evaluation

* Add options_context to check functions

This allows the parent parameter to be added the the error message if a validation
error occurs in a sub option.

* Fix bug in apply_defaults behavior of sub spec validation

* Accept options_conext in get_unsupported_parameters()

If options_context is supplied, a tuple of parent key names of unsupported parameter will be
created. This allows the full "path" to the unsupported parameter to be reported.

* Build path to the unsupported parameter for error messages.

* Remove unused import

* Update recursive finder test

* Skip if running in check mode

This was done in the _check_arguments() method. That was moved to a function that has no
way of calling fail_json(), so it must be done outside of validation.

This is a silght change in behavior, but I believe the correct one.

Previously, only unsupported parameters would cause a failure. All other checks would not be executed
if the modlue did not support check mode. This would hide validation failures in check mode.

* The great purge

Remove all methods related to argument spec validation from AnsibleModule

* Keep _name and kind in the caller and out of the validator

This seems a bit awkward since this means the caller could end up with {name} and {kind} in
the error message if they don't run the messages through the .format() method
with name and kind parameters.

* Double moustaches work

I wasn't sure if they get stripped or not. Looks like they do. Neat trick.

* Add changelog

* Update unsupported parameter test

The error message changed to include name and kind.

* Remove unused import

* Add better documentation for ArgumentSpecValidator class

* Fix example

* Few more docs fixes

* Mark required and mutually exclusive attributes as private

* Mark validate functions as private

* Reorganize functions in validation.py

* Remove unused imports in basic.py related to argument spec validation

* Create errors is module_utils

We have errors in lib/ansible/errors/ but those cannot be used by modules.

* Update recursive finder test

* Move errors to file rather than __init__.py

* Change ArgumentSpecValidator.validate() interface

Raise AnsibleValidationErrorMultiple on validation error which contains all AnsibleValidationError
exceptions for validation failures.

Return the validated parameters if validation is successful rather than True/False.

Update docs and tests.

* Get attribute in loop so that the attribute name can also be used as a parameter

* Shorten line

* Update calling code in AnsibleModule for new validator interface

* Update calling code in validate_argument_spec based in new validation interface

* Base custom exception class off of Exception

* Call the __init__ method of the base Exception class to populate args

* Ensure no_log values are always updated

* Make custom exceptions more hierarchical

This redefines AnsibleError from lib/ansible/errors with a different signature since that cannot
be used by modules. This may be a bad idea. Maybe lib/ansible/errors should be moved to
module_utils, or AnsibleError defined in this commit should use the same signature as the original.

* Just go back to basing off Exception

* Return ValidationResult object on successful validation

Create a ValidationResult class.
Return a ValidationResult from ArgumentSpecValidator.validate() when validation is successful.
Update class and method docs.
Update unit tests based on interface change.

* Make it easier to get error objects from AnsibleValidationResultMultiple

This makes the interface cleaner when getting individual error objects contained in a single
AnsibleValidationResultMultiple instance.

* Define custom exception for each type of validation failure

These errors indicate where a validation error occured. Currently they are empty but could
contain specific data for each exception type in the future.

* Update tests based on (yet another) interface change

* Mark several more functions as private

These are all doing rather "internal" things. The ArgumentSpecValidator class is the preferred
public interface.

* Move warnings and deprecations to result object

Rather than calling deprecate() and warn() directly, store them on the result object so the
caller can decide what to do with them.

* Use subclass for module arg spec validation

The subclass uses global warning and deprecations feature

* Fix up docs

* Remove legal_inputs munging from _handle_aliases()

This is done in AnsibleModule by the _set_internal_properties() method. It only makes sense
to do that for an AnsibleModule instance (it should update the parameters before performing
validation) and shouldn't be done by the validator.

Create a private function just for getting legal inputs since that is done in a couple of places.

It may make sense store that on the ValidationResult object.

* Increase test coverage

* Remove unnecessary conditional

ci_complete

* Mark warnings and deprecations as private in the ValidationResult

They can be made public once we come up with a way to make them more generally useful,
probably by creating cusom objects to store the data in more structure way.

* Mark valid_parameter_names as private and populate it during initialization

* Use a global for storing the list of additonal checks to perform

This list is used by the main validate method as well as the sub spec validation.
2021-03-19 12:09:18 -07:00
Sloane Hertel
48c0fbd1cb
Fix a bug adding unrelated candidates to the plugin loader redirect_list (#73863)
* Fix a bug adding unrelated candidates to the plugin loader redirect_list

* Add tests for the redirect list

  * test redirect list for builtin module
  * test redirect list for redirected builtin module
  * test redirect list for collection module
  * test redirect list for redirected collection module
  * test redirect list for legacy module

* changelog
2021-03-18 17:53:57 -04:00
Matt Martz
78f34786dd
Send callbacks directly from the TaskExecutor instead of TaskResults masquerading as callbacks (#73927) 2021-03-18 12:12:29 -07:00
Sloane Hertel
95dc8f2361
Add some dev documentation for constructed features (#73497) 2021-03-18 09:45:13 -04:00
Brian Coca
37cb550185 fix su localization optoin
fixes #73837
2021-03-11 11:53:28 -08:00
Brian Coca
42c3c51665
Set fact fixes (#73684)
* fixes to set_fact

  correctly give error messages that were previously ignored
  corrected and expanded docs
2021-03-03 16:23:56 -05:00
Brian Coca
935528e22e
finish migrating ssh plugin to config system (#73708)
* finish migrating ssh plugin to config system

  fixes #72739
  fixes #57220

* fix connection detection in reset
* correct options for connection meta reset

Co-authored-by: David Shrewsbury <Shrews@users.noreply.github.com>
2021-03-03 15:25:16 -05:00
Brian Coca
43300e2279
module output is only json objects (#73765)
* module output is only json objects

 remove json lists as they are not valid from modules
 fixes #73744
2021-03-03 15:11:18 -05:00
Brian Coca
2e0835b312
minor detection improvement on j2plugin loader (#73714)
* minor  improvement on j2plugin loader, comments and var names clarified
2021-03-03 11:31:29 -05:00
Brian Coca
9690512069
Pipe it to connections (#73688)
* pipelining tweaks
  added 'defaults'  entry for ini pipelining from ssh plugin
2021-02-25 15:08:11 -05:00
Syed Ali Haider
5078a0baa2
Minor Doc Update to password.py (#73468)
Adds in the caveat that Python's string modules have different constants depending on the major version (2.x vs 3.x), but preserves the fact that a fair amount of them are common. Other minor docs touchups also included.
2021-02-18 17:22:16 -06:00
Felix Fontein
ff0edca8cc Catch more potential errors when setting up curses. 2021-02-12 13:09:02 -08:00
David Shrewsbury
f0ec10dbc3
Role arg spec validation implementation (#73152)
* Initial import of modified version of alikins' code
* Add unit testing for new Role methods
* Fix validate_arg_spec module for sanity test. Add test_include_role_fails.yml integration test from orig PR.
* Add testing of suboptions
* Use new ArgumentSpecValidator class instead of AnsibleModule
* fix for roles with no tasks, use FQ name of new plugin
* Add role dep warning
2021-02-12 16:00:17 -05:00
Brian Coca
6d15e1aa6e
corrected description info (#73586)
* corrected description info
2021-02-12 13:17:44 -05:00
Brian Coca
ea2f37d253
allow constructed to use vars plugin (#73418)
Allow constructed to optionally use vars plugin data

* mostly for those looking to leverage group_vars/ and host_vars/
* limited to already processed sources
2021-02-12 11:14:50 -05:00
Alexander Sowitzki
d0fda3e901
Let vault lookup output unicode string. (#73571)
Until now, the lookup plugin returned a byte string.
Changed this to output a unicode string instead.
2021-02-11 14:27:47 -05:00
Sam Doran
7d18ea5e93
default callback - add "show_task_path_on_failure" option (#73260)
When running in verbosity <2, display the file and line number for tasks that fail. This provides
useful information without having to run at increased verbosity.

* Move _print_task_path to CallbackBase class
* Add integration tests
* Add color parameter to _print_task_path()

* Keep color output consistent for now
  Currently the path is display with COLOR_DEBUG formatting with verbosity >= 2.
  Instead of the color of the path changing based on verbosity level, just keep it at the
  currently behavior of COLOR_DEBUG. Having the color of the same information change
  based on verbosity level seems incorrect and makes visual parsing of the information
  more difficult.

Co-authored-by: tahar.jegham <jeghamseifeddine@gmail.com>
2021-02-09 12:43:59 -05:00
Matt Martz
d3441bc313
Add new split filter (#73532)
* Add version added
2021-02-09 10:37:15 -05:00
Matt Martz
d3f3784b86
Don't treat host_pinned as lockstep (#73484)
* Don't treat host_pinned as lockstep. Fixes #73364

* Add intg tests
2021-02-05 09:06:19 -06:00
jakegatsby
f271d02a9f
bugfix: use to_text(stdout) in psrp.Connection.put_file method (#73491)
Co-authored-by: jakegatsby <jakegatsby@example.com>
2021-02-05 10:01:07 +10:00
Michael Shnit
4315e18807
Fix regression introduced in b77abd0491 causing bug in inventory modu… (#73429)
* Fix regression introduced in b77abd0491 causing bug in inventory modules which break functionality in user setting  use_contrib_script_compatible_sanitization  parameter.

* Add changelog

Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
2021-02-04 17:25:08 -05:00
Sloane Hertel
840bdc1e10
Fix warning for nonexistent inventory cache (#72840)
* Fix inventory cache warning by checking if the key exists before loading it
* changelog
2021-02-01 14:29:36 -05:00
Brian Coca
2b0cd2c13f
use correct executable and options in all cases (#73323)
Use correct ssh executable and options in all cases on connection plugin

  * Also nicer naming/comments

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-01-26 20:21:24 -05:00
Martin Krizek
7f9ac0f364
Consolidate filters/tests handling into JinjaPluginIntercept (#71463)
* Consolidate filters/tests handling into JinjaPluginIntercept

ci_complete

* Postpone loading all ansible plugins

* Do we need to create an overlay?

ci_complete

* Typo

ci_complete

* Add FIXME

* conditional.py: use public Environment.parse() method

* Remove remaining occurrences of shared_loader_obj being passed to Templar

* __UNROLLED__ not needed with this change anymore

* Incorrect rebase at some point?
2021-01-21 11:22:33 +01:00
Brian Coca
30d93995dd
Use a pty for local connections (#73023)
* Use a pty for local connections

Fixes #38696

Co-authored-by: James Cammarata <jimi@sngx.net>
2021-01-18 16:02:04 -05:00
Brian Coca
1202dd000f
Allow restricting config values to enumerated list (#73162)
* Allow restricting config values to enumerated list
* dont document internal entries
* also ignore private defs for ansible-config
* remove invalid value from tests
* added porting entry
2021-01-14 15:11:30 -05:00
Felix Fontein
df9cf368c0
Always mention the name of the deprecated plugin in routing deprecation messages (#73059) 2021-01-14 17:45:02 +01:00
Sam Doran
0e6c334115
pause - adjust warning when run in background (#73182)
When the pause module is run in the background and seconds parameter is provided,
do not warn.

* Add tests
* Fix existing tests
  The test wasn't failing when it should have.
2021-01-14 09:35:39 -05:00
Jorge Rua
7e769d6dd3
Fix example with the backwards counting in the sequence lookup plugin. (#73081)
The old example failed with:
fatal: [localhost]: FAILED! => {"msg": "to count backwards make stride negative"}
I fixed this and placed a start parameter to consolidate with the rest of the examples provided.

Co-authored-by: Jorge Rua <jorua@redhat.com>
2021-01-07 16:34:05 -06:00
Brian Coca
de3844cba9
discourage hexdigits in password lookup (#73106)
* actually add new docs

* Update lib/ansible/plugins/lookup/password.py

Co-authored-by: Sandra McCann <samccann@redhat.com>
2021-01-05 15:03:27 -05:00
Matt Martz
a3b6485073
Fix reset_connection paramiko, winrm, psrp (#72688)
* Ensure we only reset the connection when one has been previously established. Fixes #65812

* Ensure psrp doesn't trace

* winrm too

* Indentation fix
2020-12-09 12:57:56 -06:00
Martin Krizek
1b70260d5a
Add implicit role_complete block instead of role._eor (#72208)
Co-authored-by: Matt Martz <matt@sivel.net>

Fixes #69848
2020-12-09 09:53:01 +01:00
David Shrewsbury
fe17cb6eba
Fix fileglob parameter order bug (#72879) 2020-12-08 16:31:34 +01:00
Sam Doran
bc37976df2
Clarify mode behavior for modules that can set file permissions (#72837) 2020-12-08 16:11:36 +01:00
Martin Krizek
ae08c6a639
Ensure Ansible's unique filter preserves order (#67856)
Fixes #63417
2020-11-23 08:55:18 +01:00
Brian Coca
d8baf4d38d
Document and warn when max_fail and free are used (#72616)
Fixes #16666
2020-11-17 10:11:15 -05:00
Sam Doran
4b8cb6582b
pause - do not hang if run in the background (#72065)
* Consolidate logic for determining whether or not session is interactive
  into a single function, is_interactive()
* Increase test coverage

I wasn't able to find a good way of simulating running a backgrounded test with CI since the
whole test is essentially run not in a TTY, which is similar enough to cause the new is_interactive()
function to always return false.
2020-11-12 12:22:57 -05:00
Brian Coca
aa4d53ccdf
ensure local exposes correct user (#72543)
* ensure local exposes correct user

  avoid corner case in which delegation relied on
  playcontext fallback which was removed

  fixes #72541
2020-11-09 16:21:17 -05:00
Sloane Hertel
0ed7bfc694
Fix task get_name to always prepend the role name (#72511)
* Fix 'role_name : tast_name' notation if task contains role name

* Add tests for notifying handler names which contain the role name

Co-authored-by: Thomas Wouters <thomaswouters@gmail.com>
2020-11-06 11:46:58 -05:00
Felix Fontein
72302dd611 Replace some more FQCNs. 2020-11-05 12:19:08 -08:00
Brian Coca
488b9d6c35 remove redundant remote_user for local setting
local action plugin already does and this also should fix
  fork/thread issue by removing use of pwd library

  fixes #59642
2020-11-05 11:17:46 -08:00
Felix Fontein
da60525610
Fix missing ansible.builtin FQCNs in hardcoded action names (#71824)
* Make sure hard-coded action names also check for FQCN.
* Use _add_internal_fqcn() to avoid hardcoded lists and typoes.
2020-11-03 08:51:31 -05:00
Brian Coca
4b673484f0
rethink wording (#70028)
* rethink wording
* removed unrequired requirement
* fix tests
* fixed versions
Co-authored-by: Sloane Hertel <shertel@redhat.com>
2020-10-30 17:13:12 -04:00
Matt Martz
e73a0b2460
Explicitly get the include task, and not assume it is the parent (#72378)
* Explicitly get the include task, and not assume it is the parent. Fixes #6571
2020-10-29 15:15:18 -04:00
Sam Doran
a51a6f4a25
reboot - add reboot_command parameter (#69847)
Fixes #51359
* Update default search paths
* Fix returns for args and command, don't allow conversion
* Reorganize tests
* Fix test for Azure Pipelines
2020-10-28 12:10:59 -04:00
Brian Coca
e05c62547b
run playbook from collections (#67435)
* fixes for collection playbooks

 - add fqcn invocation, also handles extensions
 - bring import_playbook into new normal
 - avoid adding collection playbooks to adjacency
 - deal with extensions or lack of em
 - added tests
 - fix bugs with multiple playbooks supplied
 - unicode/bytes/native fixes
 - fix modulenotfound for < py3.7
 - tests
2020-10-28 11:46:39 -04:00
Sam Doran
8f9cf456b0
url lookup - set default user agent (#72324)
* Add unit tests
* Add note about when default changed
2020-10-28 09:34:06 -04:00
Sloane Hertel
dee8a4a8f1
Use type 'path' for fact_caching_connection in jsonfile cache plugin (#72317) 2020-10-27 10:10:13 -05:00
Sloane Hertel
8b07d46166
Add a toggle for the leading separator for keyed_groups (#60882)
* Add a toggle for the leading separator for keyed_groups if no prefix is given

* changelog

* Add tests for backward compatibility and the new toggle
2020-10-23 15:57:44 -04:00