Commit graph

51609 commits

Author SHA1 Message Date
Brian Coca
232eeee206
less blocking on fact reading (#73951)
* less blocking on fact reading


Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
2021-03-23 10:27:53 -04:00
Xabier Napal
2bff120db6
Fix wrong backup directory var name in apt module (#73840)
* Fix wrong backup directory var name in apt module
2021-03-22 10:06:45 -04:00
Rick Elrod
2c40dccd85
Update Ansible release version to v2.11.0b3.post0. (#73980) 2021-03-19 23:48:17 -05:00
Rick Elrod
88a1a85ff8
New release v2.11.0b3 (#73977) 2021-03-19 22:08:27 -05:00
Matt Davis
c6cf7986ab
add porting guide info for optional module_utils (#73979) 2021-03-19 20:21:22 -05:00
Sam Doran
abacf6a108
Use ArgumentSpecValidator in AnsibleModule (#73703)
* Begin using ArgumentSpecValidator in AnsibleModule

* Add check parameters to ArgumentSpecValidator

Add additional parameters for specifying required and mutually exclusive parameters.
Add code to the .validate() method that runs these additional checks.

* Make errors related to unsupported parameters match existing behavior

Update the punctuation in the message slightly to make it more readable.
Add a property to ArgumentSpecValidator to hold valid parameter names.

* Set default values after performining checks

* FIx sanity test failure

* Use correct parameters when checking sub options

* Use a dict when iterating over check functions

Referencing by key names makes things a bit more readable IMO.

* Fix bug in comparison for sub options evaluation

* Add options_context to check functions

This allows the parent parameter to be added the the error message if a validation
error occurs in a sub option.

* Fix bug in apply_defaults behavior of sub spec validation

* Accept options_conext in get_unsupported_parameters()

If options_context is supplied, a tuple of parent key names of unsupported parameter will be
created. This allows the full "path" to the unsupported parameter to be reported.

* Build path to the unsupported parameter for error messages.

* Remove unused import

* Update recursive finder test

* Skip if running in check mode

This was done in the _check_arguments() method. That was moved to a function that has no
way of calling fail_json(), so it must be done outside of validation.

This is a silght change in behavior, but I believe the correct one.

Previously, only unsupported parameters would cause a failure. All other checks would not be executed
if the modlue did not support check mode. This would hide validation failures in check mode.

* The great purge

Remove all methods related to argument spec validation from AnsibleModule

* Keep _name and kind in the caller and out of the validator

This seems a bit awkward since this means the caller could end up with {name} and {kind} in
the error message if they don't run the messages through the .format() method
with name and kind parameters.

* Double moustaches work

I wasn't sure if they get stripped or not. Looks like they do. Neat trick.

* Add changelog

* Update unsupported parameter test

The error message changed to include name and kind.

* Remove unused import

* Add better documentation for ArgumentSpecValidator class

* Fix example

* Few more docs fixes

* Mark required and mutually exclusive attributes as private

* Mark validate functions as private

* Reorganize functions in validation.py

* Remove unused imports in basic.py related to argument spec validation

* Create errors is module_utils

We have errors in lib/ansible/errors/ but those cannot be used by modules.

* Update recursive finder test

* Move errors to file rather than __init__.py

* Change ArgumentSpecValidator.validate() interface

Raise AnsibleValidationErrorMultiple on validation error which contains all AnsibleValidationError
exceptions for validation failures.

Return the validated parameters if validation is successful rather than True/False.

Update docs and tests.

* Get attribute in loop so that the attribute name can also be used as a parameter

* Shorten line

* Update calling code in AnsibleModule for new validator interface

* Update calling code in validate_argument_spec based in new validation interface

* Base custom exception class off of Exception

* Call the __init__ method of the base Exception class to populate args

* Ensure no_log values are always updated

* Make custom exceptions more hierarchical

This redefines AnsibleError from lib/ansible/errors with a different signature since that cannot
be used by modules. This may be a bad idea. Maybe lib/ansible/errors should be moved to
module_utils, or AnsibleError defined in this commit should use the same signature as the original.

* Just go back to basing off Exception

* Return ValidationResult object on successful validation

Create a ValidationResult class.
Return a ValidationResult from ArgumentSpecValidator.validate() when validation is successful.
Update class and method docs.
Update unit tests based on interface change.

* Make it easier to get error objects from AnsibleValidationResultMultiple

This makes the interface cleaner when getting individual error objects contained in a single
AnsibleValidationResultMultiple instance.

* Define custom exception for each type of validation failure

These errors indicate where a validation error occured. Currently they are empty but could
contain specific data for each exception type in the future.

* Update tests based on (yet another) interface change

* Mark several more functions as private

These are all doing rather "internal" things. The ArgumentSpecValidator class is the preferred
public interface.

* Move warnings and deprecations to result object

Rather than calling deprecate() and warn() directly, store them on the result object so the
caller can decide what to do with them.

* Use subclass for module arg spec validation

The subclass uses global warning and deprecations feature

* Fix up docs

* Remove legal_inputs munging from _handle_aliases()

This is done in AnsibleModule by the _set_internal_properties() method. It only makes sense
to do that for an AnsibleModule instance (it should update the parameters before performing
validation) and shouldn't be done by the validator.

Create a private function just for getting legal inputs since that is done in a couple of places.

It may make sense store that on the ValidationResult object.

* Increase test coverage

* Remove unnecessary conditional

ci_complete

* Mark warnings and deprecations as private in the ValidationResult

They can be made public once we come up with a way to make them more generally useful,
probably by creating cusom objects to store the data in more structure way.

* Mark valid_parameter_names as private and populate it during initialization

* Use a global for storing the list of additonal checks to perform

This list is used by the main validate method as well as the sub spec validation.
2021-03-19 12:09:18 -07:00
Brian Coca
089d0a0508
find - set proper default based on use_regex (#73961)
When using "use_regex: yes" and setting an excludes: without
specifying a pattern: the existing code passes the file-glob '*' to
the regex matcher.  This results in an internal invalid-regex
exception being thrown.

This maintains the old semantics of a default match-all for pattern:
but switches the default to '.*' when use_regex is specified.

The code made sense as-is before excludes: was added (2.5).  In that
case, it made no sense to set use_regex but *not* set a pattern.
However, with excludes: it now makes sense to only want to exclude a
given regex but not specify a specific matching pattern.

Closes: #50067

* moved change to new location
added changelog

* Update lib/ansible/modules/find.py


Co-authored-by: Ian Wienand <iwienand@redhat.com>
2021-03-19 13:18:31 -04:00
Sloane Hertel
48c0fbd1cb
Fix a bug adding unrelated candidates to the plugin loader redirect_list (#73863)
* Fix a bug adding unrelated candidates to the plugin loader redirect_list

* Add tests for the redirect list

  * test redirect list for builtin module
  * test redirect list for redirected builtin module
  * test redirect list for collection module
  * test redirect list for redirected collection module
  * test redirect list for legacy module

* changelog
2021-03-18 17:53:57 -04:00
Baptiste Mille-Mathias
3e63eb8418 fix typo 2021-03-18 12:50:06 -07:00
Abhijeet Kasurde
3811fddede Amazon: Fix distribution facts for older release
Ansible can gather distribution facts for older Amazon Linux
with /etc/os-release data.

Fixes: #73946

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-03-18 12:42:04 -07:00
s-hertel
ad5ee1542a Remove ini and env fields for config option CONNECTION_FACTS_MODULES 2021-03-18 12:40:15 -07:00
Alicia Cozine
7a55e98d29
clarifies docs for file module, addresses issue 72372 (#73938) 2021-03-18 12:34:04 -07:00
Brian Coca
ccd9a992cf
complex data example using value and default from list of dicts (#73937) 2021-03-18 12:31:33 -07:00
sgpinkus
efe97aeb31
Update developing_inventory.rst (#73931)
* Update developing_inventory.rst

hashes/dictionaries in JSON (JavaScript Object Notation) are represented by what JSON calls "objects", so use that established terminology instead of "hash", "dictionary", or "hash/dictionary" etc.

* Update docs/docsite/rst/dev_guide/developing_inventory.rst

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
2021-03-18 12:14:21 -07:00
Matt Martz
78f34786dd
Send callbacks directly from the TaskExecutor instead of TaskResults masquerading as callbacks (#73927) 2021-03-18 12:12:29 -07:00
Ompragash Viswanathan
561cdf3ace
Updated COLLECTIONS_4.rst to include ansible-inclusion repo link for new collection submission (#73582)
Co-authored-by: Sandra McCann <samccann@redhat.com>

Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
2021-03-18 11:59:01 -04:00
Sloane Hertel
95dc8f2361
Add some dev documentation for constructed features (#73497) 2021-03-18 09:45:13 -04:00
Alicia Cozine
30c465c1a9
Promote publication (#72400)
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
2021-03-17 16:14:32 -04:00
Alicia Cozine
b57444af14
Rebased pr73824 (#73934)
Co-authored-by: Eugene <k.evgen61@gmail.com>
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
2021-03-17 16:07:38 -04:00
sommersoft
4add723107
ansible-pull: Run All Playbooks When Multiple Are Supplied (#73172)
* ansible-pull: run all playbooks when multiple are supplied

* add test for ansible-pull with multiple playbooks supplied from cli

* add changelog fragment
2021-03-17 13:52:51 -04:00
Toshio Kuratomi
0279d02980 Add porting guide for ansible 4 2021-03-17 08:41:02 -07:00
Toshio Kuratomi
e10da3c368 Porting guide changes when adding a new ansible release
* Add the ansible-4 porting guide to the index
* Need to exclude the porting guide for ansible-4 from the core docs build
2021-03-17 08:41:02 -07:00
Matt Martz
832631b888
Ensure task from the worker is finalized/squashed (#73881)
* Ensure task from the worker is finalized/squashed. Fixes #57399. Fixes #49942
2021-03-16 15:09:43 -04:00
Rick Elrod
a29fcfa995
Update Ansible release version to v2.11.0b2.post0. (#73917) 2021-03-15 20:30:15 -05:00
Rick Elrod
aff69b6b4c
New release v2.11.0b2 (#73915) 2021-03-15 17:54:49 -05:00
David Shrewsbury
1e5ccb326f
Allow for searching handler subdir for included tasks (#73809)
* Allow for searching handler subdir for included tasks
2021-03-15 15:39:59 -04:00
Ikko Ashimine
3098022146
Fix typo in porting_guide_3.rst (#73871)
paramater -> parameter
2021-03-15 15:58:41 +05:30
Martin Nečas
7099a5f448
setup - virtualization facts add Red Hat vendor (#72876)
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-03-15 15:20:44 +05:30
Paul Belanger
9ec4e08534
Don't match passive_interface for validate-modules (#73880)
This is a follow up to:

  https://github.com/ansible/ansible/pull/73508

To avoid adding no_log statements to passive_interface args.

Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2021-03-13 08:28:57 -06:00
Felix Fontein
7c0af58449
validate-modules no_log test: extend search range, add more known false-positives (#73882)
* validate-modules no_log test: extend search range, add more known false-positives.

* Mark false-positives.
2021-03-13 08:28:33 -06:00
Mark Chappell
ed18fcac3b
Ensure unit test paths for connection and inventory plugins are based on the context (#73877) 2021-03-12 12:46:40 -08:00
Matt Clay
3a8c9242e1 Use virtual host style S3 bucket names in tests. 2021-03-12 11:34:25 -08:00
Toshio Kuratomi
197e8f5640
Mention that minor releases end once the next ansible release is made. (#73836) 2021-03-11 16:12:55 -06:00
Toshio Kuratomi
780f79ecbc
Update the porting guide for a new ansible version (#73856) 2021-03-11 14:49:03 -06:00
Brian Coca
b486feaaf4
document slurp returns (#73865)
* no path on return
2021-03-11 14:28:47 -06:00
Sandra McCann
bd020dd14a
describe what ansible-core is (#73829) 2021-03-11 14:27:36 -06:00
Brian Coca
37cb550185 fix su localization optoin
fixes #73837
2021-03-11 11:53:28 -08:00
Brian Coca
8ef54759ec
moved vs deprecated world readable tmp setting (#73825) 2021-03-11 11:43:27 -08:00
Jens Heinrich
27eaab310b
Add direct link to using multiple inventory sources (#73680)
This helps users to find the right ressources more easily and prevents them from clicking the first matching link only to be redirected to a subsection on the samepage from there

Co-authored-by: Jens Heinrich <github.com/JensHeinrich>
2021-03-11 14:36:00 -05:00
Alicia Cozine
474f46ea56
Docs: Updates the release and maintenance page to reflect new cadences (#73781)
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
Co-authored-by: John R Barker <john@johnrbarker.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
Co-authored-by: John R Barker <john@johnrbarker.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
2021-03-11 14:25:18 -05:00
bylerbk
1ac2858b5a
Update developing_modules_general.rst (#73805)
##### SUMMARY
<!--- Fixing a typo. There was a double "you" in the text. -->


##### ISSUE TYPE
- Docs Pull Request

+label: docsite_pr
2021-03-11 14:11:04 -05:00
Alicia Cozine
31932f5fa3
Docs: Update installation page for package split (#73790) 2021-03-11 11:05:44 -08:00
Felix Fontein
b2015c98e2
validate-modules: make sure that options that potentially contain secret data have no_log set (#73508)
* Catch more potential errors (and increase false-positive rate).
* Flag some false-positives in lib/ansible/modules/ with no_log=False.

Co-authored-by: John Barker <john@johnrbarker.com>
2021-03-11 12:57:11 -05:00
Matt Davis
3e1f6484d7
add optional module_utils import support (#73832)
* add optional module_utils import support

Treat core and collections module_utils imports nested within any Python block statement (eg, `try`, `if`) as optional. This allows Ansible modules to implement runtime fallback behavior for missing module_utils (eg from a newer version of ansible-core), where previously, the module payload builder would always fail when unable to locate a module_util (regardless of any runtime behavior the module may implement).

* sanity test fixes

ci_complete
2021-03-10 11:08:13 -08:00
Matt Clay
15064c7a42 Update resource_prefix syntax for ansible-test. 2021-03-10 09:00:12 -08:00
Sviatoslav Sydorenko
1d5110db6f
Unify additional info + WYSIWYG as optional input in the documentation report issue form (#73796) 2021-03-09 20:55:14 +01:00
Brian Coca
9db557e431
Nonfatal facts (#73804)
continue with local facts vs at script error
 actually capture execution errors
 better error messages in general
 add more local facts tests

 fixes #52427
2021-03-08 16:20:37 -05:00
Yanis Guenane
212837defc
Documentation: Lay the ground for i18n work. (#73746)
* Documentation: Lay the ground for i18n work.

* Translating both ansible-core and ansible docs requires msgcat
  (a program from the gettext package). If msgcat is not present,
  only the ansible-core index file's strings will be extracted for translation.
  If this is a problem, a short hacking script could be written to take the
  place of msgcat (merging the various index pot files into a single pot file).

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
2021-03-05 09:31:56 -08:00
Abhijeet Kasurde
fb66b4ffbc
distribution: Add Amazon Linux distribution facts (#73767)
* Update Amazon Linux Distribution facts gathering logic
* Update tests

Fixes: #73742

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-03-05 09:16:10 -05:00
David Shrewsbury
78d3810fdf
Auto cleanup of async cache file (#73760)
* Auto cleanup of async cache file

* Add changelog
2021-03-04 11:06:27 -08:00