* fix: action _fixup_perms2 macos +a remote_paths in list() as it can be tuple
in `lib/ansible/plugin/action/__init__.py`'s `_fixup_perms2`,
`remote_paths` can be a list or tuple. however, the macos
specific attempt to use chmod +a attempts to concatenate
`remote_paths` with a list, which will fail if it is a tuple.
wrapping `remote_paths` in `list()` fixes this error.
* Update changelogs/fragments/74613-actionfixup_perms2_macos_remote_paths_ensure_list.yml
Co-authored-by: Rick Elrod <rick@elrod.me>
* support separate role argspec file in ansible-doc
* support separate role argspec file in ansible-core
* support both .yml and .yaml extensions on argspec file in ansible-doc
* fix filename building bug and rename some argspec files to test variations
* use yaml extensions from constants
* add superfluous meta/main.yml files to tests
* Update lib/ansible/cli/doc.py
Co-authored-by: Sam Doran <sdoran@redhat.com>
* update docs
* ci_complete
* add changelog and allow for main.yml variations
* add collection role testing
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Get available collection versions with page_size=100 for v2 and limit=100 for v3
* Update unit tests for larger page sizes
* Add a generic retry decorator in module_utils/api.py that accepts an Iterable of delays and a callable to determine if an exception inheriting from Exception should be retried
* Use the new decorator to handle Galaxy API rate limiting
* Add unit tests for new retry decorator
* Preserve the decorated function's metadata with functools.wraps
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Always use create_homedir when we are asked to create a home directory
in the User class. Don't use the -m and -k parameters from
useradd / luseradd as they behave differently with respect to
preexisting home directories. Instead always specify -M to ensure
that useradd / luseradd do not try to create the home directory.
This does not change potential different behaviours in child classes
of the User class.
Consider the new umask option from #73821 in create_homedir as well as
we do not let luseradd / useradd create the home directory any longer.
From the sha512sum man page:
... The default mode is to print a line with checksum, a character indicating type ('*' for binary, ' ' for text), and name for each FILE.
The distutils module is not shipped with SUNWPython on Solaris.
It's in the SUNWPython-devel package. Do not use LooseVersion.
Fixes: #74488
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* removal of deprecations and minor fix
name or cron_file is required.
reboot was removed.
updated docs to clarify usage.
now using cron_file=/etc/crontab is an error
fixes#37355
* warn show pathing issues when wallking through provided paths
moved issues from msg to actual warnings AND a specific return field
fixes#25314
Co-authored-by: Sam Doran <sdoran@redhat.com>
* pause - ensure control characters are always set appropriately
On some systems, curses.tigetstr() returns None, which does not work as a control character.
* Add unit tests
* Sort imports
* Skip on older Python
This is an action plugin and only runs on the controller, so no need to test of Python 2. Making
the import hackery work on Python 2 would required some more work which I am not sure is
worth it since we are moving away from Python 2 support on the controller.
* Make the tests work on Python 2 and 3
Document that systemd module manages units (timers, services, ...) and not only service. (#74442)
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
* Fix section for inventory cache prefix
* Redirect general config inventory cache options to inventory plugin documentation
* Update the template to fix the docs-build error
* removes 'default' entries for config settings with default: ''
* single backticks => italics, double backticks => code
* use 'Default: ~' to match other entries that have no default setting
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Change:
- After we output the newline in help text, we also added a space. This
caused either a space to show up before the shell prompt (in shells
like bash), or " %" and a newline before the next prompt on shells
like zsh that automatically force commands to end with a newline.
- This change removes the extra space, so that peace can be restored in
the minds of many.
- This only showed up when running commands with no arguments or invalid
arguments.
Test Plan:
- Ran `ansible`, `ansible -h`, `ansible-playbook`, `ansible-galaxy`, and
`ansible-galaxy -h`.
Tickets:
- Introduced in #69458
Signed-off-by: Rick Elrod <rick@elrod.me>
* dont specify default for port, allow ssh/config
also added general note on how defaults work.
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Not all GnuPG return codes were analyzed (rc != 0) and not all relevant GnuPG error information was returned by the 'ansible.builtin.apt_key' module (https://github.com/ansible/ansible/issues/74477)
* Update changelogs/fragments/74478-apt_key-gpg-error-check.yaml
Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Fix BaseFileCacheModule#keys to respect prefix
Change:
- Previously BaseFileCacheModule#keys would return keys with the cache
prefix. These keys are impossible to retrieve from the cache without
removing the prefix or using the cache without a prefix.
Now it removes the prefix from the key and only returns keys that
share the same prefix as the cache.
Test Plan:
- New unit tests
* Add explicit BaseFileCacheModule#keys test
Test that files that do not match the cache prefix are ignored.
Test that the prefix is removed from the cache key.
* dont rely on vars, task already gives us info
* ensure we always display delegation in host label
* also added parens with ansible_host to show target host vs resolved host
* delegating to self is not delegating
* delegated vars restoration for backwards compat
* tests need mock task with delegate_to
* correctly parse device from string
* check for command presence before running them
* check for command presence and return code for solaris and aix as well
* add changelog
Change:
- Instead of returning the `str` type, return the value that was
calculated.
Test Plan:
- New unit tests.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix world readable setting
now uses shell option in all cases, the old constant still exists
as 'documentation' but it not settable.
also fix the docsite link in warnings/errors
* fix typose#
* fix comment#
* added deprecated to config
* missing :
* import cleanup
Fixes#74255
* Fix call to 'unique(case_sensitive=False)' triggering error when falling back to Ansible's version which **is** case-sensitive
* Test multiple situations of 'unique' filter errors with fallback not handling specific parameters
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Rick Elrod <rick@elrod.me>
Homebrew's default install location for macOS on ARM is /opt/homebrew.
Source: https://docs.brew.sh/FAQ
On a Mac M1 (Apple Silicon), homebrew will be installed at
/opt/homebrew/bin/brew.
* Add yaml utils file and use it
* Linting fix and missing import
* Abstract a few more details
* Parser imports
* Don't use CSafeDumper in AnsibleDumper
* Move and use convert_yaml_objects_to_native when libyaml is present
* yaml_load and yaml_dump, instead of safe_*
* re-use HAS_LIBYAML from utils.yaml
* add changelog fragment
* Address recent changes
* Use representer instead of recursive type converter
* Restore needed import
* move yaml utils to module_utils
* Properly guard imports
* Update from_yaml(_all)? to handle text wrappers with CSafeLoader
* Use yaml utils for legacy_collection_loader
* Add HAS_YAML, and ignore pylint issue
* oops
* GPL->BSD
* Map Debian 8 to Python 2
If Python 3 is installed on Debian 8 Ansible cannot run, as the version
is too old (3.4)
* Add integration test for python interpreter discovery on Debian 8
* fix test issue on Debian 9, add changelog
* un"fix" not broken test :D
Co-authored-by: Fabian Klemp <fabian.klemp@elara-gmbh.de>
Co-authored-by: Matt Davis <mrd@redhat.com>
* Add ArgumentSpecValidator to docs
* Improve docs for ArgumentSpecValidator
* Document removal of private methods
* Update module_utils documentation
- Add docs for argument spec classes as well as validation and parameters files.
- preserve the order in the source for errors.py
- document DEFAULT_TYPE_VALIDATORS so it can be referenced elsewhere
- fix automodule directive for validation.py
* Update docs in arg_spec and paremeters
- This improves the generated documentation.
* Document breaking changes in porting guide.
* Update formatting in porting guide and add a Deprecated section
* Fine tune module_utils documentation
* Move instance docstring to the __init__ method
Remove optional description since it fails the sanity test and I am not 100% it is valid anyway.
* Remoe incorrect parameter from docstring
This was changed a while ago but wasn't removed from the docstring.
* Use attr rather than attribute
The py:attribute: domain only exists in newer Sphinx >= 3.1.
* Improve documentation on exceptions
* Final pass
- use args/kwargs instead of param
- fix formatting errors that didn't display examples correctly
- format TypeErrors so they are referenced as classes
- specify complex types
* Add umask option to user module
* Fail on setting both umask and local: True
* Add integration test
* Add changelog
* Run integration tests only if HOME_MODE is not set
* Run integration tests only on Linux
Co-authored-by: Matt Clay <matt@mystile.com>
Change:
- Newer Solaris drops setfacl. Add a fallback for its chmod ACL syntax.
Test Plan:
- New units
Tickets:
- Fixes#74282
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Regression introduced in #70785
- When macOS chmod ACL syntax is used, Solaris-derived chmods return
with a status of 5. This is also used for our sshpass handling,
because sshpass will return 5 on auth failure. This means on Solaris,
we incorrectly assume auth failure when we reach this branch of logic
and try to run chmod with macOS syntax.
- We now wrap this specific use of chmod in an exception handler that
looks for AnsibleAuthenticationFailure and skips over it. This adds
another authentication attempt (something we normally avoid to prevent
account lockout), but seems better than the regression of not allowing
other fallbacks to be used.
- Without this patch, if setfacl fails on Solaris (and sshpass is used),
we do not try common_remote_group or world-readable tmpdir fallbacks.
Test Plan:
- New unit
Signed-off-by: Rick Elrod <rick@elrod.me>
* Handle netconf plugin ncclient import error when running in FIPS mode
* While running in FIPS mode importing ncclient result in
InternalError raised by cryptography
* Refer https://github.com/ansible/ansible/pull/65477
This simplifies rendering the hostname (or hostname+delegated host) in
the default callback module, and reduces code duplication
I've chosen not move where in each handler the host label is rendered,
in case subsequent operations has side effects. However I'm happy to
change that if considered safe.
I've chosen not to change the formatting operator used (%), to avoid
changes in rendering that might result.
Signed-off-by: Alex Willmer <alex@moreati.org.uk>
Change:
- We were passing a directory name directly to re.compile().
If the directory isn't valid regex (or is) this can have odd side
effects, such as crashing.
- Fix a few other similar cases, but less likely to be a real issue.
Test Plan:
- New test
Signed-off-by: Rick Elrod <rick@elrod.me>
* Catch errors getting filters, and fail
* Add changelog
* Switch to warnings instead of errors, to match other plugin loader behavior
* Add tests
* Handle collections
* deprecated include
Update lib/ansible/modules/_include.py
updated version numbers in schema check (real fix in separate PR)
Co-authored-by: flowerysong <junk+github@flowerysong.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* all lookups to support config system
- added get_options to get full dict with all opts
- fixed tests to match new error messages
- kept inline string k=v parsing methods for backwards compat
- placeholder depredation for inline string k=v parsing
- updated tests and examples to also show new way
- refactored and added comments to most custom k=v parsing
- added missing docs for template_vars to template
- normalized error messages and exception types
- fixed constants default
- better details value errors
Co-authored-by: Felix Fontein <felix@fontein.de>
* various fixes to command
- Updated splitter to allow for all expected args in ad-hoc
- Ensure we always return the returns we promissed to always return (i.e stderr/stdout)
- Updated docs to clarify creates/removes precdence in checking
- Removed abspath from chdir to allow reporting to handle symlinks correctly
- Corrected tests to new output messages
Co-authored-by: Felix Fontein <felix@fontein.de>
* Support omitting the trailing separator when a dictionary key's value is an empty string
* Support a default value when the value used in the group name is an empty string
* Add tests
* change log
Change:
- When a "distro" package exists in PYTHONPATH but isn't what we expect,
fall back to our own vendored one and use it. This prevents a
traceback if someone has some random thing that provides "distro" but
isn't actually the "distro" library we need.
Test Plan:
- new tests
Tickets:
- Fixes#74228
Signed-off-by: Rick Elrod <rick@elrod.me>
* nuke playbook test file
Signed-off-by: Rick Elrod <rick@elrod.me>
* test fixes
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Tests for advanced_host_list; it is now at 100% coverage.
- There was a typo (host vs hostnames) where when the host pattern
failed to parse, instead of treating the name as a literal as
intended, it would trigger UnboundLocalError. This didn't fatal
as there's a global "Exception" handler below, but it did lead to
an ugly error and incorrect behavior.
Test Plan:
- New tests
- Local experimenting
Signed-off-by: Rick Elrod <rick@elrod.me>
* changelog
Signed-off-by: Rick Elrod <rick@elrod.me>
* fix tests
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Mostly increase error coverage for various conditions in play.py
- Also fix a string in an error, where get_name() was called before
self.name was read in, so get_name() was always ''.
Test Plan:
- new tests
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix regex for py2 and py3
Signed-off-by: Rick Elrod <rick@elrod.me>
* py2 hates me
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Variables used in ``when`` conditionals are no longer parsed and
attempted to be converted to booleans. All non-empty strings are
considered true (empty strings, false).
Test Plan:
- Updated existing tests
- Added a bunch of new tests with various kinds of truthy/falsy
values.
Tickets:
- Fixes#74134
Signed-off-by: Rick Elrod <rick@elrod.me>
use load_name of valid plugin to find config, will handle the renamed loaded plugin that collections do:
`ansible_collection.<ns>.<collname>.plugins.<type>.<name>`
Change:
- Previously when the same package name was installed twice under
different architectures, we only reported it once in changes.updated.
- This was the result of using a dict internally and keying on package
name alone.
- This change still keys on package name but turns the values into lists
which can contain multiple packages per name.
Test Plan:
- Added a lot of tests around multi-arch support
- Added some tests around virtual provides as well
Tickets:
- Fixes#73284
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Remove two deprecated features
- We now error if a playbook is an empty list instead of just skipping
- We now error if using 'include' instead of 'import_playbook'
Test Plan:
- Added new tests for new errors
Tickets:
- Fixes#74133
Signed-off-by: Rick Elrod <rick@elrod.me>
* sanity & changelog
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Require name always
- Drop 'reboot' parameter in favor of 'special_time: reboot'
Test Plan:
- CI
Tickets:
- fixes#74132
Signed-off-by: Rick Elrod <rick@elrod.me>
* --offline allows in-place verify for installed collections with manifests
* manifest hash, collection name, version, and path are now always displayed
* test updates
* Begin using ArgumentSpecValidator in AnsibleModule
* Add check parameters to ArgumentSpecValidator
Add additional parameters for specifying required and mutually exclusive parameters.
Add code to the .validate() method that runs these additional checks.
* Make errors related to unsupported parameters match existing behavior
Update the punctuation in the message slightly to make it more readable.
Add a property to ArgumentSpecValidator to hold valid parameter names.
* Set default values after performining checks
* FIx sanity test failure
* Use correct parameters when checking sub options
* Use a dict when iterating over check functions
Referencing by key names makes things a bit more readable IMO.
* Fix bug in comparison for sub options evaluation
* Add options_context to check functions
This allows the parent parameter to be added the the error message if a validation
error occurs in a sub option.
* Fix bug in apply_defaults behavior of sub spec validation
* Accept options_conext in get_unsupported_parameters()
If options_context is supplied, a tuple of parent key names of unsupported parameter will be
created. This allows the full "path" to the unsupported parameter to be reported.
* Build path to the unsupported parameter for error messages.
* Remove unused import
* Update recursive finder test
* Skip if running in check mode
This was done in the _check_arguments() method. That was moved to a function that has no
way of calling fail_json(), so it must be done outside of validation.
This is a silght change in behavior, but I believe the correct one.
Previously, only unsupported parameters would cause a failure. All other checks would not be executed
if the modlue did not support check mode. This would hide validation failures in check mode.
* The great purge
Remove all methods related to argument spec validation from AnsibleModule
* Keep _name and kind in the caller and out of the validator
This seems a bit awkward since this means the caller could end up with {name} and {kind} in
the error message if they don't run the messages through the .format() method
with name and kind parameters.
* Double moustaches work
I wasn't sure if they get stripped or not. Looks like they do. Neat trick.
* Add changelog
* Update unsupported parameter test
The error message changed to include name and kind.
* Remove unused import
* Add better documentation for ArgumentSpecValidator class
* Fix example
* Few more docs fixes
* Mark required and mutually exclusive attributes as private
* Mark validate functions as private
* Reorganize functions in validation.py
* Remove unused imports in basic.py related to argument spec validation
* Create errors is module_utils
We have errors in lib/ansible/errors/ but those cannot be used by modules.
* Update recursive finder test
* Move errors to file rather than __init__.py
* Change ArgumentSpecValidator.validate() interface
Raise AnsibleValidationErrorMultiple on validation error which contains all AnsibleValidationError
exceptions for validation failures.
Return the validated parameters if validation is successful rather than True/False.
Update docs and tests.
* Get attribute in loop so that the attribute name can also be used as a parameter
* Shorten line
* Update calling code in AnsibleModule for new validator interface
* Update calling code in validate_argument_spec based in new validation interface
* Base custom exception class off of Exception
* Call the __init__ method of the base Exception class to populate args
* Ensure no_log values are always updated
* Make custom exceptions more hierarchical
This redefines AnsibleError from lib/ansible/errors with a different signature since that cannot
be used by modules. This may be a bad idea. Maybe lib/ansible/errors should be moved to
module_utils, or AnsibleError defined in this commit should use the same signature as the original.
* Just go back to basing off Exception
* Return ValidationResult object on successful validation
Create a ValidationResult class.
Return a ValidationResult from ArgumentSpecValidator.validate() when validation is successful.
Update class and method docs.
Update unit tests based on interface change.
* Make it easier to get error objects from AnsibleValidationResultMultiple
This makes the interface cleaner when getting individual error objects contained in a single
AnsibleValidationResultMultiple instance.
* Define custom exception for each type of validation failure
These errors indicate where a validation error occured. Currently they are empty but could
contain specific data for each exception type in the future.
* Update tests based on (yet another) interface change
* Mark several more functions as private
These are all doing rather "internal" things. The ArgumentSpecValidator class is the preferred
public interface.
* Move warnings and deprecations to result object
Rather than calling deprecate() and warn() directly, store them on the result object so the
caller can decide what to do with them.
* Use subclass for module arg spec validation
The subclass uses global warning and deprecations feature
* Fix up docs
* Remove legal_inputs munging from _handle_aliases()
This is done in AnsibleModule by the _set_internal_properties() method. It only makes sense
to do that for an AnsibleModule instance (it should update the parameters before performing
validation) and shouldn't be done by the validator.
Create a private function just for getting legal inputs since that is done in a couple of places.
It may make sense store that on the ValidationResult object.
* Increase test coverage
* Remove unnecessary conditional
ci_complete
* Mark warnings and deprecations as private in the ValidationResult
They can be made public once we come up with a way to make them more generally useful,
probably by creating cusom objects to store the data in more structure way.
* Mark valid_parameter_names as private and populate it during initialization
* Use a global for storing the list of additonal checks to perform
This list is used by the main validate method as well as the sub spec validation.
When using "use_regex: yes" and setting an excludes: without
specifying a pattern: the existing code passes the file-glob '*' to
the regex matcher. This results in an internal invalid-regex
exception being thrown.
This maintains the old semantics of a default match-all for pattern:
but switches the default to '.*' when use_regex is specified.
The code made sense as-is before excludes: was added (2.5). In that
case, it made no sense to set use_regex but *not* set a pattern.
However, with excludes: it now makes sense to only want to exclude a
given regex but not specify a specific matching pattern.
Closes: #50067
* moved change to new location
added changelog
* Update lib/ansible/modules/find.py
Co-authored-by: Ian Wienand <iwienand@redhat.com>
* Fix a bug adding unrelated candidates to the plugin loader redirect_list
* Add tests for the redirect list
* test redirect list for builtin module
* test redirect list for redirected builtin module
* test redirect list for collection module
* test redirect list for redirected collection module
* test redirect list for legacy module
* changelog
Ansible can gather distribution facts for older Amazon Linux
with /etc/os-release data.
Fixes: #73946
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* ansible-pull: run all playbooks when multiple are supplied
* add test for ansible-pull with multiple playbooks supplied from cli
* add changelog fragment
* Catch more potential errors (and increase false-positive rate).
* Flag some false-positives in lib/ansible/modules/ with no_log=False.
Co-authored-by: John Barker <john@johnrbarker.com>
* add optional module_utils import support
Treat core and collections module_utils imports nested within any Python block statement (eg, `try`, `if`) as optional. This allows Ansible modules to implement runtime fallback behavior for missing module_utils (eg from a newer version of ansible-core), where previously, the module payload builder would always fail when unable to locate a module_util (regardless of any runtime behavior the module may implement).
* sanity test fixes
ci_complete
