* ACI module_utils library for ACI modules
This PR includes:
- the ACI argument_spec
- an aci_login function
- an experimental aci_request function
- an aci_response function
- included the ACI team
* New prototype using ACIModule
This PR includes:
- A new ACIModule object with various useful methods
Made the following changes:
* Removed wildcard imports
* Replaced long form of GPL header with short form
* Removed get_exception usage
* Added from __future__ boilerplate
* Adjust division operator to // where necessary
For the following files:
* web_infrastructure modules
* system modules
* linode, lxc, lxd, atomic, cloudscale, dimensiondata, ovh, packet,
profitbricks, pubnub, smartos, softlayer, univention modules
* compat dirs (disabled as its used intentionally)
Absolute path trailing slash handling in absolute directories
find_needle() isn't passing a trailing slash through verbatim. Since
copy uses that to determine if it should copy a directory or just the
files inside of it, we have to detect that and restore it after calling
find_needle()
Fixes#27439
* add_hosts.py: Fix add_host does not recognise 'host' alias
* add_hosts.py: add note in documentation that 'hosts' alias of name parameter is only available on >=2.4
Fixes#13243
** Add --vault-id to name/identify multiple vault passwords
Use --vault-id to indicate id and path/type
--vault-id=prompt # prompt for default vault id password
--vault-id=myorg@prompt # prompt for a vault_id named 'myorg'
--vault-id=a_password_file # load ./a_password_file for default id
--vault-id=myorg@a_password_file # load file for 'myorg' vault id
vault_id's are created implicitly for existing --vault-password-file
and --ask-vault-pass options.
Vault ids are just for UX purposes and bookkeeping. Only the vault
payload and the password bytestring is needed to decrypt a
vault blob.
Replace passing password around everywhere with
a VaultSecrets object.
If we specify a vault_id, mention that in password prompts
Specifying multiple -vault-password-files will
now try each until one works
** Rev vault format in a backwards compatible way
The 1.2 vault format adds the vault_id to the header line
of the vault text. This is backwards compatible with older
versions of ansible. Old versions will just ignore it and
treat it as the default (and only) vault id.
Note: only 2.4+ supports multiple vault passwords, so while
earlier ansible versions can read the vault-1.2 format, it
does not make them magically support multiple vault passwords.
use 1.1 format for 'default' vault_id
Vaulted items that need to include a vault_id will be
written in 1.2 format.
If we set a new DEFAULT_VAULT_IDENTITY, then the default will
use version 1.2
vault will only use a vault_id if one is specified. So if none
is specified and C.DEFAULT_VAULT_IDENTITY is 'default'
we use the old format.
** Changes/refactors needed to implement multiple vault passwords
raise exceptions on decrypt fail, check vault id early
split out parsing the vault plaintext envelope (with the
sha/original plaintext) to _split_plaintext_envelope()
some cli fixups for specifying multiple paths in
the unfrack_paths optparse callback
fix py3 dict.keys() 'dict_keys object is not indexable' error
pluralize cli.options.vault_password_file -> vault_password_files
pluralize cli.options.new_vault_password_file -> new_vault_password_files
pluralize cli.options.vault_id -> cli.options.vault_ids
** Add a config option (vault_id_match) to force vault id matching.
With 'vault_id_match=True' and an ansible
vault that provides a vault_id, then decryption will require
that a matching vault_id is required. (via
--vault-id=my_vault_id@password_file, for ex).
In other words, if the config option is true, then only
the vault secrets with matching vault ids are candidates for
decrypting a vault. If option is false (the default), then
all of the provided vault secrets will be selected.
If a user doesn't want all vault secrets to be tried to
decrypt any vault content, they can enable this option.
Note: The vault id used for the match is not encrypted or
cryptographically signed. It is just a label/id/nickname used
for referencing a specific vault secret.
* Fix fetch configuration in junos_command
Fixes#26358 Add support to fetch configuration
from device in `xml`, `text`, `json`, `set` display
format.
* Add error message if set display is not supported
* Throw error in case of warning for show commands
* AWS key management service fix; statement may not have a principal, and if there is only one AWS principal it will not be a list as expected. Fixes 25786.
* remove len(), only catch exception in function for json.dumps() failure
* use a defined variable and make formatting python 2.6 compatible
* Revert "Update conventions in azure modules"
This reverts commit 30a688d8d3.
* Revert "Allow specific __future__ imports in modules"
This reverts commit 3a2670e0fd.
* Revert "Fix wildcard import in galaxy/token.py"
This reverts commit 6456891053.
* Revert "Fix one name in module error due to rewritten VariableManager"
This reverts commit 87a192fe66.
* Revert "Disable pylint check for names existing in modules for test data"
This reverts commit 6ac683ca19.
* Revert "Allow ini plugin to load file using other encoding than utf8."
This reverts commit 6a57ad34c0.
- New option for ini plugins: encoding
- Add a new option encoding to _get_file_contents
- Use replace option in test/runner/lib/util.py when calling decode on stdout/err
output when diff have non-utf8 sequences
* Module argument_spec now accepts a callable for the type argument, which is passed through and called with the value when appropriate. On validation/conversion failure, the name of the callable (or its type as a fallback) is used in the error message.
* adds basic smoke tests for custom callable validator functionality
when using 'all' to load all plugins were ending in main namespace
creating conflicts with each other and random modulesr
i.e. when trying to access json callback we were getting json 'parsing' lib
* changed collection arg to argregate on 2.4 network modules
* replace users with aggregate in eos_user, junos_user, nxos_user
* added version_added to places where we replaced users with aggregate in the docs
* fix ios_static_route test
* update tests to reference aggregate instead of collection/users
xfs_info is a bash script located in /usr/sbin/ (/sbin is a symlink to
/usr/sbin/) which calls xfs_growfs command. When neither /sbin nor
/usr/sbin are in the PATH environment variable, filesystem module is able to
call xfs_info because /sbin path is hardcoded in get_bin_path method, then
xfs_growfs isn't found because neither /sbin nor /usr/sbin are in the
PATH environment variable.
"xfs_growfs -n" could be used directly instead of xfs_info, the man page
states that: "xfs_info is equivalent to invoking xfs_growfs with the -n
option".
Fixes#24823.
* Updates docs to match module behavior
* Updates docs to match module behavior
Updated using advice from jimi-c
* Removes snippet about git module before v1.8.3
At suggestion of robinro
* Enable the pylint no-name-in-module check. Checks that identifiers in
imports actually exist. When we do this, we also have to ignore
_MovedItems used in our bundled six. This means pylint won't check
for bad imports below ansible.module_utils.six.moves but that's
something that pylint punts on with a system copy of six so this is
still an improvement.
* Remove automatic use of system six. The exec in the six code which
tried to use a system library if available destroyed pylint's ability
to check for imports of identifiers which did not exist (the
no-name-in-module check). That test is important enough that we
should sacrifice the bundling detection in favour of the test.
Distributions that want to unbundle six can replace the bundled six in
ansible/module_utils/six/__init__.py to unbundle. however, be aware
that six is tricky to unbundle. They may want to base their efforts
off the code we were using:
2fff690caa/lib/ansible/module_utils/six/__init__.py
* Update tests for new location of bundled six Several code-smell tests
whitelist the bundled six library. Update the path to the library so
that they work.
* Also check for basestring in modules as the enabled pylint tests will
also point out basestring usage for us.
DNF's base.group_install() function accepts a string as its first
argument. Prior to DNF-2, compatibility code existed which allowed this
function to accept a base.comps.Group object instead. That is no longer
possible.
Pass "group.id" to base.group_install() instead of "group" to work
around this.
restoring behaviour in which a missing module is a warning that shows paths searched,
patch incorrectly bypassed detection and ended up in an exception: global name 'module' is not defined
Currently, someone writing a action plugin will also need
to have a empty file in the module path to avoid triggering
the error "no action detected in task.".
* Fix undefined vars on python3 and a whole bunch of other cleanup.
References #27193
* No need to catch exception and reraise. This just obfuscates the traceback
* Build up a list and then join at the end instead of building up a string. list.append() is faster than string concatenation
* No need to extract k, v pairs from one dict to make a second dict and then extract k, v pairs from the second dict. Iterate over the k, v pairs extracted from the first dict directly instead of building the second dict.
* No need to check if the dict is empty before iterating on it. Iterating on an empty dict will automatically go to the end of the loop
* Use isinstance instead of type(obj) is class, handles inheritance and is better style
* use to_native instead of v.encode(). We can use the surrogate_or_strict error handler to deal with more potential tracebacks. Does the right conversion on both Py2 and Py3.
* Convert bool to string before combining it with the string we're building.
* Don't reference unicode directly as unicode does not exist in Python3
* The string resulting from this function will not have a trailing comma
* Simplify the conversion to string int and bool values are now used in string formatting which will use str to transform them without an explicit invocation.
* Implement state='dist-upgrade'
Implements `zypper dist-upgrade` for the zypper module. This follows
how `zypper upgrade` is invoked, except `state='dist-upgrade'`.
Setting name to anything other than '*' would cause the module to error
out. `dist-upgrade` affects all packages and would not make sense to
apply to a specific package.
* Implement option extra_args
Add option to append additional arguments to zypper command. This
should be able to accommodate other options that are not (yet) covered
by zypper module.
Arguments are given as if written in the command line, complete with
dashes.
This patch adds cookie parsing to the fetch_url/open_url module_utils
method. The overall result will still contain the key `set_cookie`, however
an additional key (`cookies`) will also be present. This new field is a
dictionary of values. Overall, this should make looking for individual
cookies in the response much easier, as currently the `set_cookie` field
is an amalgamation of the returned set-cookie headers and can be somewhat
difficult to parse.
The OpenSSLObject class has been merged[1]. This commit makes the
openssl_publickey rely on this class and standardize the way openssl
module should be written.
[1] #26945
The OpenSSLObject class has been merged[1]. This commit makes the
openssl_privatekey rely on this class and standardize the way openssl
module should be written.
Co-Authored-By: Christian Pointner <cpointner@mgit.at>
[1] https://github.com/ansible/ansible/pull/26945
The error checking would itself cause a traceback. The load_config()
function that we'd need to check for errors from only returns None so
there's no way to check for errors via the return value. In the future
someone could rewrite the load_config function to return useful
information and restore the error checking but for now this is better as
it won't traceback on success and it will let us turn on static analysis
of undefined variables
Fixes#27255
References #27254
* Nuage module and unit tests with requested changes
* Cleanup of imports
* Adding check on python version
* Adding import try and catch wrappers
* Cleanup of requirements and adding integration tests
* Using pypi package for simulator
* Cleanup of requirements and adding integration tests
* Adding aliases for integration tests
* Adding module to import sanity test skip list
* Revert "Adding module to import sanity test skip list"
This reverts commit eab23af8c5.
* Adding check for importlib and cleanup of requirements
Crypto namespace contains the openssl modules. It has no integration
testing as of now.
This commits aims to add integration tests for the crypto namespace.
This will make it easier to spot breaking changes in the future.
This tests currently apply to:
* openssl_privatekey
* openssl_publickey
* openssl_csr
Fixing undefined vars across the codebase so that we can have pylint
catch them on every commit.
Some of the changes to this module_utils file are Python3 related => The
identifiers exist on python2 but not on Python3. The changes should be
portable to both py2 (2.6+) and py3.
References #27193
* Stringify exception with a nicer error and include traceback
* Give a nice message if boto is not installed
* Fix undefined vars. Cleaning up all undefined variables so that we
can run static analysis on new commits.
References #27193
The set_become_default() function is not needed. All this time, if it
was called it would try to set a value on a passed by value variable
(meaning that it had no effect after returning to the caller). Since
become has been working in v2, this code is not needed. Removing.
* Mark _symbolic_mode_to_octal and helper functions as classmethod and staticmethod
These helpers should be made toplevel functions in their own module.
For now, make them staticmethod/classmethod so that they can be used
(and tested) without instantiating an AnsibleModule.
* Move regex compilation out of loops
* Get rid of python-2.4 compat
* vmware_host: Small fixes and docs updates
This PR includes:
- A fix to no longer require a datacenter folder for adding a host
- Documentation improvements
- Ensure imports are specific
* Update vmware_host
Fix adds following:
* Update logic in vmware_host
* Update example documentation
* Added test case for vmware_host
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* surrogate_then_strict doesn't exist. Switch to surrogate_or_strict
instead.
* Found some bugs in the _text.py implementation
* The composed error handlers (error handlers which are made up of two
or more python encoding error handlers) had a wrong string in it,
'surrogate_or_escape' doesn't exist. Replaced that with
'surrogate_or_replace' which is the correct handler name.
* Left comment about the implicit conditions that are part of the
surrogate_then_replace code path
Fixes#23865Fixes#23861
This fixes the symbolic notation of the chmod modes, as stated in the man page of chmod (in Linux). This also takes into account that chmod a+x is different from chmod +x. As the second one should take the current umask into account.
Fixes#14634
* Add example of templating inline using copy module
The **copy** module documentation implies that `content:` only works
for 'simple values' and for complex stuff you need the **template**
module, but that is an understatement. You can use **copy** to template
anything you desire.
So I changed the wording, added an example, and also added a note
to the template module that the **copy** module could be used for
'inline templating'.
This fixes#19741.
One of the complex map + filter + lambda combinations here was causing
pylint to have a rare false positive detecting undefined variables.
Switch to list comprehensions and generator expressions which are
easi(er) to read for humans and do not confuse pylint.
References #27193
* Use atomic_move rather than writing in place in redhat_subscription and redhat utils
* Change status to curated
* Put curated in the proper field
* Add RETURN doc section
* Disable yum plugins when unregistering
* Change back to community supported
* Alphabetize imports
In the past, selinux.security_get_boolean_names did return 'bytes'
on python 3, but this was changed to return string later, cf:
b8711e2eaf
So we have to convert to bytes only if the API return us bytes.
Fix#25651
More openssl modules are about to be made, each of them rewriting
some pieces of code that can be refactored and used via a common
library.
This commit aims to create this "base" object and the common functions
one might want to reuse in order to avoid duplication.
* Added ability to specify destination OU when joining a domain
* Updated win_domain_membership documentation to include OU destination option.
* Updated python to include version
* Additional info in the docs
Two reasons to do this:
- It provides a proper error message indicating why it failed
- It conforms to what is being done in the command and shell modules
unsafe_writes currently allows updating a file that can be updated but
not removed (for instance, when docker mounted). This change also
allows unsafe_writes to write to writable files in unwritable dirs. For
instance, if a system has made a single file inside of /etc/ writable to
a specific normal user.
Fixes#14961
The HTTP User-Agent "ansible-httpget" is already kind of the default,
it being the default value provided by the `url_argument_spec` helper
method. Yet, it may not be practical for all modules to get their
argument_spec that way.
Without a default User-Agent we fall back on the library
User-Agent. That being something like "Python-urllib/2.7".
While I'm no big fan of web servers making decisions based on the
provided User-Agent I still think that part of being a well-behaved
HTTP client is to provide an informative User-Agent. Not to mention
that it's a good thing for Ansible to behave consistently.
Indirectly fixes#26239
Turns out in some commands, like 'sh bgp summary' a line containing
'Not found' can be shown as a column.
This causes those commands to error out with RC=1 .
Fixes#26577
IncludeRole objects don't use _raw_params for the name/etc. of roles
like regular incudes, but the code for finding relative includes assumed
that all includes had a _raw_params field. This fixes that by correctly
checking the parent object type and using the appropriate field.
Fixes#26525
* We need a directory walker that can handle symlinks, empty directories,
and some other odd needs. This commit contains a directory walker that
can do all that. The walker returns information about the files in the
directories that we can then use to implement different strategies for
copying the files to the remote machines.
* Add local_follow parameter to copy that follows local symlinks (follow
is for remote symlinks)
* Refactor the copying of files out of run into its own method
* Add new integration tests for copy
Fixes#24949Fixes#21513
* add unit test: nested dynamic includes
* nested dynamic includes: avoid AnsibleFileNotFound error
Error was:
Unable to retrieve file contents
Could not find or access 'include2.yml'
Before 8f758204cf, at the end of
'path_dwim_relative' method, the 'search' variable contained amongst
others paths:
'/tmp/roles/testrole/tasks/tasks/included.yml' and
'/tmp/roles/testrole/tasks/included.yml'.
The commit mentioned before removed the last one despite the method
docstrings specify 'with or without explicitly named dirname subdirs'.
* add integration test: nested includes
Fix IOS TypeError
* if flags are None, then ' '.join(flags) fails
* fixed get_defaults_flag so that it returns a list, and ignores lines with white space
Fixes#26918
* Revert change to docker_common as it's not as good as the try: except fix
* limit docker_volume fix to ImportErrors
* fix docker_secret i nthe same way
* Remove docker_secret from import tests
* Ensure that include_role properly fires handlers
include_role needs to ensure that any handlers included
with the role are added to the _notified_handler and
_listening_handler lists of the TaskQueueManager, otherwise
it fails when trying to run the handler.
Additionally, the handler needs to be added to the
PlayIterator's `_uuid_cache` or it fails after running
the handler
Add more uuid debug statements - this code was hard
to debug with existing debug statements, so add more
uuid information at little additional output cost.
Fixes#18411
* Add tests for include_role handlers
Tests for #18411
* Adding ciscowlc_command module and unit tests.
* Adding __init__.py for unit test.
* Fixing PEP8 W503.
* Renaming module from ciscowlc_command to aire_command.
* Renaming aire_command to aireos_command.
* Added the docker_volume module
* Code style fixes
* Added yours truly to the copyright statement
* Added documentation link
* Fixed YAML syntax in documentation string
* Documentation style fixes based on the code review
* Implemented requested code corrections
* Added documentation for the "labels" option
* Handled APIErrors from docker-py
* Fixed the type of the "labels" option (dict -> list)
* Fixed typo
* Import APIError from docker_common, not from docker-py
* Only use `git verify-tag` when verifying annotated tags
The command `git verify-tag` only applies to annotated tags. When
verifying lightweight tags, which are more similar to non-moving
branches, one has to use `git verify-commit` instead.
Using ':' as a separator is appropriate since that is one of the
characters not allowed in a Git reference name.
See also https://www.kernel.org/pub/software/scm/git/docs/git-check-ref-format.html
* Improve testing of the Git module's gpg verification
Public key can be extracted extracted in different format from
the PEM formatted RSA pair.
This commit allows the user to specify the format s/he wants to generate
the public key:
* PEM
* OpenSSH
* openssl_publickey: Allow one to specify file permission
Allow a user to specify file permissions on the generated publickey via
the file module common arguments.
* openssl: Add documentation regarding file_common_args
Add documentation for those modules to let the user know that he can
rely on file_common_args to specify file permissions.
