Made the following changes:
* Removed wildcard imports
* Replaced long form of GPL header with short form
* Removed get_exception usage
* Added from __future__ boilerplate
* Adjust division operator to // where necessary
For the following files:
* web_infrastructure modules
* system modules
* linode, lxc, lxd, atomic, cloudscale, dimensiondata, ovh, packet,
profitbricks, pubnub, smartos, softlayer, univention modules
* compat dirs (disabled as its used intentionally)
Absolute path trailing slash handling in absolute directories
find_needle() isn't passing a trailing slash through verbatim. Since
copy uses that to determine if it should copy a directory or just the
files inside of it, we have to detect that and restore it after calling
find_needle()
Fixes#27439
Fixes#13243
** Add --vault-id to name/identify multiple vault passwords
Use --vault-id to indicate id and path/type
--vault-id=prompt # prompt for default vault id password
--vault-id=myorg@prompt # prompt for a vault_id named 'myorg'
--vault-id=a_password_file # load ./a_password_file for default id
--vault-id=myorg@a_password_file # load file for 'myorg' vault id
vault_id's are created implicitly for existing --vault-password-file
and --ask-vault-pass options.
Vault ids are just for UX purposes and bookkeeping. Only the vault
payload and the password bytestring is needed to decrypt a
vault blob.
Replace passing password around everywhere with
a VaultSecrets object.
If we specify a vault_id, mention that in password prompts
Specifying multiple -vault-password-files will
now try each until one works
** Rev vault format in a backwards compatible way
The 1.2 vault format adds the vault_id to the header line
of the vault text. This is backwards compatible with older
versions of ansible. Old versions will just ignore it and
treat it as the default (and only) vault id.
Note: only 2.4+ supports multiple vault passwords, so while
earlier ansible versions can read the vault-1.2 format, it
does not make them magically support multiple vault passwords.
use 1.1 format for 'default' vault_id
Vaulted items that need to include a vault_id will be
written in 1.2 format.
If we set a new DEFAULT_VAULT_IDENTITY, then the default will
use version 1.2
vault will only use a vault_id if one is specified. So if none
is specified and C.DEFAULT_VAULT_IDENTITY is 'default'
we use the old format.
** Changes/refactors needed to implement multiple vault passwords
raise exceptions on decrypt fail, check vault id early
split out parsing the vault plaintext envelope (with the
sha/original plaintext) to _split_plaintext_envelope()
some cli fixups for specifying multiple paths in
the unfrack_paths optparse callback
fix py3 dict.keys() 'dict_keys object is not indexable' error
pluralize cli.options.vault_password_file -> vault_password_files
pluralize cli.options.new_vault_password_file -> new_vault_password_files
pluralize cli.options.vault_id -> cli.options.vault_ids
** Add a config option (vault_id_match) to force vault id matching.
With 'vault_id_match=True' and an ansible
vault that provides a vault_id, then decryption will require
that a matching vault_id is required. (via
--vault-id=my_vault_id@password_file, for ex).
In other words, if the config option is true, then only
the vault secrets with matching vault ids are candidates for
decrypting a vault. If option is false (the default), then
all of the provided vault secrets will be selected.
If a user doesn't want all vault secrets to be tried to
decrypt any vault content, they can enable this option.
Note: The vault id used for the match is not encrypted or
cryptographically signed. It is just a label/id/nickname used
for referencing a specific vault secret.
* Revert "Update conventions in azure modules"
This reverts commit 30a688d8d3.
* Revert "Allow specific __future__ imports in modules"
This reverts commit 3a2670e0fd.
* Revert "Fix wildcard import in galaxy/token.py"
This reverts commit 6456891053.
* Revert "Fix one name in module error due to rewritten VariableManager"
This reverts commit 87a192fe66.
* Revert "Disable pylint check for names existing in modules for test data"
This reverts commit 6ac683ca19.
* Revert "Allow ini plugin to load file using other encoding than utf8."
This reverts commit 6a57ad34c0.
We do want to allow certain from __future__ imports in modules that make
it easier to code compatible python2 and python3. Note that
unicode_literals is specifically left out and should never be allowed.
Now that python-3.4+ allows u"" there's no good reason to use
unicode_literals.
Also switch tables in the validate_modules documentation to simple table format
- New option for ini plugins: encoding
- Add a new option encoding to _get_file_contents
- Use replace option in test/runner/lib/util.py when calling decode on stdout/err
output when diff have non-utf8 sequences
* Module argument_spec now accepts a callable for the type argument, which is passed through and called with the value when appropriate. On validation/conversion failure, the name of the callable (or its type as a fallback) is used in the error message.
* adds basic smoke tests for custom callable validator functionality
* changed collection arg to argregate on 2.4 network modules
* replace users with aggregate in eos_user, junos_user, nxos_user
* added version_added to places where we replaced users with aggregate in the docs
* fix ios_static_route test
* update tests to reference aggregate instead of collection/users
* Enable the pylint no-name-in-module check. Checks that identifiers in
imports actually exist. When we do this, we also have to ignore
_MovedItems used in our bundled six. This means pylint won't check
for bad imports below ansible.module_utils.six.moves but that's
something that pylint punts on with a system copy of six so this is
still an improvement.
* Remove automatic use of system six. The exec in the six code which
tried to use a system library if available destroyed pylint's ability
to check for imports of identifiers which did not exist (the
no-name-in-module check). That test is important enough that we
should sacrifice the bundling detection in favour of the test.
Distributions that want to unbundle six can replace the bundled six in
ansible/module_utils/six/__init__.py to unbundle. however, be aware
that six is tricky to unbundle. They may want to base their efforts
off the code we were using:
2fff690caa/lib/ansible/module_utils/six/__init__.py
* Update tests for new location of bundled six Several code-smell tests
whitelist the bundled six library. Update the path to the library so
that they work.
* Also check for basestring in modules as the enabled pylint tests will
also point out basestring usage for us.
The OpenSSLObject class has been merged[1]. This commit makes the
openssl_privatekey rely on this class and standardize the way openssl
module should be written.
Co-Authored-By: Christian Pointner <cpointner@mgit.at>
[1] https://github.com/ansible/ansible/pull/26945
* Nuage module and unit tests with requested changes
* Cleanup of imports
* Adding check on python version
* Adding import try and catch wrappers
* Cleanup of requirements and adding integration tests
* Using pypi package for simulator
* Cleanup of requirements and adding integration tests
* Adding aliases for integration tests
* Adding module to import sanity test skip list
* Revert "Adding module to import sanity test skip list"
This reverts commit eab23af8c5.
* Adding check for importlib and cleanup of requirements
Crypto namespace contains the openssl modules. It has no integration
testing as of now.
This commits aims to add integration tests for the crypto namespace.
This will make it easier to spot breaking changes in the future.
This tests currently apply to:
* openssl_privatekey
* openssl_publickey
* openssl_csr
* Move tests to their own file
* Port to a pytest parametrized test so it's easier to define new tests
* Now that _symbolic_mode_to_octal is a classmethod, we don't have to
instantiate an AnsibleModule to test it.
* Add tests for #14994, having more than one operator per role and umask
chmod
* vmware_host: Small fixes and docs updates
This PR includes:
- A fix to no longer require a datacenter folder for adding a host
- Documentation improvements
- Ensure imports are specific
* Update vmware_host
Fix adds following:
* Update logic in vmware_host
* Update example documentation
* Added test case for vmware_host
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Add example of templating inline using copy module
The **copy** module documentation implies that `content:` only works
for 'simple values' and for complex stuff you need the **template**
module, but that is an understatement. You can use **copy** to template
anything you desire.
So I changed the wording, added an example, and also added a note
to the template module that the **copy** module could be used for
'inline templating'.
This fixes#19741.
Sometimes MacOSX's pwd doesn't return an expanded path. Not sure why
but this test is still valid if we expand it via a playbook filter so
go ahead and do that.
* We need a directory walker that can handle symlinks, empty directories,
and some other odd needs. This commit contains a directory walker that
can do all that. The walker returns information about the files in the
directories that we can then use to implement different strategies for
copying the files to the remote machines.
* Add local_follow parameter to copy that follows local symlinks (follow
is for remote symlinks)
* Refactor the copying of files out of run into its own method
* Add new integration tests for copy
Fixes#24949Fixes#21513
* add unit test: nested dynamic includes
* nested dynamic includes: avoid AnsibleFileNotFound error
Error was:
Unable to retrieve file contents
Could not find or access 'include2.yml'
Before 8f758204cf, at the end of
'path_dwim_relative' method, the 'search' variable contained amongst
others paths:
'/tmp/roles/testrole/tasks/tasks/included.yml' and
'/tmp/roles/testrole/tasks/included.yml'.
The commit mentioned before removed the last one despite the method
docstrings specify 'with or without explicitly named dirname subdirs'.
* add integration test: nested includes
* list failed tests for iosxr
* list failed tests for ovs
* list failed tests for junos
* list failed tests for ios
* list failed tests for eos
* list failed tests for nxos
* list failed tests for vyos
* Revert change to docker_common as it's not as good as the try: except fix
* limit docker_volume fix to ImportErrors
* fix docker_secret i nthe same way
* Remove docker_secret from import tests
* Ensure that include_role properly fires handlers
include_role needs to ensure that any handlers included
with the role are added to the _notified_handler and
_listening_handler lists of the TaskQueueManager, otherwise
it fails when trying to run the handler.
Additionally, the handler needs to be added to the
PlayIterator's `_uuid_cache` or it fails after running
the handler
Add more uuid debug statements - this code was hard
to debug with existing debug statements, so add more
uuid information at little additional output cost.
Fixes#18411
* Add tests for include_role handlers
Tests for #18411
* Adding ciscowlc_command module and unit tests.
* Adding __init__.py for unit test.
* Fixing PEP8 W503.
* Renaming module from ciscowlc_command to aire_command.
* Renaming aire_command to aireos_command.
* Only use `git verify-tag` when verifying annotated tags
The command `git verify-tag` only applies to annotated tags. When
verifying lightweight tags, which are more similar to non-moving
branches, one has to use `git verify-commit` instead.
Using ':' as a separator is appropriate since that is one of the
characters not allowed in a Git reference name.
See also https://www.kernel.org/pub/software/scm/git/docs/git-check-ref-format.html
* Improve testing of the Git module's gpg verification
* Add default description string to vyos_interface
* If `state=up` it should remove the `disable` configuration
for interface. However, if no other interface parameter is configured
this ends up deleting the interface itself which is not the desired
behaviour. Hence adding a default description field to avoid such
scenario's.
* Minor changes
* Add default description to aggregate
* Adding aruba_command module along with unit tests.
* Fixing PEP8 E303 too many blank lines.
* Adding default for timeout.
* Removing unused arguments. Moving default for timeout argument. Fixing cliconf to find hostname.
* Fixing PEP8 E302.
* Add unit tests for parted module
Test the current expected behavior of the module:
- mock parted() and get_device_info()
- Use some of the examples and test the 'script' passed to parted.
- mock check_parted_label() to return false, as if parted version is > 3.1
- assert get_device_info output is correct
Current implementation of the module runs parted several time while going
through all parameters (flags, name, ...). Between calls it uses get_device_info
to update the dictionary. Use check_mode for some of the tests to force module
to go through all the parameters even is dictionary is not updated.
* test_parted.py: add "name" param into expected results
since 78fff751ab, parse_partition_info
fetch the partition name. This commit adds 'name' key and value into
the expected results.
Consolidate the module_utils, constants, and config functions that
convert values into booleans into a single function in module_utils.
Port code to use the module_utils.validate.convert_bool.boolean function
isntead of mk_boolean.
* aci_rest: New module to access Cisco ACI
This PR includes:
- Relicense as GPLv3+
- Check-mode support
- Cosmetic changes to documentation
- Examples in YAML format
- Removal of incorrect requirements (for this module)
- Do not log passwords
- Implement native fetch_url instead of requests
- Use standard hostname, username and password parameters
- Add alias src for parameter config_file
- Add mutual exclusive content option for inline data (and show some inline examples)
- Add timeout parameter
- Add validate_certs parameter
- Handling ACI result output (identical for JSON as XML input)
- Parse/expose ACI error output to user
* Lower case method, add use_ssl, Use python dicts
This commit includes:
- Use lowercase method names
- Add `use_ssl` parameter (not the `protocol` parameter)
- Use a python dict for the request data (not a JSON string)
- Documentation improvements
* Ensure one of 'content' or 'src' is provided
* Fix issue with totalCount being a string in JSON
This fixes the problem with JSON output where totalCount is a string and
not an integer.
This fixesjedelman8/aci-ansible#7
* Improve code documentation
* Improve error handling and module response
* Small typo
* Improve documentation and examples
* Keep protocol parameter, but deprecate it
* Extrude aci functions from module_utils
* aci_rest: Add unit tests
The 'admin' word was being masked by Ansible as potential cred.
Let's just use network-operator since we are just testing here
we can create users in aggregate.
* Use Boto3 for ec2_group
Currently boto doesn't support ipv6. To support ipv6 in ec2_group, we need boto3.
boto3 has significant API changes, which caused more re-factoring for ec2_group module.
Added additional integration test to test_ec2_group role.
* Follow the standard for boto3 ansible
Fixed imports. Use boto3 ansible exception with camel_dict_to_snake_dict.
Refactored the call to authorize/revoke ingress and egress.
* Removed dependancy with module ipaddress
Added new parameter called cidr_ipv6 for specifying
ipv6 addresses inline with how boto3 handles ipv6 addresses.
* Updated integration test
* Added ipv6 integration test for ec2_group
* Set purge_rules to false for integration test
* Fixed import statements
Added example for ipv6.
Removed defining HAS_BOTO3 variable and import HAS_BOTO3 from ec2.
Cleaned up import statements.
* Fixed exception handling
* Add IAM permissions for ec2_group tests
Missing AuthorizeSecurityGroupEgress necessary for latest tests
* Wrapped botocore import in try/except block
Import just botocore to be more similar to other modules
* junos implementation for net_l3_interface module
* junos_l3_interface implementation
* junos_l3_interface integration test
* net_l3_interface integration test for junos
* Fix module name typo
* All integration commands support --continue-on-error
* The network-integration command supports --inventory
* Add landing page for compile test docs.
* Add bot documentation links.
* once smoketest is implemented in CI, will run a much smaller set of Windows module integration tests for changes to module_utils or connection subsystems
* win_secedit: Added module with tests/diff mode
* fixed up test issues
* Added missing return value
* change for win_secedit based on review
* updated win_security_policy examples for rename
These integration tests were used for testing the exact behaviour of
Ansible for YAML-style syntax and key=value syntax.
This includes fixes to win_shortcut (as `src` can be a URL too)
* win_regedit: rewrite to support edge cases and fix issues
* fix up byte handling of single bytes and minor doc fix
* removed unused method
* updated with requested changes
* vyos implementation for net_interface module
* vyos_interface implementation module
* vyos_interface integration test
* net_interface integration test for vyos
* Change collection to aggregate
* Fix the editable condition into pip module (#19028)
* Add editable to tests
Default changed to False, so now editable: True is needed explicitly in
tests
* Revert "Temporarily disable zypper tests due to timeouts."
This reverts commit 25f943e22f.
* Revert "Disable zypper_repository tests due to errors."
This reverts commit 5833197a19.
* until we can have a robust internal source for chocolatey (and/or PSScriptAnalyzer sanity tests), we want to at least make sure that the module is syntactially valid.
* vmware_guest: fixes for cache objects and datacenter association
* find_all_objs was only looking for datastores
* Clear the result if it's datacenter is not correct.
* Re-enable pyvmomi installation
Addresses #25011
Addresses #26511
* add first, last and next usable
* add usable ip filters
* add size usable, range usable and wildcard
* add ip prefix and netmask filter
* add network formatting and check if ip in subnet
* clean up order, add comments
* fix pep8
* update format by index
* clean up and updates from jmcgill298
* Slight refactor on vmware_guest to fix path searching and vcsim compatibility.
* Clean up pep8 errors
* Fix more pep8 issues
* Add assertions
* Remove debug failure
* Update docsting for folder with examples
* Revise _get_vm_prop based on suggestions
* Implement folder walker to find a folder path
* More enhancements for datacenter->datacenter cloning
* Fix a few pep8 issues
* Remove useless check for subclass of None
* Fix a few suggestions
* Move serialize_spec to a util function
* Group exceptions together
Fixes#25011
* add connection plugin for buildah
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* fixup
* create a method to invoke buildah
* mount container filesystem persistently so we can access it
during put and fetch
* use copyfile function for copying files
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* revert tests
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* fixup
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* progress
(this will be squashed into a single commit before merge)
* add docs for the conn plugin
* fix issue invoking the integration tests
* add a way to invoke commands inside the container as a different user
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* fix shellcheck warning
Signed-off-by: Tomas Tomecek <ttomecek@redhat.com>
* adds more intelligent save logic and diff to network config modules
* adds sha1 property to NetworkConfig
* adds new argument save_when to argument_spec
* adds new argument diff_against to argument_spec
* adds new argument intended_config to argument_spec
* renames config argument to running_config with alias to config
* deprecates the use of the save argument
* before and after now work with src argument
* misc module clean
Modules updated
* nxos_config
* ios_config
* eos_config
Most notably this makes the save mechanism more intelligent for config
modules for devices that need to copy the ephemeral config to
non-volatile storage.
The diff_against argument allows the playbook task to control what the
device's running-config is diff'ed against. By default it will return
the diff of the startup-config.
* removes ios_config from pep8/legacy_files.txt
* extends the ignore lines argument to the module
* clean up CI errors
* add missing list brackets
* fixes typo
* fixes unit test cases
* remove last line break when returning config contents
* encode config string to bytes before hashing
* fix typo
* addresses feedback in PR
* update unit test cases
Module was importing '*' from facts to get to TimeoutError
but that has moved to facts.timeout, so import is updated.
Also rm old style imports to new style imports at the start
of the module.
'signal' py module was used and referenced but never imported,
presumably it was using the 'signal' previously imported into
module_utils.facts. Now imported directly.
'AnsibleModule' was also from a * import, so now imported directly.
A ref to 'module' was in _delete_disks_when_detached(), so now it
is updated to raise an AzureException() with its message, and
let its caller catch it and call module.fail_json()
It was in lib/ansible/modules/system/setup.py since it
was the only thing using it, but move it back to module_utils
and add a ansible_collector.get_ansible_collector() to build
a facts collector just like the one used by setup.py
mv test_setup.py -> test_ansible_collector.py
All the code it was testing is now in ansible_collector
rm code to create 'ansible_facts' subkey from namespace
Just leave it up to the caller to do, and just return a
flat dictionary from AnsibleFactCollector.collect()
* junos_linkagg implementation and junos modules refactor
* junos_linkagg implementation
* junos_linkagg integration test
* net_linkagg integration test for junos
* decouple `load_config` and `commit` operations,
to allow single commit (in case on confirm commit) and
to perform batch commit (multiple `load_config` followed by single
`commit`)
* Other related refactor
* Fix CI issues
* Fix unit test failure
* invalid drive letter breaks new Powershell path validation (this is a separate issue that needs to be solved)
* test also exposes a minor bug in win_get_url where a nonexistent directory in the root of a drive that exists passes the "does the parent dir exist" check and causes DownloadFile to bomb (this should be fixed by dag's rewrite of win_get_url in https://github.com/ansible/ansible/pull/26612)
* Feature #2731: added postgres import and dump
* Feature #2731: be more permissive of arguments
```
hacking/test-module -m ./ppostgresql_db.py -a "db=example state=dump target=/tmp/out"`
```
failed previously since host, user, and port were required as keywords
in the pg_dump / pg_import methods.
* Feature #2731: fixed doc string for validate-modules
```
$ ansible-validate-modules database/postgresql/
```
now passes.
* Feature #2731: disable 'password' for dump/restore
* Feature #2731: bump added version to 2.3
* Feature #2731: replace db_import with db_restore
* Feature #2731: add missing version description
* Feature #2731: fix 'state' description
* Feature #2731: fix pep8 issues
* Feature #2731: put state documentation in a single string
* Bump added version from 2.3 to 2.4
* Fix pep8 and pylint errors
* Attempt yaml formatting of documentation string
* Add integration tests for postgres_db:dump/restore
* Update dump/restore logic to support new kw-args
Also attempt to support password; integration tests are
still failing.
* Revert to postgres user for dump/restore
Passing PGPASSWORD is not working for subprocesses. For the
moment, reverting to the strategy of failing if login_password
is set and using `postgres` for all testing of dump/restore.
* Various cleanups to have tests passing
* Working tests for {sql,tar} x {,bz2,gz,xz}
* Use pg_user to support FreeBSD
* Revert login_ prefixes and re-enable password support
All `login_` keywords are mapped to their non-prefix versions
so the previous changes were effectively using `postgres` for
all actions. With the proper keywords, PGPASSWORD-passing to
the subprocess is now working.
* Optionally add password
environ_update doesn't handle None values in the
dictionary to be added to the environment. Adding
check.
* Quick fixes
* Refactor login arguments after fixes from pchauncey
The fixes introduced by pchaunchy pointed to further issues
(like no --dbname on PG<=9.2) with the login parameters. This
refactors them and adds further tests.
Note: this will still not pass integration tests due to a further
issue with pg_dump as a non-admin user:
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 1925; 0 0 COMMENT EXTENSION plpgsql
pg_restore: [archiver (db)] could not execute query: ERROR: must be owner of extension plpgsql
* Introduce target_opts for passing limiting dumped/restored schemas
The current integration tests (PG version and template DBs) don't
permit a regular user (`{{ db_user1 }}`) access to plpgsql causing
restores to fail. By adding an option for passing arbitrary args to
pg_dump and pg_restore, testing is made easier. This also paves the
way for `-j` usage, once the PG version is bumped.
The iApp service module worked fine previously, but this patch
adds enhancements to it to include more fields that can be
specified when creating iapp services.
* add enhanced run option support for win_scheduled_task
* changed run_level option to runlevel
* correct merge conflicts since task path fix
* changed run_level option to runlevel
* changed do_not_store_password to store_password, and other minor fixes
* conditional logic swap, and documentation change for password
* postgresql_user module - transaction logic hacks to allow recovery from failed select
* postgresql_user - PEP8 and style fixes to make debugging easier
* postgresql_user - move password changing logic to separate function
* postgresql_user - trap failure in case where there is no access to pg_authid
* postgresql_user - further PEP8 fixes
* postgresql_user - Simplify password change logic and improve imports according to suggestions from PR review
* postgresql_user - Eliminate pep8/blank line errors introduced in merge
* Check behaviour when pg_authid relation isn't readable
TASK [postgresql : Normal user isn't allowed to access pg_authid relation:
password comparison will fail, password will be updated] ***
An exception occurred during task execution. To see the full traceback,
use -vvv. The error was: psycopg2.ProgrammingError: permission denied
for relation pg_authid
* Don't reintroduce passlib, remove useless query
This PR includes:
- Checkmode improvements
- Integration tests
- A fix for python3
- PEP8 fixes
This backports improvements from the win_wakeonlan module.
* Implementation of junos_static_route module
* junos implementation of net_static_route
* integration test for junos_static_route
* integration test for junos net_static_route
* Minor change
* Doc change
* Fix CI issue
* Move Config classes from executor.py to config.py.
* Move Environment and Test config to config.py.
* Move Coverage/CoverageReport Config to config.py.
* Clean up type hints.
* junos_user declarative module changes
* Active/Deactivate support
* junos_user integration test
* net_user intergration test for junos
* Add version_added for active param
restored 'rc' inspection but only when failed is not specified
removed redundant changed from basic.py as task_executor already adds
removed redundant filters, they are tests
added aliases to tests removed from filters
fixed test to new rc handling
* as a result of recent core engine changes to ignore rc, modules are responsible to set `failed` on nonzero RC if they want that behavior
* the `failed` filter currently triggers on nonzero RC, which caused the tests to false-pass
* updated tests to explicitly check both rc and failed keys, as well as using the failed filter.
