* Optimize file handling
Use the best practice of opening and doing operations on an opened file
Signed-off-by: Daniel Andrei Minca <mandrei17@gmail.com>
* Fix docstring to Sphinx type
- update the docstrings to Sphinx type, as suggested by Toshio
- Move the pattern object assignment outside the context manager, as
suggested by Matt
Signed-off-by: Daniel Andrei Minca <mandrei17@gmail.com>
* Handle duplicate headers, and make it easier for users to use cookies, by providing a pre-built string
* Ensure proper cookie ordering, make key plural
* Add note about cookie sort order
* Add tests for duplicate headers and cookies_string
* Extend tests, normalize headers between py2 and py3
* Add some notes in test code
* Don't use AttributeError, use six.PY3. Use better names.
* Error if a module is found to shadow a reserved keyword
* Add test for shadowed module
* Bring in functools.wraps for the decorator
* Drop the decorator, make _find_plugin the real function, find_plugin now holds the shadow logic
* Swap order of functions for bottom to top execution order
* Only error for modules
* Add test for loading a lookup plugin that shadows a keyword
* Reflect updated names of the aws_batch_job modules
* Remove mention of `lambda_invoke` module which has yet to make it to
Ansible.
* Update broken rst link in win_dsc module
* Start of tests for ansible.module_utils.urls
* Start adding file for generic functions throughout urls
* Add tests for maybe_add_ssl_handler
* Remove commented out line
* Improve coverage of maybe_add_ssl_handler, test basic_auth_header
* Start tests for open_url
* pep8 and ignore urlopen in test_url_open.py tests
* Extend auth tests, add test for validate_certs=False
* Finish tests for open_url
* Add tests for fetch_url
* Add fetch_url tests to replace-urlopen ignore
* dummy instead of _
* Add BadStatusLine test
* Reorganize/rename tests
* Add tests for RedirectHandlerFactory
* Add POST test to confirm behavior is to convert to GET
* Update tests to handle recent changes to RedirectHandlerFactory
* Special test, just to confirm that aliasing http_error_308 to http_error_307 does not cause issues with urllib2 type redirects
* Allow to set tls_ca_cert or skip verify for grafana datasources
* version_added in documentation for new options tls_skip_verify
* Added default value for tls_skip_verify option in doc
* Fixed author git account
* Updated author
For connection=local check only if the transport value in
provider is cli and the respective module support cli
transport. If not report back appropriate error message.
Currently, when ones run the module in check_mode it tries to retrieve
values from the actual certificate generated in the generate() function.
Since in check_mode we call dump() without calling generate(), self.cert
is None, leading to self.cert.get_notBefore(), self.cert.get_notAfter()
and self.cert.get_serial_number() raising an error.
> NoneType' object has no attribute 'get_notBefore'
The solution is to have two way to handle dump() method, whether its run
in check_mode=True or check_mode=False leading to different way the
information is retrieved.
* manageiq_provider: method docstrings fixes
* manageiq_provider: refactoring step towards DRY
`resource` is now same in `edit_provider()` and `create_provider()`,
to be extracted to main later.
* manageiq_provider: Fix creation of openshift provider
Fixes#38331.
Sending `api_version` to openshift provider would cause error:
"OpenShift api_version cannot be modified"
due to https://github.com/ManageIQ/manageiq-providers-kubernetes/blob/gaprindashvili-2/app/models/manageiq/providers/kubernetes/container_manager.rb#L37
In `edit_provider` we `delete_nulls()` on the whole data being sent
so `api_version` was omitted if not specified.
In `create_provider()` we only did it on endpoints list so `api_version`
was always sent - now doing on whole data.
Physical devices are listed using 'pvs' command. Then, for
'/dev/dm-*' devices 'dmsetup' command is used to find pv_name.
An error occurs when 'pvs' command list an unknown device:
$ pvs --noheadings -o pv_name,vg_name --separator ';'
/dev/dm-0;vg_var
/dev/mapper/sdb3_backups;vg_data_backups
$ dmsetup info -C --noheadings -o name /dev/dm-0
Device dm-0 not found
Then the module fails:
{
"changed": false,
"err": "Device dm-0 not found\nCommand failed\n",
"msg": "Failed executing dmsetup command.",
"rc": 1
}
This failure can be avoided when the unknown device isn't used in
module parameter 'pvs'.
* Required changes to support redirects on HTTP 307/308
This ensures HTTP 307 and 308 will redirect the request to the new
location without modification.
* Fix the unused newheaders reference
* Be more compliant
* Add integration tests for follow_redirects=all
* Improve other tests for new behaviour
* Make follow_redirects values more strict
* Added module win_pester and relatives integration tests
* Corrected issues as stated by ansible-test
* Added defaults variable in integration tests
* Added task to install Pester if needed in the integration test
* Corrected error in win_psmodule task
* Added Pester installation with Chocolatey when Powershell version < 5
* Get facts...
* Disabled invoke-pester output
* Added pester_result type
* Added jhawkesworth changes proposal
* Corrected documentation linting
* Corrected linting
* Added dagwieers recommendations
* Added dagwieers recommendations
* Corrected linting errors and task error in integration test
* Corrected error in integration test
* Added dagwieers recommendations
* Corrected requirements in the DOCUMENTATION block
This offers an optimization that allows loading larger
inventories of various structure by improving the
scaling laws involved for adding hosts and groups.
The primary speed benefit is the elimination of duplicate
recusion from traversing converging paths.
* tolerate windows line endings when loading windows module utils. Helpful for old custom windows modules.
* add test modules to demonstrate win line ending module load behaviour.
* attempt to fix sanity check failures
* pep8 fix
* explict skip of test modules from shebang check (core modules must still have expected unix style line endings)
* switch to rstrip() following core team meeting feedback
The gunicorn module has a hard-coded reference to '/tmp' which may
or may not be the actual temp directory for an operating system.
This patch replaces '/tmp' with module.tmpdir which should
resolve to the correct temp directory for the OS.
Fixes Issue #36953
Signed-off-by: Eric Brown <browne@vmware.com>
* Add aws_ses_identity_policy module for managing SES sending policies
* Add option to AnsibleAWSModule for applying a retry decorator to all calls.
* Add per-callsite opt in to retry behaviours in AnsibleAWSModule
* Update aws_ses_identity_policy module to opt in to retries at all callsites.
* Add test for aws_ses_identity_policy module with inline policy.
* Remove implicit retrys on boto resources since they're not working yet.
* Fix bug 36936
* Added version_added to argument and fixed whitespace
* Update panos_import documentation
Update parameter documentation and add note.
* Add type documentation
* added version number for documentation
For real
* Integrated recommended changes
- Added recommended changes from PR
* Changed validate_ssl default back to True considering there is a
note at the top of documentation explaining change
* Format changes based on recommendations from gundalow
* Rename validate_ssl to validate_cert
* Change description to remove SSL reference
* Change url default ih documentation
* Integrated small changes from bug report
- Renamed validate_cert to validate_certs
- Changed documentation for disabling cert validation
Consolidate waiters to a single file
* Add waiter message with token ID
* Add waiter
Add waiter for WAF change tokens
Working waiter for waf_condition module
Add support for waiters to waf_rule
* WAF data model refactor
* Fix ref to self.client
* Add custom waiters to aws_waf_web_acl
* Allow add/remove rule tasks to operate in parallel, then wait for their change tokens to complete
* Move waiter into run_func_with_change_token_backoff since it is generic to all WAF update operations
* Wait for deletes on waf_web_acl
* Remove always-wait
* Remove waiter retry catch
* Add OpenNebula one_image_facts module
`one_image_facts` - module for gathering facts about OpenNebula images
Add integration tests
* Add an alias for ids
Map values can contain commas, e.g.
- name: Configure OVN bridge mapping
openvswitch_db:
state: present table: open_vswitch record: . col: external_ids key: ovn-bridge-mappings
value: '"vmnet-static:br-vmnet-st,vmnet-dynamic:br-vmnet-dyn"'
Previous behaviour was splitting the value and raised an exception.
`get_options` is an undefined method from CallbackBase
As a result, remove the trailing `s` letter.
Signed-off-by: Daniel Andrei Minca <mandrei17@gmail.com>
This fix adds check for NMClient, NetworkManager availability
while using require_version API.
Fixes: #38042
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Add error handling for junos in case wrong connection type
Fixes#37990
If a junos module doesn't support given connection/transport type
return appropriate error message.
* Fix CI issues
* Fix review comment
* fixes issue when netconf would report ios is not supported
This change now will map ansible_network_os=ios to the correct netconf
plugin implementation. This will resolve an error where the netconf
connection plugin will report that ios is unsupported.
* stabilize ec2_vpc_subnet module
* Add waiters for ec2_vpc_subnet
Clean up integration tests
* Reenable CI for stabilized ec2_vpc_subnet tests
* rename waiters
* Use module_json_aws where applicable
Handle WaiterError first if waiting failed
* Fix traceback when tagging with keys/values that look like booleans
* Fix check mode with tags
* Add integration tests for tags that look like booleans and check mode
* Add waiter for deleting subnet
* Sleep a few seconds after using aws command line
Iterating an object and changing it at the same time is unsecure and no longer permitted in Python >= 3.6
Provisioning an instance fail with the Python error: "RuntimeError: dictionary changed size during iteration"
* Update office_365_connector_card.py
The two bullet items were about one single item; this module is not idempotent.
Therefore the two bullet items should be one.
label: docsite_pr
* Update office_365_connector_card.py
Removed trailing white space that caused the test failure.
When the URI module returns complex JSON objects, the YAML callback
fails while trying to represent these objects. The problem arises
because the filter method returns an iterator in Python 3, rather than a
str object. Therefore, the str method expandtabs() is not available,
and the callback fails with the following error:
[WARNING]: Failure using method (v2_runner_on_failed) in callback plugin (<ansible.plugins.callback.yaml.CallbackModule object at 0x7f7c7ed8aa20>): 'filter' object has no attribute 'expandtabs'
Issue can be replicated by running this playbook:
- hosts: localhost
gather_facts: false
tasks:
- uri:
url: https://jsonplaceholder.typicode.com/posts
ansible-playbook tmp.yml -v
* Use AnsibleAWSModule to simplify AWS connection
* Add Exception handling, pagination, retries and backoff
* Allow events to be switched off
* Allow details to be obtained without having to specify services
This is required if we want to ensure that #36809 doesn't cause any
important behavioral changes.
This PR changes the uri module to support follow_redirects=urllib2
It also adds a better error message when the connection closes before
any data was returned.
* module_common: set required parameter templar
Fix the following error (related to b455901):
$ ./hacking/test-module -m ./lib/ansible/modules/system/ping.py -I ansible_python_interpreter=/usr/bin/python
Traceback (most recent call last):
File "./hacking/test-module", line 268, in <module>
main()
File "./hacking/test-module", line 249, in main
(modfile, modname, module_style) = boilerplate_module(options.module_path, options.module_args, interpreters, options.check, options.filename)
File "./hacking/test-module", line 152, in boilerplate_module
task_vars=task_vars
File "ansible/lib/ansible/executor/module_common.py", line 910, in modify_module
environment=environment)
File "ansible/lib/ansible/executor/module_common.py", line 736, in _find_module_utils
shebang, interpreter = _get_shebang(u'/usr/bin/python', task_vars, templar)
File "ansible/lib/ansible/executor/module_common.py", line 452, in _get_shebang
interpreter = templar.template(task_vars[interpreter_config].strip())
AttributeError: 'NoneType' object has no attribute 'template'
* module_common.modify_module: templar is required
SGs created when a VPC ID was not specified would not necessarily
get the default egress rule, even when no explicit egress rules
were set.
Add some checks for egress rules in results from existing tests
The generated file was completely unusable by the system
therefore the fix which ensures that diffing the file
prior to changes and after only shows diffs
Furthermore the code did not work for Python 3.6
> f.writelines(to_bytes(lines, errors='surrogate_or_strict'))
E TypeError: a bytes-like object is required, not 'int'
The other modifications (lambda variable renaming) is to
comply with default flake8 rules
VMware throws error if hostname provided by user is not RFC 952
compliant. Added minor documentation fixes.
Fixes: #24225, #27096
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Currently the --rsh command arg being passed to rsync is not quoted,
but we're adding arguments to the ssh command and that causes rsync
to attempt to accept them as it's own, which is not the desired
outcome.
Fixes#35717
Signed-off-by: Adam Miller <admiller@redhat.com>
If user does not specify esxi_hostname then module
fails to detect ESXi hostsystem from given configuration.
This fixes the regression in get_all_host_objs API by
getting first host managed object from list.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Fix 'New Vault password' on vault 'edit'
ffe0ddea96 introduce a
change on 'ansible-vault edit' that tried to check
for --encrypt-vault-id in that mode. But '--encrypt-vault-id'
is not intended for 'edit' since the 'edit' should always
reuse the vault secret that was used to decrypt the text.
Change cli to not check for --encrypt-vault-id on 'edit'.
VaultLib.decrypt_and_get_vault_id() was change to return
the vault secret used to decrypt (in addition to vault_id
and the plaintext).
VaultEditor.edit_file() will now use 'vault_secret_used'
as returned from decrypt_and_get_vault_id() so that
an edited file always gets reencrypted with the same
secret, regardless of any vault id configuration or
cli options.
Fixes#35834
* continue fact gathering even without dmidecode
If dmidecode is not available we still wan to continue with fact
gathering.
On certain platforms dmidecode just won't work
Updated EXAMPLES documentation which had following 2 incorrect parameters:
'type' has been changed to 'ds_type'
'basic_auth' parameter has been removed.
label: docsite_pr
Change assign_public_ip to default to None rather than False so
that we can detect whether the value is being explicitly set or
not, and only warn if it is explicitly set to False for something
with a public_dns_name
Fixes#37985
* Make the module idempotent
* pep8 fixes.
* Made required changes for overwrite_value for idempotency.
* Fix missing error definition
* Add in missing documentation variable.
* Use arg_spec type for comparisons on default and choices
* Further improve type casting
* Make sure to capture output in more places
* Individually report invalid choices
* Update ignore.txt after resolving merge conflicts
* Make it less likely that we have to identify all the modules during
a playbook run. PluginLoader is optimized to look for modules one
directory at a time. If we find a module before we've examined all
the directories we never have to touch the other directories.
Reordering this conditional makes it so tasks which don't have
a module file will not force us to examine all the module directories
before moving on to other sources of task actions.
* Change several variables we consult to see if a task is in a certain
category from lists/tuples to frozensets. These are static lists
which we only do containment tests on so frozensets should be faster
Fixes#37208
If check_mode is enabled instead of committing th config need to
discard all the chnages to cnadidate db
In case of cli to discard changes issue `rollback 0` command
and for netconf execute `discard-changes` rpc call
The message text used to check stderr for a warning about
groupinstall in order to determine if a change occurred is specific
to the version of yum that is in RHEL7 and newer. This change simply
removes a couple words off the end in order to only use text found
in the warning message in older versions of yum.
Fixes#35982
Signed-off-by: Adam Miller <admiller@redhat.com>
This patch fixes up some English typos in the yaml inventory
plugin documentation.
* s/specifically/specific
* s/as/as an
Signed-off-by: Eric Brown <browne@vmware.com>
If a repo with `repo_gpgcheck=1` is added and the repo GPG key was never
accepted, quering this repo would throw an error `repomd.xml signature
could not be verified` and the module would fail. If that happens now
`yum -y makecache` will be run which will fetch the new repo data and
accept the repo GPG key.
* template: Add integration tests for `lstrip_blocks'
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* template: Fix passing `trim_blocks' inline
Fix passing `trim_blocks' option to the template module as inline
argument. Previously passing the `trim_blocks' option inline instead of
using the YAML dictionary format resulted in it always being set to
`True', even if `trim_blocks=False' was used.
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* template: Add option to `lstrip_blocks'
Add option to set `lstrip_blocks' when using the template module to
render Jinja templates. The Jinja documentation suggests that
`trim_blocks' and `lstrip_blocks' is a great combination and the
template module already provides an option for `trim_blocks'.
Note that although `trim_blocks' in Ansible is enabled by default since
version 2.4, in order to avoid breaking things keep `lstrip_blocks'
disabled by default. Maybe in a future version it could be enabled by
default.
This seems to address issue #10725 in a more appropriate way than the
suggested.
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* template: Add integration tests for `trim_blocks'
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* template: Check Jinja2 support for `lstrip_blocks'
Since the `lstrip_blocks' option was added in Jinja2 version 2.7, raise
an exception when `lstrip_blocks' is set but Jinja2 does not support it.
Check support for `lstrip_blocks' option by checking `jinja2.defaults'
for `LSTRIP_BLOCKS' and do not use `jinja2.__version__' because the
latter is set to `unknown' in some cases, perhaps due to bug in
`pkg_resources' in Python 2.6.6.
Also update option description to state that Jinja2 version >=2.7 is
required.
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* Decodes kv values from Consul to utf-8.
* Switches to using module utils to perform text decoding.
* Adds self to authors list to help community maintenance.
include_vars and set_fact are already updating hostvars in strategy
no need to 're add again' with lower priority the same data.
fixes#37535, mostly by avoiding reprocessing and 'cleaning'
* Fix loading of filter and test plugins
Filter and test plugins are different than other plugins in that they
can have many plugins in a single file. Therefore they need to operate
a little differently. They need to have all of the potential files
returned. Then the caller takes care of passing those onto jinja2 in
order for jinja2 to make use of them.
This problem was (most recently) introduced with f921369445
This commit also restructures how we deduplicate plugins to take paths
into account. If we want to start scoping which set of modules are
loaded (due to roles, for instance) we'll need to hang on to the path
information.
* add integration test for override
* Fix style checks for bcoca code
* Implement jinja2 plugin loader as a subclass
Having a subclass allows us to customize the overriding of jinja
plugins. We can then move common parts of common code into the Loader.
* Added missing scalable target creation
* Changed if statement
* Added support to results of all actions
* Fixed line lengths, whitespaces and blank lines between functions
* Fixed documentation formatting
* Work in progress, fixed returns from functions, still need to do exception handling
* Work in progress, still need to do exception handling
* Moved to AnsibleAWSModule, Added exception handling
* Added detailed return doc
* Fixed return doc alarms
* fixed return yaml
* Fixed function calls when creating/deleting
* fixed unnecessary blank line
* removed imports and unnecessary checks handled by AnsibleAWSModule
* removed whitespace
* [cloud] ec2_vpc_route_table: ignore routes without DestinationCidrBlock
Add module warnings rather than silently skipping
* Permit warnings for routes tables containing vpc endpoints to be turned off
* Add tests to ensure a VPC endpoint associated with a route table does not result in a traceback
As with list_users, list_vhosts can sometimes return a value that
doesn't contain a '\t' character. This appears to be the case if the
server has no vhosts, for example.
The same fix was applied to the rabbitmq_users module here:
fafb89cde5
* New generator inventory plugin
Allows construction of hosts and groups through cartesian
product of various group combinations
* Add generator plugin documentation
* adding possibility to specify resource group for referred virtual network
* fixed sanity issues
* removed trailing whitespace
* added test
* fixed documentation
* try to fix unstable test
* Tidied up the description of virtual_network_resource_group
* adding possibility of disabling public ip address
* fixed indent
* fixed whitespace
* fixed mistake
* try to create test with vm without public ip address
* try to fix test
* another attempt for test
* fixing test
* create vm with no ip with different name and delete it immediately
* a few additional fixes
* another attempt to pass test
* must be deleted
* simplified no ip test
* reorganised tests
* Wrapped choice in C()
* fixed issue when public ip address should not be created
* adding test for public ip address
* fixed samples
* another fix to sample formatting
* fixed test
* fix test
* fixed test
* another attempt to fix test
* maybe it works now
* still wrong
* improved check per customer request
* removed stupid semicolon
* updated test to match main scenario
* changed ip configurations to list
* another attempt
* msi
* add env and param
* add msi in default
* add azure_rm
* add document
* subscription param
* if not enabled msi
* remove the msi in default mode since the infinite loop will block if not enabled msi
* lint
* lint
* Update azure_rm_common.py
* fix
* catch exc and make error message more friendly
* lint
* Minor docs changes to the msi source option
