The generated file was completely unusable by the system
therefore the fix which ensures that diffing the file
prior to changes and after only shows diffs
Furthermore the code did not work for Python 3.6
> f.writelines(to_bytes(lines, errors='surrogate_or_strict'))
E TypeError: a bytes-like object is required, not 'int'
The other modifications (lambda variable renaming) is to
comply with default flake8 rules
VMware throws error if hostname provided by user is not RFC 952
compliant. Added minor documentation fixes.
Fixes: #24225, #27096
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Currently the --rsh command arg being passed to rsync is not quoted,
but we're adding arguments to the ssh command and that causes rsync
to attempt to accept them as it's own, which is not the desired
outcome.
Fixes#35717
Signed-off-by: Adam Miller <admiller@redhat.com>
If user does not specify esxi_hostname then module
fails to detect ESXi hostsystem from given configuration.
This fixes the regression in get_all_host_objs API by
getting first host managed object from list.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Fix 'New Vault password' on vault 'edit'
ffe0ddea96 introduce a
change on 'ansible-vault edit' that tried to check
for --encrypt-vault-id in that mode. But '--encrypt-vault-id'
is not intended for 'edit' since the 'edit' should always
reuse the vault secret that was used to decrypt the text.
Change cli to not check for --encrypt-vault-id on 'edit'.
VaultLib.decrypt_and_get_vault_id() was change to return
the vault secret used to decrypt (in addition to vault_id
and the plaintext).
VaultEditor.edit_file() will now use 'vault_secret_used'
as returned from decrypt_and_get_vault_id() so that
an edited file always gets reencrypted with the same
secret, regardless of any vault id configuration or
cli options.
Fixes#35834
* continue fact gathering even without dmidecode
If dmidecode is not available we still wan to continue with fact
gathering.
On certain platforms dmidecode just won't work
Updated EXAMPLES documentation which had following 2 incorrect parameters:
'type' has been changed to 'ds_type'
'basic_auth' parameter has been removed.
label: docsite_pr
Change assign_public_ip to default to None rather than False so
that we can detect whether the value is being explicitly set or
not, and only warn if it is explicitly set to False for something
with a public_dns_name
Fixes#37985
* Make the module idempotent
* pep8 fixes.
* Made required changes for overwrite_value for idempotency.
* Fix missing error definition
* Add in missing documentation variable.
* Use arg_spec type for comparisons on default and choices
* Further improve type casting
* Make sure to capture output in more places
* Individually report invalid choices
* Update ignore.txt after resolving merge conflicts
* Make it less likely that we have to identify all the modules during
a playbook run. PluginLoader is optimized to look for modules one
directory at a time. If we find a module before we've examined all
the directories we never have to touch the other directories.
Reordering this conditional makes it so tasks which don't have
a module file will not force us to examine all the module directories
before moving on to other sources of task actions.
* Change several variables we consult to see if a task is in a certain
category from lists/tuples to frozensets. These are static lists
which we only do containment tests on so frozensets should be faster
Fixes#37208
If check_mode is enabled instead of committing th config need to
discard all the chnages to cnadidate db
In case of cli to discard changes issue `rollback 0` command
and for netconf execute `discard-changes` rpc call
The message text used to check stderr for a warning about
groupinstall in order to determine if a change occurred is specific
to the version of yum that is in RHEL7 and newer. This change simply
removes a couple words off the end in order to only use text found
in the warning message in older versions of yum.
Fixes#35982
Signed-off-by: Adam Miller <admiller@redhat.com>
This patch fixes up some English typos in the yaml inventory
plugin documentation.
* s/specifically/specific
* s/as/as an
Signed-off-by: Eric Brown <browne@vmware.com>
If a repo with `repo_gpgcheck=1` is added and the repo GPG key was never
accepted, quering this repo would throw an error `repomd.xml signature
could not be verified` and the module would fail. If that happens now
`yum -y makecache` will be run which will fetch the new repo data and
accept the repo GPG key.
* template: Add integration tests for `lstrip_blocks'
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* template: Fix passing `trim_blocks' inline
Fix passing `trim_blocks' option to the template module as inline
argument. Previously passing the `trim_blocks' option inline instead of
using the YAML dictionary format resulted in it always being set to
`True', even if `trim_blocks=False' was used.
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* template: Add option to `lstrip_blocks'
Add option to set `lstrip_blocks' when using the template module to
render Jinja templates. The Jinja documentation suggests that
`trim_blocks' and `lstrip_blocks' is a great combination and the
template module already provides an option for `trim_blocks'.
Note that although `trim_blocks' in Ansible is enabled by default since
version 2.4, in order to avoid breaking things keep `lstrip_blocks'
disabled by default. Maybe in a future version it could be enabled by
default.
This seems to address issue #10725 in a more appropriate way than the
suggested.
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* template: Add integration tests for `trim_blocks'
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* template: Check Jinja2 support for `lstrip_blocks'
Since the `lstrip_blocks' option was added in Jinja2 version 2.7, raise
an exception when `lstrip_blocks' is set but Jinja2 does not support it.
Check support for `lstrip_blocks' option by checking `jinja2.defaults'
for `LSTRIP_BLOCKS' and do not use `jinja2.__version__' because the
latter is set to `unknown' in some cases, perhaps due to bug in
`pkg_resources' in Python 2.6.6.
Also update option description to state that Jinja2 version >=2.7 is
required.
Signed-off-by: Alex Tsitsimpis <alextsi@arrikto.com>
* Decodes kv values from Consul to utf-8.
* Switches to using module utils to perform text decoding.
* Adds self to authors list to help community maintenance.
include_vars and set_fact are already updating hostvars in strategy
no need to 're add again' with lower priority the same data.
fixes#37535, mostly by avoiding reprocessing and 'cleaning'
* Fix loading of filter and test plugins
Filter and test plugins are different than other plugins in that they
can have many plugins in a single file. Therefore they need to operate
a little differently. They need to have all of the potential files
returned. Then the caller takes care of passing those onto jinja2 in
order for jinja2 to make use of them.
This problem was (most recently) introduced with f921369445
This commit also restructures how we deduplicate plugins to take paths
into account. If we want to start scoping which set of modules are
loaded (due to roles, for instance) we'll need to hang on to the path
information.
* add integration test for override
* Fix style checks for bcoca code
* Implement jinja2 plugin loader as a subclass
Having a subclass allows us to customize the overriding of jinja
plugins. We can then move common parts of common code into the Loader.
* Added missing scalable target creation
* Changed if statement
* Added support to results of all actions
* Fixed line lengths, whitespaces and blank lines between functions
* Fixed documentation formatting
* Work in progress, fixed returns from functions, still need to do exception handling
* Work in progress, still need to do exception handling
* Moved to AnsibleAWSModule, Added exception handling
* Added detailed return doc
* Fixed return doc alarms
* fixed return yaml
* Fixed function calls when creating/deleting
* fixed unnecessary blank line
* removed imports and unnecessary checks handled by AnsibleAWSModule
* removed whitespace
* [cloud] ec2_vpc_route_table: ignore routes without DestinationCidrBlock
Add module warnings rather than silently skipping
* Permit warnings for routes tables containing vpc endpoints to be turned off
* Add tests to ensure a VPC endpoint associated with a route table does not result in a traceback
As with list_users, list_vhosts can sometimes return a value that
doesn't contain a '\t' character. This appears to be the case if the
server has no vhosts, for example.
The same fix was applied to the rabbitmq_users module here:
fafb89cde5
* New generator inventory plugin
Allows construction of hosts and groups through cartesian
product of various group combinations
* Add generator plugin documentation
* adding possibility to specify resource group for referred virtual network
* fixed sanity issues
* removed trailing whitespace
* added test
* fixed documentation
* try to fix unstable test
* Tidied up the description of virtual_network_resource_group
* adding possibility of disabling public ip address
* fixed indent
* fixed whitespace
* fixed mistake
* try to create test with vm without public ip address
* try to fix test
* another attempt for test
* fixing test
* create vm with no ip with different name and delete it immediately
* a few additional fixes
* another attempt to pass test
* must be deleted
* simplified no ip test
* reorganised tests
* Wrapped choice in C()
* fixed issue when public ip address should not be created
* adding test for public ip address
* fixed samples
* another fix to sample formatting
* fixed test
* fix test
* fixed test
* another attempt to fix test
* maybe it works now
* still wrong
* improved check per customer request
* removed stupid semicolon
* updated test to match main scenario
* changed ip configurations to list
* another attempt
* msi
* add env and param
* add msi in default
* add azure_rm
* add document
* subscription param
* if not enabled msi
* remove the msi in default mode since the infinite loop will block if not enabled msi
* lint
* lint
* Update azure_rm_common.py
* fix
* catch exc and make error message more friendly
* lint
* Minor docs changes to the msi source option
* added set_env_proxy function for setting proxy environment
Added set_env_proxy function, that set system http(s)_proxy
environment for fetching RPM if proxy settings set in yum.conf file
that fix Issue - #18979
* fix automatic field numbering specification
* changed if statement in setting http_proxy environment
* Change set_env_proxy function to function with decorator
That decorator set system proxy environment from yum.conf and revert
back to system configuration after fetching RPM
* Minor fix
- rename variable schem to scheme
- change 'in' to 'startswith'
* change decorator set_env_proxy to decorating through contextmanager
- added import contextmanager from contextlib
- set_env_proxy now decorating through contextmanager
- fix http/https setting environment principle
* Fix csvfile traceback on Python3
The csvfile lookup uses some custom iterators. These needed to be
ported to handle the python3 iterator protocol. In addition, the
csvfile module takes an iterator of byte strings in Python2 and an
iterator of text strings in Python3
Fixes#36808
* Fix a traceback in ansible-pull on python3 comparing output from
subprocess with a text string.
* Rename variables that hold byte strings so we are clear that those are
not text strings.
* Use to_text() to transform variable that's being displayed as it's
less fragile than str().
Fixes#36962
The user_data field is base64 encoded inside of the boto library. In
Python3, base64 must be used with byte strings. So we make sure to
encode the user_data into a byte string before passing it on to the boto
library.
Fixes#34978
An IncludedFile() object built using the original_task will have
its _task bound to the original_task. The iterative reassignment of
original_task._role_name during with_item loops leaves all returned
included_files with the same ._task._role_name (the final name from
the with_items list). This commit builds IncludedFile() objects
from an original_task.copy() to avoid the problematic binding.
NSO verify did not handle leaf-list value verification in 4.5 and
later due to changes made for configuration writing made.
map prefix for identityref types in verification.
NSO operations can take much longer than 10 seconds as they operate on
real network equipment, set default timeout to 5 minutes and allow for
user override.
* Fix redundant yaml error blurbs on ModArgs parse errors
Some of the AnsibleParserErrors from parsing.mod_args
are created with the obj=some_yaml_ds options but
some are not.
If they were, we don't want to add another yaml_ds to
it, because that will result in double yaml error blurbs.
And since we dont need to add info, we can just re raise it.
But if there is no ._obj, add it here so we get the extra
detail in the error message (see issue #14790) and raise
a new AnsibleParserError instance.
Fixes#36848
* cleanup existing test_tasks pep8/sanity issues
* Don't mask values specified in data.
This heavily improves readability of error messages such as
'Authorization for ******** returned in********: CHALLENGE: ********-01 DETAILS: Incorrect TXT record "********" (and 1 more) found at ********.********;'
(verbatim quote).
* Added documentation.
This PR makes and adjustment for Python 3 scoping rules which differ from Python 2. In Python 3, the variable __ex__ goes out of scope at the exit of the __try-except__ block. This means that when __ex__ is referred to on the lines that follow, it would be an _undefined name_ causing a __NameError__ to be raised instead of the expected __ConnectionError__.
flake8 testing of https://github.com/ansible/ansible on Python 3.6.3
The command "echo ; echo -n "flake8 testing of ${URL} on " ; python -V" exited with 0.
130.72s$ time flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics
./lib/ansible/module_utils/api.py:114:21: F823 local variable 'retry_count' (defined in enclosing scope on line 107) referenced before assignment
retry_count += 1 # pylint: disable=undefined-variable
^
./lib/ansible/module_utils/six/__init__.py:54:20: F821 undefined name 'basestring'
string_types = basestring,
^
./lib/ansible/module_utils/six/__init__.py:55:27: F821 undefined name 'long'
integer_types = (int, long)
^
./lib/ansible/module_utils/six/__init__.py:57:17: F821 undefined name 'unicode'
text_type = unicode
^
./lib/ansible/module_utils/six/__init__.py:658:16: F821 undefined name 'unicode'
return unicode(s.replace(r'\\', r'\\\\'), "unicode_escape")
^
./lib/ansible/module_utils/six/__init__.py:754:37: F821 undefined name 'basestring'
if not isinstance(data, basestring):
^
./lib/ansible/module_utils/six/__init__.py:757:32: F821 undefined name 'file'
if (isinstance(fp, file) and
^
./lib/ansible/module_utils/six/__init__.py:758:38: F821 undefined name 'unicode'
isinstance(data, unicode) and
^
./lib/ansible/module_utils/six/__init__.py:768:32: F821 undefined name 'unicode'
if isinstance(sep, unicode):
^
./lib/ansible/module_utils/six/__init__.py:774:32: F821 undefined name 'unicode'
if isinstance(end, unicode):
^
./lib/ansible/module_utils/six/__init__.py:782:36: F821 undefined name 'unicode'
if isinstance(arg, unicode):
^
./lib/ansible/module_utils/six/__init__.py:786:23: F821 undefined name 'unicode'
newline = unicode("\n")
^
./lib/ansible/module_utils/six/__init__.py:787:21: F821 undefined name 'unicode'
space = unicode(" ")
^
./lib/ansible/modules/cloud/misc/rhevm.py:594:24: F821 undefined name 'e'
setMsg(str(e))
^
./lib/ansible/modules/cloud/openstack/os_user.py:202:38: F821 undefined name 'shade'
if description and StrictVersion(shade.__version__) < StrictVersion('1.13.0'):
^
./lib/ansible/modules/cloud/openstack/os_volume.py:153:27: F821 undefined name 'shade'
StrictVersion(shade.__version__) < StrictVersion('1.22')):
^
./lib/ansible/modules/files/archive.py:391:92: F821 undefined name 'e'
module.fail_json(dest=dest, msg='Error deleting some source files: ' + str(e), files=errors)
^
./lib/ansible/modules/network/dellos9/dellos9_facts.py:379:24: F821 undefined name 'key'
parsed[key] += '\n%s' % line
^
./lib/ansible/modules/network/dellos9/dellos9_facts.py:399:24: F821 undefined name 'key'
parsed[key] += '\n%s' % line
^
./lib/ansible/modules/network/dellos9/dellos9_facts.py:403:24: F821 undefined name 'key'
parsed[key] += '\n%s' % line
^
./lib/ansible/modules/network/dellos9/dellos9_facts.py:418:24: F821 undefined name 'key'
parsed[key] += '\n%s' % line
^
./lib/ansible/modules/network/f5/bigip_device_httpd.py:461:40: F821 undefined name 'ex'
if 'Connection aborted' in str(ex) and 'redirectHttpToHttps' in params:
^
./lib/ansible/modules/network/f5/bigip_device_httpd.py:465:33: F821 undefined name 'ex'
raise F5ModuleError(str(ex))
^
./lib/ansible/modules/network/ironware/ironware_facts.py:274:24: F821 undefined name 'key'
parsed[key] += '\n%s' % line
^
./lib/ansible/modules/network/ironware/ironware_facts.py:514:24: F821 undefined name 'key'
parsed[key] += '\n%s' % line
^
24 F821 undefined name 'basestring'
1 F823 local variable 'retry_count' (defined in enclosing scope on line 107) referenced before assignment
25
* Clean up Grafana docs a bit
But more is needed, default values, missing choices, proper
descriptions, actual sentences :-)
* Remove validate-modules entries
* Added parameters to na_cdot_volume
Before this commit there were no parameter for creating a volume
and exporting it as NFS volume. The added parameters are:
- *junction-path* for specifying where to mount the volume in the namespace
- *export-policy* for specifying which export policy to apply to the given namespace
- *snapshot-policy* if required, applying a snapshot policy for scheduling data protection tasks
* Sanity fixes
* Sanity fixes (2)
* add additional test coverage for tower modules
* add test coverage for the tower_credential module
* add test coverage for the tower_user module
* fix a bug in py3 for tower_credential when ssh_key_data is specified
* add test coverage for tower_host, tower_label, and tower_project
* add test coverage for tower_inventory and tower_job_template
* add more test coverage for tower modules
- tower_job_launch
- tower_job_list
- tower_job_wait
- tower_job_cancel
* add a check mode/version assertion for tower module integration tests
* add test coverage for the tower_role module
* add test coverage for the tower_group module
* add more integration test edge cases for various tower modules
* give the job_wait module more time before failing
* randomize passwords in the tower_user and tower_group tests
* Increase delay and tries for ec2_vpc_net backoff
Wait for DHCP option to be created in ec2_vpc_dhcp_option
Wait for all modifications to the VPC
* Use the vpc_available waiter because is uses Filters
* Missed one
* Optimize retries to only occur if the functionality is available
* Increase max wait time
* Add comments to explain what the waiters are doing
* maven_artifact: add verify_checksum option - fixes#31799
* maven_artifact: some cleaning
* Remove blank lines to please the format checker
* Now targeting 2.6...
* fix typos
* use formatting functions
* use 'job template' instead of 'job_template'
* acronyms: user uppercase
* become_enabled param is about privilege escalation
* Check that connection error msg are not unsafe
* Connection error messages are unsafe: wrap them
For example, in case of error, docker connection plugin returns exception
message containing Go template. These messages weren't tagged as unsafe
and were consequently rendered:
The conditional check 'result is failed' failed. The error was:
{
'msg': u'Docker version check ([\'/usr/bin/docker\', \'version\', \'--format\', "\'{{.Server.Version}}\'"]) failed: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.35/version: dial unix /var/run/docker.sock: connect: permission denied\n',
'failed': True
}:
template error while templating string: unexpected '.'.
String: Docker version check (['/usr/bin/docker', 'version', '--format', "'{{.Server.Version}}'"]) failed: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.35/version: dial unix /var/run/docker.sock: connect: permission denied
* eos can not check config without config session support
* add testcase for check_mode without config session
* fix eos eapi to read use_session env var
This PR includes:
- A fix for multiple-choice defaults
- A fix for messed up dictionary samples
- Cleaner defaults when they don't appear part of choices
I wrote this code because we had a traceback at some point with pyyaml
not able to handle our AnsibleUnicode type (the C library was doing an
exact match to python's unicode type rather than testing with the object
inheritance hierarchy.) Neither jimi think that AnsibleUnicode gets
passed into this function anymore, though, so we think it is safe to remove.
Fixes#36979
If `abort` is not issued in the top level session prompt
the existing session goes to pending state.
The fix is to come out of config mode by issuing `end` command
and again to same config session and execute `abort` which
`abort` is issued at the top level session prompt.
When looking for installed packages we do not need to query
repositories since we only care about the rpmdb. As such, we can
disable all the repositories operations in order to improve the
performance of that step
Before this patch, when using 'state: present' in the zypper module,
the operation was taking about 12 seconds to complete:
time ansible-playbook foo.yml 1>/dev/null
real 0m12.614s
user 0m10.880s
sys 0m0.683s
After this patch:
time ansible-playbook foo.yml 1>/dev/null
real 0m4.193s
user 0m2.560s
sys 0m0.575s
see:
https://bugzilla.opensuse.org/show_bug.cgi?id=1084525
debian:9
`apt-get install python-apt` - this will install python2.7 and python3
After this operation, *88.4 MB* of additional disk space will be used.
`apt-get install --no-install-recommends python-apt` - only python2.7
After this operation, *35.0 MB* of additional disk space will be used.
(cherry picked from commit c542e62)
When commit revisions are disabled, there will be no revision items returned.
Add note about setting ANSIBLE_PERSISTENT_COMMAND_TIMEOUT to a higer value to avoid timeouts.
* Using correct content type (as per version 10 of ACME draft).
* Another incompatibility with ACME v2: body must be {} and not contain v1 data (Pebble fails otherwise).
* Fixing bug: self.args in a subclass of Exception is apparently always a tuple.
* PY3 = dnf
Red Hat are unlikely to provide a Python 2 version of the yum bindings
as they are moving to `dnf`.
If Ansible can't find the yum Python library give the user a hint about
dnf and Python 3
* Fix unarchive with strip-components in extra_opts
When unarchive is given extra_opts to strip all leading directories, it
could end up trying to change the permissions on the root directory.
Tar archives shouldn't contain absolute paths anyways so make sure that
all paths are relative as we handle them.
Fixes#21397
In Python2, `map` returns list whereas Python3, `map` function
returns iterator. This fix typecast map function for Python3.
Fixes: #37114
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Network modules to link to network guide
* Make it clear what's deprecated
* enable -> become
* Detail when provider is still needed (eAPI/NX-API)
* Link to specific sections
* rx_rate & tx_rate to link to conditional docs
* Make use of named links in documentation notes
Now that it is possible to name external links, we are making use of
this to make the documentation better.
* Add improvements to ACI documentation
* Disable QA for long line
* Add :menuselection: and :guilabel:
* Improve links on some modules
This creates a way for us to use boto3's data-driven waiter support to use custom waiters where Boto3 hasn't implemented them yet.
The only waiter implemented so far is for VPC Route Tables to check that they exist, and this replaces some custom retry code.
Fixes#35993 - Changes to update_size in commit eb4cc31 made it so
the group dict passed into update_size was not modified. As a result,
the 'replace' call does not see an updated min_size like it previously
did and doesn't pause to wait for any new instances to spin up. Instead,
it moves straight into terminating old instances. Fix is to add batch_size
to min_size when calling wait_for_new_inst.
Fixes#28087 - Make replace_all_instances and replace_instances behave
exactly the same by setting replace_instances = current list of instances
when replace_all_instances used. Root cause of issue was that without lc_check
terminate_batch will terminate all instances passed to it and after updating
the asg size we were querying the asg again for the list of instances - so terminate batch
saw the list including new ones just spun up.
When creating new asg with replace_all_instances: yes and lc_check: false
the instances that are initially created are then subsequently replaced.
This change makes it so replace only occurs if the asg already existed.
Add integration tests for #28087 and #35993.
* Remove default admin_distance and fix the idempotence thereof
Fixes#33290
* Fix tests and use yaml anchors to shorten tests
* Add test for undefined admin_distance
* Read config from `show run` if `show ip static route` fails
* Restore flags to ios.get_config & use get_config where appropriate
[cloud aws_s3] Most modes require `object` parameter, and this is easy to
get wrong (e.g. through confusion with the `dest` parameter). As
it's as easy to enforce, let's do that.
There should no longer be a need for boto documentation - it
just adds significant extra clutter to the guidelines.
AnsibleAWSModule greatly improves the codebase to such an extent
that we should mandate it for new code unless there is a
documented good reason.
* Update doc of AnsibleAWSModule to remove incorrect connect example.
Current example uses aws_connect method which no longer exists. Replace
this with the client and resource methods that do exist.
Also remove try/except block in connect example as guidelines imply that
there aren't connection errors thrown on connection, just when later
sing the connection.
* Update AWS module guidelines to include the AnsibleAWSModule connection methods.
* Add information on integration testing to the AWS module guidelines.
* Add information on common documentation fragments to AWS module
Guidelines.
* Changes as requested on PR.
* Restructure connection section to start with current best practice
* Explain the use of the CI build groups
* Use YAML Anchors for aws credentials example
* Replace guidance on use of test groups with link to aliases file.
This achieves the goal of explaining why this file is necessary without
introducing overhead of keeping documentation up to date when test
groups change.
* Clean up vmware BOTMETA
This removes everything that's already part of the module docs. So
basically the BOTMETA file only extends the information from the module.
* Updated author information and copyright statement
