* Split modules/network into two parts
Given the dedicated team we we have working on Ansible Networking a
clearer split is needed between Networking modules and "things that
happen to use the network"
* nmcli to net_tools
* nmcli moved
* Run `save` before exiting config mode.
* Fix unit tests for `save`
* Allow `save` to be on its own again and introspect success
* Introspecting `compare running` makes this a lot harder.
Move `save` tests to integration tests
Ok, so for openvswitch_db in particular we just return one command
but in the sake of consistency and code re-use, let's return a list
of "commands", even if it's just one.
* Document deprecation of fetch module validate_md5 and update --tags merging deprecation
Update the default of --tags merging config option to merge by default
* Update CHANGELOG.md
Minor edit
Added an integration test for fetch module idempotence. (Testing
that validate_checksum is doing what it's supposed to is harder as we'd
have to create a race condition with the downloaded data to trigger it.
Probably need to make that a unittest eventually).
Also give a deprecation message to the validate_md5 parameter so that we
can eventually get rid of it.
* atomic_image: add support for pull --storage=
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* atomic_container: new module
Manage atomic containers
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
* Minor junos_config module changes
* Add note for json config support
* Add example for lines argument
* Fix future warning for ElementTree
* Remove unwanted space
We fail-fast and display 'stderr' in case 'pacman' returns with 'rc != 0'.
There is no point computing 'module._diff' in such case anyway.
Fixes#23910
(cherry picked from commit 8c6a2a848c)
* Fix systemd in chroot
The 'request ignored' message is in stderr, not stdout.
* Check both stdout and stderr for systemd message
Some versions of systemd report to stderr, others to stdout. Also check
whether output could be a valid normal response to avoid false positives.
* Use sys.stdout.buffer to write vault bytes to stdout on py3
We need sys.stdout.buffer on py3 so we can write bytes to it since the plaintext
of the vaulted object could be anything/binary/etc
Before, attempting to write bytes to stdout on py3 would cause:
TypeError: write() argument must be str, not bytes
* keep unsafe .. unsafe
fixes#23734, which was broken in previous fix that allowed non string types to be templated
use new 'is_template' function vs bastardizing others
refactored clean_data to allow for arbitrary data structures to clean
fixed/removed some tests
* deal with complex data for is_template
* typos
We have a list of specific messages that we scree-scrape and flag
them as legit errors.
However, we also have a catch-all regex that matches everything
starting with %.
That can cause issues on commands that return lines with that
character, like for example the 'crypto key generate'.
Fixes#23770
In current stable (2.2), ansible galaxy install --force do erase
a role, even if the version is not set. This commit should restore
that specific behavior, in accordance to people reports:
https://github.com/ansible/ansible/issues/11266#issuecomment-273801480
It was also the behavior planned in the initial discussion:
"if you're not fixing versions in your roles file, then it's fine
to expect that the role will be reinstalled each time you run
ansible-galaxy install.", cf https://github.com/ansible/ansible/pull/12904
* Revert "fixes play context connection user (#21776)"
This reverts commit 58ee661437.
fixes#23530
* preserve original in copy
updated comment to clarify wtf is going on here
fixes#17382
alternate to #22979
deal with cases in which group/host have . in name
updated as per feedbck
only be strict about extension when doing dirs
also avoid ~ endings
without this patch, ansible-galaxy will mangle files containing
the archive parent directory name, eg 'owncloud/files/owncloud.cron'
will become 'owncloud/files/.cron'.
The previous code could affect the entire path and even filenames.
If a file path has the top level dir name as a substring, galaxy
replaces it with ''. In one example, the archive top level dir
is 'go', so 'files/go-bin.sh' becomes 'files/-bin.sh'.
Fixes#22572, #23694, #23623
* Fix vault reading from stdin (avoid realpath() on non-links)
os.path.realpath() is used to find the target of file paths that
are symlinks so vault operations happen directly on the target.
However, in addition to resolving symlinks, realpath() also returns
a full path. when reading from stdin, vault cli uses '-' as a special
file path so VaultEditor() will replace with stdin.
realpath() was expanding '-' with the CWD to something like
'/home/user/playbooks/-' causing errors like:
ERROR! [Errno 2] No such file or directory: u'/home/user/ansible/-'
Fix is to specialcase '-' to not use realpath()
Fixes#23567
* to_text decrypt output when writing to stdout
* Make warning logs consistent
Arguments outside provider with default
value should not log as warning in case
it is not mentioned in play.
* Make nxos timeout default consistent and add comments
* Make comments more verbose
update module to support more standard state=present/absent syntax
update module to use required_if, required_together, mutually_exclusive functions where possible
per ryansb review: make documentation section more clear, fix some extra quotes, remove FIXME comment
pre willthames review: force private_zone to True if vpc_id is set and fix word wrap
The commit was started before 2.3 was branched, but was only merged once
2.3 was actually branched. This leads to documentation stating this
module is new in 2.3 when it will be actually new in 2.4
Regex patterns were not being escaped properly so package names
containing characters that could be interpreted as regex symbols
were causing failures.
Fixes: #19714
* Support check mode in ec2_vpc_dhcp_options_facts
As a facts module, ec2_vpc_dhcp_options_facts supports check mode
by default
* ec2_vpc_dhcp_options_facts tidy up
Use named method imports, move imports to top of code
Use shared code to handle filters and tags
Use snake case for parameter names while retaining backward compatibility
* Code cleanup
Removed 'add' method from CustomNetworkConfig. It is identical
to the one inherited from NetworkConfig
* Removed unused CustomNetworkConfig import
* Replaced
```
def get_existing(module, args):
existing = {}
netcfg = get_config(module)
config = netcfg.get_section(parents)
```
with
```
netcfg = CustomNetworkConfig(indent=2, contents=get_config(module))
```
get_config returns a string, not an object in 2.3.
* Removed non-functioning get_object method in CustomNetworkConfig in favor of the
inherited method.
Added child_objs property so that expand_selection would work. The original
verion never worked correctly as it compared NetworkConfig obj's and str's.
* Removed ShellError method in favor or new load_config method.
* Removed ShellError method in favor or new load_config method.
fixes#20260
* nxos requires a "no" statement to change mcase group. Corrected.
Corrected changed logic.
* Corrected deleted CustomNetworkConfig import
This addresses a problem where the action plugin would ignore the
remote_addr value for the host. In this case, only the inventory values
for the hostname would be considered and populate the remote host
remote_addr value for the connection plugin.
Only a few more modules were using Set-Attr on the $result object rather
than using a normal hashtable. This PR changes the PSObject to a
hashtable and gets rid of Set-Attr.
This fixes issue when list from module contains more than one element.
Ansible and/or boto may put same elements in list in different order,
thus resulting task as changed.
Fixes#3310
* Improve Fortios IPv4 policy with logging capabilities. While there, fix typos in examples. forti_config: use the backup_filename param and dont enforce the the filename value.
* forti-typos
* Add version_added for new options in the documentation
* Make it easier to find network modules
Feedback has been it's difficult (via Google or directly) to find
modules as some people search for the company name vs product name,
therefore specify both.
* "IOS XR" (not "IOS-XR")
It's possible that if the module has a low-level failure, such as
"unable to open shell", or something else in the action plugin that
stdout and stdout_lines will not be returned.
Update the documentation to clarify this point.
When the security groups specified to the os_server module change they
should be updated on the server. This will require shade 1.19 where the
server security group commands were added.
Fixes: #23206
* allow split horizon for route53_zone and refactor
* fix documentation
remove comment
fix version_added
* Remove unused imports
* Only include zone as matching if it has the same privacy setting
* Use `.endswith` instead of indexing into a string
* Update public zone behavior to only create new if there is no matching public zone
* Remove from legacy PEP8 files
We invoke /usr/bin/apt inside of the ansible module. When that command
exits, it doesn't always include a helpful error message. Include the
exit code so that user's have all the information we can gice them as to
why apt failed.
Addresses #19128
* msg_format parameter added
error message received from telegram API is added to fail json
compatibility with python3 added
* pep8 formatted
* version_added property added for msg_format
* bot token must be set without 'bot' prefix in module parameters
* formatting options described in documentation
* six module for compatibility used
telegram.py removed from legacy-files.txt
* rpm_key: Decode bytes to string to work with Python 3 (#20326)
The read() method will return bytes we need to then decode() those bytes
to a string before trying to match() it using the re module.
* Make the rpm_key pgp regex more robust on both py2 and py3
* Check for provider values inside check_args of respective network code
* Partial revert of b9ee5aa
The no_log change is okay, but take out the action result munging
Recent Python3 versions require open() to specify binary mode if the data is anything other than text.
Python3: Use int() instead of long() in unarchive
Changes long() to int() for CRC values in the unarchive module. Affects unarchiving of zip files. Since CRC values in zipfile are 32 bits the behaviour should be unchanged even in Python 2.
When retrieving file contents for diffing we need to get the contents as
binary. Otherwise python3 will try to convert the file to text and fail
with non-decodable contents.
Fixes#23171
* Build HTTPSClientAuthHandler more similarly to how HTTPSHandler works
* Add docs for new client cert authentication
* Support older versions of python
* Simplify logic
* Initial support for client certs in urls.py
* Add an extra test
* Add a get_url test for client cert auth
* Add additional test for client cert auth, with validation and ssl mismatch
* Skip assert when http tester not available
* Update version_added for new options
Copy module was walking over files in subdirectories repeatedly (a
directory tree a few levels deep could bring the time spent into the
tens of minutes)
This was traced to the fix for this bug report: https://github.com/ansible/ansible/issues/13013Fixed#13013 a different way and added an integration test to check for
regressions of #13013 as we optimize this code.
Fixes#21513
Ansible will now automatically retry a connection if SSH returns an error:
mux_client_hello_exchange: write packet: Broken pipe
This is probably a bug in SSH, but because it's safe to retry this
connection there is no need for Ansible to fail because of it.
If we run the task with 'login' banner, the 'show banner' command
will return a dict containing key 'loginBanner'.
However for motd, it will just return 'motd'.
Yay naming consistency!
We were hard-coding the protocol, port and validate_certs on
eos EAPI via the action plugin.
Put defaults on the eos_argument_spec and pull those values from it.
* 'unable to open shell' -> direct to web help
The "unable to open shell" error is returned for a number of different,
direct people to online docs (we we can update out of band of releases)
to guide them though the various solutions.
* fix pep8 errors
* Resolve#23239:
- path is missing when state=absent
- update doc for unclear policy ID
- required params based on state
* Fix pep8 + doc
* Fix doc
* Another doc problem...
* doc
* Update doc fot policy ID
* Fix doc string
* fix required in doc
* fix required in doc Required when I(state=present).
The openvswitch_db module uses the ovs-vsctl binary to
address changes.
On other network modules we follow the pattern of returning 'commands'
as part of the result, containing the commands run on the target device.
Follow that for code consistency and maintenance.
Also, adding state param, which allows to add/remove keys on columns.
This reverts commit 97cb2016d8.
It was causing issues downstream as facts seem to be merged back for
subsequent tasks. Will restore if I ever figure out how to avoid that.
* Add workaround for evasive in apache2_module
* Fixes#22635
* Clean up workarounds for php/shib
* Add test for evasive workaround
* Remove use of re module, since all searches work with native python
* Add unit tests to apache2_module name replacements
Go back to using re package where needed
* Rename replace_name to create_apache_identifier
* Add ability to use either sec group name or sec group id or combination to ec2_lc. Also fix header size in AWS Guidelines
* Fix ec2 import to only import neccessary packages. Remove pep8 double line
The current implementation matches libreoffice-oldstable when testing for libreoffice.
So uninstalling libreoffice fails when libreoffice-oldstable is installed.
```
PS C:\WINDOWS\system32> choco list --local-only libreoffice
Chocolatey v0.10.3
libreoffice-oldstable 5.2.6
1 packages installed.
PS C:\WINDOWS\system32> choco list --local-only --exact libreoffice
Chocolatey v0.10.3
0 packages installed.
```
The solution is easy, just add `--exact`.
Apparently in some devices the filesystems gathering command can return
a dict containing a 'messages' key with the filesystems, instead of a
plain string.
Fixes#23217
* re-raise exception as expected
The current ThrottlingException handling code hides other actual exceptions from the user, and basically goes infinite loop instead. eg when the api caller doesn't have permission (example below) to use the api, ansible effectively hangs. adding the re-raise stops execution and shows the error to the command line as expected
can test by removing permission to the efs api, and calling the efs: module
The error was: botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the DescribeFileSystems operation: User: <x> is not authorized to perform: elasticfilesystem:DescribeFileSystems on the specified resource
* PEP8 fix
* Remove DryRun parameter in ec2_vpc_igw_facts check_mode
Using DryRun in check mode causes errors, and is not required
(as nothing changes when calling describe_internet_gateways)
Prevents the following error:
```
{"changed": false,
"failed": true,
"msg": "An error occurred (DryRunOperation) when calling
the DescribeInternetGateways operation: Request
would have succeeded, but DryRun flag is set."}
```
* ec2_vpc_igw_facts pep8 tidy up
Due to the number of "real world" caveats in the current runas become method, it was agreed that we'd warn that it's experimental on use. A potential future version based on LogonUser/CreateProcessAsUser will have very different requirements and limitations.
* . switched from 'user.id' to 'account.id' in REST API calls.
+ added ability to select desired account (by name or identifier) from list of accounts to which authorized user have access.
* + added account option addition version.
* Remove pubnub_blocks from PEP8-legacy list
* [cloud][tests] Create fixtures for using placebo to test boto3-using modules
* Use pytest's importorskip instead of manually skipping on missing deps
* Fix imports in cloudformation module
* Delete unused code
* Add maybe_sleep fixtures to speed up recorded test runs
* Build basic placebo-CFN tests
* Commit placebo recordings of basic stack operations
* Add placebo to test-requires
* Allow unit tests to run regardless of environment by setting a default region
* Use explicit relative import for Python 3 compat
* Use __name__ attribute that works on Python 2 and 3
These were deprecated back in the 1.x timeframe, so have been deprecated
for the 2 cycles required. They also do things wrong and should be
avoided anyway. Go ahead and remove them.
* GCP: backend service module
* GCP: rework param-checking code. Fixed a couple of bugs and changed to ValueError instead of custom tuple.
* GCP: fixed commit, spelled out Google Cloud for clarity in module description.
fixed itertools.imap busting several things that used to be lists,
profiles not being set correctly, upon create, when it was a separate
method, allowed port having the wrong lowest port (zero is allowed),
empty port value should just be interpreted as None.
The password_hash filter will generate a salt value if none is supplied.
The character set used by Ansible
(upper & lowercase letters, digits)
did not match that used by libc crypt
(upper & lowercase letters, digits, full stop, forward slash).
This resulted in a slightly smaller key space, and hence hashes would be
slightly easier to attack (e.g. by dictionary, brute force).
* Ansible 2.3 feature support for dellos6.
- With the new Ansible 2.3 infra changes, the dellos modules doesn't work
(the new infra changes are not backward compatible), so added the below
changes support it.
- Added the new terminal plugin for DellOS6
- Added the new action plugin for DellOS6
- Modified the modules to work with the new infra.
- with that it adds support for DellOS6 Persistent Connection support.
* Remove pep8 confirming files from dellos6.py and dellos6_config legacy-files
* add else statement to ensure instids is set
set res_list to None to avoid UnboundLocalError and fix iteration over a nonetype by adding an empty tuple
* make res_list empty tuple by default and check for instids before setting tags (fails otherwise)
* fix 'sequence' lookup shortcut syntax and documentation
* Update playbooks_loops.rst
Minor edits for grammar and clarity.
* Update playbooks_loops.rst
Another tweak for clarity.
If the banner is not set, the stdout of 'show configuration | begin banner <banner>'
returns empty string thus the re.search raises an exception.
Fixes#22216
The fix for leading junk in sudo output: fee6e29 causes problems with
ssh + sudo. On the initial connection using ControlPersist, the output
that we scan for the prompt contains both the command we're sending to
configure the prompt and the prompt itself. The code in fee6e29 ends up
sending the password when it sees the line configuring the prompt which
is too early.
Switch to a version that splits on lines and then checks whether the
first or last line starts with the prompt to decide if it's time to send
the password.
Fixes#23054
References #20858
The pass prompt expects an answer and compares a `str` to a binary buffer, thus crashing.
It's an obvious fix to help transitioning towards Python3 and hopes it does not need a specific test.
In old IOSXR versions, 'show commit changes diff' does not work.
Fall-back to 'show configuration' if that command fails so execution
can move forward.
Fixes#22235
When using state=latest with the package not being installled, Ansible complains that the package is not installed and fails the task.
Whereas the expected behaviour is to install the package when it is missing.
This PR fixes this behaviour.
* Fix a couple issues in synchronize with docker
* Make the rsync_opts parse as a list using the same criteria as
module_utils argumentspec parsing
* Do not quote arguments in the action plugin. The module will quote as
it knows whether it will invoke rsync with a shell or via exec.
Fixes#23046
The cron module forces changed=True when there was no real change,
but the original crontab did not contain a final newline, which is
mandatory.
When the user has no crontab or the user does not exist at all,
crontab -l exits with 1 and the cron module correctly interprets
this as "no crontab" and stores the old crontab as "".
However this triggers changed=True, even if we're not going to
change anything, e.g. when removing a crontab entry from a user
who has no crontabs at all.
Let's special-case the fact that the old crontab is empty and not
force changed=True in that case.
* Fixes bug https://github.com/ansible/ansible/issues/22455 - Git reset fails when file with the name 'origin' is present in the repository
* #22455 Integration test for git module to test if it clones a repository with a file named origin
* Fixed yaml errors for test/integration/roles/test_git_clone/tasks/main.yml
* Removed the newly added target git and moved the code to test/integration/targets/git
* Reorganised code structure as per comments
* Fixed playbook name
* Added code that didn't make it to the playbook before.
* No longer need git.yml
* Remove non tasks specific code from tasks file and use existing variables
* Removed msg var from the vars.yml
When the ios_config module has 'defaults' param it runs in the device the command
'show running-config all' but 'all' may not be available in older devices.
This change makes introspection by using the help command and run 'full' in case
'all' is not available.
Fixes#22747
* Add examples in os_server module showing userdata
Added an example using Bash and also using Cloud-init. Also, showing using {% raw %} and {% endraw %} to provide a pointer to those hitting similar issues to the ones I had.
* Removed erronious whitespace
* Ansible 2.3 feature support for dellos9 and dellos10
* Use Persistent Connection Manager
* Fix CI issue, revert the doc and metadata changes
* Reverted the meta_info (supported_by) to community from core
* Fixed the CI issues, use module_utisl.six and updated legacy-files
Set encrypted to default False, rather than None
Otherwise you get:
```
Invalid type for parameter Encrypted, value: None, type: <type 'NoneType'>, valid types: <type 'bool'>)
```
* Adding first checkin of resource_pool module
* fixing length issues and indentation multipliers
* mend
* mend
* mend
* remove * import from modules
* Add space on variable assignments
* changing line 321 to less characters
* fixing pep8 issues
* mend
* cleaning up example section
* Add RETURN section
* adding spacing to list
* all spacing consistent now
* move imports as per CI
* update integration tests for updated boto exception message
* integration tests fail on both "test credential" test cases
exception bubbles out of module. instead catch and wrap
* ec2_group does not support updating a security group's description
AWS security group descriptions are immutable.
if ec2_group finds a group that matches by name, but the descriptions do not match, the module does not support this case
previously it would check if the group was used, but would not do anything if it was
old behavior was erroneous because it could make a user expect that the description change of a group was fine when in fact it did not occur
also, it made an expensive check against all ec2 instances for no good reason
* comments not doc strings
* else must have pass w/o doc-string statement
* Catch specific BotoServerException, give context around error when fetching SGs
* python3 compatible exception blocks
* add traceback to fail_json
* two blank lines before first function
* fixes action handlers for sros
* fixes sros_config module execution to use AnsibleModule
* fixes sros_command module to use socket connection
* adds sros to constants
* key is ANSIBLE_METADATA, not METADATA
* Need to check that the node is an ast.Dict, not a real dict.
* Need to convert the ast.Dict into a real dict via literal_eval.
* synchronize: Convert cmd to list and fix handling of the copy_links argument
Converting cmd from str to list stops the pain of argument quoting/escaping.
* synchronize: Update imports according to #pullrequestreview-28758614
* fixes#15770
* When running under the UTF-8 codepage, Powershell subprocesses will fail (eg, Start-Job, others) if the input encoding is using the default BOM preamble. This fix forces it to use no preamble in leaf_exec and win_shell, and includes tests to verify that Start-Job works.
* win_command/win_shell: Clean up and make coherent
Changes include:
- Make win_command and win_shell more coherent
- Make condensed one-line more readable
* Revert to original formatting
This change to the template action plugin make template use the
platform's native newline_sequence for Jinja.
We also added the option `newline_sequence` to change the newline
sequence using by Jinja if you need to use another newline sequence than
the platform default.
This was previously discussed in
https://github.com/ansible/ansible/issues/16255#issuecomment-278289414
And also relates to issue #21128
* fixes#22441
* fixes#22655
* moves all env handling into the exec wrapper; this should work for everything but raw, which is consistent with non-Windows.
* Update module_utils.six to latest
We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility. Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
If the # is inside of quotes, it's a string. If it's outside of quotes
then an exception will be raised which we'll catch and then send to the
to_text() call at the end of the function anyhow.
Fixes#22868
The base64 stdlib functions require byte strings. So we have to
transform the strings into bytes before handing to the stdlib and then
transform them back into text before handing back to ansible.
Fixes#22873
The junos action handler was not honoring the host value in the provider
argument. This patch will now use the provider host entry if it exists
and falls back to the inventory hostname
Made ansible-doc more plugin agnostic
We can have docs in lookup, callback, connectionm strategy, etc
Use first docstring and make pepizis happy
generalized module_docs to plugin_docs
documented cartesian, ssh, default, jsonfile, etc as examples
changed lack of docs to warning when listing
made smarter about bad docstrings
better blacklisting
added handling of options/config/envs/etc
move blacklist to find_plugins, only need once
purge_subnets|routes were introduced recently.
ensure_subnet_associations now takes purge_subnets as a parameter. A
call to this function was missed when introducing this feature. With
out, results in a "got 5 expected 6" error.
Allows to health-check two services on one host. Before, asking for 2 health checks being present for `http://myhost:8888` and `http://myhost:9999` resulted in just a single health check, as this module considered the two specs as equal.
Bonus: route53_health_check now passes style guidelines
* ensure exit_json does not fail from stderr=False
- do a little bit of safety-checking in exit_json to not try to .splitlines() on a boolean
- remove the stderr=boolean from uri.py, this is the only spot that uses it (at least so obviously)
- add unit tests that call exit_json. These are useless because the stderr parsing is in _execute_module and is difficult to mock; deleting these tests after the commit.
* remove added unit tests per prev commit
exit_json doesn't do the param parsing, that is buried deep inside _execute_module.
