The stdout and stderr values returned from self._low_level_execute() are text, not bytes. This results in an error in Python 3 since str and bytes cannot be concatenated.
Changing to unicode type allows this to work without error on Python 2 and Python 3.
(cherry picked from commit 77f73f6d2a)
* reboot: Fix typo and support bare Linux systems
This fixes a problem for bare Linux systems that do not support 'who -b' or 'uptime -s'.
* Accumulate stdout and stderr information
(cherry picked from commit a7a99c5fd4)
- use context manager for dealing with the checksum file
- use loop that can tolerate zero, one, or more items return rather than the previous expression which would break if anything other than exactly one item was returned
(cherry picked from commit 03dbb1d9c4)
squash
* Fix targets that may be a list containing strings and lists which worked prior to 2.6.
* Add ec2_group integration tests for lists of nested targets
* changelog
* Add diff mode support for lists of targets containing strings and lists.
(cherry picked from commit d7ca3f2bd3)
* fix nxos_facts indefinite hang for text based output (#45845)
Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
(cherry picked from commit 1b0c4fde86)
* Fix remote checksums when paths have leading dots (#45287)
* Fix remote checksums with paths have leading dots
* Fix result recorded from the wrong file
* Add changelog fragment
(cherry picked from commit 600c7ac108)
* [stable-2.7] Fix logic to not re-download existing files when force=no (#45495)
* Fix logic to not re-download existing files when force=no. Fixes#45491
* Reduce logic complexity.
(cherry picked from commit 5785de582f)
Co-authored-by: Matt Martz <matt@sivel.net>
* Backport of get_url fix cannot use result
result was only added in 2.8+
* [AWS] cloudfront_distribution - fix method name from 'validate_distribution_id_from_caller_reference' to 'validate_distribution_from_caller_reference' and set distribution_id to the distribution's key 'Id' (#45498)
(cherry picked from commit e5269c047c)
* changelog
* Don't pass file_name to DataLoader.load in script inventory plugin. Fixes#34164
* Add changelog fragment
(cherry picked from commit 263b9fa)
Co-authored-by: Matt Martz <matt@sivel.net>
* plugins/callbacks : fix some key documentation (#45267)
These are supposed to be a dictionary with section/key entries;
correct a few typos.
(cherry picked from commit c34e0f5e11)
* backport of #45267
* ec2_vpc_route_table - allow routes to be created if the CIDR already exists but its 'Origin' is 'EnableVgwRoutePropagation' (which cannot be replaced). (#43417)
Fixes#43415
(cherry picked from commit a6c97f2243)
* Add new session parameter to win_psexec
* Indicate which version the session option was added in
* Added changelog fragment and minor edit on doc entry
* YUM4/DNF compatibility via yum action plugin
DNF does not natively support allow_downgrade as an option, instead
that is always the default (not configurable by the administrator)
so it had to be implemented
- Fixed group actions in check mode to report correct changed state
- Better error handling for depsolve and transaction errors in DNF
- Fixed group action idempotent transactions
- Add use_backend to yum module/action plugin
- Fix dnf handling of autoremove (didn't used to work nor had a
default value specified, now does work and matches default
behavior of yum)
- Enable installroot tests for yum4(dnf) integration testing, dnf
backend now supports that
- Switch from zip to bc for certain package install/remove test
cases in yum integration tests. The dnf depsolver downgrades
python when you uninstall zip which alters the test environment
and we have no control over that.
- Add changelog fragment
- Return a pkg_mgr fact if it was not previously set.
* Share the implementation of hashing for both vars_prompt and password_hash.
* vars_prompt with encrypt does not require passlib for the algorithms
supported by crypt.
* Additional checks ensure that there is always a result.
This works around issues in the crypt.crypt python function that returns
None for algorithms it does not know.
Some modules (like user module) interprets None as no password at all,
which is misleading.
* The password_hash filter supports all parameters of passlib.
This allows users to provide a rounds parameter, fixing #15326.
* password_hash is not restricted to the subset provided by crypt.crypt,
fixing one half of #17266.
* Updated documentation fixes other half of #17266.
* password_hash does not hard-code the salt-length, which fixes bcrypt
in connection with passlib.
bcrypt requires a salt with length 22, which fixes#25347
* Salts are only generated by ansible when using crypt.crypt.
Otherwise passlib generates them.
* Avoids deprecated functionality of passlib with newer library versions.
* When no rounds are specified for sha256/sha256_crypt and sha512/sha512_crypt
always uses the default values used by crypt, i.e. 5000 rounds.
Before when installed passlibs' defaults were used.
passlib changes its defaults with newer library versions, leading to non
idempotent behavior.
NOTE: This will lead to the recalculation of existing hashes generated
with passlib and without a rounds parameter.
Yet henceforth the hashes will remain the same.
No matter the installed passlib version.
Making these hashes idempotent.
Fixes#15326Fixes#17266Fixes#25347 except bcrypt still uses 2a, instead of the suggested 2b.
* random_salt is solely handled by encrypt.py.
There is no _random_salt function there anymore.
Also the test moved to test_encrypt.py.
* Uses pytest.skip when passlib is not available, instead of a silent return.
* More checks are executed when passlib is not available.
* Moves tests that require passlib into their own test-function.
* Uses the six library to reraise the exception.
* Fixes integration test.
When no rounds are provided the defaults of crypt are used.
In that case the rounds are not part of the resulting MCF output.
* Switch to LiteralPath instead of Path. Closes#44508
* add changelog fragment
* fix line endings and remove final empty line
* Minor text changes in changelog
* elb_application_lb: fix dangerous default of deleting an ELB if state is omitted by changing state to default to present to be more like other AWS modules
* expend checksum format to <algorithm>:(<checksum>|<url>)
* continue to code at office
* ALPHA - expend checksum format to <algorithm>:(<checksum>|<url>)
* clean up tmpfile and comment
* try to add test code for 27617
* try to add test code for 27617
* try to add test code for 27617
* try to fix [Could not find or access 'testserver.py']
* fix test code [Could not find or access 'testserver.py']
* fix test code [add files dir]
* fix test code [files dir not exists]
* as [connection was closed before a valid response was received]
* [connection was closed before a valid response was received]
* [connection was closed before a valid response was received]
* add test item [sha1 and sha256]
* since [connection was closed before a valid response was received]
* fix [connection was closed before a valid response was received]
* fix test code typo
* add docs for #27617
* PR #43751 is minor change
* fix pep8 issue.
* fix test code style.
* fix unexpected quote
* Add (preview) diff mode support ec2_group
* Add diff mode to some ec2_group integration tests
* Remove unnecessary arguments and add comment to the module notes
* Add changelog
In python 3.7.0, changes in `ssl.py` breaks `smtplib.SMTP_SSL`, which
then breaks `mail` module in ansible.
Run this line in python shell:
import smtplib;smtplib.SMTP_SSL().connect(host='smtp.gmail.com', port=465)
Before python 3.7.0, we will get:
(220, b'smtp.gmail.com ESMTP j13-v6sm3086685pgq.56 - gsmtp')
In python 3.7.0, we get such error at `lib/python3.7/ssl.py` line 843, method `_create`:
ValueError: server_hostname cannot be an empty string or start with a leading dot.
The ssl module is using host info on SMTP_SSL instance, which is not set.
The fix/workaround is simple, just pass host info to it:
import smtplib;smtplib.SMTP_SSL(host='smtp.gmail.com').connect(host='smtp.gmail.com', port=465)
Fixes: #44550
Signed-off-by: Guo Qiao <guoqiao@gmail.com>
Extends `module_defaults` by adding a prefix to defaults `group/` which denotes a builtin list of modules. Initial groups are: `group/aws`, `group/azure`, and `group/gcp`
* added account_alias in the response of module aws_caller_facts
* added comment to explain list_account_aliases
* renamed caller_identity to caller_facts as the content is extended
* created changelog
* security-policy needs the iam:ListAccountAliases for this module to work
* test now checks for the added field account_alias
* gracefully handle missing iam:ListAccountAliases permission
* Use newer is_sequence function instead of MutableSequence. Fixes#44327
* Add changelog for #44331
* Update changelog fragment to describe the fix in more detail
* Don't use copy.deepcopy in high workload areas, use deepishcopy. ci_complete
* Add tests
* Add changelog fragment
* rename to naive_deepcopy and add extra docs
* Rename to module_response_deepcopy and move to vars/clean
One can install alternate packages managers on debuntu machines.
However, doing so doesn't mean you want to suddenly start using them.
Add in a check similar to the fedora yum/dnf check that sets apt as the
pkg_mgr if the ansible_os_family is Debian.
* Add backup option
* Only backup shadow file when the OS has one
* Only backup shadow file for SunOS
* Update docs on backup feature
* Add changelog fragment
* Add tests for shadow backup
* Remove backup option, make it automatic
Remove the option to enable/disable backups and make it automatic. Add note to docs describing this behavior.
Change tests to account for new module behavior.
Change section name in changelog fragment since minor_features is not a valid section.
* strip additional comments from /etc/default/passwd
Strip trailling comments from /etc/default/passwd like
MINWEEKS=1 #MINWEEKS=2
MAXWEEKS=12 # MAXWEEKS=8
Which otherwise cause failures with "failed to read /etc/default/passwd: too many values to unpack"
* fix carriage return typo in commit
* yet another typo in commit
* Fix indent problem
* add changelog fragment for PR 43931
* 29891 use os.path.realpath to follow keyfile symlinks
* 29891 add parameter follow
* updated changelog fragment
* add documentation and set default to false
* Only template values in vars_prompt rather than all vars
This allows the use of variables in vars_prompt fields but allows variables entered in the prompt to affect play vars rather than throwing an undefined error.
Only post validate if there was a vars_prompt
* Add tests for vars_prompt
* Check the password format
Check the password format and notify user if they
input unencrypted password.
* Fix sanity error
* Add integration test
* Missed a task name
* Hard code the testing password
Since some testing platfrom has no passlib installed
* Add changelog fragment
* Rework some English sentences
* Fix a grammar mistake
The bundled selectors library which is used by the local and ssh
connection plugins had a bug which caused a traceback in a cornercase.
If selectors were in use and a syscall was interrupted, selectors would
attempt to restart the syscall after the interrupt was processed. if
the attempt determined that the timeout for running the syscall had
already expired, the code attempted to raise OSError. The raise was
using a Python3-ism and needed to be ported to work on Python2.
Fixes#41630
* Collecting PEM -> DER conversions.
* Using cryptography instead of OpenSSL binary in some situations.
* Moving key-to-disk writing for key content to parse_account_key.
* Rename parse_account_key -> parse_key.
* Move OpenSSL specific code for key parsing and request signing into global functions.
* Also using cryptography for key parsing and request signing.
* Remove assert statements.
* Fixing handling of key contents for cryptography code path.
* Allow to disable the use of cryptography.
* Updating documentation.
* 1.5 seems to work as well (earlier versions don't have EC sign function). Making Python 2.x adjustments.
* Changing option to select_crypto_backend.
* Python 2.6 compatibility.
* Trying to test both backends separately for acme_account.
* Also testing both backends separately for acme_certificate and acme_certificate_revoke.
* Adding changelog entry which informs about select_crypto_backend option in case autodetect fails.
* Fixing YAML.
they dropped the Linux so now it only shows as ALT, it should still be backwards compatible
pkg_mgr detection relies on `Altlinux` string, so properly setting os_distribution should take care of it as side effect
fixes#43539
* Remove use of simplejson throughout code base. Fixes#42761
* Address failing tests
* Remove simplejson from contrib and other outlying files
* Add changelog fragment for simplejson removal
Only print warning when ansible.cfg is actually skipped
* Also add unittests for the find_ini_config_file function
* Add documentation on world writable current working directory
config files can no longer be loaded from a world writable current
working directory but the end user is allowed to specify that
explicitly. Give appropriate warnings and information on how.
Fixes#42388
On Python3, these would be text strings already. On Python2, we need to
convert them from bytes.
Use a new helper function py3compat to do this.
Fixes#43207
Some distribtuions like SUSE has the rc%.d directories under /etc/init.d
Quote of /etc/rc.d.README on SLES11.
"Some people expect the system startup scripts in /etc/rc.d/.
We use a slightly different structure for better LSB compliance."
* Add Ansible.ModuleUtils.PrivilegeUtil and converted code to use it
* Changed namespace and class to be a better standard and fixed some typos
* Changes from review
* changes to avoid out of bound mem of server 2008
* changes to detect failure when setting a privileged not allowed
Now that we don't need to worry about python-2.4 and 2.5, we can make
some improvements to the way AnsiballZ handles modules.
* Change AnsiballZ wrapper to use import to invoke the module
We need the module to think of itself as a script because it could be
coded as:
main()
or as:
if __name__ == '__main__':
main()
Or even as:
if __name__ == '__main__':
random_function_name()
A script will invoke all of those. Prior to this change, we invoked
a second Python interpreter on the module so that it really was
a script. However, this means that we have to run python twice (once
for the AnsiballZ wrapper and once for the module). This change makes
the module think that it is a script (because __name__ in the module ==
'__main__') but it's actually being invoked by us importing the module
code.
There's three ways we've come up to do this.
* The most elegant is to use zipimporter and tell the import mechanism
that the module being loaded is __main__:
* 5959f11c9d/lib/ansible/executor/module_common.py (L175)
* zipimporter is nice because we do not have to extract the module from
the zip file and save it to the disk when we do that. The import
machinery does it all for us.
* The drawback is that modules do not have a __file__ which points
to a real file when they do this. Modules could be using __file__
to for a variety of reasons, most of those probably have
replacements (the most common one is to find a writable directory
for temporary files. AnsibleModule.tmpdir should be used instead)
We can monkeypatch __file__ in fom AnsibleModule initialization
but that's kind of gross. There's no way I can see to do this
from the wrapper.
* Next, there's imp.load_module():
* https://github.com/abadger/ansible/blob/340edf7489/lib/ansible/executor/module_common.py#L151
* imp has the nice property of allowing us to set __name__ to
__main__ without changing the name of the file itself
* We also don't have to do anything special to set __file__ for
backwards compatibility (although the reason for that is the
drawback):
* Its drawback is that it requires the file to exist on disk so we
have to explicitly extract it from the zipfile and save it to
a temporary file
* The last choice is to use exec to execute the module:
* https://github.com/abadger/ansible/blob/f47a4ccc76/lib/ansible/executor/module_common.py#L175
* The code we would have to maintain for this looks pretty clean.
In the wrapper we create a ModuleType, set __file__ on it, read
the module's contents in from the zip file and then exec it.
* Drawbacks: We still have to explicitly extract the file's contents
from the zip archive instead of letting python's import mechanism
handle it.
* Exec also has hidden performance issues and breaks certain
assumptions that modules could be making about their own code:
http://lucumr.pocoo.org/2011/2/1/exec-in-python/
Our plan is to use imp.load_module() for now, deprecate the use of
__file__ in modules, and switch to zipimport once the deprecation
period for __file__ is over (without monkeypatching a fake __file__ in
via AnsibleModule).
* Rename the name of the AnsiBallZ wrapped module
This makes it obvious that the wrapped module isn't the module file that
we distribute. It's part of trying to mitigate the fact that the module
is now named __main)).py in tracebacks.
* Shield all wrapper symbols inside of a function
With the new import code, all symbols in the wrapper become visible in
the module. To mitigate the chance of collisions, move most symbols
into a toplevel function. The only symbols left in the global namespace
are now _ANSIBALLZ_WRAPPER and _ansiballz_main.
revised porting guide entry
Integrate code coverage collection into AnsiballZ.
ci_coverage
ci_complete
One of the earlier implementation of unified temp for 2.4 passed the
temp diretory to the remote side using this environment variable. We
later changed it to be passed via a module parameter but forgot to
remove the environment variable.
* Support multi-doc yaml in the from_yaml filter
* Most automatic method of handling multidoc
* Only use safe_load_all
* Implement separate filter
* Update plugin docs and changelog
Allow specifying the source and destination files' encodings in the template module
* Added output_encoding to the template module, default to utf-8
* Added documentation for the new variables
* Leveraged the encoding argument on to_text() and to_bytes() to keep the implementation as simple as possible
* Added integration tests with files in utf-8 and windows-1252 encodings, testing all combinations
* fix bad smell test by excluding windows-1252 files from the utf8 checks
* fix bad smell test by excluding valid files from the smart quote test
* win_user: use different method to validate credentials that does not rely on SMB/RPC
* Use Add-Type as SetLastError on .net reflection not working on 2012 R2
* win_chocolatey: refactor module to fix bugs and add new features
* Fix some typos and only emit install warning not in check mode
* Fixes when testing out installing chocolatey from a server
* Added changelog fragment
* Enable check_mode in command module
This only works if supplying creates or removes since it needs
something to base the heuristic off. If none are supplied it will just
skip as usual.
Fixes#15828
* Add documentation for new check_mode behavior
* Update nclu.py
Stop module from running `net` on empty commands.
* Update nclu.py
Updated the copyright date
* Update nclu.py
Returned metadata version to 1.1
* Update nclu.py
Fix indentation to be a multiple of 4.
* Create changelog fragment
* Only report change when home directory is different
Add tests with home: parameter
Have to skip macOS for now since there is a bug when specifying the home directory path for an existing user that results in a module failure. That needs to be fixed in a separate PR.
* fix a (forgotten?) change in moving createhome -> create_home
Fix for following bug on FreeBSD host whith user module:
```
fatal: [webssp]: FAILED! => {"changed": false, "module_stderr": "X11 forwarding request failed
Traceback (most recent call last):
File \"/tmp/ansible_2rmlBl/ansible_module_user.py\", line 2487, in <module>
main()\n File \"/tmp/ansible_2rmlBl/ansible_module_user.py\", line 2426, in main
(rc, out, err) = user.modify_user()
File \"/tmp/ansible_2rmlBl/ansible_module_user.py\", line 1011, in modify_user
if (info[5] != self.home and self.move_home) or (not os.path.exists(self.home) and self.createhome):
AttributeError: 'FreeBsdUser' object has no attribute 'createhome'
", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1}
```
It happenned with 'createhome' AND with 'create_home' form, with python 2.7 AND python 3.6
* Add changelog
Co-authored-by: dgeo <dgeo@users.noreply.github.com>
* win_reboot: fix 2.6 issues and better handle post reboot reboot
* changed winrm _reset to reset
* Add handler to reset calls when .reset() throws an AnsibleError on older hosts
* Moving back to _reset to get the issue fixed
The ssh connection plugin is overzealous in thinking that error code 255
can only come from ssh. Python can return 255 in some circumstances and
error from php does as well.
* __file__ won't work if we want to invoke modules via -m or if we
figure out how to keep modules from hitting the disk with pipelining.
* module.tmpdir is the new way to place a file where it will be cleaned
automatically.
Change format string to not depend on __file__:
* cloud/amazon/ec2_elb_lb.py
* cloud/amazon/elb_classic_lb.py
Use module.tempdir:
* packaging/os/apt.py
* files/unarchive.py
* Add additional puppet options
Add support for puppet options --debug, --verbose, --summary,
and extend logdest to support logging to stdout and syslog at
the same time.
Fixes issue: #37986
* Fix docs
* Doc fix, add release note
* Fix silly yaml error
* Correct changelog, add C() to params in doc
* Fix tmpdir on non root become
- also avoid exception if tmpdir and remote_tmp are None
- give 'None' on deescalation so tempfile will fallback to it's default behaviour
and use system dirs
- fix issue with bad tempdir (not existing/not createable/not writeable)
i.e nobody and ~/.ansible/tmp
- added tests for blockfile case
* Revert "Temporarily revert c119d54"
This reverts commit 5c614a59a6.
* changes based on PR feedback and changelog fragment
* changes based on the review
* Fix tmpdir when makedirs failed so we just use the system tmp
* Let missing remote_tmp fail
If remote_tmp is missing then there's something more basic wrong in the
communication from the controller to the module-side. It's better to
be alerted in this case than to silently ignore it.
jborean and I have independently checked what happens if the user sets
ansible_remote_tmp to empty string and !!null and both cases work fine.
(null is turned into a default value controller-side. empty string
triggers the warning because it is probably not a directory that the
become user is able to use).
* Revert "Account for empty string regexp in lineinfile (#41451)"
This reverts commit 4b5b4a760c.
* Use context managers for interacting with files
* Store line and regexp parameters in a variable
* Add warning when regexp is an empty string
* Remove '=' from error messages
* Update warning message and add changelog
* Add tests
* Improve warning message
Offer an equivalent regexp that won't trigger the warning.
Update tests to match new warning.
* Add porting guide entry for lineinfile change
* elasticsearch_plugin - Show STDERR on module failures.
I tried to install a ES plugin without
become: yes
and found after debugging the module that the module failed ude to permission issues.
The only error message I got was
Is analysis-icu a valid plugin name?
That was strange considering I followed the example documentation by the letter.
I found out that when this module fails, it hides the real reason for failure.
This patch replaces the generic error with more meaningful diagnostics.
* elasticsearch_plugin - Show STDERR on module failures. Changelog fragment
samdoran commented 2 days ago
This looks good. Please create a changelog fragment to go along with this change. See fragments for examples.
* runas + async - get working on older hosts
* fixed up sanity issues
* Moved first task to end of test for CI race issues
* Minor change to async test to be more stable, change to runas become to not touch the disk
* moved async test back to normal spot
