Now that we don't need to worry about python-2.4 and 2.5, we can make
some improvements to the way AnsiballZ handles modules.
* Change AnsiballZ wrapper to use import to invoke the module
We need the module to think of itself as a script because it could be
coded as:
main()
or as:
if __name__ == '__main__':
main()
Or even as:
if __name__ == '__main__':
random_function_name()
A script will invoke all of those. Prior to this change, we invoked
a second Python interpreter on the module so that it really was
a script. However, this means that we have to run python twice (once
for the AnsiballZ wrapper and once for the module). This change makes
the module think that it is a script (because __name__ in the module ==
'__main__') but it's actually being invoked by us importing the module
code.
There's three ways we've come up to do this.
* The most elegant is to use zipimporter and tell the import mechanism
that the module being loaded is __main__:
* 5959f11c9d/lib/ansible/executor/module_common.py (L175)
* zipimporter is nice because we do not have to extract the module from
the zip file and save it to the disk when we do that. The import
machinery does it all for us.
* The drawback is that modules do not have a __file__ which points
to a real file when they do this. Modules could be using __file__
to for a variety of reasons, most of those probably have
replacements (the most common one is to find a writable directory
for temporary files. AnsibleModule.tmpdir should be used instead)
We can monkeypatch __file__ in fom AnsibleModule initialization
but that's kind of gross. There's no way I can see to do this
from the wrapper.
* Next, there's imp.load_module():
* https://github.com/abadger/ansible/blob/340edf7489/lib/ansible/executor/module_common.py#L151
* imp has the nice property of allowing us to set __name__ to
__main__ without changing the name of the file itself
* We also don't have to do anything special to set __file__ for
backwards compatibility (although the reason for that is the
drawback):
* Its drawback is that it requires the file to exist on disk so we
have to explicitly extract it from the zipfile and save it to
a temporary file
* The last choice is to use exec to execute the module:
* https://github.com/abadger/ansible/blob/f47a4ccc76/lib/ansible/executor/module_common.py#L175
* The code we would have to maintain for this looks pretty clean.
In the wrapper we create a ModuleType, set __file__ on it, read
the module's contents in from the zip file and then exec it.
* Drawbacks: We still have to explicitly extract the file's contents
from the zip archive instead of letting python's import mechanism
handle it.
* Exec also has hidden performance issues and breaks certain
assumptions that modules could be making about their own code:
http://lucumr.pocoo.org/2011/2/1/exec-in-python/
Our plan is to use imp.load_module() for now, deprecate the use of
__file__ in modules, and switch to zipimport once the deprecation
period for __file__ is over (without monkeypatching a fake __file__ in
via AnsibleModule).
* Rename the name of the AnsiBallZ wrapped module
This makes it obvious that the wrapped module isn't the module file that
we distribute. It's part of trying to mitigate the fact that the module
is now named __main)).py in tracebacks.
* Shield all wrapper symbols inside of a function
With the new import code, all symbols in the wrapper become visible in
the module. To mitigate the chance of collisions, move most symbols
into a toplevel function. The only symbols left in the global namespace
are now _ANSIBALLZ_WRAPPER and _ansiballz_main.
revised porting guide entry
Integrate code coverage collection into AnsiballZ.
ci_coverage
ci_complete
One of the earlier implementation of unified temp for 2.4 passed the
temp diretory to the remote side using this environment variable. We
later changed it to be passed via a module parameter but forgot to
remove the environment variable.
* Support multi-doc yaml in the from_yaml filter
* Most automatic method of handling multidoc
* Only use safe_load_all
* Implement separate filter
* Update plugin docs and changelog
Allow specifying the source and destination files' encodings in the template module
* Added output_encoding to the template module, default to utf-8
* Added documentation for the new variables
* Leveraged the encoding argument on to_text() and to_bytes() to keep the implementation as simple as possible
* Added integration tests with files in utf-8 and windows-1252 encodings, testing all combinations
* fix bad smell test by excluding windows-1252 files from the utf8 checks
* fix bad smell test by excluding valid files from the smart quote test
* win_user: use different method to validate credentials that does not rely on SMB/RPC
* Use Add-Type as SetLastError on .net reflection not working on 2012 R2
* win_chocolatey: refactor module to fix bugs and add new features
* Fix some typos and only emit install warning not in check mode
* Fixes when testing out installing chocolatey from a server
* Added changelog fragment
* Enable check_mode in command module
This only works if supplying creates or removes since it needs
something to base the heuristic off. If none are supplied it will just
skip as usual.
Fixes#15828
* Add documentation for new check_mode behavior
* Update nclu.py
Stop module from running `net` on empty commands.
* Update nclu.py
Updated the copyright date
* Update nclu.py
Returned metadata version to 1.1
* Update nclu.py
Fix indentation to be a multiple of 4.
* Create changelog fragment
* Only report change when home directory is different
Add tests with home: parameter
Have to skip macOS for now since there is a bug when specifying the home directory path for an existing user that results in a module failure. That needs to be fixed in a separate PR.
* fix a (forgotten?) change in moving createhome -> create_home
Fix for following bug on FreeBSD host whith user module:
```
fatal: [webssp]: FAILED! => {"changed": false, "module_stderr": "X11 forwarding request failed
Traceback (most recent call last):
File \"/tmp/ansible_2rmlBl/ansible_module_user.py\", line 2487, in <module>
main()\n File \"/tmp/ansible_2rmlBl/ansible_module_user.py\", line 2426, in main
(rc, out, err) = user.modify_user()
File \"/tmp/ansible_2rmlBl/ansible_module_user.py\", line 1011, in modify_user
if (info[5] != self.home and self.move_home) or (not os.path.exists(self.home) and self.createhome):
AttributeError: 'FreeBsdUser' object has no attribute 'createhome'
", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1}
```
It happenned with 'createhome' AND with 'create_home' form, with python 2.7 AND python 3.6
* Add changelog
Co-authored-by: dgeo <dgeo@users.noreply.github.com>
* win_reboot: fix 2.6 issues and better handle post reboot reboot
* changed winrm _reset to reset
* Add handler to reset calls when .reset() throws an AnsibleError on older hosts
* Moving back to _reset to get the issue fixed
The ssh connection plugin is overzealous in thinking that error code 255
can only come from ssh. Python can return 255 in some circumstances and
error from php does as well.
* __file__ won't work if we want to invoke modules via -m or if we
figure out how to keep modules from hitting the disk with pipelining.
* module.tmpdir is the new way to place a file where it will be cleaned
automatically.
Change format string to not depend on __file__:
* cloud/amazon/ec2_elb_lb.py
* cloud/amazon/elb_classic_lb.py
Use module.tempdir:
* packaging/os/apt.py
* files/unarchive.py
* Add additional puppet options
Add support for puppet options --debug, --verbose, --summary,
and extend logdest to support logging to stdout and syslog at
the same time.
Fixes issue: #37986
* Fix docs
* Doc fix, add release note
* Fix silly yaml error
* Correct changelog, add C() to params in doc
* Fix tmpdir on non root become
- also avoid exception if tmpdir and remote_tmp are None
- give 'None' on deescalation so tempfile will fallback to it's default behaviour
and use system dirs
- fix issue with bad tempdir (not existing/not createable/not writeable)
i.e nobody and ~/.ansible/tmp
- added tests for blockfile case
* Revert "Temporarily revert c119d54"
This reverts commit 5c614a59a6.
* changes based on PR feedback and changelog fragment
* changes based on the review
* Fix tmpdir when makedirs failed so we just use the system tmp
* Let missing remote_tmp fail
If remote_tmp is missing then there's something more basic wrong in the
communication from the controller to the module-side. It's better to
be alerted in this case than to silently ignore it.
jborean and I have independently checked what happens if the user sets
ansible_remote_tmp to empty string and !!null and both cases work fine.
(null is turned into a default value controller-side. empty string
triggers the warning because it is probably not a directory that the
become user is able to use).
* Revert "Account for empty string regexp in lineinfile (#41451)"
This reverts commit 4b5b4a760c.
* Use context managers for interacting with files
* Store line and regexp parameters in a variable
* Add warning when regexp is an empty string
* Remove '=' from error messages
* Update warning message and add changelog
* Add tests
* Improve warning message
Offer an equivalent regexp that won't trigger the warning.
Update tests to match new warning.
* Add porting guide entry for lineinfile change
* elasticsearch_plugin - Show STDERR on module failures.
I tried to install a ES plugin without
become: yes
and found after debugging the module that the module failed ude to permission issues.
The only error message I got was
Is analysis-icu a valid plugin name?
That was strange considering I followed the example documentation by the letter.
I found out that when this module fails, it hides the real reason for failure.
This patch replaces the generic error with more meaningful diagnostics.
* elasticsearch_plugin - Show STDERR on module failures. Changelog fragment
samdoran commented 2 days ago
This looks good. Please create a changelog fragment to go along with this change. See fragments for examples.
* runas + async - get working on older hosts
* fixed up sanity issues
* Moved first task to end of test for CI race issues
* Minor change to async test to be more stable, change to runas become to not touch the disk
* moved async test back to normal spot
* Add aws/core.py function to check for specific AWS error codes
* Use sys.exc_info to get exception object if it isn't passed in
* Allow catching exceptions with is_boto3_error_code
* Replace from_code with is_boto3_error_code
* Return a type that will never be raised to support stricter type comparisons in Python 3+
* Use is_boto3_error_code in aws_eks_cluster
* Add duplicate-except to ignores when using is_boto3_error_code
* Add is_boto3_error_code to module development guideline docs
In the process of building up the inventory by parsing each inventory
source with each available inventory plugin, there are three kinds of
possible errors (listed in order from earliest to latest):
1. One source could not be parsed by a particular plugin.
2. One source could not be parsed by any available plugin.
3. ALL sources could not be parsed by any available plugin.
The errors in (1) are a part of normal operation, e.g., the script
plugin is expected to fail to parse an ini-format source, and we will
ignore that error and try the next plugin. There is currently no way to
control this, and no known compelling use-case for a setting to control
it. This commit does not make any changes here.
We implement "any_unparsed_is_failed" to handle (2) above. If enabled,
this requires that every available source be parsed validly by at least
one plugin. In an inventory comprising a static hosts file and ec2.py,
this setting will cause a fatal error if ec2.py fails (a situation that
attracted only a warning earlier).
We clarify that the existing "unparsed_is_failed=true" setting causes a
fatal error only in (3) above, i.e., if NO inventory source could be
parsed. In other words, if there is ANY valid source in the inventory
(e.g., an ini-format static file), no combination of errors and the
setting will cause a fatal error.
If you want to execute your playbooks when your inventory is…
(a) complete, use "any_unparsed_is_failed=true".
(b) not empty, use "unparsed_is_failed=true".
The "unparsed_is_failed" setting should be renamed to
"all_unparsed_is_failed", but this commit does not do so.
Fixes#40512Fixes#40996
* Port aws_ses_identity module to use AnsibleAWSModule
* Support Check Mode in aws_ses_identity
* Add tests for check mode
* Move feedback forwarding parameter check to before any changes are made.
* Fixed check_mode status to be the same as normal execution
* Now when setting the status to `disabled` in check_mode it correctly
returns the state changed and prints a warning like it does in normal
model. Before it always returned changed even if everything was set
correctly and a reboot was required.
* Add changelog entry
Co-authored by: Strahinja Kustudic <kustodian@gmail.com>
* winrm: add better exception handling for krb5 auth with pexpect
* Added changelog fragment
* Added exception handler in case kinit path isn't valid, added test cases
* fixed for Python 2 compatibility
* win_updates: add scheduled tasks back in for older hosts
* Fixed up typo in category name error message
* Fixed up some minor issues after merge
* added changelog fragment
* Default to become but add override to use scheduled tasks
* Added basic unit tests for win_updates
* fix minor typos
* Get the str value of xmlrpc.client.DateTime
* get_all_records should be used instead of get_all
* Facts returned with 'ansible_facts'
* Remove some redundant code
* Add cheese as maintainer
* Add changelog entry
* Set encrypted as default and fix empty password reporting changed
* Starting with Postgres 10 `UNENCRYPTED` passwords are removed and
because of that this module fails with the default `encrypted=no`.
Also encrypted passwords are suported since version 7.2
(https://www.postgresql.org/docs/7.2/static/sql-createuser.html) which
went EOL in 2007 and since 7.3 it is the default. Because of this it
makes a lot more sense to make `encrypted=yes` the default. This won't
break backward compatibility, the module would just update the user's
password in the DB in the hashed format and everything else will work
like before. It's also a security bad practice to store passwords in
plain text. fixes#25823
* There was also a bug with `encrypted=yes` and an empty password always
reported as changed.
* Improved documentation for `encrypted`/`password` parameters, and
removed some obsolete notes about passlib.
* Fix clearing user's password to work with all versions of Postgres
* Add tests for clearing the user password
* Fix documentation atfer rebase
* Add changelog fragment
* Fix all cases with pause and ctrl+c
- naked:
- pause:
- with prompt
- pause: prompt=hi
- time wait
- pause: seconds=60
- time wait with prompt
- pause: seconds=60 prompt=hi
Fixes#35372
* Use curses to control stdout
* Use curses to clear lines on interactive input
* Validate input for echo parameter and fail nicely if invalid
* Add integration tests for pause module using pexpect
* Use try except when trying to determine erase sequence to account for lack of TTY in containers in tests
* Improve output validation for regular paus test
* Accept two digit precision for pause length in test
* Check for seconds when seconds is specificed, minutes when minutes is specified
* Add test for no TTY mode
Co-authored by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored by: Brian Coca <brian.coca+git@gmail.com>
* Add the ability to specify an install_dir to the gem module
* Add GEM_HOME when installing a non-global gem
* Add tests for custom gem path
* Fix sanity tests
* Add changelog entry
* Rebase and add tests for incorrect options
Co-authored by: Antoine Catton <devel@antoine.catton.fr>
* add pytest_cache to gitignore
* onepassword lookup plugin
* fix linter/style test complaints
* second pass at making pycodestyle happy
* use json module instead of jq
* update copyrights, license & version added
* fix python2 compatibility
* doh. fix spacing issue.
* use standard ansible exception
* remove potentially problematic stdin argument
* actually call assertion method
* add support for top-level fields
* make vault uuids pedantically consistent in fixture
* fix new style issues
* ability specify section & correct case handling
* improve error handling
* add onepassword_raw plugin
* Add maintainer info
* Move common code to module_utils/onepassword.py
* Load raw data JSON data for easier use in Ansible
* Put OnePass class back inside lookup plugin
There is no good place for sharing code across lookups currently.
* Remove debugging code in unit tests
* Patche proper module in raw unit tests
* Add changelog entry
Co-authored-by: Scott Buchanan <sbuchanan@ri.pn>
* Set src in the state functions rather than the toplevel
A good API should only require passing one version of a piece of data
around so do that for src
* Move the rewriting of path into additional_parameter_handling
When the path is a directory we can rewrite the path to be a file inside
of the directory
* Emit a warning when src is used with a state where it should be ignored
* base64 filter: Added ability to specify encoding
* Added unicode chars for further testing
* Removed errors to keep previous behaviour in place
* Removed surrogate pairs due to issues loading YAML in CI
* create internal loadbalancer
* fix test
* remove duplicate test
* clean up
* fix doc
* lint
* add sku support
* update version
* change to the version the same as CLI
* add pip support sku
* fix lint
* fix test
* Update main.yml
* add changelog entry
There was a traceback when setting permissions on a directory tree when
there were broken symlinks inside of the tree and follow=true. chmod -R
ignores broken symlinks inside of the tree so we've fixed the file
module to do the same.
Fixes#39456
* Fix for file module with symlinks to nonexistent target
When creating a symlink to a nonexistent target, creating the symlink
would work but subsequent runs of the task would fail because it was
trying to operate on the target instead of the symlink.
Fixes#39558
* Only change expiration date if it is different
Modify user_info() method to also return the password expiration.
Compare current and desired expiration times and only change if they are different.
* Improve formatting on user tests
* Add integration test for expiration
* Add changelog fragment
* Improve integration test
Skip macOS and use getent module for validating expiration date.
* Fix expiration change for FreeBSD
* Don't use datetime since the total_seconds method isn't available on CentOS 6
* Use better name for expiration index field
Use separate tasks for verifying expiration date on BSD
* Use calendar.timegm() rather than time.mktime()
calendar.timegm() is the inverse of time.gmtime() and returns a timestamp in UTC not localtime
Add tests that change the system timezone away from UTC
* Mark tests as destructive and use test for change status
* Fix account expiration for FreeBSD
Use DATE_FORMAT when setting expiration date on FreeBSD. Previously the argument passed to -e was an integer of days since epoch when the account will expire which was inserted directly into master.passwd. This value is interpreted as seconds since epoch by the system, meaning the account expiration was actually set to a few hours past epoch.
Greatly simply comparing desired and current expiration time by using the first three values of the struct_time tuple rather than doing a whole bunch of manipulations of the seconds since epoch.
* Fixes for mode=preserve
* Document mode=preserve for template and copy
* Make mode=preserve work with remote_src for copy
* Make mode=preserve work for template
* Integration tests for copy & template mode=preserve
Fixes#39279
* Changed mode option in win_copy to hidden option as it doesn't reflect copy mode
