Commit graph

109 commits

Author SHA1 Message Date
Toshio Kuratomi
52449cc01a AnsiballZ improvements
Now that we don't need to worry about python-2.4 and 2.5, we can make
some improvements to the way AnsiballZ handles modules.

* Change AnsiballZ wrapper to use import to invoke the module
  We need the module to think of itself as a script because it could be
  coded as:

      main()

  or as:

      if __name__ == '__main__':
          main()

  Or even as:

      if __name__ == '__main__':
          random_function_name()

  A script will invoke all of those.  Prior to this change, we invoked
  a second Python interpreter on the module so that it really was
  a script.  However, this means that we have to run python twice (once
  for the AnsiballZ wrapper and once for the module).  This change makes
  the module think that it is a script (because __name__ in the module ==
  '__main__') but it's actually being invoked by us importing the module
  code.

  There's three ways we've come up to do this.
  * The most elegant is to use zipimporter and tell the import mechanism
    that the module being loaded is __main__:
    * 5959f11c9d/lib/ansible/executor/module_common.py (L175)
    * zipimporter is nice because we do not have to extract the module from
      the zip file and save it to the disk when we do that.  The import
      machinery does it all for us.
    * The drawback is that modules do not have a __file__ which points
      to a real file when they do this.  Modules could be using __file__
      to for a variety of reasons, most of those probably have
      replacements (the most common one is to find a writable directory
      for temporary files.  AnsibleModule.tmpdir should be used instead)
      We can monkeypatch __file__ in fom AnsibleModule initialization
      but that's kind of gross.  There's no way I can see to do this
      from the wrapper.

  * Next, there's imp.load_module():
    * https://github.com/abadger/ansible/blob/340edf7489/lib/ansible/executor/module_common.py#L151
    * imp has the nice property of allowing us to set __name__ to
      __main__ without changing the name of the file itself
    * We also don't have to do anything special to set __file__ for
      backwards compatibility (although the reason for that is the
      drawback):
    * Its drawback is that it requires the file to exist on disk so we
      have to explicitly extract it from the zipfile and save it to
      a temporary file

  * The last choice is to use exec to execute the module:
    * https://github.com/abadger/ansible/blob/f47a4ccc76/lib/ansible/executor/module_common.py#L175
    * The code we would have to maintain for this looks pretty clean.
      In the wrapper we create a ModuleType, set __file__ on it, read
      the module's contents in from the zip file and then exec it.
    * Drawbacks: We still have to explicitly extract the file's contents
      from the zip archive instead of letting python's import mechanism
      handle it.
    * Exec also has hidden performance issues and breaks certain
      assumptions that modules could be making about their own code:
      http://lucumr.pocoo.org/2011/2/1/exec-in-python/

  Our plan is to use imp.load_module() for now, deprecate the use of
  __file__ in modules, and switch to zipimport once the deprecation
  period for __file__ is over (without monkeypatching a fake __file__ in
  via AnsibleModule).

* Rename the name of the AnsiBallZ wrapped module
  This makes it obvious that the wrapped module isn't the module file that
  we distribute.  It's part of trying to mitigate the fact that the module
  is now named __main)).py in tracebacks.

* Shield all wrapper symbols inside of a function
  With the new import code, all symbols in the wrapper become visible in
  the module.  To mitigate the chance of collisions, move most symbols
  into a toplevel function.  The only symbols left in the global namespace
  are now _ANSIBALLZ_WRAPPER and _ansiballz_main.

revised porting guide entry

Integrate code coverage collection into AnsiballZ.

ci_coverage
ci_complete
2018-07-26 20:07:25 -07:00
Toshio Kuratomi
6eacfecb73 ANSIBLE_REMOTE_TMP was an implementation of unified temp that was later changed
One of the earlier implementation of unified temp for 2.4 passed the
temp diretory to the remote side using this environment variable.  We
later changed it to be passed via a module parameter but forgot to
remove the environment variable.
2018-07-25 16:57:46 -07:00
Fabian von Feilitzsch
d27de6acd9 Add from_yaml_all to support multi document yaml strings (#43037)
* Support multi-doc yaml in the from_yaml filter

* Most automatic method of handling multidoc

* Only use safe_load_all

* Implement separate filter

* Update plugin docs and changelog
2018-07-25 16:12:22 -04:00
Julien Champseix
19dc267e4c Allow specifying the output encoding in the template module (#42171)
Allow specifying the source and destination files' encodings in the template module

* Added output_encoding to the template module, default to utf-8
* Added documentation for the new variables
* Leveraged the encoding argument on to_text() and to_bytes() to keep the implementation as simple as possible
* Added integration tests with files in utf-8 and windows-1252 encodings, testing all combinations
* fix bad smell test by excluding windows-1252 files from the utf8 checks
* fix bad smell test by excluding valid files from the smart quote test
2018-07-25 13:10:40 -07:00
Brian Coca
ac1f05478e Allow to specifically customize console's color 2018-07-24 13:21:58 -04:00
Jordan Borean
04431216e7
win_user: use different method to validate credentials that does not rely on SMB/RPC (#43059)
* win_user: use different method to validate credentials that does not rely on SMB/RPC

* Use Add-Type as SetLastError on .net reflection not working on 2012 R2
2018-07-24 08:16:42 +10:00
Jordan Borean
93c05074ee
win_chocolatey: refactor module to fix bugs and add new features (#43013)
* win_chocolatey: refactor module to fix bugs and add new features

* Fix some typos and only emit install warning not in check mode

* Fixes when testing out installing chocolatey from a server

* Added changelog fragment
2018-07-24 07:52:13 +10:00
Jarryd Tilbrook
460f858640 Enable check_mode in command module (#40428)
* Enable check_mode in command module

This only works if supplying creates or removes since it needs
something to base the heuristic off. If none are supplied it will just
skip as usual.
Fixes #15828

* Add documentation for new check_mode behavior
2018-07-23 14:06:41 -07:00
Matt Martz
8d933928d2 Only assume GET if no data, otherwise POST (#43133)
* Only assume GET if no data, otherwise POST. Fixes #42876 #43091 #42750
2018-07-23 07:30:10 -07:00
Barry Peddycord III
dc42b43cd1 NCLU Module: Improve performance by not operating on empty lines (#43024)
* Update nclu.py

Stop module from running `net` on empty commands.

* Update nclu.py

Updated the copyright date

* Update nclu.py

Returned metadata version to 1.1

* Update nclu.py

Fix indentation to be a multiple of 4.

* Create changelog fragment
2018-07-20 11:38:29 -04:00
Thierry Bouvet
5d23406926 Fix ssl_version default value. (#42955)
* Fix ssl_version default value.

* Add changelog
2018-07-20 11:32:04 -04:00
Brian Coca
9217fbb7dd
better error messasge (#42770)
* better error messasge
2018-07-19 12:13:09 -04:00
Sam Doran
0ca61e9d87
Only report change when home directory is different on FreeBSD (#42865)
* Only report change when home directory is different

Add tests with home: parameter

Have to skip macOS for now since there is a bug when specifying the home directory path for an existing user that results in a module failure. That needs to be fixed in a separate PR.
2018-07-19 10:07:00 -04:00
dgeo
ae96ba0d4f fix a (forgotten?) change in moving createhome -> create_home (#42711)
* fix a (forgotten?) change in moving createhome -> create_home

Fix for following bug on FreeBSD host whith user module:
```
fatal: [webssp]: FAILED! => {"changed": false, "module_stderr": "X11 forwarding request failed
Traceback (most recent call last):
  File \"/tmp/ansible_2rmlBl/ansible_module_user.py\", line 2487, in <module>
    main()\n  File \"/tmp/ansible_2rmlBl/ansible_module_user.py\", line 2426, in main
    (rc, out, err) = user.modify_user()
  File \"/tmp/ansible_2rmlBl/ansible_module_user.py\", line 1011, in modify_user
    if (info[5] != self.home and self.move_home) or (not os.path.exists(self.home) and self.createhome):
AttributeError: 'FreeBsdUser' object has no attribute 'createhome'
", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 1}
```
It happenned with 'createhome' AND with 'create_home' form, with python 2.7 AND python 3.6

* Add changelog


Co-authored-by: dgeo <dgeo@users.noreply.github.com>
2018-07-16 16:55:57 -04:00
Filippo125
5864871fc1 Zabbix inventory improvement (#42669)
* Add validate_certs option to zabbix inventory

* Add validation option

* Fix pep8

* Add changelog
2018-07-14 09:10:16 -04:00
jamessemai
dc32842573 win_security_policy: Allow setting a value to empty (#42051)
* win_security_policy: allow removing values (resolves #40869)

* Removing warning

* Adding test for remove policy setting

* Fixing string comparison

* Make idempotent

* Adding idempotency and diff test

* added changelog fragment
2018-07-13 14:08:14 +10:00
Jordan Borean
e3521776f5
win_chocolatey: add TLSv1.2 support for install phase (#41992) 2018-07-13 13:38:24 +10:00
Jordan Borean
d723b8541d changed winrm _reset to reset and make ssh reset show warning (#42651)
* changed winrm _reset to reset and make ssh reset show warning

* minor changelog update
2018-07-11 20:22:01 -07:00
Matt Davis
a5fc9a17f0
return wu result from inner job (#42647)
fixes #42423
2018-07-11 18:01:42 -07:00
Brian Coca
e115e6496f
preserve delegation info on no_log (#42577)
* preserve delegation info on no_log

fixes #42344
2018-07-11 20:41:37 -04:00
Jerry Chong
42f44b24c6 Fix NameError in pause module (#42038)
* Fix NameError in pause module

* Add comment and changelog

Co-authored-by: Jerry Chong <jchong@netbase.com>
2018-07-11 11:49:32 -04:00
Jordan Borean
940d4a0e89
win_reboot: fix 2.6 issues and better handle post reboot reboot (#42330)
* win_reboot: fix 2.6 issues and better handle post reboot reboot

* changed winrm _reset to reset

* Add handler to reset calls when .reset() throws an AnsibleError on older hosts

* Moving back to _reset to get the issue fixed
2018-07-11 09:12:29 +10:00
Andreas Calminder
577e66660d Simple file locking feature (#42024)
* class for file locking feature
2018-07-10 14:13:27 -07:00
Brian Coca
937e710485
ensure 'text' source assumptions (#42522)
* ensure 'text' source assumptions
2018-07-10 09:45:37 -04:00
Brian Coca
1c08eb8b27
fix irc module to work with py3 (#42267)
* fix irc module to work with py3

fixes #42256
2018-07-10 09:42:14 -04:00
Toshio Kuratomi
673c55f2ef Separate some 255 exit codes that are not ssh errors
The ssh connection plugin is overzealous in thinking that error code 255
can only come from ssh.  Python can return 255 in some circumstances and
error from php does as well.
2018-07-09 15:51:20 -07:00
Toshio Kuratomi
9350a81ae4 Port modules away from __file__
* __file__ won't work if we want to invoke modules via -m or if we
  figure out how to keep modules from hitting the disk with pipelining.
* module.tmpdir is the new way to place a file where it will be cleaned
  automatically.

Change format string to not depend on __file__:

* cloud/amazon/ec2_elb_lb.py
* cloud/amazon/elb_classic_lb.py

Use module.tempdir:

* packaging/os/apt.py
* files/unarchive.py
2018-07-09 15:51:20 -07:00
Matt Martz
abb05c98f3 Make sure we are comparing bytes extensions in inventory plugins (#42475)
* Ensure we are comparing text paths with extensions. Fixes #42118

* Add changelog
2018-07-09 12:24:51 -04:00
Sam Doran
1d1595b990
Fix pause module so it does not stack trace when redirecting stdout. (#42217)
* Use separate variables for stdin and stdout file descriptors

* Do not set stdout to raw mode when output is not a TTY
2018-07-06 17:19:55 -04:00
Jill R
8606fb33f0 Add additional puppet options (#42218)
* Add additional puppet options

Add support for puppet options --debug, --verbose, --summary,
and extend logdest to support logging to stdout and syslog at
the same time.

Fixes issue: #37986

* Fix docs

* Doc fix, add release note

* Fix silly yaml error

* Correct changelog, add C() to params in doc
2018-07-06 13:52:17 -04:00
Jordan Borean
8bdd04c147 Fix remote_tmp when become with non admin user (#42396)
* Fix tmpdir on non root become

 - also avoid exception if tmpdir and remote_tmp are None
 - give 'None' on deescalation so tempfile will fallback to it's default behaviour
   and use system dirs
 - fix issue with bad tempdir (not existing/not createable/not writeable)
   i.e nobody and ~/.ansible/tmp
 - added tests for blockfile case

* Revert "Temporarily revert c119d54"

This reverts commit 5c614a59a6.

* changes based on PR feedback and changelog fragment

* changes based on the review

* Fix tmpdir when makedirs failed so we just use the system tmp

* Let missing remote_tmp fail

If remote_tmp is missing then there's something more basic wrong in the
communication from the controller to the module-side.  It's better to
be alerted in this case than to silently ignore it.

jborean and I have independently checked what happens if the user sets
ansible_remote_tmp to empty string and !!null and both cases work fine.
(null is turned into a default value controller-side.  empty string
triggers the warning because it is probably not a directory that the
become user is able to use).
2018-07-06 10:49:19 -07:00
Toshio Kuratomi
ae55e9f5d6 fix changelog (#42272) 2018-07-03 14:31:19 -04:00
Abhijeet Kasurde
087fc0c20c Restore BOOLEANS import in basic.py (#42008)
This import was removed by mistake. This is required for backward
compatibility.

Fixes: #41988

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2018-07-02 10:27:16 -04:00
Sam Doran
fb55038d75 Add warning when using an empty regexp in lineinfile (#42013)
* Revert "Account for empty string regexp in lineinfile (#41451)"

This reverts commit 4b5b4a760c.

* Use context managers for interacting with files

* Store line and regexp parameters in a variable

* Add warning when regexp is an empty string

* Remove '=' from error messages

* Update warning message and add changelog

* Add tests

* Improve warning message

Offer an equivalent regexp that won't trigger the warning.
Update tests to match new warning.

* Add porting guide entry for lineinfile change
2018-06-29 17:15:43 -07:00
Brian Coca
b6f2aad600 ignore ansible.cfg in world writable cwd (#42070)
* ignore ansible.cfg in world writable cwd
 * also added 'warnings' to config
 * updated man page template
2018-06-29 16:46:10 -07:00
Brian Coca
de0e11c0d5 avoid loading vars on unspecified basedir (cwd) (#42067)
* avoid loading vars on unspecified basedir (cwd)
2018-06-29 16:45:38 -07:00
Olli-Antti Kivilahti
8eacbd0381 elasticsearch_plugin - Show STDERR on module failures. (#41954)
* elasticsearch_plugin - Show STDERR on module failures.

I tried to install a ES plugin without
  become: yes
and found after debugging the module that the module failed ude to permission issues.

The only error message I got was
  Is analysis-icu a valid plugin name?

That was strange considering I followed the example documentation by the letter.

I found out that when this module fails, it hides the real reason for failure.

This patch replaces the generic error with more meaningful diagnostics.

* elasticsearch_plugin - Show STDERR on module failures. Changelog fragment

 samdoran commented 2 days ago

This looks good. Please create a changelog fragment to go along with this change. See fragments for examples.
2018-06-29 17:28:17 -04:00
Sloane Hertel
22a6927dbd Fix file module with check_mode - Fixes #42111 (#42115)
* Fix file module check_mode
2018-06-29 11:19:34 -07:00
Lukas Beumer
26abdbf0fd Add the possiblity to force a plugin installation (#41688) 2018-06-27 12:36:51 -04:00
Jordan Borean
77526a5036
win_domain: fix typo in cmdlet call (#41993) 2018-06-27 11:29:45 +10:00
Jordan Borean
2af36412f9
runas + async - get working on older hosts (#41772)
* runas + async - get working on older hosts

* fixed up sanity issues

* Moved first task to end of test for CI race issues

* Minor change to async test to be more stable, change to runas become to not touch the disk

* moved async test back to normal spot
2018-06-26 14:40:49 +10:00
Jordan Borean
abfcc35e6f
win_iis_webapppool: do not output some cmdlet outputs (#41884) 2018-06-26 11:19:09 +10:00
Jordan Borean
9b7b564d75
Stop displaying kinit pass input on a failure (#41882)
* Stop displaying kinit pass input on a failure

* Fixed up minor logic info and added tests
2018-06-26 05:49:19 +10:00
Andrew Gaffney
9c5d40ff15
Merge various stdout callback plugins into 'default' (#41058)
This allows mixing and matching of stdout callback features
2018-06-24 23:00:07 -05:00
Jordan Borean
2e46688bca
win_chocolatey: always return the rc return value (#41883) 2018-06-25 08:23:31 +10:00
Olivier Bourdon
06b73ff8f1 Add random_mac string filter (#39775)
Add new filter to generate random MAC addresses from
string prefix. See docs/docsite/rst/user_guide/playbooks_filters.rst
for more detailed infos.
2018-06-21 15:56:54 -05:00
Toshio Kuratomi
3ea936312a Remove an unnecessary import from the ansiballz wrapper (#41747) 2018-06-21 12:04:55 +10:00
Jordan Borean
11bd3fd318 win_updates fix when win_updates is run with async (#41756) 2018-06-20 14:52:24 -07:00
Toshio Kuratomi
8bd245a6a2 Fix file state=touch not returning diff information
Fixes #41755
2018-06-20 14:51:27 -07:00
Toshio Kuratomi
ad8142fc6d Add changelog for gce_net sort fix
https://github.com/ansible/ansible/pull/41567
2018-06-16 13:10:23 -07:00