Commit graph

6831 commits

Author SHA1 Message Date
Toshio Kuratomi
0a31c24938 Merge pull request #15419 from nitzmahone/module_debug_arg_override
add _load_params debug overrides for module args/file passed on cmdline
2016-04-19 07:49:05 -07:00
Toshio Kuratomi
596f6e0894 Update core submodule ref again for another git fix 2016-04-19 07:44:26 -07:00
Toshio Kuratomi
0e63f09d5a Fix for apt module test failures 2016-04-19 07:36:46 -07:00
Brian Coca
33761efd19 Merge pull request #15417 from bcoca/copy_vaulted
Copy vaulted
2016-04-19 10:22:30 -04:00
Toshio Kuratomi
c8cb5e1042 Update submodule refs 2016-04-19 07:01:50 -07:00
Matt Davis
822f904aa6 Merge pull request #14813 from mattclay/lxd-plugin
Add lxd connection plugin.
2016-04-19 02:06:31 -07:00
James Cammarata
0e2f1b423d Merge pull request #15455 from chemikadze/devel
Fix duplicate host with different uuid created after add_host
2016-04-19 02:50:27 -04:00
James Cammarata
6eefc11c39 Make the loop variable (item by default) settable per task
Required for include+with* tasks which may include files that also
have tasks containing a with* loop.

Fixes #12736
2016-04-19 01:42:40 -04:00
Toshio Kuratomi
d2a7be142e A couple fixes for ziploader:
* Move zipcache temp dir creation into the locked section otherwise it
  races with other workers.
* Catch IOError and turn it into an AnsibleError.  IOErrors can hang
  multiprocessng.
2016-04-18 11:51:45 -07:00
nitzmahone
5b336832af add _load_params debug overrides for module args/file passed on cmdline
Updated python module wrapper explode method to drop 'args' file next to module.
Both execute() and excommunicate() debug methods now pass the module args via file to enable debuggers that are picky about stdin.
Updated unit tests to use a context manager for masking/restoring default streams and argv.
2016-04-18 11:06:46 -07:00
James Cammarata
ed35e8bce2 Don't override the http status code when successful in urls.py 2016-04-18 13:20:24 -04:00
Toshio Kuratomi
7833b5bec4 Merge pull request #15420 from robinro/distribution_version_tests
Distribution version tests
2016-04-18 10:05:55 -07:00
Chris Houseknecht
f28a0ca4bb Update Azure doc fragment (#15450)
* Add default attribute to all option doc strings.

* Fix fragment file name.
2016-04-18 11:05:30 -04:00
Brian Coca
621b98668f fixed boto.ec2 import
alternative to #15224 and fixes #11755
2016-04-18 10:34:44 -04:00
Nikolay Sokolov
d8867d7ce3 Fix duplicate host with different uuid created after add_host
If add_host is performed with hostname existing in inventory, but
not yet accessed and put in inventory cache, additional host with
same hostname and different uuid is created, causing patterns to
misbehave.
2016-04-18 00:10:09 -07:00
Ritesh Khadgaray
3608829b1b vmware.py: allow for better search functionality (#15053)
- search entity by path
 - search vm based on folder
 - search for a vm based on given id ( uuid/name/dns_name/ip/inventory_path)
 - search for a cluster by name, in a given datacenter (optionally)
 - search for objects of a given type in a folder

Note: instance uuid is unique to a VM
2016-04-17 19:10:24 +02:00
chouseknecht
ea07afe67b
Add missing AD user params. 2016-04-17 01:43:39 -04:00
Chris Houseknecht
9e6988c7b5 Merge pull request #15359 from chouseknecht/azure_common
Adding common utils module for new Azure modules.
2016-04-16 04:17:56 -04:00
chouseknecht
98d8da1e67 Adding Azure module doc fragments. 2016-04-16 01:19:59 -04:00
chouseknecht
47620737fb Adding common utils module for new Azure modules. 2016-04-16 01:13:19 -04:00
Matt Clay
5fc76df18b Add lxd connection plugin. 2016-04-15 19:10:57 -07:00
Matt Martz
a985bf6a31 Don't pass context to urlopen, instead add it to the handlers. Fixes https://github.com/ansible/ansible-modules-core/issues/3437 2016-04-15 16:10:54 -05:00
Robin Roth
37188ea336 cleanup tests
* use nose test generator
* more comments
* move facts import inside the skipped function, fix python3 warning
2016-04-15 17:14:14 +02:00
Robin Roth
2b104fe6ad fix tests for SLES and CoreOS
* include #15230
2016-04-15 17:13:59 +02:00
Brian Coca
7062e086d4 made paging marker configurable as boto differs 2016-04-15 10:37:32 -04:00
nitzmahone
e75077fb23 bump submodule refs 2016-04-14 15:57:45 -07:00
Peter Sprygada
9111efc975 adds two new regex filters for use in playbooks (#14696)
The first filter is regex_search which adds the ability to do a regex
search on a fact.  The filter supports returning either all capture groups
or a set of capture groups (either by index or named).

Example usage of regex_search filter

debug: msg="{{ out.stdout[0] | regex_search('image version. (?P<test>\d+).(\d+)', '\\g<test>', '\\2' ) }}"

The second filter is regex_findall with adds the capability to do a
regex findall on a fact or variable and return all occurances of a pattern.

debug: msg="{{ out.stdout[0] | regex_findall('vlan (\d+)' }}"

The filter supports two addtional keyword arguments, ignorecase=[true, false]
and multiline=[true, false]
2016-04-14 14:54:03 -04:00
Brian Coca
5940d3d45b fixes to vault/copy
rm _del_ as it might leak memory
renamed to tmp file cleanup
added exception handling when traversing file list, even if one fails try rest
added cleanup to finally to ensure removal in most cases
2016-04-14 14:12:48 -04:00
Cambell
cdf6e3e4bf feature/copy-vault-dataloader: Add method get_real_file(file_path) to dataloader
- get_real_file will decrypt vault encrypted files and return a path to
  a temporary file.

- cleanup_real_file will remove a temporary file created previously with
  get_real_file
2016-04-14 14:12:48 -04:00
Patrick Ogenstad
6c5ea685a2 Allow timeout to be configurable (#14973) 2016-04-14 09:35:07 -04:00
Toshio Kuratomi
78365e206f Merge pull request #15059 from willthames/remove_boto_ec2_requirement
Remove need for unnecessary boto.ec2 import
2016-04-13 19:37:56 -07:00
Toshio Kuratomi
2163a24a12 Fix when debug commands are run on the wrapper and the wrapper as a file in the current directory 2016-04-13 14:20:01 -07:00
Brian Coca
a5a6bcfbbf updated capabilities to new way of module ref 2016-04-13 14:54:34 -04:00
Brian Coca
b63238ff2c removed unused code for signal handling 2016-04-13 14:54:34 -04:00
Michael Scherer
74c83a91ed Add caps facts, fix #10360 (#15292) 2016-04-13 14:13:45 -04:00
Wolfgang Karall
7a9b8e43da get_mount_facts -- find bind mounts and add info to options field (#12036)
* get_mount_facts -- find bind mounts and add info to options field

* get_mount_facts -- only run findmnt if get_bin_path() finds the binary
2016-04-13 14:05:50 -04:00
Toshio Kuratomi
208ad36ce4 Merge pull request #15344 from abadger/ziploader
Ziploader "recursive imports" and caching
2016-04-13 10:27:01 -07:00
Jonathon Klobucar
7bee994e1c Fix for serial when percent amount is less than one host (#15396)
Ansible when there was a percentage that was calculated to be less than
1.0 would run all hosts as the value for a rolling update.

The error is due to the fact that Python will round a
float that is under 1.0 to 0, which will trigger the case of
0 hosts. The 0 host case tells ansible to run all hosts.

The fix will see if the percentage calculation after int
conversion is 0 and will else to 1 host.
2016-04-13 10:49:38 -04:00
Toshio Kuratomi
afc7ec431a Update submodule refs 2016-04-12 13:55:12 -07:00
Toshio Kuratomi
7d174910c3 Make the ziploader wrapper work with PY3 2016-04-12 11:39:22 -07:00
Toshio Kuratomi
0031e08f7d excommunicate fix 2016-04-12 09:12:58 -07:00
Toshio Kuratomi
55bb24fb7d Reenable unicode=>byte conversion for module parameters to fix integration tests 2016-04-12 08:01:07 -07:00
Toshio Kuratomi
75546678d9 Fix unittests 2016-04-12 08:01:07 -07:00
Toshio Kuratomi
452034564c python-2.4's -m is severely limited.
Extract the module and invoke it as a script to work around python-2.4's
lack of features.
2016-04-12 08:01:07 -07:00
Toshio Kuratomi
b571ecdfec Move module arg passing from the environment to stdin (from the wrapper to the module) 2016-04-12 08:01:07 -07:00
Toshio Kuratomi
dcc5dfdf81 Controller-side module caching.
This makes our recursive, ast.parse performance measures as fast as
pre-ziploader baseline.

Since this unittest isn't testing that the returned module data is
correct we don't need to worry about os.rename not having any module
data.  Should devise a separate test for the module and caching code
2016-04-12 08:01:07 -07:00
Toshio Kuratomi
b5717ef696 First whack at a recursive and ast based ziploader. I'm afraid this one may be too slow. 2016-04-12 08:01:07 -07:00
Toshio Kuratomi
b4b058794e Correct the worker.run documentation 2016-04-12 08:01:07 -07:00
Toshio Kuratomi
c929556226 Fix encoding string to show up in wrapper 2016-04-12 08:01:07 -07:00
Toshio Kuratomi
6bb0e35f85 Update submodule refs 2016-04-12 08:00:20 -07:00
Chris Houseknecht
432bfe5a4f Merge pull request #15364 from gregsidelinger/galaxy-ingore-certs-fix
Fix issue with ansible-galaxy and --ignore-certs
2016-04-12 10:32:02 -04:00
James Cammarata
8ef564176b Handle includes which may have been created in the flow of the playbook
Since 2.0 made all includes dynamic, it is now possible to create and
include a file in the course of executing a playbook. However, with the
introduction of implicit static includes this can cause problems if an
include is thought to be static but does not yet exist. For now, we're
handling missing implicit static includes as a potential dynamic include
but also adding a deprecation message to show includes like this will
need to be marked as `static: no` in the future.

Fixes #15342
2016-04-12 09:33:58 -04:00
Nathaniel Case
7290b6282d Update IOS, IOSXR, JUNOS, & OpenSwitch for environment vars. 2016-04-12 08:55:15 -04:00
Nathaniel Case
dbc49ad95b First implementation of 'fallback' parameter
Implemented & documented for EOS & NXOS
2016-04-12 08:55:15 -04:00
James Cammarata
2e55b3567b Also fix intermittent ssh error using pty's for paramiko
Related to #13876
2016-04-12 08:35:48 -04:00
Bjørnar Ness
d868cdc7b8 enable mysql connection via unix socket 2016-04-12 14:16:18 +02:00
Toshio Kuratomi
0ced20951f Use /var/tmp is user set remote_tmp below /var/tmp.
We want them to be able to influence this choice in case they want to
keep files out of /tmp (perhaps to conserve RAM if that is a tmpfs).
2016-04-11 21:03:13 -07:00
James Cammarata
44877b7c7e Don't use -tt for ssh connections when sudoable=False
Due to an apparent race condition while using pty's on a heavily loaded
system, rarely a request to create a temp directory returns an empty
string rather than the newly created path, causing an error. Disabling
forced pty's appears to resolve the issue, so this patch modifies the
mkdtemp remote call not use -tt as we're not escalating privileges and
thus no pty is required.

Fixes #13876
2016-04-11 23:22:13 -04:00
Toshio Kuratomi
6a97d6386f Always use system /tmp if system=True.
Otherwise unexpected errors can happen when the user sets remote_tmp.

Fixes #15372
2016-04-11 20:00:10 -07:00
Toshio Kuratomi
b27c424fa1 Fixes to the documentation build (#15356)
* Could only have one alias before.  Subsequent aliases overrode the
  previous ones.  Now multiple aliases work.
* Fix BLACKLISTED_MODULES.   Previously, modules were listed in the
  generated documentation despite being blacklisted
* Deprecated modules form extras were showing the (E) tag and not the
  (D) tag. Reversed that now (Probably not necessary to also show the
  E tag).
* Sort the deprecated modules alphabetically in the Category docs as
  well as the list of all modules
* Optimization: Previously rendered the modules to rst twice once in all
  group and once in individual categories.  Fixed to only render them
  once.
* Add fireball to blacklist and remove async_status (as people need to
  use that).
2016-04-11 17:11:55 -07:00
nitzmahone
336b1ae84b add win_reboot action
also includes WinRM connection plugin change to support connection reset
2016-04-11 14:48:39 -07:00
Greg Sidelinger
11ffc0bfee Fix issue with ansible-galaxy and --ignore-certs
The --ingore-certs flag was not honored for https://server/role.tar.gz roles in the requirements file.
2016-04-11 01:17:15 -04:00
Toshio Kuratomi
54acdd7ead Remove workaround for fixed bug. (#15340)
* Remove workaround for fixed bug.

The bug where PluginLoader required objects to directly inherit from
base_classes has been fixed.  Remove workaround from this strategy
plugin   Also switched to using super so that we don't have to modify
all of hte code anytime something like that happens.

* These should be to_uniocde because they're being sent to display()
2016-04-10 09:24:54 -07:00
Brian Coca
005dc84aa7 moved last utils.debug to display.debug 2016-04-08 16:00:36 -04:00
Kishin Yagami
e4a6106ea5 Add debug strategy plugin (#15125)
* Add debug strategy plugin

* Fix Python 2-3 compatiblity issue

* Add document for debug strategy
2016-04-08 14:39:08 -04:00
Brian Coca
0eb2844cc6 moved display's debug lock to cli as global 2016-04-08 13:08:24 -04:00
Matt Martz
dd39f57ae7 Merge pull request #15289 from sivel/sni-urllib3-contrib-try2
Optional Use of urllib3 for SNI verification
2016-04-08 11:26:49 -05:00
Brian Coca
a7f2588920 updated submodule ref 2016-04-08 11:44:09 -04:00
peter.jang
65856f2231 added suport rsync protocol (#12197)
* added suport rsync protocol

* use startswith method for safety
2016-04-08 11:42:07 -04:00
Toshio Kuratomi
d29b833cb1 Update extras submodule ref 2016-04-08 08:11:43 -07:00
James Cammarata
e5c4f88b71 Fix free strategy, which was incorrectly using _wait_on_pending_results
Broken in 210cf06d
2016-04-08 06:49:16 -04:00
nitzmahone
eac4cf1177 fix options_vars iteration for py3 2016-04-07 19:16:15 -07:00
nitzmahone
83a17662cc update core submodule ref 2016-04-07 18:28:53 -07:00
Strahinja Kustudić
eed6cf5dad Adds 'ansible_check_mode' boolean magic variable
* Makes it possible to pass any options variable to VariableManager
  by changing `load_options_vars(options)` in `lib/ansible/utils/vars.py`
2016-04-08 00:26:02 +02:00
Brian Coca
1942cd33dc draft add group merge priority and yaml inventory
* now you can specify a yaml invenotry file

* ansible_group_priority will now set this property on groups

* added example yaml inventory

* TODO: make group var merging depend on priority

  groups, child/parent relationships should remain unchanged.
2016-04-07 16:22:36 -04:00
Toshio Kuratomi
03ec71e5af Updae submodule ref 2016-04-07 09:43:15 -07:00
Toshio Kuratomi
19737236ee Add "excommunicate" debug option
Some debuggers are easier to work with when we do everything in a single
process.  This debug option caters to that at the expense of being
different from what Ansible will actually do to invoke a module.

When we document this we should be clear that this shouldn't be used for
general purpose debugging and that some modules may show strange
"errors" when used with this.  Those won't be considered real bugs as
it's not how ansible really invokes the modules.
2016-04-07 08:12:25 -07:00
Matt Davis
957b336cc1 skip fixup_perms for Powershell
action plugins will now skip _fixup_perms for Powershell. We'll have to come up with another way to do this at some point, but it's not necessary yet since we don't support become on Windows. Also added NotImplementedError throws to chmod/chown/set_facl operations on Powershell (instead of returning '') in case anyone tries to use them in the future.



fixes #15312
2016-04-07 07:27:01 -07:00
Brian Coca
5225048df9 capture attribute errors for invalid class plugins
also switched to py3 safe to_str
2016-04-07 10:16:35 -04:00
Toshio Kuratomi
fd9838f1a0 Add BSD license information to the ZIPLOADER_TEMPLATE string since it
may be combined with module code.
2016-04-06 21:37:04 -07:00
Toshio Kuratomi
647123ee18 Ziploader subprocess.Popen
* Run the module as a script from the wrapper instead of executing in the same process.

Fixes cornercases where the module could potentially be executed twice
if we import and then run the main() function without calling sys.exit()
somewhere.

Also fixes problem with concurrent.futures() hanging.  Not sure
precisely how this is being triggered but it is related to invoking the
main() function outside of an if __name__ == '__main__' conditional.

* Fix for python-2.6
2016-04-06 21:25:18 -07:00
Toshio Kuratomi
8d46bcaa65 Update submodule refs for git fix 2016-04-06 14:09:50 -07:00
Toshio Kuratomi
a57a32adcc Revert "Make the 'path' param type also use abspath on the value"
This reverts commit 1ffadbcc80.

Some modules seem to have path listed for things that are "commands"  --
something that may be a path to a command or a bare command that should
be looked up in PATH.  With this change, if they were formerly looked up
inPATH they are now being made into an absolute path in the cwd.
Reverting this until we can think more about whether to do this and
change those modules to not use path for those parameters.
2016-04-06 14:07:48 -07:00
James Cammarata
1ffadbcc80 Make the 'path' param type also use abspath on the value 2016-04-06 15:58:49 -04:00
Nathaniel Case
77e3dc9e6a Fix network modules for ziploader changes 2016-04-06 11:54:47 -07:00
James Cammarata
040893a677 Adding a config option to allow disabling locale settings upon module exec
Fixes #15138
2016-04-06 12:19:22 -04:00
Matt Martz
398218b6ea More intelligent building of the SSLValidationError message based on capabilities 2016-04-06 11:04:04 -05:00
Matt Martz
6e9c09d7f7 Utilize urllib3.contrib.pyopenssl functionality for SNI capability in python versions lacking SNI support
Also add SNI tests, move test_uri to destructive since we are messing with packages for SNI testing
2016-04-06 10:14:13 -05:00
Toshio Kuratomi
edab8d338d Expand tilde after expanding vars 2016-04-05 22:27:07 -07:00
Peter Sprygada
72ce1e8f14 bugfix eos shared module handling config replace
fixes an issue with doing a config replace using eapi where the configuration
wasn't being replaced.
2016-04-05 22:54:23 -04:00
Peter Sprygada
b97c8ce8e7 bugfix netcfg return config as str
fixes issue where netcfg would return out of order configuration
2016-04-05 22:53:44 -04:00
Toshio Kuratomi
4b0aa1214c Ziploader
* Ziploader proof of concept (jimi-c)

* Cleanups to proof of concept ziploader branch:

* python3 compatible base64 encoding
* zipfile compression (still need to enable toggling this off for
  systems without zlib support in python)
* Allow non-wildcard imports (still need to make this recusrsive so that
  we can have module_utils code that imports other module_utils code.)
* Better tracebacks: module filename is kept and module_utils directory
  is kept so that tracebacks show the real filenames that the errors
  appear in.

* Make sure we import modules that are used into the module_utils files that they are used in.

* Set ansible version in a more pythonic way for ziploader than we were doing in module replacer

* Make it possible to set the module compression as an inventory var

This may be necessary on systems where python has been compiled without
zlib compression.

* Refactoring of module_common code:

* module replacer only replaces values that make sense for that type of
  file (example: don't attempt to replace python imports if we're in
  a powershell module).
* Implement configurable shebang support for ziploader wrapper
* Implement client-side constants (for SELINUX_SPECIAL_FS and SYSLOG)
  via environment variable.
* Remove strip_comments param as we're never going to use it (ruins line
  numbering)

* Don't repeat ourselves about detecting REPLACER

* Add an easy way to debug

* Port test-module to the ziploader-aware modify_module()

* strip comments and blank lines from the wrapper so we send less over the wire.

* Comments cleanup

* Remember to output write the module line itself in powershell modules

* for line in lines strips the newlines so we have to add them back in
2016-04-05 11:06:17 -07:00
Matt Davis
6a3670b1f0 Merge pull request #15281 from jpic/fix_plugin_configuration
Regression fix: proper plugin configuration was reset
2016-04-05 10:00:40 -07:00
James Cammarata
0f2b1244d2 Make sure VariableManager has a view of HostVars
Fixes #15261
2016-04-05 11:32:28 -04:00
jpic
da175d0a57 Bugfix: proper plugin configuration was reset
Before this patch, if config was ['/some/path'] then it would enter the
else block and config would be set to [].

The regression this patch fixes was introduced by 700db154.
2016-04-05 14:41:49 +02:00
Peter Sprygada
cca084c89d enhancement to ios shared module connection
this enhancement will cause the module to connect to the remote ios device
the first time a command wants to run instead of building a connection
immediately
2016-04-04 21:53:05 -04:00
Peter Sprygada
c2fac6c808 enhancement to iosxr shared module to lazy connect to device
this update will only connect to the remote device when the first request
is made to run a command instead of building the connection immediately
2016-04-04 21:52:55 -04:00
Toshio Kuratomi
315cfe2f63 Update module refs 2016-04-04 13:23:29 -07:00
Peter Sprygada
39a576697d feature to localize prompt search login to iosxr shared module
this localizes the cli prompt search logic to the iosxr shared module
instead of using the common regexp list in shell.py
2016-04-04 16:07:45 -04:00
Peter Sprygada
73c3f35112 fixes ansible/ansible-modules-core#3304 2016-04-04 16:03:13 -04:00