Commit graph

7047 commits

Author SHA1 Message Date
Toshio Kuratomi
75beca22a5 Be more lenient of symlinked /bin/sh inside the chroot (#16239)
Symlinks inside of the chroot were failng because we weren't able to
    determine if they were pointing to a real file or not.  We could write
    some complicated code to walk the symlink path taking into account where
    the root of the tree is but that could be fragile.  Since this is just
    a sanity check, instead we just assume that the chroot is fine if we
    find that /bin/sh in the chroot is a symlink.  Can revisit if it turns
    out that many chroots have a /bin/sh that's a broken symlink.

    Fixes #16097
2016-06-14 12:05:31 -07:00
nitzmahone
fc8d8b5cde bump core submodule ref for windows setup fix 2016-06-14 11:35:57 -07:00
Brian Coca
97a9eaa299 ignore closing file error when file not open
fixes #16268

(cherry picked from commit 07846f7bec)
2016-06-14 09:34:57 -04:00
Peter Sprygada
6e2651ce07 fixes using ssh keyfile with junos network module
The junos network module will now properly use the ssh key file if its
passed from the playbook to authenticate to the remote device.  Prior
to this commit, the ssh keyfile was ignored.
2016-06-13 22:50:49 -04:00
Toshio Kuratomi
9cbb1a196b Workaround bad interaction with .pth files. (#16238)
When setuptools installs a python module (as is done via python setup.py
install)  It puts the module into a subdirectory of site-packages and
then creates an entry in easy-install.pth to load that directory.  This
makes it difficult for Ansiballz to function correctly as the .pth file
overrides the sys.path that the wrapper constructs.  Using
sitecustomize.py fixes this because sitecustomize overrides the
directories handled in .pth files.

Fixes #16187
2016-06-13 08:00:59 -07:00
nitzmahone
467ef193fd call base _connect() from winrm._connect()
without it, we don't get the base's free become method error check

(cherry picked from commit 445a88d3e8)
2016-06-10 14:41:11 -07:00
Brian Coca
2982b99b8e better handling of retry files
(cherry picked from commit 734bbcb1d3)
2016-06-10 14:03:53 -04:00
nitzmahone
6b286ee0c8 fix for psuedo-connection hostvars not propagating to connection
(mostly done by jimi-c, tested working)

(cherry picked from commit 9f998dbfc49ab5b7a587db6a7099096a414f5841)
2016-06-10 10:17:56 -07:00
Brian Coca
efed4e577c raw should not use default executable (#16085)
also removed unused cruft in script
(cherry picked from commit a529a60478)
2016-06-10 11:41:49 -04:00
J. Javier Maestro
9bb754c0a7 Fix: create retry_files_save_path if it doesn't exist (#15659)
* Fix: create retry_files_save_path if it doesn't exist

Ansible documentation states that retry_files_save_path directory will be
created if it does not already exist. It currently doesn't, so this patch
fixes it :)

* Use makedirs_safe to ensure thread-safe dir creation

@bcoca suggested to use the makedirs_safe helper function :)

(cherry picked from commit 828b73dd2d)
2016-06-10 09:41:30 -04:00
Toshio Kuratomi
d65e2aec81 Update submodule ref 2016-06-09 15:52:23 -07:00
Brian Coca
e971a63c33 avoid processing bad mtab entries (#16206)
fixes #16174
(cherry picked from commit c376954ecf)
2016-06-09 15:14:26 -04:00
Toshio Kuratomi
553ceefe81 Update submodule refs 2016-06-09 08:58:54 -07:00
James Cammarata
c9b125d21a Fix hostvars lookup of locahost based on 5f1bbb4
(cherry picked from commit b51d0dd69b)
2016-06-08 14:15:02 -05:00
James Cammarata
00cf1cbac1 There can be only one localhost
The changes to exclude implicit localhosts from group patterns exposed
the bug that we sometimes create multiple implicit localhosts, which
caused some bugs with things like includes, where the host was used as
an entry into a dict, so having multiple meant that the incorrect host
(with a different uuid) was found and includes were not executed for
implicit localhosts.
2016-06-08 13:01:47 -05:00
James Cammarata
2e003adbc8 Expand return code values returned by TQM and strategies
This allows the PlaybookExecutor to receive more information regarding
what happened internal to the TaskQueueManager and strategy, to determine
things like whether or not the play iteration should stop.

Fixes #15523

(cherry picked from commit fbec2d9692)
2016-06-08 10:44:59 -05:00
Peter Sprygada
d4c78b84f0 fixes issue with ssh keyfile and nxos authentication
The nxos cli provider would not properly handle ssh key files passed
from the playbook task.   The ssh_keyfile argument is now properly
passed to the ssh authentication method

This fix address the bug reported in #3862
2016-06-07 21:28:31 -04:00
nitzmahone
405f636cc7 bump submodule refs 2016-06-07 17:13:53 -07:00
James Cammarata
a2c1247dff Revert "don't tempalte register"
This reverts commit 7ba790bbaf.

Fixes #15700
2016-06-07 13:40:36 -05:00
James Cammarata
3ced6d3e90 Further tweaks to variable precedence to make it match our docs
Also removes looking at role variables from the Block, as those are
merged in separately via VariableManager
2016-06-07 12:09:05 -05:00
James Cammarata
a9c9cd773f Fix variable precedence issue where set facts beat role params
Also updates doc on variable precedence, as it was incorrect for the
order of play vars/vars_prompt/vars_files in relation to set_fact and
registered variables.

Fixes #14702
Fixes #14826
2016-06-07 11:25:26 -05:00
James Cammarata
313d94cc71 Create state in PlayIterator for unknown hosts rather than raise errors
Since we now use the PlayIterator to carry forward failures from previous
play executions, in the event that some hosts which had previously failed
are not in the current inventory we now create a stub state instead of
raising an error.
2016-06-07 09:45:22 -05:00
Krzysztof Magosa
aa66551352 fix: playbook_on_task_start expects name and not task object (#16168)
(cherry picked from commit 721da46842)
2016-06-07 09:43:39 -04:00
Ritesh Khadgaray
66d54dec58 set remote_user to default if none is found when using delegate_to (#16138)
(cherry picked from commit ef9238ab85)
2016-06-07 09:37:14 -04:00
Peter Sprygada
6fc2660194 fixes issues with authenticating using ssh-agent for ios devices
Exception was raised when trying to use ssh-agent for authentication to
ios devices.   This fix enables ssh-agent and enable use of password
protected ssh keys.  There is one additional fix to capture authentication
exceptions nicely.
2016-06-07 06:33:35 -04:00
Michael Schmid
c1fe7aa322 Fixes #15915: New HandlerTaskInclude Class which can run TaskIncludes inside Handlers 2016-06-06 16:02:27 -05:00
jctanner
4cabc94f09 Force relative role paths to be absolute and correct errors about missing path(s). (#16088)
Addresses #10811
2016-06-06 15:46:34 -05:00
James Cammarata
7725c58315 Make sure we add host/group vars files when parsing inventory
Also fixes a bug where add_host was not adding the vars files

Fixes #16117
2016-06-06 15:45:55 -05:00
James Cammarata
9936d7355c Fix recursive call of _load_inventory_file() in VariableManager
Fixes #16128
2016-06-06 14:57:59 -05:00
James Cammarata
0ba9a6a875 Fix the way host and group vars are loaded
* In the VariableManager, we were not properly tracking if a file
  had already been loaded, so we continuously append data to the end
  of the list there for host and group vars, meaning large sets of data
  are duplicated multiple times
* In the inventory, we were merging the host/group vars with the vars
  local to the host needlessly, as the VariableManager already handles that.
  This leads to needless duplication of the data and makes combining the
  vars in VariableManager take even longer.
2016-06-06 14:57:59 -05:00
Tobias Wolf
7287effb5c Solve performance issue with a large number of groups (#13957)
Ansible excessively checks the file system for the potential presence of
`group_vars` and `host_vars` files.

For large numbers of groups this leads to combinatorial performance
issues.

This commit generates a set of group_vars and host_vars filenames using
`os.listdir()` in every possible location and then checks against the sets
before making a stat of the file system.

Also included in this commit is caching of the base directory lookup
for the inventory.
2016-06-06 14:57:59 -05:00
James Cammarata
41dde7259b Mark implicitly hosts as such and exclude them from the all group
Fixes #16059
2016-06-04 18:54:50 -05:00
nitzmahone
31f6e26009 bump extras submodule ref for azure fixes 2016-06-04 16:18:01 -07:00
nitzmahone
6e6dd98b86 update azure minimum doc fragment 2016-06-04 16:06:29 -07:00
nitzmahone
a538b222dc updates for Azure SDK 2.0.0rc4 2016-06-04 15:08:52 -07:00
James Cammarata
55d40b50e4 Removing __version__ from __init__.py since it belongs in release.py 2016-06-04 11:00:08 -05:00
Toshio Kuratomi
6f5d1456bc Fix the copy action to not send the content as a param. (#16127)
Has already been transferred as a tempfile.

This fixes the error in https://github.com/ansible/ansible/issues/16125
but there may be higher level issues that should be fixed as well (other
modules might be able to cause status fields like failed and changed to
return a censored string instead of a bool).  So leaving 16125 open for
now.
2016-06-03 12:34:20 -07:00
nitzmahone
7ddf6dbbd4 bump core submodule ref 2016-06-03 10:48:30 -07:00
nitzmahone
9f32dcdd28 bump core submodule ref 2016-06-03 10:22:38 -07:00
Bede Carroll
1b5ec51988 Add validate_certs to vmware doc fragment (#16096) 2016-06-02 10:22:40 -04:00
Brian Coca
71350e022a fixed confusing reporting when no hosts are matched
it was hitting 'empty inventory' warning.
2016-06-01 08:35:30 -04:00
Brian Coca
ff9641a6a1 fix version/help when missing action
also fixed issues with galaxy cli options
2016-06-01 08:35:30 -04:00
Brian Coca
a59478f4f4 treat roles_path as the list it is for galaxy list 2016-06-01 08:35:30 -04:00
Brian Coca
2452bd2135 moved validate info from vvv to debug 2016-05-27 09:44:42 -04:00
Chris Houseknecht
0de111be00 Fix missing ignore_certs (#16018) 2016-05-27 09:43:41 -04:00
Toshio Kuratomi
2399dd32a8 Update submodule refs -- get them back onto the stable-2.1 branch instead of devel 2016-05-26 16:29:41 -07:00
Toshio Kuratomi
4304574c00 Fix unarchive failures when the destination is a symlink to a directory (#15999)
Also add integration tests for this problem and unicode filenames inside
a tarball.

Fixes #3725
2016-05-26 15:02:10 -07:00
James Cammarata
a44743d142 Use get_dep_chain() instead of directly using a blocks _dep_chain
Child blocks (whether nested or via includes) don't get a copy of the
dependency chain, so the above method should be used to ensure the block
looks at its parents dep chain.

Fixes #15996
2016-05-26 15:48:53 -04:00
Toshio Kuratomi
52c9e2ffcf Add strings 'True' and 'False' as booleans as python bools converted to strings will look that way.
Workaround for custom modules which are using choices=BOOLEANS instead
of type='bool'.
2016-05-25 10:38:35 -07:00
Adrian Likins
b3ca832025 Add ChangeLog for pkg_util bug fix. 2016-05-25 12:27:26 -04:00
James Cammarata
09fa05373b When adding nested blocks, don't pass in any parent includes
We previously changed block behavior to prefer includes over parent
blocks, which broke inheritence in nested blocks.

Fixes #15926
2016-05-25 06:59:37 -07:00
Peter Oliver
faf85ec57c Catch DistributionNotFound when pycrypto is absent (#15731)
* Catch DistributionNotFound when pycrypto is absent

On Solaris 11, module `pkg_resources` throws `DistributionNotFound` on import if `cryptography` is installed but `pycrypto` is not.  This change causes that situation to be handled gracefully.

I'm not using Paramiko or Vault, so I my understanding is that I don't
need `pycrpto`.  I could install `pycrypto` to make the error go away, but:
- The latest released version of `pycrypto` doesn't build cleanly on Solaris (https://github.com/dlitz/pycrypto/issues/184).
- Solaris includes an old version of GMP that triggers warnings every time Ansible runs (https://github.com/ansible/ansible/issues/6941).  I notice that I can silence these warnings with `system_warnings` in `ansible.cfg`, but not installing `pycrypto` seems like a safer solution.

* Ignore only `pkg_resources.DistributionNotFound`, not other exceptions.
2016-05-25 06:59:37 -07:00
Toshio Kuratomi
1867adfabc Update submodule ref to include parameter change to junos module. 2016-05-24 11:48:32 -07:00
James Cammarata
9e622dcf31 New release v2.1.0.0-0.4.rc4 2016-05-22 16:44:41 -04:00
Toshio Kuratomi
3bef107872 Fix the error handling for loop squashing to restore the name argument into the task args (#15949) 2016-05-21 07:34:36 -07:00
Toshio Kuratomi
36f2312071 Update core submodule to pull in docker documentation fix 2016-05-19 14:42:33 -07:00
Matt Clay
b970e2ca80 Add work-around for ssh pty race condition.
This should minimize loss of stdout when using
a pty and connecting with ssh or paramiko_ssh.

(cherry picked from commit bad293ae35)
2016-05-19 10:34:09 -07:00
James Cammarata
036547b4dd New release v2.1.0.0-0.3.rc3 2016-05-18 15:48:24 -04:00
James Cammarata
a6150dbc14 More playbook executor compatibility fixes
With some earlier changes, continuing to forward failed hosts on
to the iterator with each TQM run() call was causing plays with
max_fail_pct set to fail, as hosts which failed in previous plays
were counting those old failures against the % calculation.

Also changed the linear strategy's calculation to use the internal
failed list, rather than the iterator, as this now represents the
hosts failed during the current run only.
2016-05-18 14:09:04 -04:00
Toshio Kuratomi
92e4f25066 Make load_params into a function that custom modules can access (#15913) 2016-05-18 10:55:46 -07:00
machilde
41f45336a9 Fixed issue with parents havings depth of 3+ on add method of NetworkConfig 2016-05-18 09:31:54 -04:00
James Cammarata
710a96956e In TQM run() mark any entries in _failed_hosts as failed in the PlayIterator
As noted in the comment, the TQM may be used for more than one play. As such,
after creating the new PlayIterator object it is necessary to mark any failed
hosts from previous calls to run() as failed in the iterator, so they are
properly skipped during any future calls to run().
2016-05-18 08:17:53 -04:00
James Cammarata
7855612805 Re-remove checking for failed state on hosts when building list of hosts
This was re-added by 63471cd (and modified by me to use iterator again),
it simply needs to be removed.

Fixes #15395
2016-05-18 08:17:53 -04:00
Vincent Roy
b0259a8631 Make sure that serial failures are handled correctly. 2016-05-18 08:17:53 -04:00
Vincent Roy
5e9b13cb94 Don't let max_fail_percentage affect future plays. 2016-05-18 08:17:53 -04:00
Vincent Roy
46e9e4c4da Handle max_fail_percentage per task. 2016-05-18 08:17:53 -04:00
Vincent Roy
9602e43952 Don't stop executing plays after failure.
https://github.com/ansible/ansible/pull/13750/files
2016-05-18 08:17:53 -04:00
Vincent Roy
c901c9144c Backward compatibility execution failures with 1.9 2016-05-18 08:17:53 -04:00
Mike Bryant
76cd7cadfe Allow ssh agent usage for junos_netconf
By default the `Shell` class disables ssh agents. The `junos_netconf`
module uses this class, but doesn't re-enable agents.
Here it's explicitly enabled again, so an ssh agent can be used to
connect to and configure Junos devices.
2016-05-18 07:55:31 -04:00
Kei Nohguchi
94e4e4105d openswitch.py: Fix the OpenSwitch REST authentication
It's a cookie based authentication, that we get it
through /login endpoint, called by connect() method
and save the cookie for the rest of the call.
2016-05-18 07:41:04 -04:00
Peter Sprygada
309aba128c handle name resolution errors more gracefully from shell.py
This change will catch socket.gaierror exceptions from shell.py and
return a more friendly message to the user
2016-05-18 06:55:17 -04:00
Peter Sprygada
2259b90827 bugfix for issue with trying to eval contains with non-string
fixes ansible/ansible-modules-core#3502
2016-05-17 06:46:03 -04:00
Brian Coca
cd2991c02d made format more flexible and allow for non dict entries 2016-05-16 14:40:11 -04:00
Peter Sprygada
d9fa5a3b80 fixes #15496
changed to using OrderedDict to preserve order of lines
2016-05-16 14:05:09 -04:00
Michael Scherer
e0112a3db3 Fix pkgin detection on NetBSD 6 and 7 (#15834)
Since this is now the default package manager, it got moved
to another location on Netbsd :

  netbsd# type pkgin
  pkgin is a tracked alias for /usr/pkg/bin/pkgin
  netbsd# uname -a
  NetBSD netbsd.example.org 6.1.4 NetBSD 6.1.4 (GENERIC) amd64

But since the package manager is also used outside of NetBSD, we
have to keep the /opt/local path too.
2016-05-16 10:14:46 -04:00
Nathaniel Case
8f4243c3ec Check for jxmlease when using netconf on JUNOS. (#15835) 2016-05-16 10:10:26 -04:00
Kei Nohguchi
956829f0f0 net_template.py: Fix jinja2 template file search path (#15134)
The change is needed to support the multiple include statements
inside the jinja2 template file, as in '{% include ['another.j2'] %}'.
statement.  I need this capability, as OpenSwitch `switch` role needs
to handle multiple *.j2 files and supporting the include statement
inside jinja2 file is essential, otherwise I need to combine multiple
template files into a single file, which easily causes conflicts
between developers working on different parts of the teamplate, ports
and interface.
2016-05-16 10:10:04 -04:00
camradal
af5195e336 vCloud module utils error handling bug fixes (#15859)
* Fix AttributeError that hides login errors

* Typo fixes for vca error messages
2016-05-13 23:59:47 -07:00
James Cammarata
0ad0b3d83b New release v2.1.0.0-0.2.rc2 2016-05-13 17:07:03 -04:00
James Cammarata
cc347dcaa9 Adding a deprecation message for accelerated mode 2016-05-13 17:00:06 -04:00
James Cammarata
4c5dddb25a Reworking retry/until logic to fix bugs
Prior to this patch, the retry/until logic would fail any task that
succeeded if it took all of the alloted retries to succeed. This patch
reworks the retry/until logic to make things more simple and clear.

Fixes #15697
2016-05-13 16:09:17 -04:00
jctanner
b181fe0bc2 Disable sftp batch mode if sshpass (#15829)
Make use of the -oBatchMode=no option to force password prompts from sftp

Addresses #13401
2016-05-13 13:52:30 -04:00
Toshio Kuratomi
71c7476199 Update submodule refs 2016-05-13 10:15:52 -07:00
Matt Martz
9121ca2f8e Use .code instead of .getcode() as py24 does not have .getcode(). Fixes https://github.com/ansible/ansible-modules-core/issues/3608 2016-05-13 12:11:15 -05:00
Toshio Kuratomi
cae6240e5e Ship constants to the modules via internal module params rather than a secondary dict. 2016-05-13 10:09:28 -07:00
James Cammarata
2f302e26f4 Make sure setting facts with run_once makes copies of the data
When using run_once, there is only one dict of facts so passing that
to the VariableManager results in the fact cache containing the same
dictionary reference for all hosts in inventory. This patch fixes that
by making sure we pass a copy of the facts dict to VariableManager.

Fixes #14279
2016-05-13 10:05:50 -04:00
Matt Martz
1968bc5952 Cascade ssh_*args configurations in synchronize instead of limiting to just ssh_args. See https://github.com/ansible/ansible-modules-core/issues/3370 2016-05-12 19:01:09 -05:00
Matt Martz
047d62cce3 Guard against a shell profile printing extraneous data 2016-05-12 18:33:54 -05:00
James Cammarata
1c2f0ae8f7 Simply being in an always block shouldn't mean a host did not fail
Previously the changed code was necessary, however it is now problematic
as we've started using the is_failed() method in other places in the code.
Additional changes at the strategy layer should make this safe to remove
now.

Fixes #15625
2016-05-12 17:12:07 -04:00
James Cammarata
3a052654f3 Do not include params when getting role vars in certain situations
In VariableManager, we fetch the params specifically in the next step,
so including them in the prior step is unnecessary and could lead to things
being overridden in an improper order.

In Block, we should not be getting the params for the role as they are
included earlier via the VariableManager.

Fixes #14411
2016-05-12 17:12:00 -04:00
Carl
2a512affde Fixes #15745 playbook include: Conditional scoping
Fixes #15745
Applies conditional forwarding to all tasks/roles within the included playbook.
The existing line only applies forwarded conditionals to the main Task block, and misses pre_, post_, and roles.

Typo ::

Made a selection mistake when I copied over the one line change
2016-05-12 15:02:46 -04:00
Toshio Kuratomi
90fb1fb3fa If we can't squash for any reason, then simply do not optimize the items loop.
Also add more squashing testcases

Fixes #15649
2016-05-12 11:11:05 -07:00
Toshio Kuratomi
8cd0c432e7 Fixed importing the libcloud modules to give a nice error rather than a traceback. 2016-05-12 09:14:51 -07:00
Vic Iglesias
151df71464 Update GCE module to use JSON credentials (#13623)
* Update GCE module to use JSON credentials

* Ensure minimum libcloud version when using JSON crednetials for GCE

* Relax langauge around libcloud requirements
2016-05-12 09:14:51 -07:00
James Cammarata
d22898a27c Clear blocked hosts when a role duplicate task is found in free strategy
In the free strategy, we mark a host as blocked when it has work to do
(the PlayIterator returns a task) to prevent multiple tasks from being sent
to the host. However, we check for role duplicates after setting the blocked
flag, but were not clearing that when the task was skipped leading to an
infinite loop. This patch corrects that by clearing the blocked flag when
the task is skipped.

Fixes #15681
2016-05-12 10:42:51 -04:00
Lars Kellogg-Stedman
22b86bbb63 do not erroneously set gathered_facts=True
In `lib/ansible/executor/play_iterator.py`, ansible sets a host's
`_gathered_facts` property to `True` without checking to see if there
are any tasks to be executed.  In the event that the entire play is
skipped, `_gathered_facts` will be `True` even though the `setup`
module was never run.

This patch modifies the logic to only set `_gathered_facts` to `True`
when there are tasks to execute.

Closes #15744.
2016-05-12 10:23:06 -04:00
Andrew Taumoefolau
00b04ab794 Apply inventory host restrictions by host name rather than UUID.
Issue #15633 observes that a meta: inventory_refresh task causes the playbook
to exit. An inventory refresh flushes all caches and rebuilds all host
objects, assigning new UUIDs to each. These new host UUIDs currently fail to
match those on host objects stored for restrictions in the inventory, causing
the playbook to exit for having no hosts to run further tasks against.

This changeset attempts to address this issue by storing host restrictions
by name, and comparing inventory host names against these names when applying
restrictions in get_hosts.
2016-05-12 09:52:36 -04:00
James Cammarata
af257c20da Change error about loop variable in use to a warning 2016-05-11 21:50:51 -04:00
Toshio Kuratomi
8a84ef80e2 Strip junk after JSON return. (#15822)
Fixes #15601
2016-05-11 17:55:20 -07:00
Toshio Kuratomi
e9406bcfd3 Backport some python3 fixes for facts 2016-05-10 21:47:19 -07:00