Commit graph

7976 commits

Author SHA1 Message Date
Andrew Haines
35e198a616 Default include_role results to empty list in linear strategy plugin
Fixes #18544.

When a loop is over an empty list, the result is set to

    {'skipped_reason': u'No items in the list', 'skipped': True, 'changed': False}

which means that accessing `hr._result['results']` throws a `KeyError`.

(cherry picked from commit 200d6bdb23)
2016-11-18 12:14:02 -05:00
James Cammarata
0871d955fe Reworking iterator logic regarding failed states during always
Previous changes addressed a corner case, which unfortunately introduced
another bug. This patch adds a new flag to the host state (did_rescue) which
is set to true when the rescue portion of a block completes. This flag is
then checked in _check_failed_state() when the fail_state != FAILED_NONE.

This lead to the discovery of another bug - current strategies are not advancing
hosts to ITERATING_COMPLETE after doing a peek at the next task, leaving the
host state in the run_state of the final task. To address this, before gathering
the list of failed hosts in StrategyBase.run(), a final pass through the iterator
for all hosts is done to ensure each host is in its final state. This way, no
strategy derived from StrategyBase has to worry about it and it's handled.

Fixes #17983

(cherry picked from commit ca5b361ad8)
2016-11-18 11:09:36 -06:00
James Cammarata
159399fea4 Return failed instead of raising an error when an include_vars file is missing
Fixes #18508

(cherry picked from commit 937d872f4b)
2016-11-18 09:57:57 -06:00
Adrian Likins
47d5f0f0a8 Fix 'ansible-vault edit' crash on changed nonascii
ansible-vault edit was attempting to decode the file contents
and failing.

Fixes #18428

(cherry picked from commit c09060e8ff)
2016-11-18 10:11:00 -05:00
Brian Coca
bc539adddc added changes from devel needed by systemd fixes 2016-11-18 10:06:12 -05:00
Brian Coca
ecbac4cf73 fix for filter fix
(cherry picked from commit 5d043b65d3)
2016-11-17 13:42:44 -05:00
Brian Coca
1a247de0b6 remove rsync path from returned facts
(cherry picked from commit 7c960d440f)
2016-11-17 13:21:14 -05:00
James Cammarata
30d5d5fa67 Don't fail on missing vars files for delegated hosts
Fixes #17965

(cherry picked from commit f7fe6dc19c)
2016-11-17 11:59:42 -06:00
Brian Coca
6f5ec79e91 corrected service filtered option to singular
(cherry picked from commit afaec3da82)
2016-11-16 10:13:02 -05:00
Brian Coca
42fb088807 always template when called from template (#18490)
* Have template action plugin call do_template

Avoids all the magic done for 'inline templating' for ansible plays.
renamed _do_template to do_template in templar to make externally accessible.
fixes #18192

* added backwards compat as per feedback

(cherry picked from commit bd70397e24)
2016-11-15 15:17:06 -05:00
James Cammarata
7602a2a030 Catch loop eval errors and only raise them again if the task is not skipped
This should help on issues like #16222.

(cherry picked from commit 57cf5e431c)
2016-11-15 10:39:27 -06:00
James Cammarata
413f6ab7f1 Add option to prepend inherited attributes when extending values
Fixes #18483

(cherry picked from commit 435ca620b2)
2016-11-14 16:42:46 -06:00
Brian Coca
832cc5bb5b added alias to argumetns 'args' to blacklist
this should fix https://github.com/ansible/ansible-modules-core/issues/5584

(cherry picked from commit f4391d34e4)
2016-11-14 15:19:21 -05:00
Chris Church
28883975bf Increment changed stat for a failed task if changed. (#18014)
(cherry picked from commit 534bd12ae9)
2016-11-14 01:24:01 -06:00
James Cammarata
c23d99b786 Alternately track listening handlers by uuid if no name is set
Fixes #17846

(cherry picked from commit 4f06a86161)
2016-11-13 15:28:38 -06:00
Matt Davis
fad7f1de7c bump extras submodule ref for win_nssm bugfix 2016-11-11 10:44:20 -08:00
James Cammarata
debfb798dd Don't copy the parent block of TaskIncludes when loading statically
When loading an include statically, we previously were simply doing a
copy() of the TaskInclude object, which recurses up the parents creating
a new lineage of objects. This caused problems when used inside load_list_of_blocks
as the new parent Block of the new TaskInclude was not actually in the list
of blocks being operated on. In most circumstances, this did not cause a
problem as the new parent block was a proper copy, however when used in
combination with PlaybookInclude (which copies conditionals to the list of
blocks loaded) this untracked parent was not being properly updated, leading
to tasks being run improperly.

Fixes #18206

(cherry picked from commit 5b87951d6c)
2016-11-11 08:10:00 -06:00
James Tanner
c920c8bc3b ini lookup: add 'default' to the list of parsed keys
Fixes #18369

(cherry picked from commit 20fb74b1b1)
2016-11-08 11:07:50 -05:00
Brian Coca
9499ed5360 restore play_hosts variables to not show removed
Also adds ansible_play_hosts_all with original list of hosts the play targeted

(cherry picked from commit 5dd195b52f)
2016-11-08 10:51:51 -05:00
James Cammarata
caba50c778 Merge class dict with parent dict when creating meta attributes
In some situations, where the Base class defines an Attribute, the
BaseMeta class doesn't properly see the _get_parent_attribute or
_get_attr_<whatever> methods because of multiple layers of subclasses
(ie. Handler, which subclasses Task). This addresses that by merging
the __dict__ of the parent with the current classes __dict__ meaning
all future iterations see available special methods.

Fixes #18378

(cherry picked from commit 4794b5da45)
2016-11-07 22:36:09 -06:00
Toshio Kuratomi
58d8a0fca0 Text's .translate() is easier to use than bytes
Text strings and byte strings both have a translate method but the byte
string version is harder to use.  It requires a mapping of all 256 bytes
to a translation value.  Text strings only require a mapping from the
characters that are changing to the new string.  Switching to text
strings on both py2 and py3 allows us to state what we're getting rid of
simply without having to rely on the maketrans() helper function.

(cherry picked from commit ee14e0cc2a)
2016-11-07 10:24:51 -08:00
Michael Scherer
aaedf0bd73 Make facts detection work on OpenBSD with Python3
The traceback is the following:

    Traceback (most recent call last):
      File \"/tmp/ansible_8s0bj604/ansible_module_setup.py\", line 134, in <module>
        main()
      File \"/tmp/ansible_8s0bj604/ansible_module_setup.py\", line 126, in main
        data = get_all_facts(module)
      File \"/tmp/ansible_8s0bj604/ansible_modlib.zip/ansible/module_utils/facts.py\", line 3641, in get_all_facts
      File \"/tmp/ansible_8s0bj604/ansible_modlib.zip/ansible/module_utils/facts.py\", line 3584, in ansible_facts
      File \"/tmp/ansible_8s0bj604/ansible_modlib.zip/ansible/module_utils/facts.py\", line 1600, in populate
      File \"/tmp/ansible_8s0bj604/ansible_modlib.zip/ansible/module_utils/facts.py\", line 1649, in get_memory_facts
    TypeError: translate() takes exactly one argument (2 given)

And the swapctl output is this:

    # /sbin/swapctl -sk
    total: 83090 1K-blocks allocated, 0 used, 83090 available

The only use of the code is to remove prefix in case they are present, so just
replacing them with empty space is sufficient.

(cherry picked from commit df145df962)
2016-11-07 09:51:10 -08:00
Adrian Likins
366bfe14c3 Fix bug (#18355) where encrypted inventories fail 18355 (#18373)
* Fix bug (#18355) where encrypted inventories fail

This is first part of fix for #18355
* Make DataLoader._get_file_contents return bytes

The issue #18355 is caused by a change to inventory to
stop using _get_file_contents so that it can handle text
encoding itself to better protect against harmless text
encoding errors in ini files (invalid unicode text in
comment fields).

So this makes _get_file_contents return bytes so it and other
callers can handle the to_text().

The data returned by _get_file_contents() is now a bytes object
instead of a text object. The callers of _get_file_contents() have
been updated to call to_text() themselves on the results.

Previously, the ini parser attempted to work around
ini files that potentially include non-vailid unicode
in comment lines. To do this, it stopped using
DataLoader._get_file_contents() which does the decryption of
files if vault encrypted. It didn't use that because _get_file_contents
previously did to_text() on the read data itself.

_get_file_contents() returns a bytestring now, so ini.py
can call it and still special case ini file comments when
converting to_text(). That also means encrypted inventory files
are decrypted first.

Fixes #18355

(cherry picked from commit dd0189839e)
2016-11-07 10:15:09 -05:00
Dag Wieers
780d2c4bc4 Remove unnecessary warnings (#18121)
(cherry picked from commit cd784cd345)
2016-11-07 14:52:13 +01:00
Brian Coca
1cc26cf7cd resolve inventory path on init
This allows meta refresh_inventory to work with relative paths
Added option to unfrackpath to not resolv symlinks
fixes #16857

(cherry picked from commit 8217c1c39c)
2016-11-04 17:11:44 -04:00
Toshio Kuratomi
de7fbd407a Limit how much of the file we read to test if it's an encrypted vault file
Fixes memory errors reported in #16391

(cherry picked from commit ed134d81f1)
2016-11-04 12:31:28 -07:00
Jasper Lievisse Adriaanse
9d81ad6423 Rework how OpenBSD processor facts are resolved
(cherry picked from commit c17dad0def)
2016-11-04 08:58:58 -07:00
Matt Clay
5bf1269aaf Python 3 fixes for ansible-doc.
(cherry picked from commit 88dbb5a630)
2016-11-03 16:10:08 -07:00
James Cammarata
3f25088bf0 Correctly reassign implicit block parents when an include is involved
(cherry picked from commit 7ff9942ec6)
2016-11-03 15:28:35 -05:00
James Cammarata
09fc911a48 Correctly assign parent when squashing adjacent implict Blocks
Related to: #18315
Related to: #18206
Related to: #17810

(cherry picked from commit d12475c98b)
2016-11-03 15:28:28 -05:00
Robin Roth
182943f3b3 Exclude floppy disks from lsblk call for uuids
Fixes #18326

(cherry picked from commit 6ca6a9a291)
2016-11-03 10:52:28 -07:00
Brian Coca
5c4a4703d9 only validate extensions when using dir loading
fixes #18223

(cherry picked from commit 32a7b4ce71)
2016-11-03 07:31:32 -07:00
Patrick Uiterwijk
06599f49eb Fix adding the same trusted certificates multiple times (#18296)
If there is an intermittent network failure, we might be trying to reach
an URL multiple times. Without this patch, we would be re-adding the same
certificate to the OpenSSL default context multiple times.
Normally, this is no big issue, as OpenSSL will just silently ignore them,
after registering the error in its own error stack.
However, when python-cryptography initializes, it verifies that the current
error stack of the default OpenSSL context is empty, which it no longer is
due to us adding the certificates multiple times.
This results in cryptography throwing an Unknown OpenSSL Error with details:

OpenSSLErrorWithText(code=185057381L, lib=11, func=124, reason=101,
reason_text='error:0B07C065:x509 certificate routines:X509_STORE_add_cert:cert already in hash table'),

Signed-off-by: Patrick Uiterwijk <puiterwijk@redhat.com>
(cherry picked from commit 77af3a68de)
2016-11-02 10:41:14 -07:00
Toshio Kuratomi
6824b1ea1e Update submodule refs 2016-11-02 07:42:07 -07:00
Daniel Menet
00bdada50e fix iteritems for python 3
(cherry picked from commit 19fdb58948)
2016-11-01 12:47:46 -07:00
Adrian Likins
32971e8639 Fix 'vault rekey' with vault secret env var
if ANSIBLE_VAULT_PASSWORD_FILE is set, 'ansible-vault rekey myvault.yml'
will fail to prompt for the new vault password file, and will use
None.

Fix is to split out 'ask_vault_passwords' into 'ask_vault_passwords'
and 'ask_new_vault_passwords' to make the logic simpler. And then
make sure new_vault_pass is always set for 'rekey', and if not, then
call ask_new_vault_passwords() to set it.

ask_vault_passwords() would return values for vault_pass and new
vault_pass, and vault cli previously would not prompt for new_vault_pass
if there was a vault_pass set via a vault password file.

Fixes #18247

(cherry picked from commit 309f54b709)
2016-11-01 13:13:10 -04:00
Toshio Kuratomi
70824e06b5 Update submodule refs 2016-11-01 08:04:24 -07:00
Steve Kuznetsov
e13f3e3c07 Change v2_playbook_on_start logic to positively detect legacy plugins
In order to support legacy plugins, the following two method signatures
are allowed for `CallbackBase.v2_playbook_on_start`:

def v2_playbook_on_start(self):
def v2_playbook_on_start(self, playbook):

Previously, the logic to handle this divergence checked to see if the
callback plugin being called supported an argument named `playbook`
in its `v2_playbook_on_start` method. This was fragile in a few ways:
 - if a plugin author did not use the literal `playbook` to name their
   method argument, their plugin would not be called correctly
 - if a plugin author wrapped their `v2_playbook_on_start` method and
   by doing so changed the argspec to no longer expose an argument
   with that literal name, their plugin would not be called correctly

In order to continue to support both types of callback for backwards
compatibility while making the call more robust for plugin authors,
the logic can be reversed in order to have a positive check for the old
method signature instead of a positive check for the new one.

Signed-off-by: Steve Kuznetsov <skuznets@redhat.com>
(cherry picked from commit 0bc35354ce)
2016-11-01 07:53:04 -07:00
jasdeep-hundal
6adbc7d64a Fix OpenSSH-related ssh process exit race
Mitigate the effects of observing the ssh process still running
after seeing an EOF on stdout when using OpenSSH with
ControlPersist, since it does not close the stderr file descriptor
in this case.

(cherry picked from commit 679da00236)
2016-11-01 07:53:04 -07:00
Matt Robinson
325bf617e9 Set ansible_os_family correctly under KDE neon
As neon is derived from Ubuntu, ansible_os_family should have the value
"Debian" instead of "Neon".  Add a test case for KDE neon and set
os_family correctly for it.

(cherry picked from commit 4ff8890ec1)
2016-11-01 07:53:04 -07:00
Michael Riss
2c572ba786 Improved caching for urls
- When there is no file at the destination yet, we have no modification time for the `If-Modified-Since`-Header. In this case trust the cache to make the right decision to either serve a cached version or to refresh from origin. This should help with mass-deployment scenarios where you want to use a local cache to relieve your uplink.
- If you don't trust the cache to make the right decision you can still force it to refresh by providing the `force: yes` option.

(cherry picked from commit c05bad9f74)
2016-11-01 07:53:04 -07:00
James Cammarata
cdec853e37 New release v2.2.0.0-1 2016-10-31 22:20:38 -05:00
Nathaniel Case
1f80e35312 Exception.message gone in 3.x (#18221)
* Exception.message gone in 3.x
(cherry picked from commit 4a067c3f50)
2016-10-31 11:17:06 -07:00
Toshio Kuratomi
c07f6d1bdd Update submodule refs 2016-10-31 11:11:07 -07:00
Sijis Aviles
219a20277f Simplify surrogate check in to_text() (#18211)
* Simplify surrogate check in to_text()

* Simplify surrogateescape check even further

(cherry picked from commit b365f44fa1)
2016-10-29 09:12:39 -07:00
Michael Scherer
3de9d8373b Cleanup StringIO import for module_utils/shell.py
(cherry picked from commit 6052c1294b)
2016-10-28 08:01:42 -07:00
James Cammarata
9d4ce0a94e New release v2.2.0.0-0.4.rc4 2016-10-27 13:41:06 -05:00
Toshio Kuratomi
c5d4134f37 Add hint that python3 might be too old
This limitation of python-3.4 mkstemp() is the final reason we made
python-3.5 our minimum version.  Since we know about it, give a nice
error to the user with a hint that Python3.4 could be the issue.

Fixes #18160

(cherry picked from commit fda933723c)
2016-10-27 07:45:34 -07:00
jctanner
95a8bbdbda iterate through task results only if the key is not at the root level (#18214)
Fixes https://github.com/ansible/ansible-modules-core/issues/5396
(cherry picked from commit 5a0621db55)
2016-10-27 09:46:06 -04:00
Rene Moser
1ebc94f290 tasks_queue_manager: fix fork calculation if serial in %
(cherry picked from commit 3763283d01)
2016-10-26 15:27:05 -04:00