Commit graph

15923 commits

Author SHA1 Message Date
Brian Coca
23a22397bf remove closing connections after every task, this goes against conneciton caching and was not expected behaviuor nor inhertited from v1 2015-09-04 10:11:25 -04:00
Toshio Kuratomi
a6de4b86a8 Merge pull request #12244 from mgedmin/py3k
Python 3: use six.text_type instead of unicode
2015-09-04 00:16:01 -07:00
Marius Gedminas
0eb0b56722 Replace type() checks with isinstance()
Fixes the bug I introduced in my previous commit (six.string_types is
(basestring,) on Python 2).  Thanks @abadger for noticing!
2015-09-04 09:47:17 +03:00
Marius Gedminas
37be9539ff Python 3: use six.text_type instead of unicode
Replace 'unicode' with six.text_type, everywhere but in module_utils.
2015-09-04 08:40:10 +03:00
James Cammarata
1840906f74 Merge pull request #12236 from amenonsen/ssh-cpdir
Go back to using ~/.ansible/cp as the ControlPath
2015-09-03 21:12:45 -04:00
Toshio Kuratomi
e218bf8da9 Do not escape backslashes when processing a template file.
Fixes #12198
Fixes #12241
2015-09-03 17:45:17 -07:00
Toshio Kuratomi
21ee4b02da Add more details to the changelog -- may move this to a porting guide
later
2015-09-03 15:24:16 -07:00
Abhijit Menon-Sen
7aa6cd3f63 Go back to using ~/.ansible/cp as the ControlPath
This was commented out earlier because of the lack of interprocess
locking and prepare_writeable_dir in v2.

The locking was not needed: it could only protect against other siblings
of this process (since they were all locking a temporary file that was
opened in the parent), and those would be running as the same user and
with the same umask. Also, os.makedirs() tolerates intermediate paths
being created by other processes. For any other kind of error, both
locking and non-locking code paths would fail in the same way.

So all we really need to do is make sure we have write permissions.

(We also move the cp_dir handling code to where we actually set the
ControlPath ourselves; if the user has set it via ssh_*args already,
we don't need to bother.)
2015-09-04 00:35:13 +05:30
James Cammarata
b2cb3153f1 Fix unit tests for PlayIterator changes regarding fact gathering 2015-09-03 14:25:43 -04:00
James Cammarata
4ac2bafc4b Set hosts fact gathering flag based on fact cache entries
Fixes #12213
2015-09-03 14:11:19 -04:00
Toshio Kuratomi
7ece767767 Merge pull request #12234 from ansible/travis-var-precedence
Enable the var precedence test in travis
2015-09-03 09:41:44 -07:00
James Cammarata
2a32384a2c Merge pull request #12227 from arbabnazar/patch-1
Update developing_modules.rst
2015-09-03 11:42:14 -04:00
James Cammarata
6650ba7654 Squashed commit of the following:
commit 9921bb9d20
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Mon Aug 10 20:19:44 2015 +0530

    Document --ssh-extra-args command-line option

commit 8b25595e7b
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Thu Aug 13 13:24:57 2015 +0530

    Don't disable GSSAPI/Pubkey authentication when using --ask-pass

    This commit is based on a bug report and PR by kolbyjack (#6846) which
    was subsequently closed and rebased as #11690. The original problem was:

        «The password on the delegated host is different from the one I
        provided on the command line, so it had to use the pubkey, and the
        main host doesn't have a pubkey on it yet, so it had to use the
        password.»

    (This commit is revised and included here because #11690 would conflict
    with the changes in #11908 otherwise.)

    Closes #11690

commit 119d032389
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Thu Aug 13 11:16:42 2015 +0530

    Be more explicit about why SSH arguments are added

    This adds vvvvv log messages that spell out in detail where each SSH
    command-line argument is obtained from.

    Unfortunately, we can't be sure if, say, self._play_context.remote_user
    is obtained from ANSIBLE_REMOTE_USER in the environment, remote_user in
    ansible.cfg, -u on the command line, or an ansible_ssh_user setting in
    the inventory or on a task or play. In some cases, e.g. timeout, we
    can't even be sure if it was set by the user or just a default.

    Nevertheless, on the theory that at five v's you can use all the hints
    available, I've mentioned the possible sources in the log messages.

    Note that this caveat applies only to the arguments that ssh.py adds by
    itself. In the case of ssh_args and ssh_extra_args, we know where they
    are from, and say so, though we can't say WHERE in the inventory they
    may be set (e.g. in host_vars or group_vars etc.).

commit b605c285ba
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Tue Aug 11 15:19:43 2015 +0530

    Add a FAQ entry about ansible_ssh_extra_args

commit 49f8edd035
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Mon Aug 10 20:48:50 2015 +0530

    Allow ansible_ssh_args to be set as an inventory variable

    Before this change, ssh_args could be set only in the [ssh_connection]
    section of ansible.cfg, and was applied to all hosts. Now it's possible
    to set ansible_ssh_args as an inventory variable (directly, or through
    group_vars or host_vars) to selectively override the global setting.

    Note that the default ControlPath settings are applied only if ssh_args
    is not set, and this is true of ansible_ssh_args as well. So if you want
    to override ssh_args but continue to set ControlPath, you'll need to
    repeat the appropriate options when setting ansible_ssh_args.

    (If you only need to add options to the default ssh_args, you may be
    able to use the ansible_ssh_extra_args inventory variable instead.)

commit 37c1a5b679
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Mon Aug 10 19:42:30 2015 +0530

    Allow overriding ansible_ssh_extra_args on the command-line

    This patch makes it possible to do:

        ansible somehost -m setup \
            --ssh-extra-args '-o ProxyCommand="ssh -W %h:%p -q user@bouncer.example.com"'

    This overrides the inventory setting, if any, of ansible_ssh_extra_args.

    Based on a patch originally by @Richard2ndQuadrant.

commit b023ace8a8
Author: Abhijit Menon-Sen <ams@2ndQuadrant.com>
Date:   Mon Aug 10 19:06:19 2015 +0530

    Add an ansible_ssh_extra_args inventory variable

    This can be used to configure a per-host or per-group ProxyCommand to
    connect to hosts through a jumphost, e.g.:

        inventory:
            [gatewayed]
            foo ansible_ssh_host=192.0.2.1

        group_vars/gatewayed.yml:
            ansible_ssh_extra_args: '-o ProxyCommand="ssh -W %h:%p -q bounceuser@gateway.example.com"'

    Note that this variable is used in addition to any ssh_args configured
    in the [ssh_connection] section of ansible.cfg (so you don't need to
    repeat the ControlPath settings in ansible_ssh_extra_args).
2015-09-03 11:26:56 -04:00
Toshio Kuratomi
4921f52036 Merge pull request #12232 from praveenkumar/doc_fix
Fix typo to avoid traceback during testing
2015-09-03 07:05:46 -07:00
James Cammarata
8774ff5f57 Make sure PlayContext is copied when iterating in a with_ loop 2015-09-03 08:11:30 -04:00
James Cammarata
c16961db60 Fix two PlayIterator bugs
* When iterating over a child state, a failure should be propagated
  up so parent blocks don't continue iterating
* Make sure a child state exists before trying to search it

Fixes #12210
2015-09-03 07:48:42 -04:00
Praveen Kumar
6e1f7cd503 Fix typo to avoid traceback during testing 2015-09-03 16:35:17 +05:30
Toshio Kuratomi
7f0cd14e0f Merge pull request #12230 from mgedmin/py3k
Python 3: avoid iteritems() in a template
2015-09-03 00:14:55 -07:00
Marius Gedminas
38a96d7e8f Python 3: avoid iteritems() in a template
I don't think six.iteritems is available here, but I also don't expect
there to be enough platforms to ever make the speed difference between
.items() and .iteritems() noticeable.
2015-09-03 10:11:32 +03:00
Toshio Kuratomi
364313c01f Simplify and speed up _split_role_params() by moving an invariant outside of a loop. 2015-09-03 00:06:34 -07:00
Toshio Kuratomi
1ae28f606b Refactor to replace filter and lambda with a generator expression 2015-09-03 00:06:34 -07:00
Toshio Kuratomi
dac562a132 Add note about the trailing newlines change 2015-09-03 00:06:34 -07:00
Toshio Kuratomi
9e023fedf2 Merge pull request #12228 from mgedmin/py3k
Simplify FactCache.copy()
2015-09-02 23:55:31 -07:00
Marius Gedminas
a32bf1ec18 Simplify FactCache.copy()
Also fix the bug (missing from six import iteritems) I introduced in
823677b490.
2015-09-03 09:44:44 +03:00
Toshio Kuratomi
ce97874523 Merge pull request #12226 from mgedmin/py3k
Replace .iteritems() with six.iteritems()
2015-09-02 23:40:58 -07:00
Arbab Nazar
d2e547c31f Update developing_modules.rst 2015-09-03 11:27:56 +05:00
Marius Gedminas
823677b490 Replace .iteritems() with six.iteritems()
Replace .iteritems() with six.iteritems() everywhere except in
module_utils (because there's no 'six' on the remote host).  And except
in lib/ansible/galaxy/data/metadata_template.j2, because I'm not sure
six is available there.
2015-09-03 09:23:27 +03:00
James Cammarata
9e1bc434c7 Further cleanup from locking changes 2015-09-03 01:07:29 -04:00
James Cammarata
b93f27e260 Move the lockfile back to tqm to make sure it stays unique 2015-09-03 00:45:42 -04:00
James Cammarata
ba658ff3a9 Merge branch 'amenonsen-connection-locking' into devel 2015-09-03 00:20:33 -04:00
James Cammarata
b9afbf0ee4 Reorganizing the way the connection lockfile is created 2015-09-03 00:18:52 -04:00
James Cammarata
7034bbef30 Merge branch 'connection-locking' of https://github.com/amenonsen/ansible into amenonsen-connection-locking 2015-09-02 23:54:52 -04:00
James Cammarata
3e5b90c6fb Merge pull request #12195 from amenonsen/ssh-lesswork
Make lock_host_keys a real noop, instead of half-a-noop
2015-09-02 23:53:53 -04:00
Abhijit Menon-Sen
9378c8e2da Make the paramiko plugin use locking 2015-09-03 08:43:21 +05:30
Abhijit Menon-Sen
5887e96b27 Introduce a connection locking infrastructure
The lock file is (a temporary file) opened in the parent process, whose
open fd is inherited by the workers after fork, and passed down through
the PlayContext. Connection grows lock/unlock methods which can be used
by individual connection plugins.
2015-09-03 08:43:21 +05:30
Abhijit Menon-Sen
12d8f9132e Make lock_host_keys a real noop, instead of half-a-noop
Right now, we don't do any locking, but we still scan known_hosts files
twice per connection. That's completely unnecessary, and the proposed
solutions to the locking problem wouldn't need known_hosts scanning
anyway, so this code can go away.
2015-09-03 05:52:57 +05:30
Brian Coca
88d3751c28 Merge pull request #12217 from thyming/hacking-remove-pyc
Delete all compiled python files when running hacking/env-setup
2015-09-02 17:13:04 -04:00
Luke Rohde
cba4642d8d Use find ... -delete instead of non-portable globbing 2015-09-02 17:05:16 -04:00
Brian Coca
099d1797f2 added new path config setting handling 2015-09-02 16:14:40 -04:00
Brian Coca
a006e85947 removed printing module args as it violates no_log settings 2015-09-02 16:14:40 -04:00
Toshio Kuratomi
1b64702b9a Fix templar unittests for change in preserve_trailing_newline parameter's default value 2015-09-02 11:58:01 -07:00
Toshio Kuratomi
1998eddf8d Keep newlines by default now.
In v1, a trailing newline was kept if the parameter was passed as key=value.  If
the parameter was passed as yaml dict the trailing newline was
discarded.  Since key-value and yaml dict were unified in v2 we have to
make a choice as to which behaviour we want.  Decided that keeping trailing
newlines by default made the most sense.

Fixes #12200
Fixes #12199
2015-09-02 11:55:45 -07:00
Luke Rohde
647e48f776 Delete all compiled python files when running hacking/env-setup 2015-09-02 14:49:02 -04:00
James Cammarata
45aac6a739 Adding unit tests for safe_eval 2015-09-02 14:12:06 -04:00
Toshio Kuratomi
d0d40dff9e Enable the var precedence test in travis 2015-09-02 11:04:13 -07:00
Toshio Kuratomi
41da8de094 Speedup for counting newlines 2015-09-02 10:59:51 -07:00
James Cammarata
843138e7d9 Merge pull request #12202 from mgedmin/py3k
Use rich comparisons for Attribute and FieldAttribute
2015-09-02 13:57:04 -04:00
James Cammarata
5434ed973b Make sure locals in safe_eval is a plain dict when running eval
Fixes #12206
2015-09-02 13:54:58 -04:00
Toshio Kuratomi
9ecfc30f9a * Mark all strings as unicode -- shouldn't matter for this dataset but
ansible is passing unicode arond internally so we should test the same
  data.
* Add a zero length test for _count_newlines and fix the zero newlines
  test to have no newlines.
2015-09-02 09:43:42 -07:00
Toshio Kuratomi
417bf1c805 Unittest the _count_trailing_newlines function 2015-09-02 08:59:31 -07:00