Commit graph

371 commits

Author SHA1 Message Date
James Cammarata
d2b3b2c03e Performance improvements 2016-08-08 15:58:46 -05:00
Brian Coca
e2f17f8d9b set cwd to task's basedir (#16805)
* switch cwd to basedir of task

This restores previous behaviour in pre 2.0 and allows for 'local type' plugins
and actions to have a more predictable relative path.

fixes #14489

* removed FIXME since prev commit 'fixes' this

* fix tests, now they need a loader (thanks jimi!)
2016-07-25 08:11:45 -04:00
jctanner
238c6461f6 Add a function to check for killed processes in all strategies (#16684)
* Add a function to check for killed processes so that if any
threads are sigkilled or sigtermed, the entire playbook execution is aborted.
2016-07-14 16:37:35 -04:00
James Cammarata
221520cbad Fixing type in 293723f (mock_handler -> mock_handler_task) 2016-07-12 17:00:29 -05:00
James Cammarata
0ea2b2a6af Fix unit test for base strategy in regards to handler changes 2016-07-12 16:50:10 -05:00
nitzmahone
de549ad675 update action unit test for powershell shebang behavior 2016-07-11 12:38:18 -07:00
jctanner
4ba60d00c8 Refactor the parameter splitting in ini lookup to handle more path formats (#16477)
Refactor the parameter splitting in ini lookup to handle more path formats.

Fixes #16468
2016-06-28 13:24:57 -04:00
James Cammarata
ca6ee4c789 FEATURE: handler listeners
Fixes ansible/proposals#8
2016-06-20 09:30:20 -05:00
James Cammarata
8218591fec Track notified handlers by object rather than simply their name
Due to the fact that roles may be instantiated with different sets of
params (multiple inclusions of the same role or via role dependencies),
simply tracking notified handlers by name does not work. This patch
changes the way we track handler notifications by using the handler
object itself instead of just the name, allowing for multiple internal
instances. Normally this would be bad, but we also modify the way we
search for handlers by first looking at the notifying tasks dependency
chain (ensuring that roles find their own handlers first) and then at
the main list of handlers, using the first match it finds.

This patch also modifies the way we setup the internal list of handlers,
which should allow us to correctly identify if a notified handler exists
more easily.

Fixes #15084
2016-06-20 09:30:20 -05:00
jctanner
587d3c368b Fix synchronize+vagrant dest assertions that were incorrect. (#16291)
Addresses #16284
2016-06-14 19:56:50 -04:00
Brian Coca
a529a60478 raw should not use default executable (#16085)
also removed unused cruft in script
2016-06-10 11:37:58 -04:00
James Cammarata
fbec2d9692 Expand return code values returned by TQM and strategies
This allows the PlaybookExecutor to receive more information regarding
what happened internal to the TaskQueueManager and strategy, to determine
things like whether or not the play iteration should stop.

Fixes #15523
2016-06-08 10:21:57 -05:00
Toshio Kuratomi
5a3493be5f Port urls.py to python3 and other byte vs text fixes (#16124)
* Port urls.py to python3

Fixes (largely normalizing byte vs text strings) for python3

* Rework what we do with attributes that aren't set already.

* Comments
2016-06-04 16:19:57 -07:00
Matt Clay
7a533406f9 Merge pull request #13145 from Mic92/devel
lxc connection plugin
2016-06-03 10:09:04 -07:00
Saran Ahluwalia
fda927a14b added failing test for testing if executable is not empty string 2016-06-02 16:04:15 -07:00
jctanner
c52cea8db3 Add a framework, fixtures and test for common synchronize scenarios. (#15983)
* Add a framework, fixtures and test for common synchronize scenarios.

Addresses #15905
2016-05-31 22:35:59 -04:00
Jörg Thalheim
88482234e6
lxc connection plugin 2016-05-28 00:15:09 +02:00
Toshio Kuratomi
b41b6b2ec5 Update test off make_become for ssh pty race mitigation (#15931) 2016-05-19 15:17:28 -07:00
Matt Martz
0faddfa168 Move binary module detection into executor/module_common.py 2016-05-12 12:25:09 -05:00
Matt Martz
3466e73c50 Resolve test failures 2016-05-12 12:25:09 -05:00
Toshio Kuratomi
8a184381af Strip junk after JSON return. (#15822)
Fixes #15601
2016-05-11 17:54:01 -07:00
Toshio Kuratomi
487e6562ca Fix ziploader for the cornercase of ansible invoking ansible.
* Make ziploader's ansible and ansible.module_utils libraries into
  namespace packages.
* Move __version__ and __author__ from ansible/__init__ to
  ansible/release.py.  This is because namespace packages only load one
  __init__.py.  If that is not the __init__.py with the author and
  version info then those won't be available.
* In ziplaoder, move the version ito ANSIBLE_CONSTANTS.
* Change PluginLoader to properly construct the path to the plugins even
  when namespace packages are present.
2016-04-29 08:47:49 -07:00
Toshio Kuratomi
dcc5dfdf81 Controller-side module caching.
This makes our recursive, ast.parse performance measures as fast as
pre-ziploader baseline.

Since this unittest isn't testing that the returned module data is
correct we don't need to worry about os.rename not having any module
data.  Should devise a separate test for the module and caching code
2016-04-12 08:01:07 -07:00
Toshio Kuratomi
4b0aa1214c Ziploader
* Ziploader proof of concept (jimi-c)

* Cleanups to proof of concept ziploader branch:

* python3 compatible base64 encoding
* zipfile compression (still need to enable toggling this off for
  systems without zlib support in python)
* Allow non-wildcard imports (still need to make this recusrsive so that
  we can have module_utils code that imports other module_utils code.)
* Better tracebacks: module filename is kept and module_utils directory
  is kept so that tracebacks show the real filenames that the errors
  appear in.

* Make sure we import modules that are used into the module_utils files that they are used in.

* Set ansible version in a more pythonic way for ziploader than we were doing in module replacer

* Make it possible to set the module compression as an inventory var

This may be necessary on systems where python has been compiled without
zlib compression.

* Refactoring of module_common code:

* module replacer only replaces values that make sense for that type of
  file (example: don't attempt to replace python imports if we're in
  a powershell module).
* Implement configurable shebang support for ziploader wrapper
* Implement client-side constants (for SELINUX_SPECIAL_FS and SYSLOG)
  via environment variable.
* Remove strip_comments param as we're never going to use it (ruins line
  numbering)

* Don't repeat ourselves about detecting REPLACER

* Add an easy way to debug

* Port test-module to the ziploader-aware modify_module()

* strip comments and blank lines from the wrapper so we send less over the wire.

* Comments cleanup

* Remember to output write the module line itself in powershell modules

* for line in lines strips the newlines so we have to add them back in
2016-04-05 11:06:17 -07:00
jpic
4985411816 Test other use-cases for PluginLoader's config 2016-04-05 14:45:40 +02:00
jpic
da175d0a57 Bugfix: proper plugin configuration was reset
Before this patch, if config was ['/some/path'] then it would enter the
else block and config would be set to [].

The regression this patch fixes was introduced by 700db154.
2016-04-05 14:41:49 +02:00
Toshio Kuratomi
52e9209491 Don't create world-readable module and tempfiles without explicit user permission 2016-03-23 09:52:19 -07:00
Brian Coca
db61e9be0c add ansible_executable inventory var
also handle the overrides appropriately
also new executable to set shell type
2016-03-17 18:54:37 -07:00
Toshio Kuratomi
2b8a0cdddc Fix ssh connection unittests for python3.4 2016-03-09 11:14:02 -08:00
James Cammarata
f0d3284ead Adding more unit tests for ssh connection plugin 2016-03-08 14:50:35 -05:00
James Cammarata
2f472fd4e9 Adding start of ssh connection unit tests 2016-03-08 01:07:16 -05:00
James Cammarata
e011f52557 Expanding unit tests for module_utils/basic.py 2016-03-01 13:53:13 -05:00
James Cammarata
a9d25f455c More unit tests for ActionBase 2016-02-29 12:59:58 -05:00
James Cammarata
e588437067 Adding some unicode params to the ActionBase module formatting test 2016-02-27 10:05:22 -05:00
Toshio Kuratomi
b70bf3b056 Use io.StringIO and io.BytesIO instead of StringIO.StringIO for compat with py3 2016-02-26 16:43:05 -08:00
Toshio Kuratomi
c29f51804b Fix mixing of bytes and str in module replacer (caused traceback on python3) 2016-02-26 16:43:05 -08:00
James Cammarata
01d835700b Cleaning up some py version problems with ActionBase unit tests 2016-02-26 16:22:35 -05:00
James Cammarata
3518a05db6 Starting to expand unit tests for ActionBase plugin class 2016-02-26 12:47:59 -05:00
Brian Coca
6dc910c13a shell + become fixes
1 less level of shell + quoting
simplified become commands, less quote and shell
2016-02-15 13:00:13 -05:00
Brian Coca
c09c01a1f5 go back to defaulting wrapping commands in shell
this was taken out in an effort to default to the user's shell but creates issues as this is not known ahead of time
and its painful to set executable and shell_type for all servers, it should only be needed for those that restrict the user
to specific shells and when /bin/sh is not available. raw and command may still bypass this by explicitly passing None.
fixes #13882

still conditional
2016-01-19 22:09:17 -05:00
Brian Coca
27f4730c29 correctly deals with non serializable type
combine_vars shoudl really be data types, but some just get in
in test, add dict to mock and avoid combine_vars using object
2016-01-18 15:17:43 -05:00
Toshio Kuratomi
1ed3a018eb Revert "Fix make tests-py3 on devel. Fix for https://github.com/ansible/ansible/issues/13638."
This reverts commit e70061334a.

Going to do this in the connection plugins
2016-01-04 19:25:40 -08:00
James Cammarata
2d11cfab92 Squashed commit of the following:
commit 24efa310b58c431b4d888a6315d1285da918f670
Author: James Cammarata <jimi@sngx.net>
Date:   Tue Dec 29 11:23:52 2015 -0500

    Adding an additional test for copy exclusion

    Adds a negative test for the situation when an exclusion doesn't
    exist in the target to be copied.

commit 643ba054877cf042177d65e6e2958178bdd2fe88
Merge: e6ee59f 66a8f7e
Author: James Cammarata <jimi@sngx.net>
Date:   Tue Dec 29 10:59:18 2015 -0500

    Merge branch 'speedup' of https://github.com/chrismeyersfsu/ansible into chrismeyersfsu-speedup

commit 66a8f7e873
Author: Chris Meyers <chris.meyers.fsu@gmail.com>
Date:   Mon Dec 28 09:47:00 2015 -0500

    better api and tests added

    * _copy_results = deepcopy for better performance
    * _copy_results_exclude to deepcopy but exclude certain fields. Pop
    fields that do not need to be deep copied. Re-assign popped fields
    after deep copy so we don't modify the original, to be copied, object.
    * _copy_results_exclude unit tests

commit 93490960ff
Author: Chris Meyers <chris.meyers.fsu@gmail.com>
Date:   Fri Dec 25 23:17:26 2015 -0600

    remove uneeded deepcopy fields
2015-12-29 11:40:18 -05:00
Yannig Perré
b22d998d1d Fix make tests-py3 on devel. Fix for https://github.com/ansible/ansible/issues/13638. 2015-12-22 16:47:27 -05:00
Brian Coca
e957335b0d Merge pull request #13607 from mattclay/test-cache-typo
Fixed import typo for memcache module in tests.
2015-12-21 10:23:38 -05:00
Matt Clay
d2ad17e88f Fixed import typo for memcache module in tests.
The typo caused the test for the memcached cache plugin to be skipped
even when the necessary memcache python module was installed.
2015-12-19 00:08:49 -08:00
James Cammarata
d7f2f606e1 Add has_hostkey to mock objects to fix broken unit tests 2015-12-16 21:49:33 -05:00
James Cammarata
6109f70397 Attempt at fixing strategy unit test failures on py2.6 and py3 2015-12-16 14:02:25 -05:00
Toshio Kuratomi
457f86f61a Minor: Correct type pyhton => python 2015-12-14 08:50:37 -08:00
James Cammarata
279c5a3596 Cleanup strategy tests broken by new forking strategy 2015-12-14 03:07:20 -05:00
Toshio Kuratomi
2e87c1f74e Two fixes to action plugins
* Fix the task_vars parameter to not default to a mutable type (dict)
* Implement invocation in the base class's run() method have each action
  module call the run() method's implemention in the base class.
* Return values from the action plugins' run() method takes the return
  value from the base class run() method into account so that invocation
  makes its way to the output.

Fixes #12869
2015-10-22 16:07:26 -07:00
Toshio Kuratomi
a1428d6bed Remove tmp as a parameter to the connection plugins
There doesn't appear to be anything that actually uses tmp_path in the
connection plugins so we don't need to pass that in to exec_command.
That change also means that we don't need to pass tmp_path around in
many places in the action plugins any more.  there may be more cleanup
that can be done there as well (the action plugin's public run() method
takes tmp as a keyword arg but that may not be necessary).

As a sideeffect of this patch, some potential problems with chmod and
the patch, assemble, copy, and template modules has been fixed (those
modules called _remote_chmod() with the wrong order for their
parameters.  Removing the tmp parameter fixed them.)
2015-09-24 13:33:57 -07:00
Toshio Kuratomi
03127dcfae remove the stdin return value from connection plugin exec_command() methods
The value was useless -- unused by the callers and always hardcoded to
the empty string.
2015-09-24 08:57:19 -07:00
Florian Apolloner
d9f873495e Ported over #7158 to support SELinux context switches. 2015-09-17 15:03:46 +02:00
Florian Apolloner
8548690ca3 Added a test to ensure that sudo is only used if remote and become user difer. 2015-09-17 14:25:52 +02:00
Brian Coca
b6d6c2e4db corrected all missing paths changes 2015-09-15 11:57:54 -04:00
Abhijit Menon-Sen
065bb52109 Be systematic about parsing and validating hostnames and addresses
This adds a parse_address(pattern) utility function that returns
(host,port), and uses it wherever where we accept IPv4 and IPv6
addresses and hostnames (or host patterns): the inventory parser
the the add_host action plugin.

It also introduces a more extensive set of unit tests that supersedes
the old add_host unit tests (which didn't actually test add_host, but
only the parsing function).
2015-09-11 21:47:18 +05:30
James Cammarata
444987cde6 Fix unit test for plugins after basedir/push_basedir removal 2015-09-10 17:01:52 -04:00
Toshio Kuratomi
e2c49b4ef4 Fix problem with "=" in the initial file path. 2015-09-09 11:14:31 -07:00
Marius Gedminas
a32bf1ec18 Simplify FactCache.copy()
Also fix the bug (missing from six import iteritems) I introduced in
823677b490.
2015-09-03 09:44:44 +03:00
Ryan Petrello
1886307845 Fix a parsing bug that prevents IPv6 addresses from being used with add_host
Closes #8682
2015-08-23 22:50:47 +05:30
James Cammarata
8146a193eb Fix strategy base unit tests after 88e1ef8 2015-07-24 10:53:54 -04:00
James Cammarata
e64989beb4 Moving ConnectionInformation -> PlayContext
Also making PlayContext a child class of the Playbook Base class,
which gives it access to all of the FieldAttribute code to ensure
field values are correctly typed after post_validation

Fixes #11381
2015-07-21 12:13:50 -04:00
James Cammarata
fadd8f23f1 Fix broken strategy unit test from earlier change 2015-07-20 15:47:29 -04:00
James Cammarata
3b913943b2 Updating base strategy unit test regarding bad file loads based on earlier change 2015-07-14 11:09:22 -04:00
James Cammarata
bfbb88b4a9 Fix strategy plugin unit tests related to earlier changes 2015-07-07 16:26:24 -04:00
Matt Martz
49d19e82ab Get tests passing
The largest failure in the tests was due to selinux not being installed.
The tests don't require it to be installed, so mock the import.
2015-06-05 16:05:39 -05:00
James Cammarata
0828028c71 Fixing unit test for included file changes 2015-05-29 00:15:14 -05:00
James Cammarata
9921a1d2be Unit tests for base strategy class (v2) 2015-05-21 02:03:59 -05:00
James Cammarata
21fa385ce7 Reorganizing plugin unit tests and adding start of strategy tests (v2) 2015-05-18 17:26:59 -07:00
James Cammarata
ce3ef7f4c1 Making the switch to v2 2015-05-03 21:47:26 -05:00