Commit graph

780 commits

Author SHA1 Message Date
Rick Elrod
2abaf320d7
[ansiballz] ensure that '' is not in sys.path (#69342)
Change:
On OpenBSD when using pipelining, we do not set cwd which results in a
permissions fatal. Ensure that `''` - cwd - is not in `sys.path`.

Test Plan:
Tested against local OpenBSD VM

Tickets:
Fixes #69320

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-06-01 03:43:20 -05:00
Karl Fogel
225ae65b0f
Make Python path warning say what it means to say (#69669) 2020-05-29 13:15:11 -04:00
Felix Fontein
40f21dfd3c
Version source tagging (automatic and manual) for version_added and deprecation versions (#69680)
* Track collection for version_added.
Validate *all* version numbers in validate-modules.
For tagged version numbers (i.e. version_added), consider source collection to chose validation.

* Make tagging/untagging functions more flexible.

* Tag all versions in doc fragments.

* Tag all deprecation versions issued by code.

* Make Display.deprecated() understand tagged versions.

* Extend validation to enforce tagged version numbers.

* Tag versions in tests.

* Lint and fix test.

* Mention collection name in collection loader's deprecation/removal messages.

* Fix error IDs.

* Handle tagged dates in Display.deprecated().

* Also require that removed_at_date and deprecated_aliases.date are tagged.

* Also automatically tag/untag removed_at_date; fix sanity module removal version check.

* Improve error message when invalid version number is used (like '2.14' in collections).
2020-05-28 22:46:16 -07:00
Martin Krizek
061c6c7c6f
Prevent losing error info by including both loop and cond error msgs (#68485)
Fixes #66529
2020-05-28 17:13:39 -04:00
Rick Elrod
8b6c02fc69
Make connection plugins use password from config (#69670)
Change:
Rather than connection plugins directly accessing play_context and pulling
the password from there, have them pull it from the config system, and
have TaskExecutor store it there for now.

Internally, it still routes through play_context for now, but this is
the first step away from that.

Test Plan:
- Local test with `ansible -c ssh`
- grep -R play_context.pass lib/ansible/plugins/connection/
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-27 09:59:55 -05:00
Matt Davis
f7dfa817ae
collection routing (#67684)
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
2020-05-26 09:42:06 -07:00
Rick Elrod
fe9696be52
Fix ansible -K become_pass regression (#69629)
* Fix `ansible -K` become_pass regression

Change:
- This fixes a breaking change introduced in
  2165f9ac40

Test Plan:
- Local VM for now, with plans to add an integration test for -K going
  forward.

Tickets:
Refs #69244
2020-05-22 09:34:26 -04:00
Brian Coca
df4e83deda
added 'task timeout' feature (#69284)
* added 'task timeout' feature


Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-05-19 17:20:08 -04:00
Matt Clay
d0484fbf74 Cleanup for Python 3.9 and pycodestyle compat. 2020-05-14 21:06:16 -07:00
Brian Coca
2165f9ac40
fix delegation vars usage (debug still shows inventory_hostname) (#69244)
* fix delegation vars usage and reporting

 - just pass delegated host vars + task vars to plugins
   and avoid poluting with original host vars
 - updated tests
2020-05-14 16:36:13 -04:00
Felix Fontein
9767d592a8
Add resource_from_fqcr utility function. (#68474) 2020-05-14 13:50:29 -04:00
Brian Coca
0aa76503dc
avoid fatal tb on bad fqcn callback name (#69440) 2020-05-13 12:02:31 -04:00
David Shrewsbury
af44bd4ddd
Fix --start-at-task when skipping tasks with no name (#68951)
Using --start-at-task on a playbook with tasks with no name would fail
if those unnamed tasks were encountered before the targetted start task.
2020-04-21 09:39:17 +02:00
Jordan Borean
702949e64c
windows - fix up env var quoting for unicode single quotes (#68968)
* windows - fix up env var quoting for unicode single quotes

* Add sanity ignore check for smart quotes
2020-04-16 20:24:23 +10:00
Jordan Borean
6db66bcadd
Add relative module_util support for powershell (#68321)
* Add relative module_util support for powershell

* Added ansible-test classification support
2020-04-01 09:01:18 +10:00
Matt Clay
27fc049993
Fix ansible-test code coverage collection. (#68502)
The last task in a play should now properly report code coverage.

This change should also eliminate empty coverage files, as well as incomplete coverage files resulting from early worker termination.
2020-03-27 12:24:08 -07:00
Brian Coca
6452a82452
fix vault temp file handling (#68433)
* fix vault tmpe file handling

 * use local temp dir instead of system temp
 * ensure each worker clears dataloader temp files
 * added test for dangling temp files
 * added notes to data loader

CVE-2020-10685
2020-03-25 15:24:04 -04:00
Matt Martz
26da443fd2 Address compat issue for collection loading on py26 (#68219)
* Address compat issue for collection loading on py26

* Move import_module shim to utils for compat across the codebase

* Enable collection tests on py2.6

* Update changelog fragment

* Simplify code using sys.moduls

* Move compat to module_utils/compat/importlib

* Add back errantly deleted newline

* Remove hack comment

Co-Authored-By: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
2020-03-23 11:14:21 -05:00
Jordan Borean
61f2ae1c26
Fix runas as collection become plugin (#67638) 2020-02-21 19:23:53 +10:00
Jordan Borean
36def8bf03
Use Ansible to load ps module utils in arg spec checker (#67596)
* Use Ansible to load ps module utils in arg spec checker

* Added changelog and note
2020-02-20 14:32:21 +10:00
Nathaniel Case
3dbc03d58a
Try to load action plugin from the same collection as the module (#66701)
* Try to load network action plugin from the same collection as the module

* Alter tests to match

Just make sure the action plugin is as qualified as the module it is paired with
2020-01-31 12:49:37 -05:00
Xyon
5c721e8a47 Add new configuration parameter to make Windows async startup timeout configurable (#66670)
* Add new configuration parameter to make this timeout configurable

* Rename keys to be more correct and reformat exception message for whitespace handling

* Rename config away from default prefix. Add vars element and associated changes to support.

* Update 65001-allow_configuring_async_startup_timeout.yml

Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
2020-01-24 06:04:52 +10:00
Jordan Borean
e459eac565
Use UTF-8 without a BOM for win coverage (#66510) 2020-01-16 09:52:05 +10:00
Matt Martz
f8654de851
Ensure that data within a tuple is marked as unsafe (#65918)
* Use is_sequence, and Mapping throughout, add support for tuples. Fixes #65722

* Address tests

* Remove unused import

* Add changelog

* Add docstring for clarity

* Argh, linting fix

* Not chasing this rabbit

* wrap_var doesn't return a ref to the original item

* no ref tests

* Remove unused import
2020-01-07 08:41:37 -06:00
Rémi REY
7888eafb82 Fix callback set_options issue with collections (#66128)
Fixes: #66127
2020-01-06 11:32:36 -05:00
Ganesh Nalawade
74e9b1e219
Fix network action plugin load in collection (#65849)
* Fix network action plugin load in collection

Fixes https://github.com/ansible/ansible/issues/65071

*  Load network action plugin that matches the module
   prefix name from list of collections.

* Update changelog

* Fix unit test
2019-12-16 19:15:41 +05:30
Matt Martz
03a4edb477 Add method to automatically clean up after an action plugin (#65509)
* Use correct var, move cleanup for async
* Add changelog and tests. Fixes #65393. Fixes #65277.
* Kill off all long running async tasks from listen_ports_facts
* Update task to work with older jinja2
2019-12-06 17:29:26 -05:00
Ganesh Nalawade
ee3f8d28a4
Fix cli context check for network_cli connection (#64697)
* Fix cli context check for network_cli connection

Fixes #64575

*  Check cli context for network_cli connection
   at the start of new task run only.

* Pass task_uuid around to identify start of new task run

* Handle for local connection
2019-11-26 08:40:49 +05:30
Jordan Borean
200fb00310
Windows coverage - output temp files with UTF-8 BOM (#65086)
* Windows coverage - output temp files with UTF-8 BOM
ci_complete ci_coverage

* Remove testing changes to get ready for merge
2019-11-20 11:07:07 +10:00
Matt Martz
b93d92ef9a
Set alter_sys=True instead of False to address backwards incompat (#64670)
* Set alter_sys=True instead of False to address backwards incompat

* ci_complete

* Add integration test

* ci_complete

* sanity

* ci_complete

* Changelog fragment

* Update import test and validate-modules to match
2019-11-11 16:47:35 -06:00
Matt Martz
254788bc22
unsafe wrapping should only happen for with_ loops (#64401)
* unsafe wrapping should only happen for with_ lookups. Fixes #64379. Addresses #64169

* edit porting guide entry

* typo in changelog fragment

* typo

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* punctuation

Co-Authored-By: Sandra McCann <samccann@redhat.com>
2019-11-06 10:31:55 -06:00
Matt Martz
82ee341fe0
Explicitly use multiprocessing fork start method (#63581)
* Explicitly use multiprocessing fork start method

* Remove unused import

* Remove unused import

* Fallback to just multiprocessing on py2
2019-10-16 15:30:50 -05:00
Jordan Borean
d4ec9422a3
Add proper error when trying to run on older PS versions (#62634) 2019-09-24 08:43:14 -04:00
Jordan Borean
bad338aca6
win_exec_wrapper - Be more defensive when trying to get output errors (#62376)
* win_exec_wrapper - Be more defensive when trying to get output errors

* Fix up property search
2019-09-19 06:53:56 +10:00
Matt Martz
5be0668fb0
Ensure we don't erase unsafe context in TaskExecutor.run on bytes (#62287)
* Ensure we don't erase unsafe context in TaskExecutor.run on bytes. Fixes #62237

* Remove unused import

* Add missing import

* use args splatting for to_unsafe_text/bytes

* Add security issue to changelog

* fix yaml linting issue
2019-09-13 13:14:35 -05:00
Nathaniel Case
9d93076a99
Debug location of ansible-connection when found (#61998) 2019-09-09 12:06:54 -04:00
Paul Belanger
f23bb6f7bd Properly load module_prefix for collections in task_executor.py (#60420)
* Properly load module_prefix for collections

Now that we are using collections for tasks, we need to properly split
the name and load the prefix properly.

Signed-off-by: Paul Belanger <pabelanger@redhat.com>

* Add unit tests for task_executor

This commit adds missing unit tests for action handler in test_task_executor.

Signed-off-by: Daniel Mellado <dmellado@redhat.com>
2019-08-29 09:04:26 -07:00
Matt Clay
f510d59943
Support relative imports in AnsiballZ. (#61196) 2019-08-27 18:11:21 -07:00
Jordan Borean
66f52b74b1 Change collection PS util import pattern (#61307)
* Change collection PS util import pattern

* Add changes for py2 compat

* fix up regex and doc errors

* fix up import analysis

* Sanity fix for 2.6 CI workers

* Get collection util path for coverage collection
2019-08-27 16:02:27 -07:00
Jordan Borean
faaa669764
PowerShell - Added coverage collector (#59009)
* Added coverage collection for PowerShell - ci_complete ci_coverage

* uncomment out coverage uploader call

* Generate XML for PowerShell coverage

* Use whitelist to exclude coverage run on non content plugins

* Remove uneeded ignore entry

* Try to reduce diff in cover.py

* Fix up coverage report package - ci_complete ci_coverage
2019-08-28 07:03:23 +10:00
Matt Martz
0ff9978bc5
Ensure has_dead_workers functions (#60653)
* Ensure has_dead_workers functions

* Fix up tests

* Add changelog. Fixes #29124
2019-08-22 14:32:49 -05:00
Matt Davis
bfa004930a
add subdir support to collection loading (#60682)
* add subdir support to collection loading

* collections may now load plugins from subdirs under a plugin type or roles dir, eg `ns.coll.subdir1.subdir2.myrole`->ns.coll's roles/subdir1/subdir2/myrole, `ns.coll.subdir1.mymodule`->ns.coll's plugins/modules/subdir1/mymodule.py
* centralize parsing/validation in AnsibleCollectionRef class
* fix issues loading Jinja2 plugins from multiple sources 
* resolves #59462, #59890,

* sanity test fixes

* string fixes

* add changelog entry
2019-08-21 10:45:04 +01:00
Jordan Borean
dbd082efe4
Ansible.AccessToken - Added shared util for managing a Windows access token (#60302)
* Ansible.AccessToken - Added shared util for managing a Windows access token

* Fix tests when running in CI

* More fixes for older servers

* More fixes for Server 2008
2019-08-09 21:10:44 +10:00
Martin Krizek
164881d871 Remove UnsafeProxy (#59711)
* Remove UnsafeProxy

Move the work from UnsafeProxy to wrap_var and add support for bytes.
Where wrap_var is not needed, use AnsibleUnsafeBytes/AnsibleUnsafeText
directly.

Fixes #59606

* item is not always text

* Address issues from reviews

* ci_complete
2019-08-07 10:39:01 -05:00
Nathaniel Case
80d8058f5c
Change sub_plugin check to completely ignore napalm (#59298)
This also fixes cases where sub_plugin might not be loaded in executor
2019-07-31 08:46:17 -04:00
Martin Krizek
08d7905be2
Remove incorrect usage of UnsafeProxy (#59633)
This removes couple of uses of UnsafeProxy that relies on the
implementation that is no longer present in the codebase.
2019-07-29 09:48:09 +02:00
Martin Krizek
1010363c0b
Preserve original variables when using a loop (#59426)
Fixes #59414
2019-07-24 11:35:14 +02:00
Martin Krizek
a752e2a467
Clear 'connection related' plugin vars for next loop iteration (#59024)
Fixes #58876
2019-07-19 08:39:05 +02:00
Matt Davis
d28f25d118
patch ansible-connection collection plugin loading (#59119) 2019-07-16 13:46:15 -07:00
Martin Krizek
aea5b795c0 Make ansible_index_var accessible as a magic var (#59030) 2019-07-12 08:50:19 -04:00