Commit graph

307 commits

Author SHA1 Message Date
chkp-orso
3a0c6454a9 Enable logging to any domain in the check point machine (#63976)
* enable using any domain in the check point machine

* Update checkpoint.py

* trying to checge `test_chrckpoint` according to `test_ftd` in order to pass the tests

* Update test_checkpoint.py
2019-11-05 17:44:34 +05:30
Pierre Prinetti
2444fae208 Add a named parameter to the filter to_uuid (#64057)
Let the caller choose a namespace for `to_uuid` and document the
behaviour of both the default case, and the new explicit case.

This PR does not change the existing behaviour of the `to_uuid` UUIDv5
filter.
2019-11-01 13:11:34 -04:00
Mark Chappell
ef7d060a3f AWS module_utils: Clear out Sanity Test issues (#63991) 2019-10-28 09:21:40 +00:00
Sam Doran
987265a6ef
Account for empty strings when splitting the host pattern (#62442)
Improve tests
- add more unit test cases
- add specific integration test with more cases

Testing shows no major downside to calling .strip() twice in a comprehension vs. using a regular for loop and only calling .strip() once. Going with the comprehension for ease of maintenance and because comprehensions are optimized in CPython.
2019-09-20 16:03:51 -04:00
Evgeni Golov
8d0c193b25 allow before/after diff to be NoneType (#62582)
when creating or deleting an object (e.g. via an API), before/after can
be `None` (or at least represented as such by the used library). to
avoid modules havig to do

    diff={'before': before or '', 'after': after or ''}

let's just convert `None` to an empty string that can be diffed properly
2019-09-20 15:47:18 -04:00
Trishna Guha
5cd3be9129
gather_facts action plugin: Fix loading network facts modules for smart gathering (#59856)
* fix smart gathering for network_os in gather_facts action plugin

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>

* Fix detection of network_os for smart gathering

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>

* Add unittest

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>

* make pep8 happy

Signed-off-by: Trishna Guha <trishnaguha17@gmail.com>
2019-09-17 18:35:44 +05:30
Nathaniel Case
50e09be14f
Allow the use of _paramiko_conn even if the connection hasn't been started. (#61570)
* Allow the use of _paramiko_conn even if the connection hasn't been started.

I'm not sure what the benefit is of Noneing paramiko_conn on close, but will keep for now

* Fix test

* Try to fix up net_put & net_get

* Add changelog
2019-09-09 16:59:20 -04:00
Vitalii Kostenko
6f7cd8370a Ansible FTD Module improvements and tests update. (#60640)
* Ansible FTD Module improvements and tests update.

* Fix sanity tests

* Fix formatting
2019-09-03 08:38:11 +05:30
Nathaniel Case
41e4ece44b
Defer passlib dependency from plugins/filter/network.py (#59304)
* Defer passlib dependency from plugins/filter/network.py to what utils/encrypt.py provides.

* Revert test changes now that passlib is optional
2019-08-29 09:01:30 -04:00
James Cammarata
51b33b79c0
T woerner max concurrent (#60702)
* play, block, task: New attribute forks

With this it is possible to limit the number of concurrent task runs.
forks can now be used in play, block and task. If forks is set in different
levels in the chain, then the smallest value will be used for the task.

The attribute has been added to the Base class as a list to easily provide
all the values that have been set in the different levels of the chain.

A warning has been added because of the conflict with run_once. forks will
be ignored in this case.

The forks limitation in StrategyBase._queue_task is not used for the free
strategy.

Signed-off-by: Thomas Woerner <twoerner@redhat.com>

* Handle forks in free strategy

The forks attribute for the free strategy is handled in run in the free
StrategyModule. This is dony by counting the amount of tasks where the uuid
is the same as the current task, that should be queued next. If this amount
is bigger or equal to the forks attribute from the chain (task, block,
play), then it will be skipped to the next host. Like it is also done with
blocked_hosts.

Signed-off-by: Thomas Woerner <twoerner@redhat.com>

* Test cases for forks with linear and free strategy

With ansible_python_interpreter defined in inventory file using
ansible_playbook_python.

Signed-off-by: Thomas Woerner <twoerner@redhat.com>

* Changing forks keyword to throttle and adding some more docs
2019-08-28 18:47:39 -05:00
Ruben-Bosch
5549788c8d win_updates: Add flag to only download updates without installing them (#58631)
* win_updates: Add flag to only download updates without installing them

* Fix test

* Fixes ansible-test (pep8)

* Fix integration test

* Fix actual fix.
2019-08-28 09:46:59 +10:00
Nathaniel Case
e8586114ad
Move instantiation of terminal plugin earlier (#61261)
* Move instantiation of terminal plugin earlier

* Move test for invalid network_os to where the failure now lives
2019-08-27 14:33:05 -04:00
Matt Martz
c81a1057e1
Speed up a few more tests by mocking time.sleep (#61186) 2019-08-22 14:33:34 -05:00
Matt Martz
0ff9978bc5
Ensure has_dead_workers functions (#60653)
* Ensure has_dead_workers functions

* Fix up tests

* Add changelog. Fixes #29124
2019-08-22 14:32:49 -05:00
Felix Fontein
cd4736d893 missing_required_lib: add information how to change Python interpreter (#60736)
* Add information how to change Python interpreter used by Ansible.

* Update lib/ansible/module_utils/basic.py

Co-Authored-By: Abhijeet Kasurde <akasurde@redhat.com>

* Make test less dependent on exact message.
2019-08-20 19:44:38 +10:00
Ganesh Nalawade
49736b6b27
Add support to configure network_cli terminal related options (#60086)
* Add support for configurable terminal plugin options

Fixes #59404

*  Add terminal options to support platform specific login menu
*  Add terminal options to support configurable options for stdout
   and stderr regex list

* Fix CI failures

* Fix CI issues

* Fix review comments and add integration test

* Fix sanity test failures

* Fix review comments

* Fix integration test case

* Fix integration test failure

* Add support to configure terminal related options

Fixes https://github.com/ansible/ansible/issues/59404

*  Add network_cli configurable options to support platform specific login menu
*  Add network_cli configurable options to support configurable options for stdout
   and stderr regex list

* Fix review comment

* Fix review comment
2019-08-19 18:56:20 +05:30
Nathaniel Case
7d3c4a8882
Delay persistent connection until needed (#59153)
* Delay calling connect() until absolutely necessary

* Implement transport_test to enable wait_for_connection

* plugin might be connected already for some reason?

* ensure_connect for httpapi

There's some become shenanigans still needing to be ironed out

* Fix tests for network_cli
2019-08-14 16:58:03 -04:00
Andrey Klychkov
21863d48f3 unit tests: remove unused imports (#60462) 2019-08-13 19:21:43 +02:00
Sam Doran
2d266fbc87
Fix unit tests to work with pytest >= 5.0 (#60246)
pytest made a change in the way the message from ExceptionInfo objects was returned.

https://docs.pytest.org/en/latest/changelog.html#pytest-5-0-0-2019-06-28
2019-08-09 14:58:46 -04:00
Andrey Klychkov
64e3cb0fa8 Unit tests: remove unused imports (#59812)
* Unit tests: remove unused imports

* Unit tests: remove unused imports, fix
2019-08-02 17:18:14 +02:00
Andrey Klychkov
32de51adac Unit tests: remove unused imports (#59740) 2019-07-29 22:10:13 +02:00
Sandeep Bandi
1d82d25ea2 Adding Avi ansible lookup module (#58667)
* Adding Avi ansible lookup module

(cherry picked from commit 77b8951f68cbc889e6595b2a359ca27b84a43c0d)

* Added description for examples

* Added debug logs and unit tests

* Fix __builtin__ import and restting super

* Fix pep8 errors

* Updated as per review comments on IP address
2019-07-24 11:35:04 -04:00
Toshio Kuratomi
8a880d6032 Fix the pytest match test for python-2.6 2019-07-23 14:07:37 -07:00
Toshio Kuratomi
67fb3a8215 Fix root filter test
On python-2.6 the error message is different
2019-07-23 09:50:47 -07:00
Matt Martz
284dafe476
Perfy McPerferton (#58400)
* InventoryManager start of perf improvements

* 0 not 1

* More startswith to [0] improvements

* Remove unused var

* The hash doesn't need to be a string, start as a list, make it into a tuple

* set actually appears faster than frozenset, and these don't need to be frozen

* Cache hosts lists, to avoid extra get_hosts calls, pass to get_vars too

* negligible perf improvement, it could help with memory later

* Try the fast way, fallback to the safe way

* Revert to previous logic, linting fix

* Extend pre-caching to free

* Address test failures

* Hosts are strings

* Fix unit test

* host is a string

* update test assumption

* drop SharedPluginLoaderObj, pre-create a set, instead of 2 comparisons in the list comprehension

* Dedupe code

* Change to _hosts and _hosts_all in get_vars

* Add backwards compat for strategies that don't do set host caches

* Add deprecation message to SharedPluginLoaderObj

* Remove unused SharedPluginLoaderObj import

* Update docs/comments

* Remove debugging

* Indicate what patterh_hash is

* That won't work

* Re-fix tests

* Update _set_hosts_cache to accept the play directly, use without refresh in get_hosts_remaining and get_failed_hosts for backwards compat

* Rename variable to avoid confusion

* On add_host only manipulate _hosts_cache_all

* Add warning docs around _hosts and _hosts_all args
2019-07-22 14:25:20 -05:00
Toshio Kuratomi
87601969a3 Correct places where match was intended
pytest.raises has two parameters, message and match.  message is meant
to be the error message that pytest gives when the tested code does not
raise the expected exception.  match is the string that pytest expects
to be a match for the repr of the exception.  Unfortunately, it seems
that message is often mistakenly used where match is meant.  Fix those
cases.

message is also deprecated so removed our usage of it.  Perhaps we
should write a sanity test later that prevents the use of
pytest.raises(message) to avoid this mistake.

seealso: https://docs.pytest.org/en/4.6-maintenance/deprecations.html#message-parameter-of-pytest-raises

Also update the exception message tested for as we're now properly
detecting that the messages have changed.
2019-07-18 17:10:58 -07:00
Toshio Kuratomi
8acf71f78f
Skip tests with unsatisfied deps (#55853)
* Skip gitlab tests if dependencies aren't met

* Skip certain unittests if passlib is not installed

* Fix tests with deps on paramiko to skip if paramiko is not installed

* Use pytest to skip for cloudstack

If either on Python-2.6 or the cs library is not installed we cannot run
this test so skip it
2019-06-28 13:09:36 -07:00
Olivier Scherler
7ed7d374e4 Fix onepassword lookup plugin and onepassword_facts module when a field has no name. (#58308)
* Fix onepassword lookup plugin crashing on fields with no 'name' or 't' property.

* Fix onepassword_facts module crashing on fields with no 'name' or 't' property.

* Add unit test for onepassword lookup plugin failing on entries without a name.

* Add changelog fragment for onepassword lookup plugin and onepassword_facts module fixes on fields without a name.
2019-06-28 12:37:14 -04:00
Toshio Kuratomi
eb7e4591ae Fix ios test for python2 non-ascii paths 2019-06-21 11:32:28 -07:00
Sumit Jaiswal
58d446e61a
PR to fix where ansible_net_model was not being populated (#58159)
* fix bug 57285

Signed-off-by: Sumit Jaiswal <sjaiswal@redhat.com>

* minor fix

Signed-off-by: Sumit Jaiswal <sjaiswal@redhat.com>

* adding TC fix related

Signed-off-by: Sumit Jaiswal <sjaiswal@redhat.com>

* fix shippable error

Signed-off-by: Sumit Jaiswal <sjaiswal@redhat.com>
2019-06-21 16:18:59 +05:30
markafarrell
43872d6b88 Fix netconf guess_network_os to use ssh_config if supplied. (#55199)
* feature: use network_os=auto to trigger network os guessing

* doc: add debug messages

* fix linting

* fix test_netconf_init default network_os is auto

* add documentation

* fix rst errors

* use init lexer

* Update docs/docsite/rst/network/user_guide/platform_netconf_enabled.rst

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* Update docs/docsite/rst/network/user_guide/platform_netconf_enabled.rst

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* Update docs/docsite/rst/network/user_guide/platform_netconf_enabled.rst

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* Update docs/docsite/rst/network/user_guide/platform_netconf_enabled.rst

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* Update docs/docsite/rst/network/user_guide/platform_netconf_enabled.rst

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* Update docs/docsite/rst/network/user_guide/platform_netconf_enabled.rst

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* Update docs/docsite/rst/network/user_guide/platform_netconf_enabled.rst

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* Update lib/ansible/plugins/connection/netconf.py

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* Update lib/ansible/plugins/connection/netconf.py

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* Update lib/ansible/plugins/connection/netconf.py

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* Update docs/docsite/rst/network/user_guide/platform_netconf_enabled.rst

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* Update docs/docsite/rst/network/user_guide/platform_netconf_enabled.rst

Co-Authored-By: Sandra McCann <samccann@redhat.com>

* Update docs/docsite/rst/network/user_guide/platform_netconf_enabled.rst

Co-Authored-By: Sandra McCann <samccann@redhat.com>
2019-06-21 09:04:13 +05:30
Rob
feae35ab7e add IAM role assumption to aws_ec2 inventory (#41637)
* add IAM role assumption to aws_ec2 inventory
* Ensure inventory._options has necessary option keys populated since the plugin docs parser isn't accessible to unit tests yet
2019-06-18 15:34:41 -07:00
bobdoah
33d0d36eaa AWS: Fix KeyError in aws_secret lookup (#54792) 2019-06-13 17:11:44 -07:00
Enno Gröper
1ed7a654ba Improve testing of FactCache (#55739)
* Improve testing of FactCache
2019-06-10 16:26:24 -04:00
Sloane Hertel
598a058afe
[FactCache] define first_order_merge method (#55781)
* Add first_order_merge method

Add test for updating a key that already exists

* changelog
2019-06-10 14:34:06 -04:00
Abhijeet Kasurde
ee8a607cca ipaddr: Handle network address in ipmath (#57149)
* Updated testcase, documentation

Fixes: #54457

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-06-04 14:01:18 +05:30
Martin Krizek
34e9d6781b Templar: encapsulate _available_variables (#55435)
Ensure variables are reset between iterations
2019-05-20 11:49:54 -04:00
Jordan Borean
ebd4462c23
Fix tests for the psrp connection plugin (#55061) 2019-04-10 08:22:48 +10:00
Vitalii Kostenko
2176b53a55 Add latest updates from FTD Ansible downstream repository. (#53638)
* Add latest updates from FTD Ansible downstream repository.
 - add a better implementation of the upsert operation;
 - add API version lookup functionality;
 - add filter which remove duplicated references from the list of references;
 - fix minor bugs.

* fix issues outlined by ansibot

* fix argument name for _check_enum_method
2019-04-01 18:08:01 +05:30
Matt Davis
f86345f777
Collection content loading (#52194)
* basic plugin loading working (with many hacks)

* task collections working

* play/block-level collection module/action working

* implement PEP302 loader

* implicit package support (no need for __init.py__ in collections)
* provides future options for secure loading of content that shouldn't execute inside controller (eg, actively ignore __init__.py on content/module paths)
* provide hook for synthetic collection setup (eg ansible.core pseudo-collection for specifying built-in plugins without legacy path, etc)

* synthetic package support

* ansible.core.plugins mapping works, others don't

* synthetic collections working for modules/actions

* fix direct-load legacy

* change base package name to ansible_collections

* note

* collection role loading

* expand paths from installed content root vars

* feature complete?

* rename ansible.core to ansible.builtin

* and various sanity fixes

* sanity tweaks

* unittest fixes

* less grabby error handler on has_plugin

* probably need to replace with a or harden callers

* fix win_ping test

* disable module test with explicit file extension; might be able to support in some scenarios, but can't see any other tests that verify that behavior...

* fix unicode conversion issues on py2

* attempt to keep things working-ish on py2.6

* python2.6 test fun round 2

* rename dirs/configs to "collections"

* add wrapper dir for content-adjacent

* fix pythoncheck to use localhost

* unicode tweaks, native/bytes string prefixing

* rename COLLECTION_PATHS to COLLECTIONS_PATHS

* switch to pathspec

* path handling cleanup

* change expensive `all` back to or chain

* unused import cleanup

* quotes tweak

* use wrapped iter/len in Jinja proxy

* var name expansion

* comment seemingly overcomplicated playbook_paths resolution

* drop unnecessary conditional nesting

* eliminate extraneous local

* zap superfluous validation function

* use slice for rolespec NS assembly

* misc naming/unicode fixes

* collection callback loader asks if valid FQ name instead of just '.'
* switch collection role resolution behavior to be internally `text` as much as possible

* misc fixmes

* to_native in exception constructor
* (slightly) detangle tuple accumulation mess in module_utils __init__ walker

* more misc fixmes

* tighten up action dispatch, add unqualified action test

* rename Collection mixin to CollectionSearch

* (attempt to) avoid potential confusion/conflict with builtin collections, etc

* stale fixmes

* tighten up pluginloader collections determination

* sanity test fixes

* ditch regex escape

* clarify comment

* update default collections paths config entry

* use PATH format instead of list

* skip integration tests on Python 2.6

ci_complete
2019-03-28 10:41:39 -07:00
Kyryl Galanov
4a5b9bd8eb New lookup module: manifold (#50435)
* New lookup module: manifold

Add Manifold.co integration. The plugin fetches resource credentials
from Manifold service.

* module manifold: fix ansible lint warnings

* module manifold: fix false warning - split test assertion

* manifold module: fix unittest import

* manifold module: fix unittest patch

* manifold module: fix python3 requests getheader error
2019-03-27 07:20:15 -04:00
Alan Rominger
f2f11341a3 Avoid erroring on host group confusion in keyed groups (#54099) 2019-03-25 16:57:46 -04:00
Sloane Hertel
54be769e8d fix AWS plugin credential precedence for environment variables (#52945)
* fix AWS plugin credential precedence for environment variables

* Allow aliases in direct plugins options

Consolidate precedence fix just in the doc fragment using aliases for mismatched options

* Access options with the option name rather than alias

* fix indentation

* update unit tests

* Improve readability
2019-03-25 16:53:11 -04:00
Matt Clay
3ea05f7757 Add pypsrp to unit tests and disable failing test. 2019-03-20 14:10:21 -07:00
Sloane Hertel
87ebc56de6 Allow parent groups to be variables or literal (#53649)
* Allow parent groups to be variables or literal, requires {{ }}
* Check strict before failing on templating errors
* Don't add a group if an invalid parent group was provided
2019-03-14 14:22:18 -04:00
Alan Rominger
8c2037aaa9 do not error in case of empty list constructed group (#53748) 2019-03-14 12:27:56 -04:00
Jordan Borean
8ef2e6da05 Add support for Windows hosts in the SSH connection plugin (#47732)
* Add support for Windows hosts in the SSH connection plugin

* fix Python 2.6 unit test and sanity issues

* fix up connection tests in CI, disable SCP for now

* ensure we don't pollute the existing environment during the test

* Add connection_windows_ssh to classifier

* use test dir for inventory file

* Required powershell as default shell and fix tests

* Remove exlicit become_methods on connection

* clarify console encoding comment

* ignore recent SCP errors in integration tests

* Add cmd shell type and added more tests

* Fix some doc issues

* revises windows faq

* add anchors for windows links

* revises windows setup page

* Update changelogs/fragments/windows-ssh.yaml

Co-Authored-By: jborean93 <jborean93@gmail.com>
2019-03-07 16:38:02 -08:00
Sloane Hertel
9687879840
Fix inventory cache interface (#50446)
* Replace InventoryFileCacheModule with a better developer-interface

Use new interface for inventory plugins with backwards compatibility

Auto-update the backing cache-plugin if the cache has changed after parsing the inventory plugin

* Update CacheModules to use the config system and add a deprecation warning if they are being imported directly rather than using cache_loader

* Fix foreman inventory caching

* Add tests

* Add integration test to check that fact caching works normally with cache plugins using ansible.constants and inventory caching provides a helpful error for non-compatible cache plugins

* Add some developer documentation for inventory and cache plugins

* Add user documentation for inventory caching

* Add deprecation docs

* Apply suggestions from docs review

* Add changelog
2019-03-06 12:12:35 -06:00
Alan Rominger
56e3597856 Add keyed_groups feature (#52045)
This implements:
- Allow creating keyed group parents
2019-03-05 12:34:34 -06:00
Matt Davis
4d3a6123d5
Python interpreter discovery (#50163)
* Python interpreter discovery

* No longer blindly default to only `/usr/bin/python`
* `ansible_python_interpreter` defaults to `auto_legacy`, which will discover the platform Python interpreter on some platforms (but still favor `/usr/bin/python` if present for backward compatibility). Use `auto` to always use the discovered interpreter, append `_silent` to either value to suppress warnings.
* includes new doc utility method `get_versioned_doclink` to generate a major.minor versioned doclink against docs.ansible.com (or some other config-overridden URL)

* docs revisions for python interpreter discovery

(cherry picked from commit 5b53c0012ab7212304c28fdd24cb33fd8ff755c2)

* verify output on some distros, cleanup
2019-02-27 23:52:02 -08:00