Commit graph

410 commits

Author SHA1 Message Date
Andrey Klychkov
333953117c Add human_to_bytes isbits arg unit tests&docstring
PR #58623

Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2019-07-23 14:56:56 +02:00
Andrey Klychkov
59e647910d Add unit tests for bytes_to_human formatter
PR #58664

Co-Authored-By: Sam Doran <sdoran@redhat.com>
2019-07-22 15:45:07 +02:00
Felix Fontein
4a574c4d0c Option parsing: warn if both an option and its alias are specified for a module (#53698)
* Print warning when both an option and its alias is specified.

* Improve output.

* Put warnings into self._warnings directly, resp. use self.warn() when handling subspecs.

* Add changelog.

* Add unit test.
2019-07-19 13:11:41 -04:00
Sam Doran
f231f21669
Handle situation where ansible_architecure may not be defined when gathering facts (#55466) 2019-07-19 11:33:05 -04:00
Andrey Klychkov
88ec5e14cb Add unit tests for lenient_lowercase function (#58586) 2019-07-08 18:33:15 -04:00
Diane Wang
9c140d26d5 VMware: add new module vmware_guest_network (#52075) 2019-07-08 07:54:54 +05:30
Abhijeet Kasurde
96eccda23b
VMware: Add module vmware_vm_storage_policy_info (#56263)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-07-03 16:01:08 +05:30
Brian Coca
ee4cba1b85 ensure bytes in run_command (#58411)
* ensure we pass bytes to run_command

* changed tests for new behaviour

* dont b the pytest

* fixes by sivel
2019-07-02 17:07:48 -05:00
Andrey Klychkov
10af2636d1 Add unit tests for human_to_bytes (#58584) 2019-07-01 13:50:54 -04:00
Sam Doran
ab6a9ef130
Get minor version number for CentOS and Debian (#57814)
* Get the most detailed version number from distro.version() for CentOS and Debian
* Update tests and fixtures
* Update fixture generation script to gather distro info and work with Python 3
* Update LinuxMint fixtures
* Cleanup fixture formatting
* Improvements based on feedback from abadger:
    - use unicode since that is what distro returns
    - use frozenset with a tuple
    - include link Debian to bug
2019-06-28 19:21:35 -04:00
Mariusz Mazur
06292566ec kubevirt: more unit tests (#57739)
* Add tests for KubeAPIVersion

* Legibility improvements for KubevirtVM tests

* Create units.utils.kubevirt with common stuff

* Add some VMIRS unit tests
2019-06-25 20:06:13 +10:00
Andrey Klychkov
64d0559e9f Postgres module_utils: add get_connect_params + unit tests (#58067)
* add get_conn_params

* add get_conn_params: add to the modules
2019-06-19 06:49:19 -07:00
Mariusz Mazur
51add5aa79 kubevirt: enable/update tests + fix merge_dicts() (#57685)
* Actually run the unit tests and separate them into two files

* Re-add recursion to merge_dicts()

* Update tests to work with current code
2019-06-13 11:07:16 +02:00
Anton Nikulin
4fa93d5b9b FTD configuration module: fix a bug with response parsing (#57480)
* Update dependence's name to firepower-kickstart

* Check response type before getting attributes

* Add unit test for construct_ansible_facts method

* Update error message
2019-06-10 18:30:35 +05:30
Sajna Shetty
e25269e1b3 Adding dell ome device_info module (#53438)
* Adding device_facts module for contribution

* changes added for pylint error

* Updated code to solve ansible-test compile error

* Changes to avoide comile error added

* Review Comments changes update

* Avoided blank line

* pylint error changes

* Removed ansible_facts return in error case

* Updated description

* modules renamed

* changing from ansible_facts to device_info

* avoide pep8 error

* Updated sample output

* version changed to 2.9

* Changed Copyright license to BSD

* Changed 3-clause BSD license to 2-clause BSD

* Added unit test support for ome_device_info

* version change

* removed pylint error in unit test modules

* Avoid Sanity error for unit test modules

* updated version
2019-06-10 06:41:48 -04:00
Andrey Klychkov
3e06013eb6 unittests for module_utils.common.removed.removed_module function (#57214) 2019-06-03 11:12:12 -04:00
Matt Martz
8bd4e2a144 cert validation fixes - Attempt 2 (#55953)
* Attempt 2 of cert validation fixes

* Remove unused code

* Cleanup the tmp cert using atexit

* Fix linting issues

* Only add SSLValidationHandler when not HAS_SSLCONTEXT

* Catch value errors on non PEM certs

* Only catch NotImplementedError to avoid masking issues

* set self._context even with PyOpenSSLContext for conformity

* Fix error building

* normalize how we interact with the context we create

* Remove unused code

* Address test for py3.7 message difference

* open_url should pass the ca_path through

* Account for new error in url lookup test

* Guard some code behind whether or not we are validating certs

* Make _make_context public

* Move atexit.register up to where the tmp file is created
2019-05-31 16:35:25 -04:00
Sviatoslav Sydorenko
4c7fff315e
Fix module_utils.common.test_collections testsuite
PR #57181
2019-05-31 13:40:18 +02:00
Andrey Klychkov
bf25889bfd test_postgres: fix typos 2019-05-30 07:18:30 -07:00
Andrey Klychkov
01ed7a489a Add unit tests for lib/ansible/module_utils/common/text/converters.py (#56915)
Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2019-05-30 09:22:53 -04:00
Andrey Klychkov
86354ff1fb postgresql: remove depricated lib (#56641) 2019-05-24 15:08:16 -04:00
Sam Doran
8f4f3750fe
Ensure uri module always returns status even on failure (#56240)
- Also return url and update docs for other values to indicate they are only returned on success.
- Add integration tests
- Use info variable for common return values
- Use -1 as default status rather than None. This is lines up with with existing code in urls.py
- Add unit tests to ensure status and url are returned on failure
2019-05-23 10:17:17 -04:00
Matt Martz
493cf817a9 Don't rely on netloc for determining hostname and port, just use hostname and port (#56270)
* Add changelog fragment
* Fix IPv6 address parsing for py2.6, and add tests
* make sure hostname isn't None
2019-05-20 16:50:54 -04:00
Andrey Klychkov
386cef18ed module_utils.postgres: added unittests (#56381) 2019-05-17 23:33:45 -07:00
Gonéri Le Bouder
cf78759f5b vmware_vm_facts: fix the support with regular ESXi
Ensure the module still work with the ESXi where CustomFieldsManager
does not exist.

From: https://www.vmware.com/support/developer/converter-sdk/conv60_apireference/vim.CustomFieldsManager.html

    The CustomFieldsManager object is used to add and remove custom fields to
    managed entities.
    The custom fields values set on managed entities are available through the
    customValue property and through the summary objects for VirtualMachine
    and HostSystem. They are not available directly through this managed object.
    This functionality is only available through VirtualCenter.

Fixes: #56071
2019-05-13 17:25:16 -04:00
Felix Fontein
bd47e64bc7 Hetzner failover IP: refactoring (#56203)
* Rename helper function.

* Extract hetzner.py module_utils.

* Rewrite docs.

* Add module docs fragment.

* Split up get_failover function.

* Add tests for new function.

* hetzner_pass -> hetzner_password

* Move common argspec to module_utils.
2019-05-10 08:33:51 +02:00
zengchen
d8314e1a45 refactor moudule utils of hwc_utils.py (#55858)
* use navigate_value instead navigate_hash

* add async wait method

* update dict compare

* remove unuse methods

* not all modules have timeouts parameter

* navigate_value, the input data may be None
2019-05-09 09:04:51 -04:00
Anatoly Pugachev
6e73150244 refactor iscsi network facts module, remove external grep call, add unit test (#55643)
* remove external grep call and parse with python
* use function for repeated code
* use module.get_bin_path() for iscsiutil on HPUX
* some code opt for HPUX
* clean up non-module code, module being defined is a requirement for this code
* import get_bin_path() directly and use without module prefix
* Add integration tests for AIX and HP-UX
* add changelog fragment
* Apply suggestions from code review
Co-Authored-By: mator <matorola@gmail.com>
* Apply suggestions from code review #2
Co-Authored-By: Sam Doran <sdoran@redhat.com>
* Remove strict requirement on executable to exist for get_bin_path() as
it will allow facts gathering to continue without an error. Almost all
other files under facts do not have "required=True" (except 2 files,
which should be probably fixed). And check return value for
get_bin_path() , before run attempt.

* add check for AIX lsattr run_command return code
2019-05-07 12:26:20 -04:00
Anatoly Pugachev
b9af6847c2 network facts, add unit test for FC WWN (#55848)
* use 'None' as return value for get_bin_path and set return code to non-zero on run_command error
2019-04-30 17:26:00 -04:00
Matt Clay
272bb8da7a Fix test_postgresql dependency analysis. 2019-04-25 10:00:49 -07:00
Anatoly Pugachev
de3bd8b791 extends linux cpuinfo test unit with sparc64 data (#55394)
* added debian sparc64 ldom cpuinfo

* updated linux data test unit with sparc64 cpuinfo
2019-04-17 12:00:17 -04:00
Anatoly Pugachev
b28c73af62 Fixes solaris (sunos) uptime in ansible facts module (#54626)
* correct uptime on solaris by using system boot_time instead of snaptime
* add unit test
2019-04-15 18:01:24 -04:00
Sam Doran
6761fc1475
Update distro unit test (#55003)
Remove test comparing output to platform.linux_distribution() since we are relying on distro.id() and are not concerned about matching the output of platform.linux_distribution()
2019-04-08 16:26:31 -04:00
Bruno Inec
c637104078 Allow Netbox device modification (#53631)
* netbox_device: Allow device modification

* Add ability to update and existing device
* Allow check_mode
* Fail when device name is missing
* Fail when cannot resolve ID instead of taking ID 1 by default

* netbox_device: Add diff output

* netbox: Some refactoring

* Add diff output and check_mode to netbox_ip_address
* Deduplicate redundant code into netbox_utils

* netbox_utils: A few unit tests
2019-04-07 15:47:11 +05:30
Claes Nästén
339f6cfcd1 NSO modules now work as expected with NSO 5.X (#54766)
Update NSO modules for NSO 5.0 which change how prefix mapping is made
as a single prefix can have multiple meanings depending on device
being managed
2019-04-03 20:56:35 +01:00
Vitalii Kostenko
2176b53a55 Add latest updates from FTD Ansible downstream repository. (#53638)
* Add latest updates from FTD Ansible downstream repository.
 - add a better implementation of the upsert operation;
 - add API version lookup functionality;
 - add filter which remove duplicated references from the list of references;
 - fix minor bugs.

* fix issues outlined by ansibot

* fix argument name for _check_enum_method
2019-04-01 18:08:01 +05:30
zhongjun2
5599b0484f cloud: huawei: Add new module hwc_network_vpc (#54102) 2019-03-30 17:20:54 +01:00
James Cassell
bc4ef99533 standardize TLS connection properties (#54315)
* openstack: standardize tls params

* tower: tower_verify_ssl->validate_certs

* docker: use standard tls config params

- cacert_path -> ca_cert
- cert_path -> client_cert
- key_path -> client_key
- tls_verify -> validate_certs

* k8s: standardize tls connection params

- verify_ssl -> validate_certs
- ssl_ca_cert -> ca_cert
- cert_file -> client_cert
- key_file -> client_key

* ingate: verify_ssl -> validate_certs

* manageiq: standardize tls params

- verify_ssl -> validate_certs
- ca_bundle_path -> ca_cert

* mysql: standardize tls params

- ssl_ca -> ca_cert
- ssl_cert -> client_cert
- ssl_key -> client_key

* nios: ssl_verify -> validate_certs

* postgresql: ssl_rootcert -> ca_cert

* rabbitmq: standardize tls params

- cacert -> ca_cert
- cert -> client_cert
- key -> client_key

* rackspace: verify_ssl -> validate_certs

* vca: verify_certs -> validate_certs

* kubevirt_cdi_upload: upload_host_verify_ssl -> upload_host_validate_certs

* lxd: standardize tls params

- key_file -> client_key
- cert_file -> client_cert

* get_certificate: ca_certs -> ca_cert

* get_certificate.py: clarify one or more certs in a file

Co-Authored-By: jamescassell <code@james.cassell.me>

* zabbix: tls_issuer -> ca_cert

* bigip_device_auth_ldap: standardize tls params

- ssl_check_peer -> validate_certs
- ssl_client_cert -> client_cert
- ssl_client_key -> client_key
- ssl_ca_cert -> ca_cert

* vdirect: vdirect_validate_certs -> validate_certs

* mqtt: standardize tls params

- ca_certs -> ca_cert
- certfile -> client_cert
- keyfile -> client_key

* pulp_repo: standardize tls params

remove `importer_ssl` prefix

* rhn_register: sslcacert -> ca_cert

* yum_repository: standardize tls params

The fix for yum_repository is not straightforward since this module is
only a thin wrapper for the underlying commands and config.  In this
case, we add the new values as aliases, keeping the old as primary,
only due to the internal structure of the module.

Aliases added:
- sslcacert -> ca_cert
- sslclientcert -> client_cert
- sslclientkey -> client_key
- sslverify -> validate_certs

* gitlab_hook: enable_ssl_verification -> hook_validate_certs

* Adjust arguments for docker_swarm inventory plugin.

* foreman callback: standardize tls params

- ssl_cert -> client_cert
- ssl_key -> client_key

* grafana_annotations: validate_grafana_certs -> validate_certs

* nrdp callback: validate_nrdp_certs -> validate_certs

* kubectl connection: standardize tls params

- kubectl_cert_file -> client_cert
- kubectl_key_file -> client_key
- kubectl_ssl_ca_cert -> ca_cert
- kubectl_verify_ssl -> validate_certs

* oc connection: standardize tls params

- oc_cert_file -> client_cert
- oc_key_file -> client_key
- oc_ssl_ca_cert -> ca_cert
- oc_verify_ssl -> validate_certs

* psrp connection: cert_trust_path -> ca_cert

TODO: cert_validation -> validate_certs (multi-valued vs bool)

* k8s inventory: standardize tls params

- cert_file -> client_cert
- key_file -> client_key
- ca_cert -> ca_cert
- verify_ssl -> validate_certs

* openshift inventory: standardize tls params

- cert_file -> client_cert
- key_file -> client_key
- ca_cert -> ca_cert
- verify_ssl -> validate_certs

* tower inventory: verify_ssl -> validate_certs

* hashi_vault lookup: cacert -> ca_cert

* k8s lookup: standardize tls params

- cert_file -> client_cert
- key_file -> client_key
- ca_cert -> ca_cert
- verify_ssl -> validate_certs

* laps_passord lookup: cacert_file -> ca_cert

* changelog for TLS parameter standardization
2019-03-28 00:19:28 -05:00
dgadmin
8de00e3e1c To ipv6 network (#48572)
* Add to_ipv6_subnet function

* Use the correct function for subnet

* Corrected code style and tests

* Corrected testcase assertion

64 bits make 8 octets, or 4 hextets

* Import from correct module directly
2019-03-27 11:20:27 -04:00
Alex Stephen
a191ed7b58 More GCP Utils Tests (#53568)
* navigate_hash tests

* remove_nones_from_dict tests

* replace_resource_dict tests
2019-03-27 08:23:58 +00:00
Bojan Vitnik
54d23a8297 XenServer: Minor changes to error messages (#54434)
Changed "." for "!" in some error messages to make error messages uniform across the code.
2019-03-27 09:57:08 +05:30
Anton Nikulin
c231fc5a7c New module to install images on Cisco FTD devices (#53467)
* Add ftd_install module

* Remove shebangs

* Avoid using enum package

* Update module docs

* Update ftd_install docs

* Update PropertyMock import

* Fixing unit tests

* Move get_system_info and FtdOperations to module_utils

* Update dependency name

* Move Kick assertion to module_utils

* Add a note about Python interpreter for this module
2019-03-26 19:35:53 +05:30
Sam Doran
ff88bd82b5
Move type checking methods out of basic.py and add unit tests (#53687)
* Move check_type_str() out of basic.py

* Move check_type_list() out of basic.py

* Move safe_eval() out of basic.py

* Move check_type_dict() out of basic.py

* Move json importing code to common location

* Move check_type_bool() out of basic.py

* Move _check_type_int() out of basic.py

* Move _check_type_float() out of basic.py

* Move _check_type_path() out of basic.py

* Move _check_type_raw() out of basic.py

* Move _check_type_bytes() out of basic.py

* Move _check_type_bits() out of basic.py

* Create text.formatters.py

Move human_to_bytes, bytes_to_human, and _lenient_lowercase out of basic.py into text.formatters.py
Change references in modules to point to function at new location

* Move _check_type_jsonarg() out of basic.py

* Rename json related functions and put them in common.text.converters

Move formatters.py to common.text.formatters.py and update references in modules.

* Rework check_type_str()

Add allow_conversion option to make the function more self-contained.
Move the messaging back to basic.py since those error messages are more relevant to using this function in the context of AnsibleModule and not when using the function in isolation.

* Add unit tests for type checking functions

* Change _lenient_lowercase to lenient_lowercase per feedback
2019-03-21 09:40:19 -04:00
Sam Doran
43a44e6f35
Move utility functions out of basic.py (#51715)
Move the following methods to lib/anisble/module_utils/common/validation.py:

- _count_terms()
- _check_mutually_exclusive()
- _check_required_one_of()
- _check_required_together()
- _check_required_by()
- _check_required_arguments()
- _check_required_if
- fail_on_missing_params() --> create check_missing_parameters()
2019-03-14 21:29:55 -04:00
Brian Coca
e280f2f7b0
Try to get correct buffer size to avoid races (#53547)
* Try to get correct buffer size to avoid races

  fixes #51393

* fix test, mock buffer function since all is mocked
2019-03-14 11:04:56 -04:00
Sam Doran
55306906cf
Correctly count processors on ARM systems. (#52884)
- Add unit tests for Linux CPU info
- Add cpuinfo output from several systems for unit tests

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-03-14 10:31:47 -04:00
Felix Fontein
c2cb82ec14 ACME: add support for IP identifiers (#53660)
* Adding support for IP identifiers according to https://tools.ietf.org/html/draft-ietf-acme-ip-05.

* Add changelog.

* Make sure that the authorizations return value is unchanged for CSRs with DNS-only SANs.

* Remove unneeded import.

* type -> identifier_type

* Python 2.6 compatibility.

* Fix unit tests.

* Add IP address normalization.

* Extend tests.

* Move data into fixtures.

* Adjust BOTMETA.
2019-03-13 10:16:56 +01:00
Bojan Vitnik
e775434a52 XenServer: Documentation improvements and fixes for XenServer related modules (#53498)
- More info added to module docs.
 - Fixed errors and typos in module docs.
 - Added parameter types to module docs.
 - Some error messages are fixed and/or changed to be more helpful.
 - Some code comments changed and added.
 - Updated unit tests that test changed error messages.
 - Improved module examples.
 - Improved docs for custom_params xenserver_guest module parameter
2019-03-11 08:56:17 +05:30
ndswartz
26d87a912b Added eseries base with tests (#49269) 2019-03-08 18:13:45 +00:00
Sam Doran
1d91e03119
Ensure Clear Linux parsing is actually parsing a Clear Linux host and all others fall back to NA (#53298)
Fixes a bug where parse_distribution_file_ClearLinux() was called on CoreOS (and probably many other distros) and it returned True since it successfully parses the distribution file. Since this file exists on many Linux distributions and they are a very similar format, add an additional check to make sure it is Clear Linux.

Change the order in which distribution files are processed so NA is last. This prevents a match on CoreOS hosts since they also have /etc/os-release and the called matching function for NA is very general and will match CoreOS.

* Add changelog

* Add unit tests

Only add tests for Clear Linux parsing since that was the cause of this issue.
2019-03-08 10:40:49 -05:00
Sam Doran
bf28b5ceca
Use distro.id() instead of distro.name() (#52199)
* Update sys_info unit tests

* Correct distribution values in hostname.py

* Normalize Rhel to Redhat
2019-03-07 13:25:59 -05:00
Felix Fontein
07fcb60d55 Python 2: accept both long and int for type=int (module options) (#53289)
* Added unit test
2019-03-05 09:39:03 -05:00
Sam Doran
aba4bed803
Move _handle_no_log_values() out of basic.py (#48628)
* Rename method and make private
* Use is_iterable, combine transformations
* Remove unused return_values from network modules
* Improve docstrings in new functions
* Add new PASS_VAR
* Add unit tests for list_no_log_values
* Fix unit tests for Python 2.6
2019-03-04 17:02:44 -05:00
Sam Doran
2a98faee2b
Move _handle_aliases() out of basic.py (#48578)
Refinements:
- return legal_inputs and update class properties
- remove redundant arguments from method and handle in caller
- add better exception types to method

* Add unit tests for handle_aliases
2019-02-28 16:43:19 -05:00
Dag Wieers
4e6c113bf0 uri/win_uri: Make method a free text field (#49719)
* uri/win_uri: Make method a free text field

Since various interfaces introduce their own HTTP method (e.g. like
PROPFIND, LIST or TRACE) it's better to leave this up to the user.

* Fix HTTP method check in module_utils urls

* Add integration test for method UNKNOWN

* Clarify the change as requested during review
2019-02-28 15:55:18 -05:00
Theron Savery
1f7ffe2619 NSO - added validate_certs parameter to allow for ignoring of SSL certificates (#51981)
* added validate_certs paramter to allow for ignoring of SSL certificates

* formatting

* updated NSO unit tests with validate_certs variable

* fixed NSO tests
2019-02-25 14:17:56 +00:00
Bojan Vitnik
922af44d7d Fix a bug that caused exception on XenServer 7.1 with Cummulative Update (#52303)
- xenserver module_util: fixed a bug in gather_vm_params function where
   an exception was generated if XenServer product_version_text_short
   parameter contained non numeric characters, e.g. "7.1 CU1" on
   XenServer version 7.1 with Cummulative Update 1. Code was changed
   to use product_version parameter instead which is all numeric.
 - xenserver module_util: get_xenserver_version function is changed
   to return a list of integers for major, minor and update version
   instead of list of strings.
 - xenserver module_util: unit tests are updated according to changes.
 - xenserver module_util: removed unused imports.
2019-02-19 12:36:57 +00:00
Ganesh Nalawade
41e2bd1df5
Add support for elements validation in argspec (#50335)
* Add support for elements validation in argspec

Fixes #48473

*  Add support to validate the elements value in argspec
   when type is `list`

* Fix unit test failures

* Add unit test for elements validation

* Fix CI failures

* Fix review comments

* Fix unit test and CI failures after rebase
2019-02-19 17:35:20 +05:30
Abhijeet Kasurde
57d85031d7
Parse multiple values for single key in cmdline facts (#49591)
* Facts parsing for cmdline can now handle multiple values for a single key.
* Unit tests for cmdline fact parsing
* Review comments

Fixes: #22766

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-02-19 15:47:06 +05:30
Matt Martz
77bfcad33a
Support unix socket with urls/uri (#43560)
* First pass at allowing unix socket with urls/uri. See #42341

* Only insert handler as needed

* Fix and add tests

* Add HTTPS functionality for unix sockets

* Additional test fixes

* Create context manager for monkey patching HTTPConnection.connect, de-dupe code, raise better errors

* doc

* Add a few more tests

* Fix __call__

* Remove unused import

* Patch HTTPConnection.connect with the functionality we want, instead of duplicating code and disabling

* Fix var name

* Remove unused var

* Add changelog fragment

* Update uri docs

* Fix rebase indentation issue
2019-02-18 10:21:42 -06:00
Hannes Ljungberg
18b968d486 docker_swarm_service: Add healthcheck option (#52419)
* Define yaml versions as strings

* Add healthcheck option

* Add changelog fragment

* Don’t set version_added as strings

* Use single quoted string

* Disable healthcheck tests on python 2.6

* Bring back quoting on already quoted version-added

* Python 2.6 compat

* Move functions to docker-common

* Move parse_healthcheck to docker-common

* Cast exception to str before printing failure

* Extend parse_healthcheck tests
2019-02-18 09:46:14 +00:00
Abhijeet Kasurde
fad6443e58
Distribution: Add support for Linux Mint 18.3 (#52224)
* Distribution: Add support for Linux Mint 18.3
* Review comments

Fixes: #52210

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-02-18 08:28:04 +05:30
Sviatoslav Sydorenko
f1e67c3328 Bugfix/ fix mocker patch in tests (#52372)
* 🐛 Fix invalid os.stat mock in tests

* 🐛 Fix leaking mock patch in tests

Closes #52347
2019-02-15 13:58:26 -08:00
Abhijeet Kasurde
db03f88b63
distribution: Add support for Cumulus Linux (#52309)
* Refactored code
* Added support for Cumulus Linux 2.5.4
* Added support for Cumulus Linux 3.7.3
* Test added

Fixes: #29969

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-02-15 22:30:51 +05:30
Dag Wieers
cd9471ef17 Introduce new 'required_by' argument_spec option (#28662)
* Introduce new "required_by' argument_spec option

This PR introduces a new **required_by** argument_spec option which allows you to say *"if parameter A is set, parameter B and C are required as well"*.

- The difference with **required_if** is that it can only add dependencies if a parameter is set to a specific value, not when it is just defined.
- The difference with **required_together** is that it has a commutative property, so: *"Parameter A and B are required together, if one of them has been defined"*.

As an example, we need this for the complex options that the xml module provides. One of the issues we often see is that users are not using the correct combination of options, and then are surprised that the module does not perform the requested action(s).

This would be solved by adding the correct dependencies, and mutual exclusives. For us this is important to get this shipped together with the new xml module in Ansible v2.4. (This is related to bugfix https://github.com/ansible/ansible/pull/28657)

```python
    module = AnsibleModule(
        argument_spec=dict(
            path=dict(type='path', aliases=['dest', 'file']),
            xmlstring=dict(type='str'),
            xpath=dict(type='str'),
            namespaces=dict(type='dict', default={}),
            state=dict(type='str', default='present', choices=['absent',
'present'], aliases=['ensure']),
            value=dict(type='raw'),
            attribute=dict(type='raw'),
            add_children=dict(type='list'),
            set_children=dict(type='list'),
            count=dict(type='bool', default=False),
            print_match=dict(type='bool', default=False),
            pretty_print=dict(type='bool', default=False),
            content=dict(type='str', choices=['attribute', 'text']),
            input_type=dict(type='str', default='yaml', choices=['xml',
'yaml']),
            backup=dict(type='bool', default=False),
        ),
        supports_check_mode=True,
        required_by=dict(
            add_children=['xpath'],
            attribute=['value', 'xpath'],
            content=['xpath'],
            set_children=['xpath'],
            value=['xpath'],
        ),
        required_if=[
            ['count', True, ['xpath']],
            ['print_match', True, ['xpath']],
        ],
        required_one_of=[
            ['path', 'xmlstring'],
            ['add_children', 'content', 'count', 'pretty_print', 'print_match', 'set_children', 'value'],
        ],
        mutually_exclusive=[
            ['add_children', 'content', 'count', 'print_match','set_children', 'value'],
            ['path', 'xmlstring'],
        ],
    )
```

* Rebase and fix conflict

* Add modules that use required_by functionality

* Update required_by schema

* Fix rebase issue
2019-02-15 10:57:45 +10:00
Juan Antonio Osorio
9f081ca04f identity: Add GSSAPI suport for FreeIPA authentication (#52031)
* identity: Add GSSAPI suport for FreeIPA authentication

This enables the usage of GSSAPI for authentication, instead of having
to pass the username and password as part of the playbook run.

If there is GSSAPI support, this makes the password optional, and will
be able to use the KRB5_CLIENT_KTNAME or the KRB5CCNAME environment
variables; which are standard when using kerberos authentication.

Note that this depends on the urllib_gssapi library, and will only
enable this if that library is available.

* identity: Add documentation for GSSAPI authentication for FreeIPA

This documentation describes how to use GSSAPI authentication with the
IPA identity modules.

* identity: Add changelog for GSSAPI support for IPA

This adds the changelog entry for the GSSAPI authentication feature for
the IPA identity module.
2019-02-13 15:38:13 +00:00
Matt Clay
467d2ebecd Fix unit tests on Python 3.8. 2019-02-13 01:03:06 -08:00
Felix Fontein
0c2bb3da04 docker modules: various adjustments (#51700)
* Move docker_ module_utils into subpackage.

* Remove docker_ prefix from module_utils.docker modules.

* Adding jurisdiction for module_utils/docker to $team_docker.

* Making docker* unit tests community supported.

* Linting.

* Python < 2.6 is not supported.

* Refactoring docker-py version comments. Moving them to doc fragments. Cleaning up some indentations.
2019-02-08 08:16:11 +00:00
Jordan Borean
a39c4ad464
Final round of moving modules to new import error msg (#51852)
* Final round of moving modules to new import error msg

* readd URL to jenkins install guide

* fix unit tests
2019-02-08 10:07:01 +10:00
Bojan Vitnik
64084d593e Initial commit for xenserver module_util unit tests (#50850) 2019-02-05 09:41:26 -08:00
mrmagooey
8eff4cae10 VMware: vmware_guest - allow existing vmdk files to be attached to guest (#45953) 2019-01-30 15:35:24 +01:00
Anatoly Pugachev
fe8412128b facts: solaris: introduce distribution_major version detection for Solaris (#43978)
* facts: solaris: introduce distribution_major version detection for Solaris

Currently, there's no distribution_major in facts module on Solaris OS.
Use "uname -r" output to report major version.

Before the patch we get this on Solaris 11.3 :

$ ansible -o solaris11 -m setup -a filter=ansible_distribution_major_version
solaris11 | SUCCESS => {"ansible_facts": {}, "changed": false}

and after this patch, output is the following:

$ ansible -o solaris11 -m setup -a filter=ansible_distribution_major_version
solaris11 | SUCCESS => {"ansible_facts": {"ansible_distribution_major_version": "11"}, "changed": false}

Tested with Solaris 11.3 and Solaris 10 (both are x86_64 VMs)

Includes patch for test/units.

Fixes #18197

* Try to fix test unit

* should work now...

* fixes for W291 (trailing whitespace) and E265 (block comment)

* mock uname_release for solaris 10 and solaris 11

* facts: solaris: introduce distribution_major version detection for Solaris

Currently, there's no distribution_major in facts module on Solaris OS.
Use "uname -r" output to report major version.

Before the patch we get this on Solaris 11.3 :

$ ansible -o solaris11 -m setup -a filter=ansible_distribution_major_version
solaris11 | SUCCESS => {"ansible_facts": {}, "changed": false}

and after this patch, output is the following:

$ ansible -o solaris11 -m setup -a filter=ansible_distribution_major_version
solaris11 | SUCCESS => {"ansible_facts": {"ansible_distribution_major_version": "11"}, "changed": false}

Tested with Solaris 11.3 and Solaris 10 (both are x86_64 VMs)

Includes patch for test/units.

Fixes #18197

* Try to fix test unit

* should work now...

* fixes for W291 (trailing whitespace) and E265 (block comment)

* mock uname_release for solaris 10 and solaris 11

* typo uname_v -> uname_r

* rebase

* fix pep8 E302: 2 blank lines

* remove int() cast to match test case

* use single function for uname_r and uname_v

* add solaris 11.4 OS to distribution test unit

* fix pep8 sanity - E231 missing whitespace

* distribution_major_version variable strip newline

* mocker test function for mock_get_uname with parameters instead of two different functions

* failed to make one fuction with test unit, revert to use 2 different functions

* try to use single get_uname function

* fix pep8: E703
2019-01-23 20:33:32 -05:00
Brian Coca
42c35a2e01
parallelize getting mount info (#49398)
* parallelize getting mount info

* fixed timeout and made 8 max thread count

  - minor cleanup
  - avoid empty mount entries
  - set timeout on get
  - enforce timeout per mount/thread
  - make note on failure per mount
  - make note on timeout per mount
  - ensure proper pool control
  - minor fixes
  - less vars, simpler code
  - move filter 'pre threading'
  - remove timeout for all mounts, now per mount
  -  also use cpu count from multiprocessing lib
  -  moved 'bind' options out of thread as per comments
  - warn on error, more info on failure to get info
2019-01-23 12:33:59 -05:00
Johannes Brunswicker
c6d404cbe9 Add optional headers to utm modules (#49856)
* removed info declaration from documentation fragment as this is not implemented
* added optional headers for POST and PUT requests
* updated documentation
* added missing headers field decalaration
* removed info choice from state field
* added tests for the new utm_utils function
* fixed class invocation
* added missing required params
* fixed the pytests
2019-01-17 19:06:50 +05:30
Roberto Polli
caa0c9d4cc Fixes #25725: strip quotes from lsb_release and distribution description. (#31143) 2019-01-11 12:07:26 -05:00
vpiserchia
774a667591 Add Devuan Support for ascii release - resolve 49615 (#49616)
* Add Devuan Support for ascii release - resolve 49615

* Devuan Ascii does not have the VERSION_ID in the /etc/os-release file, we need to set NA
2019-01-07 17:08:01 -05:00
Toshio Kuratomi
c18da65089 Add a CLIArgs Singleton class that will hold the parse cli arguments 2019-01-03 18:12:23 -08:00
Toshio Kuratomi
5844c8c7f0 Cleanups to the common.sys_info API
* Move get_all_subclasses out of sys_info as it is unrelated to system
  information.
* get_all_subclasses now returns a set() instead of a list.
* Don't port get_platform to sys_info as it is deprecated.  Code using
  the common API should just use platform.system() directly.
* Rename load_platform_subclass() to get_platform_subclass and do not
  instantiate the rturned class.
* Test the compat shims in module_utils/basic.py separately from the new
  API in module_utils/common/sys_info.py and module_utils/common/_utils.py
2019-01-03 16:21:09 -08:00
Joseph Benden
d810acf7a5 Add support for Kali Linux detection. (#50331)
Signed-off-by: Joseph Benden <joe@benden.us>
2019-01-03 09:43:24 +05:30
Toshio Kuratomi
bd072fe83a
Make the timeout decorator raise an exception out of the function's scope (#49921)
* Revert "allow caller to deal with timeout (#49449)"

This reverts commit 63279823a7.

Flawed on many levels

* Adds poor API to a public function
* Papers over the fact that the public function is doing something bad
  by catching exceptions it cannot handle in the first place
* Papers over the real cause of the issue which is a bug in the timeout
  decorator
* Doesn't reraise properly
* Catches the wrong exception

Fixes #49824
Fixes #49817

* Make the timeout decorator properly raise an exception outside of the function's scope

signal handlers which raise exceptions will never work well because the
exception can be raised anywhere in the called code.  This leads to
exception race conditions where the exceptions could end up being
hanlded by unintended pieces of the called code.

The timeout decorator was using just that idiom.  It was especially bad
because the decorator syntactically occurs outside of the called code
but because of the signal handler, the exception was being raised inside
of the called code.

This change uses a thread instead of a signal to manage the timeout in
parallel to the execution of the decorated function.  Since raising of
the exception happens inside of the decorator, now, instead of inside of
a signal handler, the timeout exception is raised from outside of the
called code as expected which makes reasoning about where exceptions are
to be expected intuitive again.

Fixes #43884

* Add a common case test.

Adding an integration test driven from our unittests.  Most of the time
we'll timeout in run_command which is running things in a subprocess.
Create a test for that specific case in case anything funky comes up
between threading and execve.

* Don't use OSError-based TimeoutError as a base class

Unlike most standard exceptions, OSError has a specific parameter list
with specific meanings.  Instead follow the example of other stdlib
functions, concurrent.futures and multiprocessing and define a separate
TimeoutException.

* Add comment and docstring to point out that this is not hte Python3 TimeoutError
2018-12-18 18:01:46 -08:00
Toshio Kuratomi
61b1daa65f Port from plaform.dist to ansible.module_utils.distro.linux_distribution
ci_complete
2018-12-17 11:01:01 -08:00
Adrian Likins
e872811b8d WIP: add tests to compare to platform.*dist
Can be removed once everything is ported.
2018-12-17 11:01:01 -08:00
Adrian Likins
5e1f8a48f3 Bundle a copy of github.com/nir0s/distro/distro.py
Since the 'platform.dist()' and 'platform.linux_distribution()'
methods will be removed from future versions of python, this
provides an alternative to replace ansibles use of those
methods.

lib/ansible/module_utils/distro.py is a copy of
https://github.com/nir0s/distro/blob/master/distro.py

This module is originally from https://github.com/nir0s/distro
and is license under the Apache License, Version 2.0.
2018-12-17 11:01:01 -08:00
Toshio Kuratomi
3fba006207 Update bare exceptions to specify Exception.
This will keep us from accidentally catching program-exiting exceptions
like KeyboardInterupt and SystemExit.
2018-12-16 15:03:19 -08:00
Josue David Hernandez
9202ef60b0 set ansible_os_family from name variable in os-release for clearlinux OS (#49639)
* set ansible_os_family from name variable in os-release for clearlinux system

Signed-off-by: Josue David Hernandez Gutierrez <josue.d.hernandez.gutierrez@intel.com>

* Add os_family for clear linux and clear linux mixes

Signed-off-by: Josue David Hernandez Gutierrez <josue.d.hernandez.gutierrez@intel.com>
2018-12-11 16:12:38 -05:00
Andreas Calminder
876b637208 move some of basic into common (#48078)
* move file functions into common.file

* move *_PERM_BITS and mark as private (_*_PERM_BITS)

* move get_{platform, distribution, distribution_version} get_all_subclasses and load_platform_subclass into common.sys_info

* forgot get_distribution_version, properly rename get_all_subclasses

* add common/sys_info.py to recursive finder test

* update module paths in test_platform_distribution.py

* update docstrings, _get_all_subclasses -> get_all_subclasses

* forgot to update names

* remove trailing whitespace
2018-12-07 10:21:11 -08:00
Josue David Hernandez
6d42c5020a Fix for changes in clearlinux (#49344)
* Fix for changes in clearlinux

clearlinux is now providing /etc/os-release file and ansible is identifying as NA
then this change allow ansible to find it

Signed-off-by: Josue David Hernandez Gutierrez <josue.d.hernandez.gutierrez@intel.com>

* Add changelog fragment for clearlinux changes

Signed-off-by: Josue David Hernandez Gutierrez <josue.d.hernandez.gutierrez@intel.com>
2018-12-05 13:13:40 +00:00
Abhijeet Kasurde
1b04571ea0
VMware: Fix module usages in module_utils (#49421)
* VMware: Fix module usages in module_utils
* Skip test for Python 2.6 as SSL context is not available in Python 2.6

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2018-12-04 10:11:29 +05:30
Felix Fontein
3726d467ec Code style adjustment. (#49329) 2018-11-29 21:57:32 +01:00
Felix Fontein
1e0ab16247 Add basic unit tests for module_utils/acme.py. (#49103) 2018-11-29 09:32:15 +00:00
Toshio Kuratomi
61e5d04a60 Replace nose with pytest 2018-11-26 13:41:57 -08:00
Anton Nikulin
9770ac70f9 FTD modules: upsert functionality and bug fixes (#47747)
* FTD modules: bug fixes and upsert functionality

* Fix sanity checks

* Fix unit tests for Python 2.6

* Log status code for login/logout

* Use string formatting in logging
2018-11-16 11:55:36 +05:30
Matt Clay
feb5b0b299 Fix unit test issues with pytest >= 4.0.0. 2018-11-15 10:06:07 -08:00
Lars
b6b238ae43 facts: suse: fix SLES4SAP12 detection (#44388) 2018-11-06 08:12:38 -05:00
Felix Fontein
73533d3fc2 docker_* modules: simplify idempotency comparisons (#47709)
* More generic comparison code from docker_container to docker_common.

* More flexibility if a is None and method is allow_to_present.

Note that this odes not affect docker_container, as there a is never None.

* Update docker_secret and docker_config: simplify labels comparison.

* Added unit tests.

* Use proper subsequence test for allow_more_present for lists.

Note that this does not affect existing code in docker_container, since lists
don't use allow_more_present. Using allow_more_present will only be possible
in Ansible 2.8.

* pep8
2018-10-30 08:50:34 +00:00
Matt Clay
37b013aca3
Enable additional pylint rules and resolve issues found. (#47221)
* Resolve unneeded-not.
* Resolve global-at-module-level.
* Resolve useless-import-alias.
* Resolve bad-whitespace.
* Resolve global-variable-not-assigned.
* Resolve logging-not-lazy.
* Resolve comparison-with-itself.
2018-10-18 13:38:08 -07:00
Matt Clay
3033fd96b0
Move unit test compat code out of lib/ansible/. (#46996)
* Move ansible.compat.tests to test/units/compat/.
* Fix unit test references to ansible.compat.tests.
* Move builtins compat to separate file.
* Fix classification of test/units/compat/ dir.
2018-10-12 20:01:14 -07:00
Matt Clay
a11f631ee4 Python 3.8 collections compatibility fixes.
Includes a new pylint blacklist plugin to prevent regressions.
2018-10-08 11:26:37 -07:00
Matt Clay
1a28898a00 Fix parametrize warning in unit tests. 2018-10-05 17:29:02 -07:00
Sloane Hertel
27534e9b47 Move network utils that are used by AWS modules (#45579)
* Separate networking tools that may be used by modules outside of networking so changes to networking-only utilities don't trigger AWS integration tests

* Add unit tests for moved network utils

* Add comment to prevent imports from being mistakenly removed

* Move to_bits as well
2018-10-02 15:08:00 -04:00
Felix Fontein
bc69aeca7f Fixing HTTPError case of fetch_url for Python 3 compatibility. (#45628)
* Fixing HTTPError case of fetch_url for Python 3 compatibility.

* Adding unit test.

* PEP8.

* Changelog.
2018-09-19 10:53:16 -05:00
Matt Clay
53b230ca74 Fix unit test parametrize order on Python 3.5. 2018-09-17 22:45:42 -07:00
Matt Clay
c0bf9815c9 Fix ACI unit test on Python 3.7.0.
The previous logic was only needed for pre-release versions of 3.7.
2018-09-11 10:58:46 -07:00
Abhijeet Kasurde
4e9b8136c2 Add support for Devuan (#45148)
Devuan is part of Debian family. This fix adds support for Devuan OS.

Fixes: #45047

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2018-09-04 08:38:57 -04:00
Alex Stephen
276ad32a45 removing libcloud secrets support for legacy gcp modules (#44932)
* removing libcloud secrets support for legacy gcp modules

* test fixes
2018-08-31 08:28:32 -04:00
Alex Stephen
3550f342e0 Gcp auth deprecation (#44866) 2018-08-30 17:14:39 -04:00
Matt Martz
0221d1ad20
Introduce and use locale-naive rfc2822 date format function (#44868)
* Introduce and use locale-naive rfc2822 date format function. Fixes #44857

* Adjust test expected response
2018-08-30 08:34:37 -05:00
Anton Nikulin
40a97d43d1 New modules and updated HTTP API plugin for FTD devices (#44578)
* Add common and Swagger client utils for FTD modules

* Update FTD HTTP API plugin and add unit tests for it

* Add configuration layer handling object idempotency

* Add ftd_configuration module with unit tests

* Add ftd_file_download and ftd_file_upload modules with unit tests

* Validate operation data and parameters

* Fix ansible-doc, boilerplate and import errors

* Fix pip8 sanity errors

* Update object comparison to work recursively

* Add copyright
2018-08-29 15:48:10 +02:00
Lindsay Hill
5981a7489b new nos_command module (#43056) 2018-08-10 13:50:02 -07:00
Matt Martz
c1c229c6d4
Remove use of simplejson throughout code base (#43548)
* Remove use of simplejson throughout code base. Fixes #42761

* Address failing tests

* Remove simplejson from contrib and other outlying files

* Add changelog fragment for simplejson removal
2018-08-10 11:13:29 -05:00
Joren Vrancken
b954917761 Surround top-level function and class definitions with two blank lines. 2018-07-31 12:06:56 -07:00
Toshio Kuratomi
52449cc01a AnsiballZ improvements
Now that we don't need to worry about python-2.4 and 2.5, we can make
some improvements to the way AnsiballZ handles modules.

* Change AnsiballZ wrapper to use import to invoke the module
  We need the module to think of itself as a script because it could be
  coded as:

      main()

  or as:

      if __name__ == '__main__':
          main()

  Or even as:

      if __name__ == '__main__':
          random_function_name()

  A script will invoke all of those.  Prior to this change, we invoked
  a second Python interpreter on the module so that it really was
  a script.  However, this means that we have to run python twice (once
  for the AnsiballZ wrapper and once for the module).  This change makes
  the module think that it is a script (because __name__ in the module ==
  '__main__') but it's actually being invoked by us importing the module
  code.

  There's three ways we've come up to do this.
  * The most elegant is to use zipimporter and tell the import mechanism
    that the module being loaded is __main__:
    * 5959f11c9d/lib/ansible/executor/module_common.py (L175)
    * zipimporter is nice because we do not have to extract the module from
      the zip file and save it to the disk when we do that.  The import
      machinery does it all for us.
    * The drawback is that modules do not have a __file__ which points
      to a real file when they do this.  Modules could be using __file__
      to for a variety of reasons, most of those probably have
      replacements (the most common one is to find a writable directory
      for temporary files.  AnsibleModule.tmpdir should be used instead)
      We can monkeypatch __file__ in fom AnsibleModule initialization
      but that's kind of gross.  There's no way I can see to do this
      from the wrapper.

  * Next, there's imp.load_module():
    * https://github.com/abadger/ansible/blob/340edf7489/lib/ansible/executor/module_common.py#L151
    * imp has the nice property of allowing us to set __name__ to
      __main__ without changing the name of the file itself
    * We also don't have to do anything special to set __file__ for
      backwards compatibility (although the reason for that is the
      drawback):
    * Its drawback is that it requires the file to exist on disk so we
      have to explicitly extract it from the zipfile and save it to
      a temporary file

  * The last choice is to use exec to execute the module:
    * https://github.com/abadger/ansible/blob/f47a4ccc76/lib/ansible/executor/module_common.py#L175
    * The code we would have to maintain for this looks pretty clean.
      In the wrapper we create a ModuleType, set __file__ on it, read
      the module's contents in from the zip file and then exec it.
    * Drawbacks: We still have to explicitly extract the file's contents
      from the zip archive instead of letting python's import mechanism
      handle it.
    * Exec also has hidden performance issues and breaks certain
      assumptions that modules could be making about their own code:
      http://lucumr.pocoo.org/2011/2/1/exec-in-python/

  Our plan is to use imp.load_module() for now, deprecate the use of
  __file__ in modules, and switch to zipimport once the deprecation
  period for __file__ is over (without monkeypatching a fake __file__ in
  via AnsibleModule).

* Rename the name of the AnsiBallZ wrapped module
  This makes it obvious that the wrapped module isn't the module file that
  we distribute.  It's part of trying to mitigate the fact that the module
  is now named __main)).py in tracebacks.

* Shield all wrapper symbols inside of a function
  With the new import code, all symbols in the wrapper become visible in
  the module.  To mitigate the chance of collisions, move most symbols
  into a toplevel function.  The only symbols left in the global namespace
  are now _ANSIBALLZ_WRAPPER and _ansiballz_main.

revised porting guide entry

Integrate code coverage collection into AnsiballZ.

ci_coverage
ci_complete
2018-07-26 20:07:25 -07:00
Toshio Kuratomi
5b4a7cc283 Fix journald unittests
d7df072b96 changed how we call
journal.send() from positional arguments to keyword arguments.  So we
need to update the test to check for the arguments it was called with in
the keyword args, not in the positional args.
2018-07-25 11:20:24 -07:00
Adam Miller
562ff66a98 Fix pkg_mgr_name fact finding for Fedora (#40922)
* Properly handle default package manager vs apt

For distros where apt might be installed but is not the default
package manager for the distro, properly identify the default distro
package manager during fact finding and re-use fact finding from
DistributionFactCollector and instead of reimplementing small
portions of it in PkgMgrFactCollector

Add unit test to always check the apt + Fedora combination to test
the new code.

Fixes #34014

Signed-off-by: Adam Miller <admiller@redhat.com>

* remove q debugging output I accidentally left behind

Signed-off-by: Adam Miller <admiller@redhat.com>

* add os_family to the conditional so we're only hitting that code path when needed

Signed-off-by: Adam Miller <admiller@redhat.com>

* setup for a _check* pattern for general os_family group pkg_mgr checking

Signed-off-by: Adam Miller <admiller@redhat.com>

* use Mock.patch decorator for os.path.exists in TestPkgMgrFactsAptFedora

Signed-off-by: Adam Miller <admiller@redhat.com>
2018-07-23 10:56:09 -04:00
Will Thames
42eaa00371 Provide kubernetes definition diffs in check_mode (#41471)
Move dict_merge from azure_rm_resource module to
module_utils.common.dict_transformations and add tests.

Use dict_merge to provide a fairly realistic, reliable
diff output when k8s-based modules are run in check_mode.

Rename unit tests so that they actually run and reflect
the module_utils they're based on.
2018-07-11 16:32:03 +10:00
Jordan Borean
8bdd04c147 Fix remote_tmp when become with non admin user (#42396)
* Fix tmpdir on non root become

 - also avoid exception if tmpdir and remote_tmp are None
 - give 'None' on deescalation so tempfile will fallback to it's default behaviour
   and use system dirs
 - fix issue with bad tempdir (not existing/not createable/not writeable)
   i.e nobody and ~/.ansible/tmp
 - added tests for blockfile case

* Revert "Temporarily revert c119d54"

This reverts commit 5c614a59a6.

* changes based on PR feedback and changelog fragment

* changes based on the review

* Fix tmpdir when makedirs failed so we just use the system tmp

* Let missing remote_tmp fail

If remote_tmp is missing then there's something more basic wrong in the
communication from the controller to the module-side.  It's better to
be alerted in this case than to silently ignore it.

jborean and I have independently checked what happens if the user sets
ansible_remote_tmp to empty string and !!null and both cases work fine.
(null is turned into a default value controller-side.  empty string
triggers the warning because it is probably not a directory that the
become user is able to use).
2018-07-06 10:49:19 -07:00
Eitan Akman
fb7882bf86 Issue #39860: Add 'not_contains' method to parsing.py (#39874)
* Issue #39860: Add 'not_contains' method to parsing.py

* Issue #30860 Adds self.negate to Conditional class in lib/ansible/module_utils/network/common/parsing.py

* Issue #39860 Fix singleton-comparison issue per sanity tests

* Issue #39860 'test/integration/targets/nxos_command/tests/cli/not_comparison_operator.yaml' integration test

* Issue #39860 Add unit tests to '../../test/units/module_utils/network/common/test_parsing.py'

* Issue #39860 Fix singleton comparison issue

* Fix E302 expected 2 blank lines, found 1

* Issue #39860 Add license header to unit tests

* Issue #39860 Move integration test to 'test/integration/targets/nxos_command/tests/common/'; remove unnecessary comment from unit test

* Issue #39860 remove unnecessary comment from unit test
2018-06-27 10:21:17 +05:30
Markos Chandras
1737b7be3e facts: suse: Fix major version detection for SUSE (#41416)
When parsing the distribution files such as /etc/os-release, we extract
the full distribution version but not the major version. As such, the
ansible_distribution_major_version ends up being 'NA' whereas the
ansible_distribution_version contains the full version.

Before this patch we get this on openSUSE Leap 15

ansible -o localhost -m setup -a filter=ansible_distribution_major_version
localhost | SUCCESS => {"ansible_facts": {"ansible_distribution_major_version": "NA"}, "changed": false}

After this patch we get this

ansible -o localhost -m setup -a filter=ansible_distribution_major_version
localhost | SUCCESS => {"ansible_facts": {"ansible_distribution_major_version": "15"}, "changed": false}

This also fixes the Tumbleweed distribution test to report a proper
major version and also adds a test for openSUSE Leap 15.0 to avoid
potential future regressions.

Fixes: #41410
2018-06-18 09:53:50 -04:00
Matt Martz
d4930e6692
Add requests.Session like class (#37622)
* Adds requests.Session like class

* py2 syntax fix

* Add a few examples to the Request docstrings

* Add helper methods and docs

* Fix test failures

* Switch tests to test Request instead of open_url, add simple open_url test to validate funcitonality

* Fix filename in replace-urlopen code smell test
2018-06-01 11:44:20 -05:00
Sumit Jaiswal
81510970ae
Fix for updating the name in case of host record and network view and also display meaningful error in case of connection timeout (#40597)
* To fix following github issues 35774, 36574 and 39494

* To fix following github issues 35774, 36574 and 39494

* To fix following github issues 35774, 36574 and 39494

* To fix following github issues 35774, 36574 and 39494

* To fix following github issues 35774, 36574 and 39494

* To fix following github issues 35774, 36574 and 39494

* removed old_name new entry to make ui cleaner

* removed old_name new entry to make ui cleaner

* removed old_name new entry to make ui cleaner

* removed old_name new entry to make ui cleaner

* removed old_name new entry to make ui cleaner

* removed old_name new entry to make ui cleaner

* to resolve the bug 40709

* reslove shippable error

* reslove shippable error

* reslove shippable error

* reslove shippable error

* reslove shippable error

* reslove shippable error

* reslove shippable error

* reslove shippable error

* reslove shippable error

* to fix shippable nios automation error

* modified the name input parsing method

* modified the name input parsing method

* modified the name input parsing method

* modified the name input parsing method

* modified the name input parsing method

* modified the name input parsing method

* modified the name input parsing method

* modified the name input parsing method

* modified the name input parsing method

* shippable error fix

* shippable error fix

* shippable error fix

* shippable error fix

* shippable error fix

* review comment fix

* shippable error fix

* shippable error fix
2018-05-31 12:25:15 +05:30
Sviatoslav Sydorenko
6cafa73441 Add is_iterable utility function 2018-05-27 02:23:57 +02:00
Sviatoslav Sydorenko
e6eec79bca Add tests for module_utils.common.collections 2018-05-27 02:23:57 +02:00
Yanis Guenane
496d10f7a8 Remove support for SSLv2 in test suite when not defined. (#39183)
When running the test test/units/module_utils/urls/test_open_url.py
test_open_url_no_validate_certs, the test fails because of the SSLv2
check.

Test is run on a machine using openssl 1.1.0g. By reading the openssl
man page[1], one can see that support for SSLv2 has been removed.

> Support for SSLv2 and the corresponding SSLv2_method(),
> SSLv2_server_method() and SSLv2_client_method() functions where removed
> in OpenSSL 1.1.0.
>
> SSLv23_method(), SSLv23_server_method() and SSLv23_client_method() were
> deprecated and the preferred TLS_method(), TLS_server_method() and
> TLS_client_method() functions were introduced in OpenSSL 1.1.0.

Hence this commit remove the uses of this flag when it is not defined.

[1] https://www.openssl.org/docs/man1.1.0/ssl/SSLv23_method.html
2018-05-25 12:50:19 -05:00
Jordan Borean
5c39c3b2d1
Module basic.py to create parent dirs of tmpdir if needed (#40201)
* Module basic.py to create parent dirs of tmpdir if needed

* Added warning to dir creation

* Assert if make_dirs was called or not in unit tests
2018-05-17 09:52:46 +10:00
Jordan Borean
44ab948e5d
create module tmpdir based on remote_tmp (#39833)
* create module tmpdir based on remote_tmp

* Source remote_tmp from controller if possible

* Fixed sanity test and not use lambda

* Added expansion of env vars to the remote tmp

* Fixed sanity issues

* Added note around shell remote_tmp option

* Changed fallback tmp dir to ~/.ansible/tmp to make shell defaults
2018-05-15 09:31:21 +10:00
Matt Martz
1663b64e18
Allow subspec defaults to be processed when the parent argument is not supplied (#38967)
* Allow subspec defaults to be processed when the parent argument is not supplied

* Allow this to be configurable via apply_defaults on the parent

* Document attributes of arguments in argument_spec

* Switch manageiq_connection to use apply_defaults

* add choices to api_version in argument_spec
2018-05-07 11:23:13 -05:00
Matthew Stone
d030032b47 Extreme Networks SLXOS Config Module (#38607)
* Adding slxos_config module and supporing util functions.

* Adding slxos module_utils load_config test

* Adding slxos_config module tests

* Removing unneeded required false statements from slxos_config module

* Removing version_aded from slxos_config module

* Removing force and save from slxos config module

* Removing save test
2018-04-20 08:52:20 +01:00
Alex Stephen
9de9633cac [GCP] New module GCP DNS Resource Record Set (#35920) 2018-04-12 07:31:43 -04:00
Matthew Stone
5b1a8ee1e8 Adding slx_command module and supporting module_utils. (#38235)
* Adding slx_command module and supporting module_utils.

This commit adds the slx_command module and tests as well as the
required slxos module_utils.

* Update copyright in header

* Adding missing module init

* Cleaning up shebangs/licensing.

* Incorporating feedback

Removing reference to `waitfor` alias in `slxos_command` module.
Adding `Extreme Networks` to `short_description` of `slxos_command` module.

* Adding cliconf tests

* Fixing 3.X tests

* Adding docstrings to test methods for slxos cliconf tests

* Adding slxos terminal tests

* Adding slxos module_utils tests

* Adding Extreme Networks team members to BOTMETA.yml
2018-04-11 12:59:34 +01:00
Matt Martz
450cfa8776
Handle duplicate headers in the uri module (#33792)
* Handle duplicate headers, and make it easier for users to use cookies, by providing a pre-built string

* Ensure proper cookie ordering, make key plural

* Add note about cookie sort order

* Add tests for duplicate headers and cookies_string

* Extend tests, normalize headers between py2 and py3

* Add some notes in test code

* Don't use AttributeError, use six.PY3. Use better names.
2018-04-10 09:26:51 -05:00
Matt Martz
6332beef65
Add unit tests for ansible.module_utils.urls (#38059)
* Start of tests for ansible.module_utils.urls

* Start adding file for generic functions throughout urls

* Add tests for maybe_add_ssl_handler

* Remove commented out line

* Improve coverage of maybe_add_ssl_handler, test basic_auth_header

* Start tests for open_url

* pep8 and ignore urlopen in test_url_open.py tests

* Extend auth tests, add test for validate_certs=False

* Finish tests for open_url

* Add tests for fetch_url

* Add fetch_url tests to replace-urlopen ignore

* dummy instead of _

* Add BadStatusLine test

* Reorganize/rename tests

* Add tests for RedirectHandlerFactory

* Add POST test to confirm behavior is to convert to GET

* Update tests to handle recent changes to RedirectHandlerFactory

* Special test, just to confirm that aliasing http_error_308 to http_error_307 does not cause issues with urllib2 type redirects
2018-04-09 10:17:43 -05:00
Alexander Bethke
eebe728ae8 Fix SteamOS ansible_distribution fact (Fixes #33628) (#37576)
Fixes #33628
2018-03-21 11:50:54 -04:00
Claes Nästén
458a07d5a4 add timeout parameter and increase default timeout for nso modules (#37391)
NSO operations can take much longer than 10 seconds as they operate on
real network equipment, set default timeout to 5 minutes and allow for
user override.
2018-03-19 15:12:58 +00:00
Claes Nästén
042c111563 nso_config break cycles in dependency sorting (#36828)
False assumption that values can not have cyclic dependencies. Fix by
removing dependency on self and look for cycles, if found remove
dependency to get a partial sort done.
2018-02-28 11:16:49 +00:00
Claes Nästén
e75989ec88 nso_config work around ordering issues (#36774)
Include dependencies when sorting entries to avoid issues with certain
versions of NSO.
2018-02-27 15:22:57 +00:00
Claes Nästén
2789cc5c09 NSO ValueBuilder improvements. 4.5 leaf-list compatability. (#36583)
Fix issues in ValueBuilder used in nso_config and nso_verify so that it
can handle leaf-list in NSO 4.5 and detect identityref types from
unions.

Fail gracefully if a type is not found.
2018-02-26 13:18:20 +00:00
Peter Sprygada
93b795baf0
allows ib_spec attrs to be filtered in update (#36673)
* allows ib_spec attrs to be filtered in update

This change will allow the ib_spec entries to be be filtered on a change
object by setting the update keyword to false.  The default value for
update is true.  When the update keyword is set to false, the keyed
entry will be removed from the update object before it is sent to the
api endpoint.

fixes #36563

* fix up pep8 issues
2018-02-26 07:55:26 -05:00
Martin Krizek
3a5263a92f facts: fix device uuid's on el6 (#36128)
* facts: fix device uuid's on el6

Fixes #36077
2018-02-16 10:52:51 -05:00
René Moser
2f36b9e5ce basic: allow one or more when param list having choices (#34537)
* basic: allow one or more when param list having choices

* add unit tests

* optimize a bit

* re-add get_exception import

* a number of existing modules expect to be able to get it from basic.py
2018-02-07 22:59:21 -08:00
Dag Wieers
bee765fa6b
ACI: Change RETURN output as discussed (#35617)
* ACI: Change result output as discussed

* Update all modules to use new aci.exit_json()

* Update output_level spec and docs

* Fix integration tests

* Small PEP8 fix

* Asorted fixes to tests and aci_rest

* More test fixes and support for ANSIBLE_DEBUG

* Fix another PEP8 issues

* Move response handling inside ACI module

* Reform of ACI error handling and error output

* Diff multiline json output

* Fix a few more tests

* Revert aci_bd tests

* Small correction

* UI change: existing->current, original->previous

* UI change: config->sent

* Update all modules with RETURN values

* Fix a few more tests

* Improve docstring and add 'raw' return value

* Fix thinko

* Fix sanity/pep8 issues

* Rewrite unit tests to comply with new design
2018-02-03 00:41:56 +01:00
Clement Trebuchet
287f3a609e Fixes #35468: ADD max_results to the nios api (#35473)
* Fixes #35468: ADD max_results to the nios api

* Fixes #35468: fix nios unittest

* Fixes #35468: document the module docs fragments for nios
2018-01-30 22:21:45 +00:00
Peter Sprygada
c2d3b9cbd5
refactors nios api shared code to handle provider better (#35393)
* refactors nios api shared code to handle provider better

This change refactors the shared code to be easily shared between
modules, plugins and dynamic inventory scripts.  All parts now implement
the provider arguments uniformly.

This also provides a centralized fix to suppress urllib3 warnings coming
from the requests library implemented by infoblox_client

* fix up pep8 errors

* fix missing var name
2018-01-27 08:03:06 -05:00
Claes Nästén
0582521f96 NSO: handle types for leaf-lists in nso_config (ValueBuilder) (#34986)
Fixes: #34985
2018-01-24 11:21:17 +00:00
Claes Nästén
b7f815ba89 nso_config handle data.not_found in exists for nested lists (#35267)
Recent NSO can return data.not_found error on exists when parent list
entries does not exist. Handle this and return as false.

Fixes: #35264
2018-01-24 11:21:05 +00:00
Adrian Likins
6e585bdf24
Fact collector ordering deps (#31362)
Add deps/requires for fact collectors

Fact collectors can now set a required_facts
class attribute that will be a set of the names
of fact collectors they require to be run first.

ie, if a collector needs to know the ansible_distribution,
it should set it's required_facts to include 'distribution'

        required_facts = set(['distribution'])

If a collector requires another collector, it gets added
to the selected collector names.

We then topological sort the ordering of the collectors
so that deps work out (ie, 'distribution' will run before
'service_mgr')

required_facts were added to the collectors for:

        - network (requires 'distribution', 'platform')
        - hardware (requires 'platform')
        - service_mgr (requires 'distribution', 'platform')

Fix name references for facts (need 'ansible_' prefix)
is service_mgr

Fixes #30753
2018-01-22 18:23:40 -05:00
Peter Sprygada
8d775a332e Add support for Infoblox NIOS (#35097)
* WIP: initial commit

* update documentation in modules

* fix up sanity check issues

* add unit tests for nios api

* fix up sanity check failures

* fix up EXAMPLES in modules

* mock infoblox_client.connector

* remove unsupported assert statement
2018-01-22 14:25:13 +00:00
Adrian Likins
08f92a9f0f
Fix fact deps when 'filter=ansible_fact' is used. (#33441)
The accumulated collected_facts was being update
with new facts _after_ filtering them. So only
facts that pass the filter would ever be passed
to other fact collectors.

For 'filter=ansible_service_mgr', even though it requires
the platform and distribution facts and even collects them,
they would get filtered out and never passed to the other
collectors that need them (service_mgr for ex).

Fix is just to add the unfiltered facts to collected_facts.

Adds unit tests for fact filter and collected_facts.

Fixes #32286
2018-01-20 15:07:27 -05:00
Adrian Likins
ec9769c82f
Facts distribution clear linux 31501 (#32453)
The search string used to look for Clear Linux
was changed in 45a9f96774 to
be more specific, but was too specific. Now finding
a substring match for 'Clear Linux' in /usr/lib/os-release
is enough to consider a match.

Since the details of the full name in os-release varies
('Clear Linux Software for Intel Architecture',
 'Clear Linux OS for Intel Architecture', etc) the
search string match was failing and would fall back to the
'first word in the release file' method resulting in
ansible_distribution='NAME="Clear'

Also add a meta fact indicating which search string
was matched.

Test case info from:
        https://github.com/ansible/ansible/issues/31501#issuecomment-340861535

Fixes #31501
2018-01-20 15:05:53 -05:00
Matt Clay
30093dc4ed
Add Python 3.7 to CI unit test matrix. (#34680)
* Add Python 3.7 to CI unit test matrix.
* Fix `os.errno` reference to be `errno`.
* Update test_aci unit test for Python 3.7.
2018-01-10 00:57:11 -08:00
Will Thames
a685b621cd Route table boto3 (#32059)
* Allow protection of certain keys during camel_to_snake

Create an `ignore_list` parameter that preserves the case
of the contents of certain dictionaries. Most valuable
for `tags` but other uses might arise.

* Port ec2_vpc_route_table to boto3

Update tests to reflect fixes in boto3.

* Add RETURN documentation to ec2_vpc_route_table

* Update DOCUMENTATION to be valid yaml

* Add check mode tests
2018-01-09 20:09:25 -05:00