Commit graph

386 commits

Author SHA1 Message Date
Dag Wieers
7fa1e4f70f Fix sanity checks (#62782)
This PR fixes all new ACI sanity issues.
2019-09-24 09:00:00 -04:00
Gonéri Le Bouder
3ea8e0a144 VMware: not ssl.SSLContext if validate_certs false (#57185)
Python < 2.7.9 does not have the ssl.SSLContext attribute.
ssl.SSLContext is only required when we want to validate the SSL
connection. If `validate_certs` is false, we don't initialize the
`ssl_context` variable.

Add unit-test coverage and a little refactoring:

- avoid the use of `mocker`, when we can push `monkeypatch` which is
  `pytest`'s default.
- use `mock.Mocker()` when possible

closes: #57072
2019-09-09 21:41:46 +05:30
Mark Chappell
1f38a12057 Fix behaviour of module_utils/ec2 compare_policies when dealing with bare bools and ints. (#61115)
* module_utils/ec2: (unit tests) Move unit tests for module_utils/ec2.py into test/units/module_utils

- compare_policies was refactored from s3_bucket
- "ec2_utils" doesn't seem to have ever existed

* module_utils/ec2: (unit tests) Add unit test for comparing quoted and unquoted bools and numbers within policies

As per https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_grammar.html

"Values are enclosed in quotation marks. Quotation marks are optional for numeric
and Boolean values."

* module_utils/ec2: Explicitly convert bools and ints to strings when comparing policies

See also: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_grammar.html
2019-09-09 12:08:25 -04:00
Jill R
d49d52eb5f
Add tests for new alias deprecation functionality (#61476)
* Add tests for the alias deprecation added in #61245
2019-09-05 15:47:42 -07:00
Vitalii Kostenko
6f7cd8370a Ansible FTD Module improvements and tests update. (#60640)
* Ansible FTD Module improvements and tests update.

* Fix sanity tests

* Fix formatting
2019-09-03 08:38:11 +05:30
ndclt
649d612d64 keycloak: improve testability of authentification (#57611) 2019-08-29 10:15:47 +02:00
Sam Doran
48a67f4465 Fix race condition in test_sunos_get_uptime_facts
Mock time.time() to return consistent time value.
2019-08-29 00:22:48 -07:00
Kevin Breit
489156378c Meraki - Enable API call rate limiting for requests (#54827)
* Initial commit for rate limiting
- Detects if error code is 429
- Pauses for random time between .5 and 5 seconds before retrying
- If it fails 10 times, give up and tell user

* Redo structure of request() to support rate limiting

* Hold down timer is now a sliding scale
- 3 * number of retries
- Fails after the 30 second wait

* Whitespace fixes

* Redo implementation using decorators
- Errors aren't tested but code works for regular calls

* Unit tests work for error handling

* Add integration tests for successful retries

* Add condition for 502 errors and retry

* Move _error_report out of the class

* PEP8 fixes

* Add changelog entry
2019-08-27 16:32:09 -04:00
Matt Martz
adb886e4ce
Speed up units using time.sleep by mocking or shortening sleep time (#61064)
* Speed up units using time.sleep by mocking or shortening sleep time

* Only use durations when not a collection
2019-08-21 16:52:58 -05:00
Jordan Borean
e747487720
ansible-galaxy - define multiple galaxy instances in ansible.cfg (#60553)
* ansible-galaxy: support multiple servers on install

* Added docs for the server configuration file

* Fix up doc string for requirements file format

* Fix bugs after testing

* Fix kwarg doc and added version

* Fix typo and doc improvement

* Fix base64 encoding and allow --server to override list
2019-08-21 07:49:05 +10:00
Felix Fontein
26b9c81a8e
crypto modules: fix sanity errors (#60046)
* Remove sanity warnings.

* Linting.

* More linting.

* Forgot one place.

* no_bytes -> num_bytes
2019-08-18 05:37:57 +02:00
Kevin Breit
f52657fb7d Meraki - Initial unit tests (#55251)
* Initial commit for the most basic of unit tests

* Rewrote unit test to actually work
- Uses pytest's fixtures structure, not classes
- Added a test file for importing

* Whitespace fixes

* Draft version of the mock unit test

* Modify code to actually work!

* Add 429 testing

* ansible-test fixes

* Resort lines

* Fix import for 2.x compatibility
2019-08-14 17:00:27 -04:00
Andrey Klychkov
21863d48f3 unit tests: remove unused imports (#60462) 2019-08-13 19:21:43 +02:00
Matt Martz
697b566971
Update units to pass on macOS (#60435)
* Update units to pass on macOS. Fixes #27810

* raising=False
2019-08-12 15:13:07 -05:00
Felix Fontein
3290b8343c
docker: fix sanity errors (#60047)
* Remove sanity check errors.

* More linting.

* Forgot to update places.

* Remove choices which aren't provided in argspec.
2019-08-09 23:50:11 +02:00
Sam Doran
2d266fbc87
Fix unit tests to work with pytest >= 5.0 (#60246)
pytest made a change in the way the message from ExceptionInfo objects was returned.

https://docs.pytest.org/en/latest/changelog.html#pytest-5-0-0-2019-06-28
2019-08-09 14:58:46 -04:00
Yaakov Selkowitz
93d9d64038 facts: fix double-counting of CPUs on POWER systems (#58360)
On POWER systems, /proc/cpuinfo provides a 'processor' entry as a
counter, and a 'cpu' entry with a description (similar to 'model name'
on x86). Support for POWER in get_cpu_facts was added via the 'cpu'
entry in commit 8746e692c1.  Subsequent
support for ARM64 in commit ce4ada93f9
used the 'processor' entry, resulting in double-counting of cores on
POWER systems.

When unit tests were later written for this code in
commit 55306906cf, the erroneous values
were just accepted in the test instead of being diagnosed.

Signed-off-by: Yaakov Selkowitz <yselkowi@redhat.com>
2019-08-09 13:01:29 -04:00
Andrew Gaffney
b38cb37728
Transform octal escape sequences in mtab fields (#60122) 2019-08-08 16:31:42 -05:00
Sam Doran
aa717661af Move assertion in unit test inside loop (#60232) 2019-08-08 14:27:49 -05:00
Matt Clay
9fd244319e
Clean up ansible-test references in tests. (#60108)
* Clean up comments in integration tests.

Tests reference soon to be outdated paths and implementation details.

* Remove unused test/runner/ reference in test.
2019-08-05 14:55:30 -07:00
Andrey Klychkov
64e3cb0fa8 Unit tests: remove unused imports (#59812)
* Unit tests: remove unused imports

* Unit tests: remove unused imports, fix
2019-08-02 17:18:14 +02:00
Nathan Swartz
26fff6f5c3 Improved netapp module utility for E-Series. (#59527)
Add header option to request method in NetAppESeriesModule
Add multipart formdata builder function
Fix issue with url port change
2019-08-01 13:24:38 -04:00
Abhijeet Kasurde
025e30ea0c VMware: add support for http_proxy in connection API
This commit allows users to access a vCenter or a ESXi through a
HTTP CONNECT based proxy.

To do so, the users have to set the `proxy_host` and `proxy_port`
variables.

The can also use the `VMWARE_PROXY_HOST` and `VMWARE_PROXY_PORT`
environment variables.

This feature depends on pyvmomi > v6.7.1.2018.12.

Fixes: #42221

Co-Author: Abhijeet Kasurde <akasurde@redhat.com>
Co-Author: Gonéri Le Bouder <goneri@redhat.com>
2019-08-01 15:57:26 +02:00
Andrey Klychkov
32de51adac Unit tests: remove unused imports (#59740) 2019-07-29 22:10:13 +02:00
Andrey Klychkov
4e8df9a4b8 unit tests: remove unused imports (#59636) 2019-07-26 10:53:31 -04:00
Chaitanya Deshpande
f988c23f82 Added unit tests for avi API utils (#58382)
* Added unit tests for ansibble_utils.py

* Removed unnecessary print statement

* Updated Ip address as review comment

* Updated as per review comments on IP address and test asserts
2019-07-24 11:05:28 -04:00
Andrey Klychkov
333953117c Add human_to_bytes isbits arg unit tests&docstring
PR #58623

Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2019-07-23 14:56:56 +02:00
Andrey Klychkov
59e647910d Add unit tests for bytes_to_human formatter
PR #58664

Co-Authored-By: Sam Doran <sdoran@redhat.com>
2019-07-22 15:45:07 +02:00
Felix Fontein
4a574c4d0c Option parsing: warn if both an option and its alias are specified for a module (#53698)
* Print warning when both an option and its alias is specified.

* Improve output.

* Put warnings into self._warnings directly, resp. use self.warn() when handling subspecs.

* Add changelog.

* Add unit test.
2019-07-19 13:11:41 -04:00
Sam Doran
f231f21669
Handle situation where ansible_architecure may not be defined when gathering facts (#55466) 2019-07-19 11:33:05 -04:00
Andrey Klychkov
88ec5e14cb Add unit tests for lenient_lowercase function (#58586) 2019-07-08 18:33:15 -04:00
Diane Wang
9c140d26d5 VMware: add new module vmware_guest_network (#52075) 2019-07-08 07:54:54 +05:30
Abhijeet Kasurde
96eccda23b
VMware: Add module vmware_vm_storage_policy_info (#56263)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-07-03 16:01:08 +05:30
Brian Coca
ee4cba1b85 ensure bytes in run_command (#58411)
* ensure we pass bytes to run_command

* changed tests for new behaviour

* dont b the pytest

* fixes by sivel
2019-07-02 17:07:48 -05:00
Andrey Klychkov
10af2636d1 Add unit tests for human_to_bytes (#58584) 2019-07-01 13:50:54 -04:00
Sam Doran
ab6a9ef130
Get minor version number for CentOS and Debian (#57814)
* Get the most detailed version number from distro.version() for CentOS and Debian
* Update tests and fixtures
* Update fixture generation script to gather distro info and work with Python 3
* Update LinuxMint fixtures
* Cleanup fixture formatting
* Improvements based on feedback from abadger:
    - use unicode since that is what distro returns
    - use frozenset with a tuple
    - include link Debian to bug
2019-06-28 19:21:35 -04:00
Mariusz Mazur
06292566ec kubevirt: more unit tests (#57739)
* Add tests for KubeAPIVersion

* Legibility improvements for KubevirtVM tests

* Create units.utils.kubevirt with common stuff

* Add some VMIRS unit tests
2019-06-25 20:06:13 +10:00
Andrey Klychkov
64d0559e9f Postgres module_utils: add get_connect_params + unit tests (#58067)
* add get_conn_params

* add get_conn_params: add to the modules
2019-06-19 06:49:19 -07:00
Mariusz Mazur
51add5aa79 kubevirt: enable/update tests + fix merge_dicts() (#57685)
* Actually run the unit tests and separate them into two files

* Re-add recursion to merge_dicts()

* Update tests to work with current code
2019-06-13 11:07:16 +02:00
Anton Nikulin
4fa93d5b9b FTD configuration module: fix a bug with response parsing (#57480)
* Update dependence's name to firepower-kickstart

* Check response type before getting attributes

* Add unit test for construct_ansible_facts method

* Update error message
2019-06-10 18:30:35 +05:30
Sajna Shetty
e25269e1b3 Adding dell ome device_info module (#53438)
* Adding device_facts module for contribution

* changes added for pylint error

* Updated code to solve ansible-test compile error

* Changes to avoide comile error added

* Review Comments changes update

* Avoided blank line

* pylint error changes

* Removed ansible_facts return in error case

* Updated description

* modules renamed

* changing from ansible_facts to device_info

* avoide pep8 error

* Updated sample output

* version changed to 2.9

* Changed Copyright license to BSD

* Changed 3-clause BSD license to 2-clause BSD

* Added unit test support for ome_device_info

* version change

* removed pylint error in unit test modules

* Avoid Sanity error for unit test modules

* updated version
2019-06-10 06:41:48 -04:00
Andrey Klychkov
3e06013eb6 unittests for module_utils.common.removed.removed_module function (#57214) 2019-06-03 11:12:12 -04:00
Matt Martz
8bd4e2a144 cert validation fixes - Attempt 2 (#55953)
* Attempt 2 of cert validation fixes

* Remove unused code

* Cleanup the tmp cert using atexit

* Fix linting issues

* Only add SSLValidationHandler when not HAS_SSLCONTEXT

* Catch value errors on non PEM certs

* Only catch NotImplementedError to avoid masking issues

* set self._context even with PyOpenSSLContext for conformity

* Fix error building

* normalize how we interact with the context we create

* Remove unused code

* Address test for py3.7 message difference

* open_url should pass the ca_path through

* Account for new error in url lookup test

* Guard some code behind whether or not we are validating certs

* Make _make_context public

* Move atexit.register up to where the tmp file is created
2019-05-31 16:35:25 -04:00
Sviatoslav Sydorenko
4c7fff315e
Fix module_utils.common.test_collections testsuite
PR #57181
2019-05-31 13:40:18 +02:00
Andrey Klychkov
bf25889bfd test_postgres: fix typos 2019-05-30 07:18:30 -07:00
Andrey Klychkov
01ed7a489a Add unit tests for lib/ansible/module_utils/common/text/converters.py (#56915)
Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2019-05-30 09:22:53 -04:00
Andrey Klychkov
86354ff1fb postgresql: remove depricated lib (#56641) 2019-05-24 15:08:16 -04:00
Sam Doran
8f4f3750fe
Ensure uri module always returns status even on failure (#56240)
- Also return url and update docs for other values to indicate they are only returned on success.
- Add integration tests
- Use info variable for common return values
- Use -1 as default status rather than None. This is lines up with with existing code in urls.py
- Add unit tests to ensure status and url are returned on failure
2019-05-23 10:17:17 -04:00
Matt Martz
493cf817a9 Don't rely on netloc for determining hostname and port, just use hostname and port (#56270)
* Add changelog fragment
* Fix IPv6 address parsing for py2.6, and add tests
* make sure hostname isn't None
2019-05-20 16:50:54 -04:00
Andrey Klychkov
386cef18ed module_utils.postgres: added unittests (#56381) 2019-05-17 23:33:45 -07:00
Gonéri Le Bouder
cf78759f5b vmware_vm_facts: fix the support with regular ESXi
Ensure the module still work with the ESXi where CustomFieldsManager
does not exist.

From: https://www.vmware.com/support/developer/converter-sdk/conv60_apireference/vim.CustomFieldsManager.html

    The CustomFieldsManager object is used to add and remove custom fields to
    managed entities.
    The custom fields values set on managed entities are available through the
    customValue property and through the summary objects for VirtualMachine
    and HostSystem. They are not available directly through this managed object.
    This functionality is only available through VirtualCenter.

Fixes: #56071
2019-05-13 17:25:16 -04:00
Felix Fontein
bd47e64bc7 Hetzner failover IP: refactoring (#56203)
* Rename helper function.

* Extract hetzner.py module_utils.

* Rewrite docs.

* Add module docs fragment.

* Split up get_failover function.

* Add tests for new function.

* hetzner_pass -> hetzner_password

* Move common argspec to module_utils.
2019-05-10 08:33:51 +02:00
zengchen
d8314e1a45 refactor moudule utils of hwc_utils.py (#55858)
* use navigate_value instead navigate_hash

* add async wait method

* update dict compare

* remove unuse methods

* not all modules have timeouts parameter

* navigate_value, the input data may be None
2019-05-09 09:04:51 -04:00
Anatoly Pugachev
6e73150244 refactor iscsi network facts module, remove external grep call, add unit test (#55643)
* remove external grep call and parse with python
* use function for repeated code
* use module.get_bin_path() for iscsiutil on HPUX
* some code opt for HPUX
* clean up non-module code, module being defined is a requirement for this code
* import get_bin_path() directly and use without module prefix
* Add integration tests for AIX and HP-UX
* add changelog fragment
* Apply suggestions from code review
Co-Authored-By: mator <matorola@gmail.com>
* Apply suggestions from code review #2
Co-Authored-By: Sam Doran <sdoran@redhat.com>
* Remove strict requirement on executable to exist for get_bin_path() as
it will allow facts gathering to continue without an error. Almost all
other files under facts do not have "required=True" (except 2 files,
which should be probably fixed). And check return value for
get_bin_path() , before run attempt.

* add check for AIX lsattr run_command return code
2019-05-07 12:26:20 -04:00
Anatoly Pugachev
b9af6847c2 network facts, add unit test for FC WWN (#55848)
* use 'None' as return value for get_bin_path and set return code to non-zero on run_command error
2019-04-30 17:26:00 -04:00
Matt Clay
272bb8da7a Fix test_postgresql dependency analysis. 2019-04-25 10:00:49 -07:00
Anatoly Pugachev
de3bd8b791 extends linux cpuinfo test unit with sparc64 data (#55394)
* added debian sparc64 ldom cpuinfo

* updated linux data test unit with sparc64 cpuinfo
2019-04-17 12:00:17 -04:00
Anatoly Pugachev
b28c73af62 Fixes solaris (sunos) uptime in ansible facts module (#54626)
* correct uptime on solaris by using system boot_time instead of snaptime
* add unit test
2019-04-15 18:01:24 -04:00
Sam Doran
6761fc1475
Update distro unit test (#55003)
Remove test comparing output to platform.linux_distribution() since we are relying on distro.id() and are not concerned about matching the output of platform.linux_distribution()
2019-04-08 16:26:31 -04:00
Bruno Inec
c637104078 Allow Netbox device modification (#53631)
* netbox_device: Allow device modification

* Add ability to update and existing device
* Allow check_mode
* Fail when device name is missing
* Fail when cannot resolve ID instead of taking ID 1 by default

* netbox_device: Add diff output

* netbox: Some refactoring

* Add diff output and check_mode to netbox_ip_address
* Deduplicate redundant code into netbox_utils

* netbox_utils: A few unit tests
2019-04-07 15:47:11 +05:30
Claes Nästén
339f6cfcd1 NSO modules now work as expected with NSO 5.X (#54766)
Update NSO modules for NSO 5.0 which change how prefix mapping is made
as a single prefix can have multiple meanings depending on device
being managed
2019-04-03 20:56:35 +01:00
Vitalii Kostenko
2176b53a55 Add latest updates from FTD Ansible downstream repository. (#53638)
* Add latest updates from FTD Ansible downstream repository.
 - add a better implementation of the upsert operation;
 - add API version lookup functionality;
 - add filter which remove duplicated references from the list of references;
 - fix minor bugs.

* fix issues outlined by ansibot

* fix argument name for _check_enum_method
2019-04-01 18:08:01 +05:30
zhongjun2
5599b0484f cloud: huawei: Add new module hwc_network_vpc (#54102) 2019-03-30 17:20:54 +01:00
James Cassell
bc4ef99533 standardize TLS connection properties (#54315)
* openstack: standardize tls params

* tower: tower_verify_ssl->validate_certs

* docker: use standard tls config params

- cacert_path -> ca_cert
- cert_path -> client_cert
- key_path -> client_key
- tls_verify -> validate_certs

* k8s: standardize tls connection params

- verify_ssl -> validate_certs
- ssl_ca_cert -> ca_cert
- cert_file -> client_cert
- key_file -> client_key

* ingate: verify_ssl -> validate_certs

* manageiq: standardize tls params

- verify_ssl -> validate_certs
- ca_bundle_path -> ca_cert

* mysql: standardize tls params

- ssl_ca -> ca_cert
- ssl_cert -> client_cert
- ssl_key -> client_key

* nios: ssl_verify -> validate_certs

* postgresql: ssl_rootcert -> ca_cert

* rabbitmq: standardize tls params

- cacert -> ca_cert
- cert -> client_cert
- key -> client_key

* rackspace: verify_ssl -> validate_certs

* vca: verify_certs -> validate_certs

* kubevirt_cdi_upload: upload_host_verify_ssl -> upload_host_validate_certs

* lxd: standardize tls params

- key_file -> client_key
- cert_file -> client_cert

* get_certificate: ca_certs -> ca_cert

* get_certificate.py: clarify one or more certs in a file

Co-Authored-By: jamescassell <code@james.cassell.me>

* zabbix: tls_issuer -> ca_cert

* bigip_device_auth_ldap: standardize tls params

- ssl_check_peer -> validate_certs
- ssl_client_cert -> client_cert
- ssl_client_key -> client_key
- ssl_ca_cert -> ca_cert

* vdirect: vdirect_validate_certs -> validate_certs

* mqtt: standardize tls params

- ca_certs -> ca_cert
- certfile -> client_cert
- keyfile -> client_key

* pulp_repo: standardize tls params

remove `importer_ssl` prefix

* rhn_register: sslcacert -> ca_cert

* yum_repository: standardize tls params

The fix for yum_repository is not straightforward since this module is
only a thin wrapper for the underlying commands and config.  In this
case, we add the new values as aliases, keeping the old as primary,
only due to the internal structure of the module.

Aliases added:
- sslcacert -> ca_cert
- sslclientcert -> client_cert
- sslclientkey -> client_key
- sslverify -> validate_certs

* gitlab_hook: enable_ssl_verification -> hook_validate_certs

* Adjust arguments for docker_swarm inventory plugin.

* foreman callback: standardize tls params

- ssl_cert -> client_cert
- ssl_key -> client_key

* grafana_annotations: validate_grafana_certs -> validate_certs

* nrdp callback: validate_nrdp_certs -> validate_certs

* kubectl connection: standardize tls params

- kubectl_cert_file -> client_cert
- kubectl_key_file -> client_key
- kubectl_ssl_ca_cert -> ca_cert
- kubectl_verify_ssl -> validate_certs

* oc connection: standardize tls params

- oc_cert_file -> client_cert
- oc_key_file -> client_key
- oc_ssl_ca_cert -> ca_cert
- oc_verify_ssl -> validate_certs

* psrp connection: cert_trust_path -> ca_cert

TODO: cert_validation -> validate_certs (multi-valued vs bool)

* k8s inventory: standardize tls params

- cert_file -> client_cert
- key_file -> client_key
- ca_cert -> ca_cert
- verify_ssl -> validate_certs

* openshift inventory: standardize tls params

- cert_file -> client_cert
- key_file -> client_key
- ca_cert -> ca_cert
- verify_ssl -> validate_certs

* tower inventory: verify_ssl -> validate_certs

* hashi_vault lookup: cacert -> ca_cert

* k8s lookup: standardize tls params

- cert_file -> client_cert
- key_file -> client_key
- ca_cert -> ca_cert
- verify_ssl -> validate_certs

* laps_passord lookup: cacert_file -> ca_cert

* changelog for TLS parameter standardization
2019-03-28 00:19:28 -05:00
dgadmin
8de00e3e1c To ipv6 network (#48572)
* Add to_ipv6_subnet function

* Use the correct function for subnet

* Corrected code style and tests

* Corrected testcase assertion

64 bits make 8 octets, or 4 hextets

* Import from correct module directly
2019-03-27 11:20:27 -04:00
Alex Stephen
a191ed7b58 More GCP Utils Tests (#53568)
* navigate_hash tests

* remove_nones_from_dict tests

* replace_resource_dict tests
2019-03-27 08:23:58 +00:00
Bojan Vitnik
54d23a8297 XenServer: Minor changes to error messages (#54434)
Changed "." for "!" in some error messages to make error messages uniform across the code.
2019-03-27 09:57:08 +05:30
Anton Nikulin
c231fc5a7c New module to install images on Cisco FTD devices (#53467)
* Add ftd_install module

* Remove shebangs

* Avoid using enum package

* Update module docs

* Update ftd_install docs

* Update PropertyMock import

* Fixing unit tests

* Move get_system_info and FtdOperations to module_utils

* Update dependency name

* Move Kick assertion to module_utils

* Add a note about Python interpreter for this module
2019-03-26 19:35:53 +05:30
Sam Doran
ff88bd82b5
Move type checking methods out of basic.py and add unit tests (#53687)
* Move check_type_str() out of basic.py

* Move check_type_list() out of basic.py

* Move safe_eval() out of basic.py

* Move check_type_dict() out of basic.py

* Move json importing code to common location

* Move check_type_bool() out of basic.py

* Move _check_type_int() out of basic.py

* Move _check_type_float() out of basic.py

* Move _check_type_path() out of basic.py

* Move _check_type_raw() out of basic.py

* Move _check_type_bytes() out of basic.py

* Move _check_type_bits() out of basic.py

* Create text.formatters.py

Move human_to_bytes, bytes_to_human, and _lenient_lowercase out of basic.py into text.formatters.py
Change references in modules to point to function at new location

* Move _check_type_jsonarg() out of basic.py

* Rename json related functions and put them in common.text.converters

Move formatters.py to common.text.formatters.py and update references in modules.

* Rework check_type_str()

Add allow_conversion option to make the function more self-contained.
Move the messaging back to basic.py since those error messages are more relevant to using this function in the context of AnsibleModule and not when using the function in isolation.

* Add unit tests for type checking functions

* Change _lenient_lowercase to lenient_lowercase per feedback
2019-03-21 09:40:19 -04:00
Sam Doran
43a44e6f35
Move utility functions out of basic.py (#51715)
Move the following methods to lib/anisble/module_utils/common/validation.py:

- _count_terms()
- _check_mutually_exclusive()
- _check_required_one_of()
- _check_required_together()
- _check_required_by()
- _check_required_arguments()
- _check_required_if
- fail_on_missing_params() --> create check_missing_parameters()
2019-03-14 21:29:55 -04:00
Brian Coca
e280f2f7b0
Try to get correct buffer size to avoid races (#53547)
* Try to get correct buffer size to avoid races

  fixes #51393

* fix test, mock buffer function since all is mocked
2019-03-14 11:04:56 -04:00
Sam Doran
55306906cf
Correctly count processors on ARM systems. (#52884)
- Add unit tests for Linux CPU info
- Add cpuinfo output from several systems for unit tests

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-03-14 10:31:47 -04:00
Felix Fontein
c2cb82ec14 ACME: add support for IP identifiers (#53660)
* Adding support for IP identifiers according to https://tools.ietf.org/html/draft-ietf-acme-ip-05.

* Add changelog.

* Make sure that the authorizations return value is unchanged for CSRs with DNS-only SANs.

* Remove unneeded import.

* type -> identifier_type

* Python 2.6 compatibility.

* Fix unit tests.

* Add IP address normalization.

* Extend tests.

* Move data into fixtures.

* Adjust BOTMETA.
2019-03-13 10:16:56 +01:00
Bojan Vitnik
e775434a52 XenServer: Documentation improvements and fixes for XenServer related modules (#53498)
- More info added to module docs.
 - Fixed errors and typos in module docs.
 - Added parameter types to module docs.
 - Some error messages are fixed and/or changed to be more helpful.
 - Some code comments changed and added.
 - Updated unit tests that test changed error messages.
 - Improved module examples.
 - Improved docs for custom_params xenserver_guest module parameter
2019-03-11 08:56:17 +05:30
ndswartz
26d87a912b Added eseries base with tests (#49269) 2019-03-08 18:13:45 +00:00
Sam Doran
1d91e03119
Ensure Clear Linux parsing is actually parsing a Clear Linux host and all others fall back to NA (#53298)
Fixes a bug where parse_distribution_file_ClearLinux() was called on CoreOS (and probably many other distros) and it returned True since it successfully parses the distribution file. Since this file exists on many Linux distributions and they are a very similar format, add an additional check to make sure it is Clear Linux.

Change the order in which distribution files are processed so NA is last. This prevents a match on CoreOS hosts since they also have /etc/os-release and the called matching function for NA is very general and will match CoreOS.

* Add changelog

* Add unit tests

Only add tests for Clear Linux parsing since that was the cause of this issue.
2019-03-08 10:40:49 -05:00
Sam Doran
bf28b5ceca
Use distro.id() instead of distro.name() (#52199)
* Update sys_info unit tests

* Correct distribution values in hostname.py

* Normalize Rhel to Redhat
2019-03-07 13:25:59 -05:00
Felix Fontein
07fcb60d55 Python 2: accept both long and int for type=int (module options) (#53289)
* Added unit test
2019-03-05 09:39:03 -05:00
Sam Doran
aba4bed803
Move _handle_no_log_values() out of basic.py (#48628)
* Rename method and make private
* Use is_iterable, combine transformations
* Remove unused return_values from network modules
* Improve docstrings in new functions
* Add new PASS_VAR
* Add unit tests for list_no_log_values
* Fix unit tests for Python 2.6
2019-03-04 17:02:44 -05:00
Sam Doran
2a98faee2b
Move _handle_aliases() out of basic.py (#48578)
Refinements:
- return legal_inputs and update class properties
- remove redundant arguments from method and handle in caller
- add better exception types to method

* Add unit tests for handle_aliases
2019-02-28 16:43:19 -05:00
Dag Wieers
4e6c113bf0 uri/win_uri: Make method a free text field (#49719)
* uri/win_uri: Make method a free text field

Since various interfaces introduce their own HTTP method (e.g. like
PROPFIND, LIST or TRACE) it's better to leave this up to the user.

* Fix HTTP method check in module_utils urls

* Add integration test for method UNKNOWN

* Clarify the change as requested during review
2019-02-28 15:55:18 -05:00
Theron Savery
1f7ffe2619 NSO - added validate_certs parameter to allow for ignoring of SSL certificates (#51981)
* added validate_certs paramter to allow for ignoring of SSL certificates

* formatting

* updated NSO unit tests with validate_certs variable

* fixed NSO tests
2019-02-25 14:17:56 +00:00
Bojan Vitnik
922af44d7d Fix a bug that caused exception on XenServer 7.1 with Cummulative Update (#52303)
- xenserver module_util: fixed a bug in gather_vm_params function where
   an exception was generated if XenServer product_version_text_short
   parameter contained non numeric characters, e.g. "7.1 CU1" on
   XenServer version 7.1 with Cummulative Update 1. Code was changed
   to use product_version parameter instead which is all numeric.
 - xenserver module_util: get_xenserver_version function is changed
   to return a list of integers for major, minor and update version
   instead of list of strings.
 - xenserver module_util: unit tests are updated according to changes.
 - xenserver module_util: removed unused imports.
2019-02-19 12:36:57 +00:00
Ganesh Nalawade
41e2bd1df5
Add support for elements validation in argspec (#50335)
* Add support for elements validation in argspec

Fixes #48473

*  Add support to validate the elements value in argspec
   when type is `list`

* Fix unit test failures

* Add unit test for elements validation

* Fix CI failures

* Fix review comments

* Fix unit test and CI failures after rebase
2019-02-19 17:35:20 +05:30
Abhijeet Kasurde
57d85031d7
Parse multiple values for single key in cmdline facts (#49591)
* Facts parsing for cmdline can now handle multiple values for a single key.
* Unit tests for cmdline fact parsing
* Review comments

Fixes: #22766

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-02-19 15:47:06 +05:30
Matt Martz
77bfcad33a
Support unix socket with urls/uri (#43560)
* First pass at allowing unix socket with urls/uri. See #42341

* Only insert handler as needed

* Fix and add tests

* Add HTTPS functionality for unix sockets

* Additional test fixes

* Create context manager for monkey patching HTTPConnection.connect, de-dupe code, raise better errors

* doc

* Add a few more tests

* Fix __call__

* Remove unused import

* Patch HTTPConnection.connect with the functionality we want, instead of duplicating code and disabling

* Fix var name

* Remove unused var

* Add changelog fragment

* Update uri docs

* Fix rebase indentation issue
2019-02-18 10:21:42 -06:00
Hannes Ljungberg
18b968d486 docker_swarm_service: Add healthcheck option (#52419)
* Define yaml versions as strings

* Add healthcheck option

* Add changelog fragment

* Don’t set version_added as strings

* Use single quoted string

* Disable healthcheck tests on python 2.6

* Bring back quoting on already quoted version-added

* Python 2.6 compat

* Move functions to docker-common

* Move parse_healthcheck to docker-common

* Cast exception to str before printing failure

* Extend parse_healthcheck tests
2019-02-18 09:46:14 +00:00
Abhijeet Kasurde
fad6443e58
Distribution: Add support for Linux Mint 18.3 (#52224)
* Distribution: Add support for Linux Mint 18.3
* Review comments

Fixes: #52210

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-02-18 08:28:04 +05:30
Sviatoslav Sydorenko
f1e67c3328 Bugfix/ fix mocker patch in tests (#52372)
* 🐛 Fix invalid os.stat mock in tests

* 🐛 Fix leaking mock patch in tests

Closes #52347
2019-02-15 13:58:26 -08:00
Abhijeet Kasurde
db03f88b63
distribution: Add support for Cumulus Linux (#52309)
* Refactored code
* Added support for Cumulus Linux 2.5.4
* Added support for Cumulus Linux 3.7.3
* Test added

Fixes: #29969

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-02-15 22:30:51 +05:30
Dag Wieers
cd9471ef17 Introduce new 'required_by' argument_spec option (#28662)
* Introduce new "required_by' argument_spec option

This PR introduces a new **required_by** argument_spec option which allows you to say *"if parameter A is set, parameter B and C are required as well"*.

- The difference with **required_if** is that it can only add dependencies if a parameter is set to a specific value, not when it is just defined.
- The difference with **required_together** is that it has a commutative property, so: *"Parameter A and B are required together, if one of them has been defined"*.

As an example, we need this for the complex options that the xml module provides. One of the issues we often see is that users are not using the correct combination of options, and then are surprised that the module does not perform the requested action(s).

This would be solved by adding the correct dependencies, and mutual exclusives. For us this is important to get this shipped together with the new xml module in Ansible v2.4. (This is related to bugfix https://github.com/ansible/ansible/pull/28657)

```python
    module = AnsibleModule(
        argument_spec=dict(
            path=dict(type='path', aliases=['dest', 'file']),
            xmlstring=dict(type='str'),
            xpath=dict(type='str'),
            namespaces=dict(type='dict', default={}),
            state=dict(type='str', default='present', choices=['absent',
'present'], aliases=['ensure']),
            value=dict(type='raw'),
            attribute=dict(type='raw'),
            add_children=dict(type='list'),
            set_children=dict(type='list'),
            count=dict(type='bool', default=False),
            print_match=dict(type='bool', default=False),
            pretty_print=dict(type='bool', default=False),
            content=dict(type='str', choices=['attribute', 'text']),
            input_type=dict(type='str', default='yaml', choices=['xml',
'yaml']),
            backup=dict(type='bool', default=False),
        ),
        supports_check_mode=True,
        required_by=dict(
            add_children=['xpath'],
            attribute=['value', 'xpath'],
            content=['xpath'],
            set_children=['xpath'],
            value=['xpath'],
        ),
        required_if=[
            ['count', True, ['xpath']],
            ['print_match', True, ['xpath']],
        ],
        required_one_of=[
            ['path', 'xmlstring'],
            ['add_children', 'content', 'count', 'pretty_print', 'print_match', 'set_children', 'value'],
        ],
        mutually_exclusive=[
            ['add_children', 'content', 'count', 'print_match','set_children', 'value'],
            ['path', 'xmlstring'],
        ],
    )
```

* Rebase and fix conflict

* Add modules that use required_by functionality

* Update required_by schema

* Fix rebase issue
2019-02-15 10:57:45 +10:00
Juan Antonio Osorio
9f081ca04f identity: Add GSSAPI suport for FreeIPA authentication (#52031)
* identity: Add GSSAPI suport for FreeIPA authentication

This enables the usage of GSSAPI for authentication, instead of having
to pass the username and password as part of the playbook run.

If there is GSSAPI support, this makes the password optional, and will
be able to use the KRB5_CLIENT_KTNAME or the KRB5CCNAME environment
variables; which are standard when using kerberos authentication.

Note that this depends on the urllib_gssapi library, and will only
enable this if that library is available.

* identity: Add documentation for GSSAPI authentication for FreeIPA

This documentation describes how to use GSSAPI authentication with the
IPA identity modules.

* identity: Add changelog for GSSAPI support for IPA

This adds the changelog entry for the GSSAPI authentication feature for
the IPA identity module.
2019-02-13 15:38:13 +00:00
Matt Clay
467d2ebecd Fix unit tests on Python 3.8. 2019-02-13 01:03:06 -08:00
Felix Fontein
0c2bb3da04 docker modules: various adjustments (#51700)
* Move docker_ module_utils into subpackage.

* Remove docker_ prefix from module_utils.docker modules.

* Adding jurisdiction for module_utils/docker to $team_docker.

* Making docker* unit tests community supported.

* Linting.

* Python < 2.6 is not supported.

* Refactoring docker-py version comments. Moving them to doc fragments. Cleaning up some indentations.
2019-02-08 08:16:11 +00:00
Jordan Borean
a39c4ad464
Final round of moving modules to new import error msg (#51852)
* Final round of moving modules to new import error msg

* readd URL to jenkins install guide

* fix unit tests
2019-02-08 10:07:01 +10:00
Bojan Vitnik
64084d593e Initial commit for xenserver module_util unit tests (#50850) 2019-02-05 09:41:26 -08:00
mrmagooey
8eff4cae10 VMware: vmware_guest - allow existing vmdk files to be attached to guest (#45953) 2019-01-30 15:35:24 +01:00
Anatoly Pugachev
fe8412128b facts: solaris: introduce distribution_major version detection for Solaris (#43978)
* facts: solaris: introduce distribution_major version detection for Solaris

Currently, there's no distribution_major in facts module on Solaris OS.
Use "uname -r" output to report major version.

Before the patch we get this on Solaris 11.3 :

$ ansible -o solaris11 -m setup -a filter=ansible_distribution_major_version
solaris11 | SUCCESS => {"ansible_facts": {}, "changed": false}

and after this patch, output is the following:

$ ansible -o solaris11 -m setup -a filter=ansible_distribution_major_version
solaris11 | SUCCESS => {"ansible_facts": {"ansible_distribution_major_version": "11"}, "changed": false}

Tested with Solaris 11.3 and Solaris 10 (both are x86_64 VMs)

Includes patch for test/units.

Fixes #18197

* Try to fix test unit

* should work now...

* fixes for W291 (trailing whitespace) and E265 (block comment)

* mock uname_release for solaris 10 and solaris 11

* facts: solaris: introduce distribution_major version detection for Solaris

Currently, there's no distribution_major in facts module on Solaris OS.
Use "uname -r" output to report major version.

Before the patch we get this on Solaris 11.3 :

$ ansible -o solaris11 -m setup -a filter=ansible_distribution_major_version
solaris11 | SUCCESS => {"ansible_facts": {}, "changed": false}

and after this patch, output is the following:

$ ansible -o solaris11 -m setup -a filter=ansible_distribution_major_version
solaris11 | SUCCESS => {"ansible_facts": {"ansible_distribution_major_version": "11"}, "changed": false}

Tested with Solaris 11.3 and Solaris 10 (both are x86_64 VMs)

Includes patch for test/units.

Fixes #18197

* Try to fix test unit

* should work now...

* fixes for W291 (trailing whitespace) and E265 (block comment)

* mock uname_release for solaris 10 and solaris 11

* typo uname_v -> uname_r

* rebase

* fix pep8 E302: 2 blank lines

* remove int() cast to match test case

* use single function for uname_r and uname_v

* add solaris 11.4 OS to distribution test unit

* fix pep8 sanity - E231 missing whitespace

* distribution_major_version variable strip newline

* mocker test function for mock_get_uname with parameters instead of two different functions

* failed to make one fuction with test unit, revert to use 2 different functions

* try to use single get_uname function

* fix pep8: E703
2019-01-23 20:33:32 -05:00
Brian Coca
42c35a2e01
parallelize getting mount info (#49398)
* parallelize getting mount info

* fixed timeout and made 8 max thread count

  - minor cleanup
  - avoid empty mount entries
  - set timeout on get
  - enforce timeout per mount/thread
  - make note on failure per mount
  - make note on timeout per mount
  - ensure proper pool control
  - minor fixes
  - less vars, simpler code
  - move filter 'pre threading'
  - remove timeout for all mounts, now per mount
  -  also use cpu count from multiprocessing lib
  -  moved 'bind' options out of thread as per comments
  - warn on error, more info on failure to get info
2019-01-23 12:33:59 -05:00
Johannes Brunswicker
c6d404cbe9 Add optional headers to utm modules (#49856)
* removed info declaration from documentation fragment as this is not implemented
* added optional headers for POST and PUT requests
* updated documentation
* added missing headers field decalaration
* removed info choice from state field
* added tests for the new utm_utils function
* fixed class invocation
* added missing required params
* fixed the pytests
2019-01-17 19:06:50 +05:30