Commit graph

382 commits

Author SHA1 Message Date
Felix Fontein
40f21dfd3c
Version source tagging (automatic and manual) for version_added and deprecation versions (#69680)
* Track collection for version_added.
Validate *all* version numbers in validate-modules.
For tagged version numbers (i.e. version_added), consider source collection to chose validation.

* Make tagging/untagging functions more flexible.

* Tag all versions in doc fragments.

* Tag all deprecation versions issued by code.

* Make Display.deprecated() understand tagged versions.

* Extend validation to enforce tagged version numbers.

* Tag versions in tests.

* Lint and fix test.

* Mention collection name in collection loader's deprecation/removal messages.

* Fix error IDs.

* Handle tagged dates in Display.deprecated().

* Also require that removed_at_date and deprecated_aliases.date are tagged.

* Also automatically tag/untag removed_at_date; fix sanity module removal version check.

* Improve error message when invalid version number is used (like '2.14' in collections).
2020-05-28 22:46:16 -07:00
Abhijeet Kasurde
564907d8ac
distribution: Refactor test_distribution_version testcases (#69300) 2020-05-27 14:01:31 +05:30
Felix Fontein
ea04e0048d
Allow to deprecate options and aliases by date (#68177)
* Allow to deprecate options and aliases by date instead of only by version.

* Update display.deprecate().

* Adjust behavior to conform to tested behavior, extend tests, and improve C# style.

* Parse date and fail on invalid date.

This is mainly to make sure that people start using invalid dates, and we eventually have a mess to clean up.

* C# code: improve validation and update/extend tests.

* Make sure that deprecate() is not called with both date and version.

* Forgot to remove no longer necessary formatting.

* Adjust order of warnings in C# code.

* Adjust unrelated test.

* Fix grammar (and make that test pass).

* Don't parse date, and adjust message to be same as in #67684.

* Sanity tests: disable date in past test.

* Validate-modules: validate ISO 8601 date format.

* Validate-modules: switch schema declaration for deprecated_aliases to improve error messages for invalid dates.

* Use DateTime instead of string for date deprecation.

* Validate that date in deprecated_aliases is actually a DateTime.

* Fix tests.

* Fix rebasing error.

* Adjust error codes for pylint, and add removed_at_date and deprecated_aliases.date checks to validate-modules.

* Make deprecation date in the past error codes optional.

* Make sure not both version and date are specified for AnsibleModule.deprecate() calls.

* Stop using Python 3.7+ API.

* Make sure errors are actually reported. Re-add 'ansible-' prefix.

* Avoid crashing when 'name' isn't there.

* Linting.

* Update lib/ansible/module_utils/csharp/Ansible.Basic.cs

Co-authored-by: Jordan Borean <jborean93@gmail.com>

* Adjust test to latest change.

* Prefer date over version if both end up in Display.deprecated().

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2020-05-26 20:23:56 -04:00
Matt Martz
e0f25a2b1f
Strip no log values from module response keys. Fixes #68400 (#69653) 2020-05-21 16:17:57 -04:00
Matt Martz
b748edea45
Add multipart/form-data functionality (#69376)
* Add multipart/form-data functionality

* Fix some linting issues

* Fix error message

* Allow filename to be provided with content

* Add integration test

* Update examples

* General improvements to multipart handling

* Use prepare_multipart for galaxy collection publish

* Properly account for py2 vs py3, ensuring no max header length

* Address test assumptions

* Add unit tests

* Add changelog

* Ensure to use CRLF instead of NL

* Ignore line-endings in fixture

* Consolidate code, add comment

* Bump fallaxy container version

* ci_complete
2020-05-20 15:44:01 -05:00
Lukas Pirl
34db57a47f
introduce fact "ansible_processor_nproc": number of usable vcpus (#66569)
This fact reflects the number of usable vcpus (which might be different
from ansible_processor_vcpus, e.g., in containers with limits). See
also #51504.

* Add fixture data and update unit tests

Co-authored-by: Sam Doran <sdoran@redhat.com>
2020-05-15 09:38:56 -04:00
Matt Clay
d0484fbf74 Cleanup for Python 3.9 and pycodestyle compat. 2020-05-14 21:06:16 -07:00
Bob Weinand
f200487414
Fix filedescriptor out of range in select() when running commands (#65058)
* Fix filedescriptor out of range in select() when running commands

* Simplify the run_command() code

Now that we're using selectors in run_command(), we can simplify some of
the code.

* Use fileobj.read() instead of os.read()
* No longer use get_buffer_size() as we can just slurp all of the data
  instead.

Also use a simpler conditional check of whether the selector map is
empty

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
2020-05-14 11:46:34 -04:00
Abhijeet Kasurde
794d269a4d
Distribution: Add PopOS as Debian OS_FAMILY (#69294)
PopOS is a Debian based OS distribution, added support to detect
ansible_os_family as 'debian' instead of 'Pop!_OS'

Fixes: #69286

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-05-05 11:43:28 -05:00
Clément Notin
9d48884e36
Fix support for Kali Linux detection (#69194)
* Fix support for Kali Linux detection

* Add test for Kali Linux detection

* Improve path matching with "in list"

Co-Authored-By: Abhijeet Kasurde <akasurde@redhat.com>

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-05-05 10:31:32 -05:00
Jordan Borean
79fff7da69
Expose to_<str> as a public function (#68965)
* Expose to_<str> as a public function

* Fix sanity checks

* Move docstring to start of util
2020-04-17 07:54:00 +10:00
Brian Coca
ac509d489b
Revert "stricter permissions on atomic_move when creating new file (#68970)" (#68983)
This reverts commit 566f2467f6.
2020-04-16 12:52:15 -04:00
Brian Coca
566f2467f6
stricter permissions on atomic_move when creating new file (#68970)
fixes #67794
  updated some tests that expected previous defaults
  CVE-2020-1736
2020-04-16 09:06:18 -04:00
Zhanwei Wang
1097694355
get_url pass incorrect If-Modified-Since header(#67417) (#67419)
Fix #67417. HTTP header value of `If-Modified-Since` set by `get_url` does not follow HTTP protocol.
2020-04-10 12:17:55 -05:00
Toshio Kuratomi
184f540056 Add some more unittests for fail_json
Test that order of msg as a keyword arg doesn't matter
2020-04-08 07:56:11 -07:00
Toshio Kuratomi
6531ba38f8 Allow the msg argument to fail_json() to be a positional argument
fial_json() requires a message be given to it to inform the end user of
why the module failed.  Prior to this commit, the message had to be a
keyword argument:

    module.fail_json(msg='Failed due to error')

Since this is a required parameter, this commit allows the message to be
given as a positional argument instead:

   module.fail_json('Failed due to an error')
2020-04-08 07:56:11 -07:00
Rick Elrod
3c3ffc09c2
Fix and add tests for some module_utils.common.validation (#67771)
* Fix test_check_mutually_exclusive exception-checking

Asserting inside of the `with` context of `pytest.raises`
doesn't actually have any effect. So we move the assert
out, using the exception that gets placed into the scope
after we leave the context, and ensure that it actually gets
checked.

This is also what the pytest documentation says to do:
https://docs.pytest.org/en/latest/assert.html#assertions-about-expected-exceptions

Signed-off-by: Rick Elrod <rick@elrod.me>

* Add some tests for check_required_together

Signed-off-by: Rick Elrod <rick@elrod.me>

* use to_native instead of str, for consistency

Signed-off-by: Rick Elrod <rick@elrod.me>

* Add newlines for pep8

Signed-off-by: Rick Elrod <rick@elrod.me>

* Add tests for check_required_arguments

Signed-off-by: Rick Elrod <rick@elrod.me>

* Sort missing keys in error message, since hashes are unsorted and this can be random

Signed-off-by: Rick Elrod <rick@elrod.me>

* Add changelog entry

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-03-26 18:18:56 -05:00
Matt Clay
7e8dae5790 Remove unused unit test files. 2020-03-23 11:14:21 -05:00
Ansible Core Team
e0e57b4beb migration final cleanup 2020-03-23 11:14:21 -05:00
Ansible Core Team
8a3f3e41f8 Migrated to cisco.meraki 2020-03-23 11:14:21 -05:00
Ansible Core Team
3f1cea89fb Migrated to cisco.aci 2020-03-23 11:14:21 -05:00
Ansible Core Team
f0cf2c159b Migrated to netapp.ontap 2020-03-23 11:14:21 -05:00
Ansible Core Team
bdd82adf61 Migrated to netbox.netbox 2020-03-23 11:14:21 -05:00
Ansible Core Team
9a7a574ffc Migrated to ansible.netcommon 2020-03-23 11:14:21 -05:00
Ansible Core Team
6d910034a0 Migrated to ansible.amazon 2020-03-23 11:14:21 -05:00
Ansible Core Team
40218535ee Migrated to community.vmware 2020-03-23 11:14:21 -05:00
Ansible Core Team
0ed8af9021 Migrated to community.crypto 2020-03-23 11:14:21 -05:00
Ansible Core Team
4e488d8435 Migrated to community.general 2020-03-23 11:14:21 -05:00
Matt Clay
2c0be65f44
Relocate netbox module_utils unit tests. (#68056)
* Relocate netbox module_utils unit tests.

This puts them where migration expects them.

* Update sanity ignores.
2020-03-05 16:01:55 -08:00
Matt Clay
7c493577ba
Unit test cleanup to assist with migration. (#67920)
* Move linode unit tests to match module layout.

* Fix location of netapp module_utils unit tests.

* Update sanity ignores.
2020-03-02 10:40:34 -08:00
Matt Clay
8446553953 Fix ACI unit test on Python 3.8+ 2020-03-01 13:59:58 -08:00
Matt Clay
04666c9fa1
Clean up unit tests to assist with migration. (#67875)
* Split out cache plugin unit tests.

* Rename unit tests to match code under test.

* Relocate unit test code to match code under test.

* Another rename.

* Update sanity ignores.
2020-02-28 20:29:41 -08:00
Rick Elrod
ed37efb217
"Fix" test_get_bin_path by changing mock order (#67730)
pytest-mock 2.0.0, when run locally, gets grumpy when os.path.exists is
messed with and then another method is patched afterwards. Likely
something in the pytest-mock chain uses os.path.exists internally, and
since pytest-mock prohibits context-specific patching, there's not a
good solution. For now, just patch os.path.exists last.

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-02-28 10:39:27 -05:00
Mark Chappell
90898132e4
CloudRetry/AWSRetry : Disable catching of NotFound exceptions (#67281)
* CloudRetry/AWSRetry : Remove default catching of NotFound exceptions

* Add docs

* Changelog updates from review

* Update unit tests after removing 'NotFound' from default retries
2020-02-13 16:59:00 -05:00
Rick Elrod
236773bfea
Improve distribution unit tests (#67300)
* Include distro.{os,lsb}_release_info() in distro fixture generation script
* Add distro release information to existing fixtures
* Add some new fixtures for newer distribution versions
* Add fixture for Arch Linux with lsb-release
2020-02-11 16:46:48 -05:00
Felix Fontein
fe454d27a1
Fix removed_in_version to support honor suboptions (#66918)
* Add unit tests.
* Fix reporting for removed_in_version.
* Add changelog.
2020-02-04 10:14:55 -05:00
Sam Doran
5112feeace
Make get_bin_path() always raise an exception (#56813)
This makes it behave in a more idiomatic way

* Fix bug in Darwin facts for free memory
    If the vm_stat command is not found, fact gathering would fail with an unhelpful 
    error message. Handle this gracefully and return a default value for free memory.

* Add unit tests
2020-01-30 12:54:25 -05:00
Rick Elrod
e2a57414f4 Remove with statement for pytest-mock unit tests
As per:
https://github.com/pytest-dev/pytest-mock#note-about-usage-as-context-manager
pytest-mock is not meant to be used within a `with` context or as a
decorator. Instead, pytest-mock will automatically unpatch the mocked
methods when each test is complete.

In newer pytest-mock, this use actually throws an exception and causes
the tests to fail.

This hasn't been hit in Ansible's CI yet, because the docker image
that the tests run in uses an older version of pytest-mock. However,
there is no constraint on the upper bound of pytest-mock in
test/lib/ansible_test/_data/requirements/constraints.txt which means
that when running the tests locally, outside of that docker image, the
tests never pass.

This patch removes the `with` context in each such case.

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-01-29 13:29:40 -08:00
Sam Doran
3461c682c3
Add mechanism for storing warnings and deprecations outside of AnsibleModule (#58993)
* Move warn() and deprecate() methods out of basic.py
* Use _global_warnings and _global_deprications and create accessor functions
    - This lays the foundation for future functions being moved outside of AnsibleModule
       that need an interface to warnings and deprecations without modifying them.
* Add unit tests for new warn and deprecate functions
2020-01-28 12:12:14 -05:00
Sam Doran
617fbad743
synchronize - fix password authentication (#66542)
On Python 2, leave all fds open since there is no mechanism to close specific fds with subprocess.Popen() on Python 2

Add unit tests.

Co-authored-by: Matt Martz <matt@sivel.net>
2020-01-21 10:43:56 -05:00
kaorihinata
3ca4580cb4 Allow no_log=False to silence the no_log warnings for module parameters (#64733)
As AnsibleModule._log_invocation is currently implemented, any parameter
with a name that matches PASSWORD_MATCH triggers the no_log warning as a
precaution against parameters that may contain sensitive data, but have not
been marked as sensitive by the module author.

This patch would allow module authors to explicitly mark the aforementioned
parameters as not sensitive thereby bypassing an erroneous warning message,
while still catching parameters which have not been marked at all by the
author.

Adds tests for various no_log states including True, False, and None (as
extracted by AnsibleModule._log_invocation) when applied to an argument with
a name that matches PASSWORD_MATCH.

Fixes: #49465 #64656
2020-01-09 16:47:57 -05:00
Mads Jensen
0f491c0289 Replace TestCase.assertEquals with TestCase.assertEqual. 2019-11-11 10:24:21 -08:00
Mark Chappell
e6607d611a Sanity test fixups - AWS related tests (#64093)
* AWS tests Sanity Test fixups (add boilerplate)

* Remove sanity ignore.txt entries
2019-10-30 09:48:59 -07:00
Bojan Vitnik
7afba9420a XenServer: Update docs with recent changes in XenAPI python lib and branding (#63728)
- xenserver module_util: XenAPI lib import error message handling is now
   ported to missing_required_lib(). Updated unit tests.
 - xenserver_guest, xenserver_guest_info, xenserver_guest_powerstate modules:
   docs are updated to reflect recent changes in availability of XenAPI lib
   on PyPI.
 - xenserver_guest, xenserver_guest_info, xenserver_guest_powerstate modules:
   docs are updated to reflect recent Citrix rebranding of XenServer. Broken
   URLs to external resources are fixed.
 - xenserver_guest, xenserver_guest_info, xenserver_guest_powerstate modules:
   more tested platforms are mentioned in docs.
2019-10-22 21:04:16 +05:30
Sam Doran
e9d29b1fe4 Properly mask no_log values is sub parameters during failure (#63405)
* Get no_log parameters from subspec

* Add changelog and unit tests

* Handle list of dicts in suboptions

Add fancy error message (this will probably haunt me)

* Update unit tests to test for list of dicts in suboptions

* Add integration tests

* Validate parameters in dict and list

In case it comes in as a string

* Make changes based on feedback, fix tests

* Simplify validators since we only need to validate dicts

Add test for suboptions passed in as strings to ensure they get validated properly and turned into a dictionary.

ci_complete

* Add a few more integration tests
2019-10-14 09:20:07 -07:00
Matt Martz
7f4befdea7
Wrap CLI Passwords with AnsibleUnsafeText, ensure unsafe context is not lost during encode/decode (#63351)
* Wrap .encode and .decode on AnsibleUnsafe objects

* runme.sh needs to be executable

* ci_complete

* Update changelog with CVE
2019-10-11 09:17:10 -05:00
Dag Wieers
7fa1e4f70f Fix sanity checks (#62782)
This PR fixes all new ACI sanity issues.
2019-09-24 09:00:00 -04:00
Gonéri Le Bouder
3ea8e0a144 VMware: not ssl.SSLContext if validate_certs false (#57185)
Python < 2.7.9 does not have the ssl.SSLContext attribute.
ssl.SSLContext is only required when we want to validate the SSL
connection. If `validate_certs` is false, we don't initialize the
`ssl_context` variable.

Add unit-test coverage and a little refactoring:

- avoid the use of `mocker`, when we can push `monkeypatch` which is
  `pytest`'s default.
- use `mock.Mocker()` when possible

closes: #57072
2019-09-09 21:41:46 +05:30
Mark Chappell
1f38a12057 Fix behaviour of module_utils/ec2 compare_policies when dealing with bare bools and ints. (#61115)
* module_utils/ec2: (unit tests) Move unit tests for module_utils/ec2.py into test/units/module_utils

- compare_policies was refactored from s3_bucket
- "ec2_utils" doesn't seem to have ever existed

* module_utils/ec2: (unit tests) Add unit test for comparing quoted and unquoted bools and numbers within policies

As per https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_grammar.html

"Values are enclosed in quotation marks. Quotation marks are optional for numeric
and Boolean values."

* module_utils/ec2: Explicitly convert bools and ints to strings when comparing policies

See also: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_grammar.html
2019-09-09 12:08:25 -04:00
Jill R
d49d52eb5f
Add tests for new alias deprecation functionality (#61476)
* Add tests for the alias deprecation added in #61245
2019-09-05 15:47:42 -07:00