Commit graph

423 commits

Author SHA1 Message Date
Sam Doran
b6811dfb61
Add argument spec validator ()
Add argument spec validator class
2021-02-11 19:17:14 -05:00
Matt Davis
4c5ce5a1a9
module compat for py3.8+ controller ()
* module compat for py3.8+ controller

* replaced internal usages of selinux bindings with internal ctypes binding (allows basic selinux operations from any Python interpreter), plus tests

* added new respawn_module API to allow modules to import Python packages that are only available under a well-known interpreter, plus tests

* added respawn logic to modules that need Python libs from a specific system interpreter (apt, apt_repository, dnf, yum)

minimize internal HAVE_SELINUX usage

spurious junk

pep8

* pylint fixes

* add RHEL8 Python 3.8 testing

* more pylint

* import sanity

* unit tests

* changelog update

* fix a bunch of stuff

* tweak changelog

* fix setup_rpm_repo on EL8

* misc sanity/test fixes

* misc feedback tweaks

* fix import fallback in test module

* fix selinux MU test

* fix dnf tests to avoid python-dependent test packages

* add trailing LFs to aliases

* fix yum tests to avoid test package with Python deps

* hack create_repo for EL6 to create noarch package
2021-02-10 21:32:59 -08:00
Sam Doran
2f5c83dfb1
Add AlmaLinux to the family of Red Hat-like operating systems ()
* Add changelog and fixtures for AlmaLinux support

Co-authored-by: Christoph Schug <com+github@schug.net>
2021-02-09 13:04:39 -05:00
Rick Elrod
9a9272305a
Correct pkg_mgr for Fedora-derived OSTree distros ()
Change:
- Remove check that states that only Fedora can be an OSTree
  distribution.
- This allows us to correctly return "atomic_container" as the pkg_mgr
  fact for RHEL for Edge, Fedora/RHEL/CentOS Atomic Host, etc.

Test Plan:
- Created local RHEL for Edge image and tested against it.
- Tested against regular RHEL 8 and still got `dnf` as expected.
- Tested against RHEL 7 Atomic Host and got `atomic_container` now.
- New unit tests.

Tickets:
- Fixes 

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-02 15:09:30 -05:00
Alexander Sowitzki
218f5c3648
user - Prevent user fact lookup failure if LOGNAME is set () ()
The `UserFactCollector` queries the user login name via
`getpass.getuser()` and looks up the corresponding entry
in the password database.
The login name may differ from the actual user name,
eg. if the `LOGNAME` env variable is set. The lookup
fails in this case. Added a fallback in this case that
tries to get the entry via the user ID.
2021-02-02 10:36:02 -05:00
Alexander Sowitzki
e6da544310 Let command always return stdout & stderr ()
The `command` module does not return stdout & sterr when calling
a non existing executable or an unknown exception arises. This fix
lets the module return empty byte strings in those cases.
2021-01-22 08:40:53 +01:00
Sam Doran
df451636e7
facts - properly report virtualization facts for Linux guests on bhyve () 2021-01-14 10:53:03 -05:00
Rick Elrod
7f0eb7ad79
[facts] Differentiate CentOS vs CentOS Stream ()
Change:
- On CentOS Stream, make distribution_release be "Stream"
- On CentOS Core, it continues to be "Core"
- Implement custom distribution file parser for CentOS, so we can look
  for "CentOS Linux" and "CentOS Stream"
- Two new fixtures introduced (CentOS Linux 8.1 and CentOS Stream 8)
- Removed two dicts from `Distribution` class that were seemingly not
  used anywhere.

Test Plan:
- ci_complete
- New test fixtures

Tickets:
- Fixes 

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-13 17:54:04 -05:00
Alexander Sowitzki
6459fbb7bc Make test_prepare_multipart succeed on non Debian distros
`test_prepare_multipart` fails in non Debian environments since
Debian installations map the file ending `.key` to the MIME type
`application/pgp-keys`, which is not IANA conformant. This commit
explicitly sets the corresponding file type to
`application/octet-stream` and adjusts the expected serialized
result.
2021-01-13 21:21:33 +01:00
Rick Elrod
20509b6507
[facts] fix version facts on FreeBSD RC/PRERELEASE ()
Change:
- The FreeBSD release can contain -RC or -PRERELEASE in addition to
  -RELEASE, -STABLE, or -CURRENT.

Test Plan:
- Added new fixed from an RC version of TrueNAS which uses a -PRERELEASE
  version of FreeBSD.

Tickets:
- Fixes 

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-05 10:16:59 -05:00
Brian Coca
932ba36160
ensure unsafe writes fallback ()
* Ensure we actually fallback to unsafe_writes when set to true

 add integration test
 add fix for get_url not passing the parameter from args
2020-12-21 11:20:52 -05:00
Pilou
b464d18fd1
AnsibleModule.set_mode_if_different: handle symlink is in a sticky directory ()
* file: add symlink is in a sticky directory tests
* file: handle symlink in a sticky directory

Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Add changelog and fix unit test
The builtins import was removed since it was unused, but it is now needed.
2020-12-04 14:21:51 -05:00
Abhijeet Kasurde
a223ea5185
distribution - handle NetBSD OS Family ()
Fixes: 

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-12-01 10:01:25 -05:00
Sam Doran
e889b1063f
arg_spec - rework _check_arguments() ()
* Move _syslog_facitily to __init__
  No good reason it should not be set for each object

* Move internal property setting to private method
* Create check_arguments() function
* Remove unused import
* Rename function to better match its behavior
  Change the behavior to return a set, either empty or populated, with unsupported keys.
  Accept legal_inputs as optional which will not required calling handle_aliases before calling
  get_unsupported_parameters().

* Add changelog
* Rework function behavior and documentation
  I realized I missed the original intent of this method when moving it to a function. It
  is meant to compared the parameter keys to legal inputs always, not compare
  parameter keys to argument spec keys, even though the argument spec keys should
  be a subset of legal inputs.

* Add tests
* Fix typo.
* Set internal properties when handling suboptions
2020-11-18 14:15:33 -05:00
Alexei Znamensky
5654de6fce
Return error if cwd directory does not exist ()
* Return warning or error if cwd directory does not exist, in AnsibleModule.run_command()
2020-11-04 14:33:39 -05:00
Jordan Borean
caba47dd3f
Add support for GSSAPI/Kerberos to urls.py ()
* Add support for GSSAPI/Kerberos to urls.py

* Test out changes with the latest test container

* Get remote hosts working

* Fix up httptester_krb5_password reader

* Fix tests for opensuse and macOS

* Hopefully last lot of testing changes

* Dont do CBT on macOS

* Fixes from review
2020-10-13 14:16:07 +10:00
Rick Elrod
709484969c
sysctl/openbsd fact fixes ()
Change:
- Use `sysctl -n` for openbsd uptime information
- Allow `get_sysctl()` to account for multi-line sysctl settings
- Add unit tests for `get_sysctl()`

Test Plan:
- New unit tests

Tickets:
- Fixes 
- Refs 
- Refs 

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Brian Coca <brian.coca+git@gmail.com>
Co-authored-by: Brian Coca <bcoca@ansible.com>
2020-10-02 18:36:31 -05:00
Christian Loos
ea119d3089
fix distribution fact for SLES4SAP ()
b6b238a fixed the SLES4SAP detection, which was at this time ok.
Sadly Suse changed with SLES 15 the /etc/os-release file, so the above
change will no longer work.

This commit updates the SLES4SAP detection regarding
https://www.suse.com/support/kb/doc/?id=000019341.

The symlink realpath is matched with endswith, because in SLES 12+ the
link target is SLES_SAP.prod, but in SLES 11 the link target is
SUSE_SLES_SAP.prod.
2020-09-30 10:52:41 -04:00
Jorge Vallecillo
94522b7c10
Fix typos () 2020-09-28 14:33:24 -04:00
Rick Elrod
5cd489af06
Let get_file_attributes() work without lsattr -v ()
* Let get_file_attributes() work without `lsattr -v`

Change:
- module_utils's get_file_attributes() expects `lsattr -v` to work, but
  in some cases, it may not.
- The function now takes an optional include_version bool parameter,
  which removes this expectation.
- Places where we call get_file_attributes() without using the 'version'
  it returns, we now call it with include_version=False.

Test Plan:
- New unit tests

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-09-22 16:15:47 -05:00
Abhijeet Kasurde
173091e2e3
distro: Add support for Pardus distribution ()
Fixes: 

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-09-15 09:47:24 -04:00
Matt Martz
48f12c14e9
AnsibleVaultEncryptedUnicode should be considered a string ()
* AnsibleVaultEncryptedUnicode should be considered a string

* linting fix

* clog frag
2020-09-03 14:54:00 -05:00
Sam Doran
888be697cb
Revert "Change default file permissions so they are not world readable ()" ()
* Revert "Change default file permissions so they are not world readable ()"

This reverts commit 5260527c4a.

* Revert "Fix warning for new default permissions when mode is not specified ()"

This reverts commit dc79528cc6.
2020-08-12 12:29:04 -05:00
Sam Doran
c4f442ed5a
facts - fix incorrect time for some date_time_facts ()
The iso8601_micro and iso8601 facts incorrectly called now.utcnow(), resulting
in a new timestamp at the time it was called, not a conversion of the previously
stored timestamp.

Correct this by capturing the UTC timestamp once then calculating the local
time using the UTC offset of the current system.

* Use time.time() for getting the current time
* Convert from that stored epoch timestamp to local and UTC times
* Used existing timestamp for epoch time
* Add unit tests that validate the formate of the return value rather than an exact value since mocking time and timezone is non-trivial
2020-07-30 11:39:58 -04:00
Abhijeet Kasurde
e439194c8c
basic: Add name of aliases in error message ()
With this patch, user will be notified with available aliases
of arg parameter.

Fixes: 

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-23 16:02:18 +05:30
Sam Doran
5260527c4a
Change default file permissions so they are not world readable ()
* Change default file permissions so they are not world readable

CVE-2020-1736

Set the default permissions for files we create with atomic_move() to 0o0660. Track
which files we create that did not exist and warn if the module supports 'mode'
and it was not specified and the module did not call set_mode_if_different(). This allows the user to take action and specify a mode rather than using the defaults.

A code audit is needed to find all instances of modules that call atomic_move()
but do not call set_mode_if_different(). The findings need to be documented in
a changelog since we are not warning. Warning in those instances would be frustrating
to the user since they have no way to change the module code.

- use a set for storing list of created files
- just check the argument spac and params rather than using another property
- improve the warning message to include the default permissions
2020-07-22 17:05:38 -04:00
David Shrewsbury
bf98f031f3
Sanitize URI module keys with no_log values ()
* Add sanitize_keys() to module_utils.

* More robust tests

* Revert 69653 change

* Allow list or dict

* fix pep8

* Sanitize lists within dict values

* words

* First pass at uri module

* Fix insane sanity tests

* fix integration tests

* Add changelog

* Remove unit test introduced in 69653

* Add ignore_keys param

* Sanitize all-the-things

* Ignore '_ansible*' keys

* cleanup

* Use module.no_log_values

* Avoid deep recursion issues by using deferred removal structure.

* Nit cleanups

* Add doc blurb

* spelling

* ci_complete
2020-07-22 14:49:37 -05:00
Abhijeet Kasurde
4f96f9826c
distribution: Add support for DragonFly ()
partially fixes 

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-21 13:47:11 -04:00
Eduard Rozenberg
566c5e6ce1
Handle Slackware OS version strings containing a plus (“+”) ()
A couple of years ago Slackware -current began using a plus (“+”) at the end of the distribution version string to indicate a future version work-in-progress.

Rearrange distribution_files unit tests to easily support more tests
  - add conftest with common fixtures 
  - use parametrize for testing multiple scenarios

* Add changelog
* Add unit tests for Slackware distribution parsing
* Use correct fixtures for Slackware
Data comes from /etc/slackware-version

Co-authored-by: Sam Doran <sdoran@redhat.com>
Co-authored-by: <Eduard Rozenberg <eduardr@pobox.com>>
2020-07-17 16:39:26 -04:00
Abhijeet Kasurde
055871cbb8
api: time.clock compatible code ()
time.clock is removed in Python 3.8. Add time.clock
compatible code.

Fixes: 

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-16 12:24:22 +05:30
Mykola Grygoriev
fe86a93482
Add a new date_time fact to provide DST timezone
PR 
2020-07-14 18:22:51 +02:00
jabdr
0690b68bd3
Support datetime.date object in module result ()
* Support datetime.date object in module result

Fixes 

* change blank lines for pep8 sanity test
2020-07-14 11:42:40 -04:00
Mark Chappell
8d160b1881
Remove remaining examples of 1.1.1.1 ()
* Remove remaining examples of 1.1.1.1

* Update ec2_group.py
2020-07-14 11:19:59 -04:00
Abhijeet Kasurde
a8ae8f4d72
setup: Handle CIFS share with backward slash ()
CIFS can be mounted using backward slash as well in /etc/fstab like

\\Windows\share /data/ cifs credentials=/root/.creds 0 0

Handle this condition while gather mount information in Linux.

Fixes: 

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-06-30 16:09:35 -04:00
s-hamann
e39a9bf583
Add support for Parrot Linux, a Debian derivate ()
Co-authored-by: black <invalid>
2020-06-29 20:22:44 +05:30
jctanner
b019029bf3
Add intentional unit tests for basic._set_cwd and common.dict_merge ()
* Add unit tests for basic._set_cwd

* incidental coverage for dict_merge

* add test for async stderr inclusion
2020-06-25 14:45:13 -05:00
Matt Clay
98a0995fd0 Clean up unit test boilerplate. 2020-06-22 14:20:33 -07:00
Sam Doran
bc05415109
Only pass kwargs to our string checker not callable checkers ()
Since only check_type_str() accepts extra param, only pass to our checker and
do not pass kwargs to custom checkers.

* Add unit tests
2020-06-19 09:52:05 -04:00
Felix Fontein
a862ff2d43
Deprecation revisited ()
* Allow to specify collection_name separately for deprecation.

* Use new functionality in Ansible.

* Use new functionality in tests.

* Update tagging/untagging functions.

* Update pylint deprecated sanity test.

* Update validate-modules. Missing are basic checks for version_added (validate semantic version format for collections).

* Improve version validation. Re-add version_added validation.

* Make sure collection names are added to return docs before schema validation.

* Extra checks to avoid crashes on bad data.

* Make C# module utils code work, and update/extend tests.

* Add changelog fragment.

* Stop extracting collection name from potentially tagged versions/dates.

* Simplify C# code.

* Update Windows modules docs.

* Forgot semicolons.
2020-06-09 15:21:19 -07:00
Dongsu Park
598e3392a9
Discover Flatcar Linux properly for hostname ()
To avoid issues with Flatcar Container Linux being unable to be found,
detect Flatcar distro name especially for hostname, just like CoreOS
Container Linux was supported.

See also https://github.com/ansible/ansible/issues/69516
2020-06-02 18:41:53 +05:30
Abhijeet Kasurde
7b3d9cbcb8
distribution: Added support for openEuler OS ()
Detect os_family for openEuler OS as 'RedHat', instead of 'openEuler'.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-05-29 12:43:16 +05:30
Felix Fontein
40f21dfd3c
Version source tagging (automatic and manual) for version_added and deprecation versions ()
* Track collection for version_added.
Validate *all* version numbers in validate-modules.
For tagged version numbers (i.e. version_added), consider source collection to chose validation.

* Make tagging/untagging functions more flexible.

* Tag all versions in doc fragments.

* Tag all deprecation versions issued by code.

* Make Display.deprecated() understand tagged versions.

* Extend validation to enforce tagged version numbers.

* Tag versions in tests.

* Lint and fix test.

* Mention collection name in collection loader's deprecation/removal messages.

* Fix error IDs.

* Handle tagged dates in Display.deprecated().

* Also require that removed_at_date and deprecated_aliases.date are tagged.

* Also automatically tag/untag removed_at_date; fix sanity module removal version check.

* Improve error message when invalid version number is used (like '2.14' in collections).
2020-05-28 22:46:16 -07:00
Abhijeet Kasurde
564907d8ac
distribution: Refactor test_distribution_version testcases () 2020-05-27 14:01:31 +05:30
Felix Fontein
ea04e0048d
Allow to deprecate options and aliases by date ()
* Allow to deprecate options and aliases by date instead of only by version.

* Update display.deprecate().

* Adjust behavior to conform to tested behavior, extend tests, and improve C# style.

* Parse date and fail on invalid date.

This is mainly to make sure that people start using invalid dates, and we eventually have a mess to clean up.

* C# code: improve validation and update/extend tests.

* Make sure that deprecate() is not called with both date and version.

* Forgot to remove no longer necessary formatting.

* Adjust order of warnings in C# code.

* Adjust unrelated test.

* Fix grammar (and make that test pass).

* Don't parse date, and adjust message to be same as in .

* Sanity tests: disable date in past test.

* Validate-modules: validate ISO 8601 date format.

* Validate-modules: switch schema declaration for deprecated_aliases to improve error messages for invalid dates.

* Use DateTime instead of string for date deprecation.

* Validate that date in deprecated_aliases is actually a DateTime.

* Fix tests.

* Fix rebasing error.

* Adjust error codes for pylint, and add removed_at_date and deprecated_aliases.date checks to validate-modules.

* Make deprecation date in the past error codes optional.

* Make sure not both version and date are specified for AnsibleModule.deprecate() calls.

* Stop using Python 3.7+ API.

* Make sure errors are actually reported. Re-add 'ansible-' prefix.

* Avoid crashing when 'name' isn't there.

* Linting.

* Update lib/ansible/module_utils/csharp/Ansible.Basic.cs

Co-authored-by: Jordan Borean <jborean93@gmail.com>

* Adjust test to latest change.

* Prefer date over version if both end up in Display.deprecated().

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2020-05-26 20:23:56 -04:00
Matt Martz
e0f25a2b1f
Strip no log values from module response keys. Fixes () 2020-05-21 16:17:57 -04:00
Matt Martz
b748edea45
Add multipart/form-data functionality ()
* Add multipart/form-data functionality

* Fix some linting issues

* Fix error message

* Allow filename to be provided with content

* Add integration test

* Update examples

* General improvements to multipart handling

* Use prepare_multipart for galaxy collection publish

* Properly account for py2 vs py3, ensuring no max header length

* Address test assumptions

* Add unit tests

* Add changelog

* Ensure to use CRLF instead of NL

* Ignore line-endings in fixture

* Consolidate code, add comment

* Bump fallaxy container version

* ci_complete
2020-05-20 15:44:01 -05:00
Lukas Pirl
34db57a47f
introduce fact "ansible_processor_nproc": number of usable vcpus ()
This fact reflects the number of usable vcpus (which might be different
from ansible_processor_vcpus, e.g., in containers with limits). See
also .

* Add fixture data and update unit tests

Co-authored-by: Sam Doran <sdoran@redhat.com>
2020-05-15 09:38:56 -04:00
Matt Clay
d0484fbf74 Cleanup for Python 3.9 and pycodestyle compat. 2020-05-14 21:06:16 -07:00
Bob Weinand
f200487414
Fix filedescriptor out of range in select() when running commands ()
* Fix filedescriptor out of range in select() when running commands

* Simplify the run_command() code

Now that we're using selectors in run_command(), we can simplify some of
the code.

* Use fileobj.read() instead of os.read()
* No longer use get_buffer_size() as we can just slurp all of the data
  instead.

Also use a simpler conditional check of whether the selector map is
empty

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
2020-05-14 11:46:34 -04:00
Abhijeet Kasurde
794d269a4d
Distribution: Add PopOS as Debian OS_FAMILY ()
PopOS is a Debian based OS distribution, added support to detect
ansible_os_family as 'debian' instead of 'Pop!_OS'

Fixes: 

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-05-05 11:43:28 -05:00