Commit graph

2990 commits

Author SHA1 Message Date
Abhijeet Kasurde
3811fddede Amazon: Fix distribution facts for older release
Ansible can gather distribution facts for older Amazon Linux
with /etc/os-release data.

Fixes: #73946

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-03-18 12:42:04 -07:00
Matt Martz
78f34786dd
Send callbacks directly from the TaskExecutor instead of TaskResults masquerading as callbacks (#73927) 2021-03-18 12:12:29 -07:00
Abhijeet Kasurde
fb66b4ffbc
distribution: Add Amazon Linux distribution facts (#73767)
* Update Amazon Linux Distribution facts gathering logic
* Update tests

Fixes: #73742

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-03-05 09:16:10 -05:00
Brian Coca
935528e22e
finish migrating ssh plugin to config system (#73708)
* finish migrating ssh plugin to config system

  fixes #72739
  fixes #57220

* fix connection detection in reset
* correct options for connection meta reset

Co-authored-by: David Shrewsbury <Shrews@users.noreply.github.com>
2021-03-03 15:25:16 -05:00
Matt Davis
d0e991e892
Clarify CLI version number as core version (#72287)
* clarify CLI version number as core version

* reduce confusion with `ansible` PyPI package >= 2.10 drifting from core version

* fix units
2021-03-01 11:14:56 -06:00
Alexei Znamensky
920b68f5f2
Fixed/improved regular expresssion for collection names (#73577)
* added changelog fragment
* added a couple of tests to coll name validation
2021-03-01 10:40:38 -05:00
Sam Doran
2377a0a776
Improve ArgumentSpecValidator unit tests (#73642)
* Add more scenarios to basic valid testing
* Update invalid tests
* Fix test for Python 2
* Condense data
* Add tests for missing required and invalid-elements
* Update aliases tests
* Add invalid scenarios for aliases
* Add tests for _add_error() method
* Fix sanity test failure
2021-02-26 09:30:16 -05:00
Sloane Hertel
00bd0b893d
ansible-galaxy - set the cache file after getting all collection versions (#73557)
* Manage the in-memory cache in _call_galaxy but let the caller set the file cache after getting paginated results

* Add a test for caching successful and not caching unsuccessful paginated results

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-02-15 09:45:01 -05:00
Sviatoslav Sydorenko
f327e65d11
Use iskeyword and str.isidentifier for "is FQCN" (#73279)
* Use valid FQCN in test_verbosity_arguments

* Use iskeyword and str.isidentifier for "is FQCN"
2021-02-11 18:23:57 -08:00
Sam Doran
b6811dfb61
Add argument spec validator (#73335)
Add argument spec validator class
2021-02-11 19:17:14 -05:00
Brian Coca
ec8a556538
only add data when there is data to add (#54559)
Only add data when there is data to add

  also avoid clobbering existing data with empty file
  fixes #45843

* remove redundant code, update comments
* fix mock dataloader, original does not return None
* added test
2021-02-11 10:07:20 -05:00
Matt Davis
4c5ce5a1a9
module compat for py3.8+ controller (#73423)
* module compat for py3.8+ controller

* replaced internal usages of selinux bindings with internal ctypes binding (allows basic selinux operations from any Python interpreter), plus tests

* added new respawn_module API to allow modules to import Python packages that are only available under a well-known interpreter, plus tests

* added respawn logic to modules that need Python libs from a specific system interpreter (apt, apt_repository, dnf, yum)

minimize internal HAVE_SELINUX usage

spurious junk

pep8

* pylint fixes

* add RHEL8 Python 3.8 testing

* more pylint

* import sanity

* unit tests

* changelog update

* fix a bunch of stuff

* tweak changelog

* fix setup_rpm_repo on EL8

* misc sanity/test fixes

* misc feedback tweaks

* fix import fallback in test module

* fix selinux MU test

* fix dnf tests to avoid python-dependent test packages

* add trailing LFs to aliases

* fix yum tests to avoid test package with Python deps

* hack create_repo for EL6 to create noarch package
2021-02-10 21:32:59 -08:00
Sam Doran
2f5c83dfb1
Add AlmaLinux to the family of Red Hat-like operating systems (#73541)
* Add changelog and fixtures for AlmaLinux support

Co-authored-by: Christoph Schug <com+github@schug.net>
2021-02-09 13:04:39 -05:00
Shane McDonald
67f5bb39c7
galaxy - add format options for collection list (#73474)
* Include all collections in single json object / yaml document
* Add tests
* For galaxy list yaml/json output, use dictionary of dictionaries instead of list
* Add tests for listing single collection in yaml / output format
* --output -> --format
* Add explicit test for listing collection in human format
* Fix bug where empty json object was emitted + add test
2021-02-05 13:24:59 -05:00
Sloane Hertel
9759e0ca49
add option to upgrade collections (#73336)
* Add a flag to ansible-galaxy to update collections


Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-02-04 07:08:02 +10:00
Rick Elrod
9a9272305a
Correct pkg_mgr for Fedora-derived OSTree distros (#73445)
Change:
- Remove check that states that only Fedora can be an OSTree
  distribution.
- This allows us to correctly return "atomic_container" as the pkg_mgr
  fact for RHEL for Edge, Fedora/RHEL/CentOS Atomic Host, etc.

Test Plan:
- Created local RHEL for Edge image and tested against it.
- Tested against regular RHEL 8 and still got `dnf` as expected.
- Tested against RHEL 7 Atomic Host and got `atomic_container` now.
- New unit tests.

Tickets:
- Fixes #73084

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-02 15:09:30 -05:00
Alexander Sowitzki
218f5c3648
user - Prevent user fact lookup failure if LOGNAME is set (#17029) (#73439)
The `UserFactCollector` queries the user login name via
`getpass.getuser()` and looks up the corresponding entry
in the password database.
The login name may differ from the actual user name,
eg. if the `LOGNAME` env variable is set. The lookup
fails in this case. Added a fallback in this case that
tries to get the entry via the user ID.
2021-02-02 10:36:02 -05:00
Sviatoslav Sydorenko
595413d113
Replace the inhouse collection dependency resolver with resolvelib
PR #72591

This change:

  * Adds an artifacts manager that abstracts away extracting the
    metadata from artifacts, downloading and caching them in a
    temporary location.

  * Adds `resolvelib` to direct ansible-core dependencies[0].

  * Implements a `resolvelib`-based dependency resolver for
    `collection` subcommands that replaces the legacy
    in-house code.

    This is a dependency resolution library that pip 20.3+ uses
    by default. It's now integrated for use for the collection
    dependency resolution in ansible-galaxy CLI.

  * Refactors of the `ansible-galaxy collection` CLI.
    In particular, it:

      - reimplements most of the `download`, `install`, `list` and
        `verify` subcommands from scratch;

      - reuses helper bits previously moved out into external modules;

      - replaces the old in-house resolver with a more clear
        implementation based on the resolvelib library[0][1][2].

  * Adds a multi Galaxy API proxy layer that abstracts accessing the
    version and dependencies via API or local artifacts manager.

  * Makes `GalaxyAPI` instances sortable.

  * Adds string representation methods to `GalaxyAPI`.

  * Adds dev representation to `GalaxyAPI`.

  * Removes unnecessary integration and unit tests.

  * Aligns the tests with the new expectations.

  * Adds more tests, integration ones in particular.

[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing

Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>

Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
2021-01-27 22:23:22 +01:00
Brian Coca
2b0cd2c13f
use correct executable and options in all cases (#73323)
Use correct ssh executable and options in all cases on connection plugin

  * Also nicer naming/comments

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-01-26 20:21:24 -05:00
Alexander Sowitzki
e6da544310 Let command always return stdout & stderr (#73004)
The `command` module does not return stdout & sterr when calling
a non existing executable or an unknown exception arises. This fix
lets the module return empty byte strings in those cases.
2021-01-22 08:40:53 +01:00
Joshua Bayfield
823c72bcb5
Shadow input for encrypt_string by default unless asked (fixes #71618) (#73263)
* Shadow input for encrypt_string by default unless asked (fixes #71618)
2021-01-20 15:50:24 -05:00
Jordan Borean
bc60d8ccda
Galaxy - make versions list consistent across versions (#72932)
* Galaxy - make versions list consistent across versions

* Fix up unit tests
2021-01-20 21:19:29 +01:00
Sam Doran
e8d4b62b41
Fix YAML error message when error is at the end of the file (#73241)
* Fix YAML error message when error is at the end of the file

If a YAML file fails to load due to a syntax error in a file, or there is an error in the last line of a
file, PyYAML reports the last line number of the file as the index where the error occurred.

When reading the file lines, we use that index to the get the relevant line.  If the index value is out
of range, the relevant line is lost for error reporting.

Subtract one from the index value to avoid the IndexError in this specific scenario. It is possible
to still get an IndexError, which will be handled as it is currently.

* Update existing tests and add new tests
2021-01-19 10:07:36 -05:00
Sam Doran
df451636e7
facts - properly report virtualization facts for Linux guests on bhyve (#73204) 2021-01-14 10:53:03 -05:00
Rick Elrod
7f0eb7ad79
[facts] Differentiate CentOS vs CentOS Stream (#73034)
Change:
- On CentOS Stream, make distribution_release be "Stream"
- On CentOS Core, it continues to be "Core"
- Implement custom distribution file parser for CentOS, so we can look
  for "CentOS Linux" and "CentOS Stream"
- Two new fixtures introduced (CentOS Linux 8.1 and CentOS Stream 8)
- Removed two dicts from `Distribution` class that were seemingly not
  used anywhere.

Test Plan:
- ci_complete
- New test fixtures

Tickets:
- Fixes #73027

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-13 17:54:04 -05:00
Alexander Sowitzki
3e8c8cd536 Make test_adhoc succeed from within a git checkout
The test currently only expects the literal `ansible` followed
by a semver in the first output line of `ansible --version`.
When running from within a git checkout, additional information
like the currently checkout branch are attached, which lets
the test fail. This commit allows arbitrary information to
follow the semver.
2021-01-13 21:22:21 +01:00
Alexander Sowitzki
6459fbb7bc Make test_prepare_multipart succeed on non Debian distros
`test_prepare_multipart` fails in non Debian environments since
Debian installations map the file ending `.key` to the MIME type
`application/pgp-keys`, which is not IANA conformant. This commit
explicitly sets the corresponding file type to
`application/octet-stream` and adjusts the expected serialized
result.
2021-01-13 21:21:33 +01:00
Martin Zimmermann
be59caa59b
iptables: Added set module/match_set support (#72984) 2021-01-11 16:32:53 -05:00
Abhijeet Kasurde
25792d2388
galaxy: handle plus sign in user token appearing in role url (#73057)
Fixes: #45475

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-01-06 10:50:44 -05:00
Rick Elrod
20509b6507
[facts] fix version facts on FreeBSD RC/PRERELEASE (#73020)
Change:
- The FreeBSD release can contain -RC or -PRERELEASE in addition to
  -RELEASE, -STABLE, or -CURRENT.

Test Plan:
- Added new fixed from an RC version of TrueNAS which uses a -PRERELEASE
  version of FreeBSD.

Tickets:
- Fixes #72331

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-05 10:16:59 -05:00
Brian Coca
932ba36160
ensure unsafe writes fallback (#70722)
* Ensure we actually fallback to unsafe_writes when set to true

 add integration test
 add fix for get_url not passing the parameter from args
2020-12-21 11:20:52 -05:00
Brian Coca
d22804c4fb
saner path dir management (#72648)
* saner path dir management

   fixes #72628

   ensure we always store paths w/o a_c

Co-authored-by: Sloane Hertel <19572925+s-hertel@users.noreply.github.com>
2020-12-14 19:30:13 -05:00
Martin Zimmermann
83fb24b923
iptables: Adding multiport module support (#72928) 2020-12-14 10:23:54 -05:00
Martin Krizek
1b70260d5a
Add implicit role_complete block instead of role._eor (#72208)
Co-authored-by: Matt Martz <matt@sivel.net>

Fixes #69848
2020-12-09 09:53:01 +01:00
Sven Wegener
08cc6edc64
vault: Read stdin data as binary on python3 (#52229)
On python3 sys.stdin is an encoded file object that does not support
reading raw binary data. Use the supplied buffer object to do so.

Signed-off-by: Sven Wegener <sven.wegener@inovex.de>

Co-authored-by: Sven Wegener <sven.wegener@inovex.de>
2020-12-09 09:46:32 +01:00
David Shrewsbury
be2c376ab8
Extract embedded function to RoleMixin method and add tests (#72754)
* Add changelog
* Simplify return
2020-12-07 13:07:23 -05:00
Pilou
b464d18fd1
AnsibleModule.set_mode_if_different: handle symlink is in a sticky directory (#45198)
* file: add symlink is in a sticky directory tests
* file: handle symlink in a sticky directory

Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Add changelog and fix unit test
The builtins import was removed since it was unused, but it is now needed.
2020-12-04 14:21:51 -05:00
Abhijeet Kasurde
a223ea5185
distribution - handle NetBSD OS Family (#70799)
Fixes: #43739

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-12-01 10:01:25 -05:00
Martin Krizek
ae08c6a639
Ensure Ansible's unique filter preserves order (#67856)
Fixes #63417
2020-11-23 08:55:18 +01:00
Matt Martz
f8ef34672b
Provide better decryption errors for single vault values (#72362)
Fixes #72276
Fixes #72281
2020-11-18 14:25:32 -06:00
Sam Doran
e889b1063f
arg_spec - rework _check_arguments() (#72447)
* Move _syslog_facitily to __init__
  No good reason it should not be set for each object

* Move internal property setting to private method
* Create check_arguments() function
* Remove unused import
* Rename function to better match its behavior
  Change the behavior to return a set, either empty or populated, with unsupported keys.
  Accept legal_inputs as optional which will not required calling handle_aliases before calling
  get_unsupported_parameters().

* Add changelog
* Rework function behavior and documentation
  I realized I missed the original intent of this method when moving it to a function. It
  is meant to compared the parameter keys to legal inputs always, not compare
  parameter keys to argument spec keys, even though the argument spec keys should
  be a subset of legal inputs.

* Add tests
* Fix typo.
* Set internal properties when handling suboptions
2020-11-18 14:15:33 -05:00
Jordan Borean
de5858f48d
Added caching mechanism for Galaxy API requests (#71904)
* Added caching mechanism for Galaxy API requests

* Add cache options and split up code

* Added unit tests

* Fix sanity test

* Use modified date and fix up caching for explicit servers

* Make sure credentials are not in cached server name

* Added test for getting updated cache version

* Changes from review
2020-11-10 06:50:32 +10:00
Amin Vakil
c1da427a5e
iptables: Reorder comment postition (#71496)
* Reorder comment postition

* Add comment unit test

* Fix unit test

* Fix unit test

* Add changelog

* Add paramaters which would be problematic without this fix

* Fix typo

* Fix unit test

* Fix unit test
2020-11-09 19:40:55 +01:00
Martin Krizek
a2593b5e27 CLI - Specify jinja version in --version output 2020-11-05 11:37:55 -08:00
Alexei Znamensky
5654de6fce
Return error if cwd directory does not exist (#72390)
* Return warning or error if cwd directory does not exist, in AnsibleModule.run_command()
2020-11-04 14:33:39 -05:00
Brian Coca
cb94c0cc55
added timeout options to adhoc and console (#71230)
* added timeout options to adhoc and console

* added test

* fix typosesz

* fix conflict

* task_timeout

* fix timeout option, added extra vars to console

* actually use right cli switch .. DUH!

* added timeout to include 'valid' but ignored keys

* fix default

* fixes per review
2020-10-30 10:09:30 -04:00
Brian Coca
e05c62547b
run playbook from collections (#67435)
* fixes for collection playbooks

 - add fqcn invocation, also handles extensions
 - bring import_playbook into new normal
 - avoid adding collection playbooks to adjacency
 - deal with extensions or lack of em
 - added tests
 - fix bugs with multiple playbooks supplied
 - unicode/bytes/native fixes
 - fix modulenotfound for < py3.7
 - tests
2020-10-28 11:46:39 -04:00
Sam Doran
8f9cf456b0
url lookup - set default user agent (#72324)
* Add unit tests
* Add note about when default changed
2020-10-28 09:34:06 -04:00
Sloane Hertel
8b07d46166
Add a toggle for the leading separator for keyed_groups (#60882)
* Add a toggle for the leading separator for keyed_groups if no prefix is given

* changelog

* Add tests for backward compatibility and the new toggle
2020-10-23 15:57:44 -04:00
Matt Davis
83909bfa22
Remove ansible-galaxy login (#72288)
* GitHub is removing the underlying API used to implement the `login` command. Since the general consensus seems to be that relatively nobody currently uses this command (in favor of explicit token passing), support was simply removed for interactive login. If a future need arises, this command should be reimplemented via OAuth Device Auth Grants.
* login or role login commands now produce a fatal error with a descriptive message
* updated 2.10 and 2.11 porting guide entries

* remove dead code/config, update messages and porting guides
2020-10-23 09:11:45 -07:00
Rick Elrod
3b40c6f3b7
[config] coerce more to string when 'type: str' (#72172)
Change:
- When a plugin defines `type: str` on a parameter, treat more kinds of
  input as a string instead of whatever it is parsed as.

Test Plan:
- New unit tests
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-10-14 12:53:08 -04:00
dberg1
cdf62edc65
service_facts: return service state information on AIX (#72073)
* service_facts: return service state information on AIX

AIX uses the System Resource Controller (SRC) to manage services.
See https://www.ibm.com/support/knowledgecenter/ssw_aix_72/osmanagement/sysrescon.htm
Use lssrc command on AIX to return service state information.


Co-authored-by: dberg1 <dberg1@github.com>
2020-10-13 11:28:59 -04:00
Jordan Borean
caba47dd3f
Add support for GSSAPI/Kerberos to urls.py (#72113)
* Add support for GSSAPI/Kerberos to urls.py

* Test out changes with the latest test container

* Get remote hosts working

* Fix up httptester_krb5_password reader

* Fix tests for opensuse and macOS

* Hopefully last lot of testing changes

* Dont do CBT on macOS

* Fixes from review
2020-10-13 14:16:07 +10:00
Rick Elrod
0332046699
[ansible-test] attempt to work around podman (#72096)
Change:
- podman > 2 && < 2.2 does not support "images --format {{json .}}"
- podman also now outputs images JSON differently than docker
- Work around both of the above.

Test Plan:
- Tested with podman 2.0.6 in Fedora 31.

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2020-10-05 20:13:52 -05:00
Evan Van Dam
3db08adbb1
Add optional attribute arg for min and max filters (#50909)
* Pass **kwargs to min and max filters

* Use the jinja2 filters if available

* Add unit tests

* Add examples to docs passing attribute
2020-10-05 10:40:37 -04:00
Rick Elrod
709484969c
sysctl/openbsd fact fixes (#72070)
Change:
- Use `sysctl -n` for openbsd uptime information
- Allow `get_sysctl()` to account for multi-line sysctl settings
- Add unit tests for `get_sysctl()`

Test Plan:
- New unit tests

Tickets:
- Fixes #71968
- Refs #72025
- Refs #72067

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Brian Coca <brian.coca+git@gmail.com>
Co-authored-by: Brian Coca <bcoca@ansible.com>
2020-10-02 18:36:31 -05:00
Matt Martz
fb03ac7019
Add libyaml info to version output, and restore git info (#71948)
* Add libyaml info to version, restore git functionality

* Add clog fragment

* Update tests to reflect new version output
2020-10-02 14:04:49 -05:00
Felix Fontein
1107aace1b
Add deprecation collection name to plugin options (#71735)
* Add changelog fragment.

* Use correct field that's expected by lib/ansible/cli/__init__.py..

* Add basic unit tests.
2020-10-02 09:43:23 -04:00
Christian Loos
ea119d3089
fix distribution fact for SLES4SAP (#71559)
b6b238a fixed the SLES4SAP detection, which was at this time ok.
Sadly Suse changed with SLES 15 the /etc/os-release file, so the above
change will no longer work.

This commit updates the SLES4SAP detection regarding
https://www.suse.com/support/kb/doc/?id=000019341.

The symlink realpath is matched with endswith, because in SLES 12+ the
link target is SLES_SAP.prod, but in SLES 11 the link target is
SUSE_SLES_SAP.prod.
2020-09-30 10:52:41 -04:00
Jorge Vallecillo
94522b7c10
Fix typos (#71970) 2020-09-28 14:33:24 -04:00
Rick Elrod
5cd489af06
Let get_file_attributes() work without lsattr -v (#71845)
* Let get_file_attributes() work without `lsattr -v`

Change:
- module_utils's get_file_attributes() expects `lsattr -v` to work, but
  in some cases, it may not.
- The function now takes an optional include_version bool parameter,
  which removes this expectation.
- Places where we call get_file_attributes() without using the 'version'
  it returns, we now call it with include_version=False.

Test Plan:
- New unit tests

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-09-22 16:15:47 -05:00
Abhijeet Kasurde
173091e2e3
distro: Add support for Pardus distribution (#71663)
Fixes: #71636

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-09-15 09:47:24 -04:00
Matt Davis
e813b0151c
fix coverage output from synthetic packages (#71727)
* fix coverage output from synthetic packages

* synthetic packages (eg, implicit collection packages without `__init__.py`) were always created at runtime with empty string source, which was compiled to a code object and exec'd during the package load. When run with code coverage, it created a bogus coverage entry (since the `__synthetic__`-suffixed `__file__` entry didn't exist on disk).
* modified collection loader `get_code` to preserve the distinction between `None` (eg synthetic package) and empty string (eg empty `__init__.py`) values from `get_source`, and to return `None` when the source is `None`. This allows the package loader to skip `exec`ing things that truly have no source file on disk, thus not creating bogus coverage entries, while preserving behavior and coverage reporting for empty package inits that actually exist.

* add unit test
2020-09-14 09:14:23 -07:00
Matt Martz
48f12c14e9
AnsibleVaultEncryptedUnicode should be considered a string (#71609)
* AnsibleVaultEncryptedUnicode should be considered a string

* linting fix

* clog frag
2020-09-03 14:54:00 -05:00
Martin Krizek
fbdfb85c20
Remove duplicate unit test (#71488)
test_dict_undefined_values is a duplicate of test_dict_undefined_values_is_defined.
It seems originally test_dict_undefined_values was supposed
to do what test_dict_undefined_values_bare does so it is safe to remove.
2020-08-27 15:55:30 -04:00
Matt Martz
fdf5dd02b3
Updates for password hashing (#71120)
* Validate salt when using crypt. Respect salt_size in password lookup. Repair salt for bcrypt. Fixes #71107. Fixes #53750. Fixes #36129.

* Handle algorithms we don't know about, and make sure to return the salt

* Account for old passlib

* Add tests for salt constraints

* Add changelog fragment

* Add test for #36129
2020-08-26 14:54:38 -05:00
Jordan Borean
8897d7e2ff
powershell - fix nested CLIXML parser (#71412) 2020-08-26 07:06:19 +10:00
Sam Doran
888be697cb
Revert "Change default file permissions so they are not world readable (#70221)" (#71231)
* Revert "Change default file permissions so they are not world readable (#70221)"

This reverts commit 5260527c4a.

* Revert "Fix warning for new default permissions when mode is not specified (#70976)"

This reverts commit dc79528cc6.
2020-08-12 12:29:04 -05:00
Toshio Kuratomi
fb144c4414
Update ansible doc formats (#71070)
* Fix tty_ify bugs and refactor

* Move tty_ify() and supporting attributes to the DocCLI class as that's
  the only thing using it.
* Add unittest for the code.
* Fix a bug where the substitution macros can be detected when they are
  a part of another word.
* Add support for L(), R(), and HORIZONTALLINE which were added to the
  website docs many years ago.

* Update test/units/cli/test_doc.py

Co-authored-by: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
2020-08-05 10:53:25 -07:00
Rick Elrod
0d7c144ce4
Allow macOS ACLs to work for unpriv -> unpriv (#70785)
Change:
- Use `chmod +a` in the fallback chain to allow MacOS to use ACLs to
  allow an unprivileged user to become an unprivileged user.

Test Plan:
- CI, new tests

Tickets:
- Fixes #70648

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-08-04 14:32:48 -04:00
Matt Martz
4e27569347
Add more include and yaml parsing tests (#70506)
These additional tests should provide coverage for features currently tested by the postgres incidental tests.
2020-08-04 11:24:52 -07:00
Rick Elrod
7f62a9d7b5
Make cache adjudicator's flush call plugin flush (#70987)
Change:
- Previously CachePluginAdjudicator#flush only removed entries from the
  cache backend that it knew about by using them earlier. Now it calls
  the underlying plugin's flush() method.

Test Plan:
- New unit tests

Tickets:
- Fixes #68770

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-08-03 17:16:15 -05:00
Sam Doran
c4f442ed5a
facts - fix incorrect time for some date_time_facts (#70665)
The iso8601_micro and iso8601 facts incorrectly called now.utcnow(), resulting
in a new timestamp at the time it was called, not a conversion of the previously
stored timestamp.

Correct this by capturing the UTC timestamp once then calculating the local
time using the UTC offset of the current system.

* Use time.time() for getting the current time
* Convert from that stored epoch timestamp to local and UTC times
* Used existing timestamp for epoch time
* Add unit tests that validate the formate of the return value rather than an exact value since mocking time and timezone is non-trivial
2020-07-30 11:39:58 -04:00
Rick Elrod
14dc4de424
Update docs for --tags default, and add some tests (#70939)
Change:
- Clarify that not passing `--tags` will cause `ansible_run_tags` to
  default to `["all"]`.
- Add some extra coverage around `ansible_run_tags`

Test Plan:
- New integration and unit tests

Tickets:
- Fixes #69619

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-29 17:16:57 -05:00
Brian Coca
1223ce656a
reset logging to INFO (#70878)
- due to CVE-2019-14846
 - also added comments and test to avoid 'oportunistic' reversion
2020-07-24 15:53:17 -04:00
Matt Davis
4c0af6c808
fix internal cases of actions calling unqualified module names (#70818)
* fix internal cases of actions calling unqualified module names

* add porting_guide entry
* misc other fixes around action/module resolution broken by redirection

ci_complete

* Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst

Co-authored-by: Rick Elrod <rick@elrod.me>

* Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst

Co-authored-by: Rick Elrod <rick@elrod.me>

* address review feedback

* pep8

* unit test fixes

* win fixes

* gather_facts fix module args ignores

* docs sanity

* pep8

* fix timeout test

* fix win name rewrites

Co-authored-by: Rick Elrod <rick@elrod.me>
2020-07-23 09:02:57 -07:00
Abhijeet Kasurde
e439194c8c
basic: Add name of aliases in error message (#69427)
With this patch, user will be notified with available aliases
of arg parameter.

Fixes: #58752

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-23 16:02:18 +05:30
Sam Doran
5260527c4a
Change default file permissions so they are not world readable (#70221)
* Change default file permissions so they are not world readable

CVE-2020-1736

Set the default permissions for files we create with atomic_move() to 0o0660. Track
which files we create that did not exist and warn if the module supports 'mode'
and it was not specified and the module did not call set_mode_if_different(). This allows the user to take action and specify a mode rather than using the defaults.

A code audit is needed to find all instances of modules that call atomic_move()
but do not call set_mode_if_different(). The findings need to be documented in
a changelog since we are not warning. Warning in those instances would be frustrating
to the user since they have no way to change the module code.

- use a set for storing list of created files
- just check the argument spac and params rather than using another property
- improve the warning message to include the default permissions
2020-07-22 17:05:38 -04:00
David Shrewsbury
bf98f031f3
Sanitize URI module keys with no_log values (#70762)
* Add sanitize_keys() to module_utils.

* More robust tests

* Revert 69653 change

* Allow list or dict

* fix pep8

* Sanitize lists within dict values

* words

* First pass at uri module

* Fix insane sanity tests

* fix integration tests

* Add changelog

* Remove unit test introduced in 69653

* Add ignore_keys param

* Sanitize all-the-things

* Ignore '_ansible*' keys

* cleanup

* Use module.no_log_values

* Avoid deep recursion issues by using deferred removal structure.

* Nit cleanups

* Add doc blurb

* spelling

* ci_complete
2020-07-22 14:49:37 -05:00
Matt Martz
96b74d3e0b
Ensure single vaulted values aren't counted as sequences. Fixes #70784 (#70786) 2020-07-21 16:48:35 -05:00
Abhijeet Kasurde
4f96f9826c
distribution: Add support for DragonFly (#70748)
partially fixes #43739

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-21 13:47:11 -04:00
Rick Elrod
69472a5f8d
Refactor _fixup_perms2 to remove way-nested logic (#70701)
Change:
- Refactoring to make it harder to get wrong and easier to read.
- Generalize become_unprivileged tests and fix some that never worked
  but also never failed.

Test Plan:
- CI, new units/integration tests

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-20 18:46:47 -05:00
Eduard Rozenberg
566c5e6ce1
Handle Slackware OS version strings containing a plus (“+”) (#68142)
A couple of years ago Slackware -current began using a plus (“+”) at the end of the distribution version string to indicate a future version work-in-progress.

Rearrange distribution_files unit tests to easily support more tests
  - add conftest with common fixtures 
  - use parametrize for testing multiple scenarios

* Add changelog
* Add unit tests for Slackware distribution parsing
* Use correct fixtures for Slackware
Data comes from /etc/slackware-version

Co-authored-by: Sam Doran <sdoran@redhat.com>
Co-authored-by: <Eduard Rozenberg <eduardr@pobox.com>>
2020-07-17 16:39:26 -04:00
Matt Davis
c616e54a6e
refactor Python module_utils locator (#70610)
* refactor Python module_utils locator

* no longer recursive
* embed special-case module code internally
* share common code between collections/not cases
* fixes #70134
* properly support subpackage redirection
* adds support for FQCN redirect targets used by migration (expands to FQ Python name)
* add tests

* add changelog
2020-07-16 17:57:47 -07:00
Abhijeet Kasurde
055871cbb8
api: time.clock compatible code (#70650)
time.clock is removed in Python 3.8. Add time.clock
compatible code.

Fixes: #70649

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-16 12:24:22 +05:30
Mykola Grygoriev
fe86a93482
Add a new date_time fact to provide DST timezone
PR #70449
2020-07-14 18:22:51 +02:00
Jordan Borean
e22e103cdf
winrm - Added kinit_args to control the args for kinit calls (#70624) 2020-07-14 12:05:11 -04:00
jabdr
0690b68bd3
Support datetime.date object in module result (#70595)
* Support datetime.date object in module result

Fixes #70583

* change blank lines for pep8 sanity test
2020-07-14 11:42:40 -04:00
Mark Chappell
8d160b1881
Remove remaining examples of 1.1.1.1 (#70552)
* Remove remaining examples of 1.1.1.1

* Update ec2_group.py
2020-07-14 11:19:59 -04:00
Brian Coca
cf89ca8a03
Make filter type errors 'loop friendly' (#70417)
- ensure we preserve the typeerror part of the exception so loop defereed error handling
 can postpone those caused by undefined variables until the when check is done.
 - fix tests to comply with the 'new normal'

 - human_to_bytes and others can issue TypeError not only on 'non string'
 but also bad string that is not convertable.

Co-authored-by: Sloane Hertel <shertel@redhat.com>

Co-authored-by: Sloane Hertel <shertel@redhat.com>
2020-07-10 18:49:57 -04:00
Felix Fontein
24dcaf8974
plugin loader: return collection name; ansible-doc: handle ansible.builtin correctly (#70026)
* Determine collection in plugin loader.

* Fix test.

* Use PluginPathContext objects in PluginLoader._plugin_path_cache instead of tuples.
2020-07-11 00:24:08 +02:00
Rick Elrod
f3ef4ed076
Remove some constants.py deprecated items (#70466)
Change:
- Remove mk_boolean
- Remove BECOME_METHODS
- Remove get_config

Test Plan:
- CI, removed mk_boolean unit tests

Tickets:
- Fixes #69678

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-07 18:41:19 -05:00
Abhijeet Kasurde
f5037314e3
sanity: remove ansible-bad-function (#70431)
* Updated docs and ignore.txt
* Replaced with correct function calls

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-07 11:46:28 -04:00
Abhijeet Kasurde
a8ae8f4d72
setup: Handle CIFS share with backward slash (#70006)
CIFS can be mounted using backward slash as well in /etc/fstab like

\\Windows\share /data/ cifs credentials=/root/.creds 0 0

Handle this condition while gather mount information in Linux.

Fixes: #48813

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-06-30 16:09:35 -04:00
Martin Krizek
b05e00e99a
Fix delegate_facts with interpreter not being set (#70293)
Fixes #70168

ci_complete

Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
Co-authored-by: Matt Clay <matt@mystile.com>
2020-06-30 15:49:09 +02:00
Hideki Saito
2d59e548f6
Improve ansible-galaxy STDOUT messages for collections (#70040)
- Fix issue #70010
- Add installation successful message
- This feature targets "collection" sub-command and does not affect "role" sub-command

Signed-off-by: Hideki Saito <saito@fgrep.org>
2020-06-29 15:18:30 -04:00
s-hamann
e39a9bf583
Add support for Parrot Linux, a Debian derivate (#69158)
Co-authored-by: black <invalid>
2020-06-29 20:22:44 +05:30
jctanner
b019029bf3
Add intentional unit tests for basic._set_cwd and common.dict_merge (#70283)
* Add unit tests for basic._set_cwd

* incidental coverage for dict_merge

* add test for async stderr inclusion
2020-06-25 14:45:13 -05:00
Abhijeet Kasurde
3fe48ecba2
Add collection path in CLI version info (#68633)
This will provide user default path of collection

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-06-25 01:58:56 -04:00
Rick Elrod
0073ab5fa6
Nuke strategy.SharedPluginLoaderObj, depr. 2.11 (#70235)
* Nuke strategy.SharedPluginLoaderObj, depr. 2.11

Change:
- Nuke SharedPluginLoaderObj class
- Update tests (which seemingly didn't use it anyway)
- Changelog

Test Plan:
CI, grep

Signed-off-by: Rick Elrod <rick@elrod.me>

* Nuke from ignore.txt

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-06-23 13:09:26 -04:00