Commit graph

51124 commits

Author SHA1 Message Date
Sam Doran
7d18ea5e93
default callback - add "show_task_path_on_failure" option (#73260)
When running in verbosity <2, display the file and line number for tasks that fail. This provides
useful information without having to run at increased verbosity.

* Move _print_task_path to CallbackBase class
* Add integration tests
* Add color parameter to _print_task_path()

* Keep color output consistent for now
  Currently the path is display with COLOR_DEBUG formatting with verbosity >= 2.
  Instead of the color of the path changing based on verbosity level, just keep it at the
  currently behavior of COLOR_DEBUG. Having the color of the same information change
  based on verbosity level seems incorrect and makes visual parsing of the information
  more difficult.

Co-authored-by: tahar.jegham <jeghamseifeddine@gmail.com>
2021-02-09 12:43:59 -05:00
Felix Fontein
ca448f7c35
Update collection routing (#73046)
* Docker content from community.general migrated to community.docker.

* Hetzner Robot content from community.general migrated to community.hrobot.

* RouterOS content from community.network migrated to community.routeros.

* PostgreSQL content from community.general migrated to community.postgresql.

* OC content from community.general migrated to community.okd.

* Cisco NSO content from community.network migrated to cisco.nso.

* Add changelog fragment.

* Google content from community.general migrated to community.google.

* Hashi Vault content from community.general migrated to community.hashi_vault.

* KubeVirt content from community.general migrated to community.kubevirt.

* FortiOS content from community.network migrated to community.fortios.

* Update changelog fragment.

* DellEMC content from community.general migrated to dellemc.openmanage.
2021-02-09 09:31:35 -08:00
Eugene Vilensky
65fc8c9e35
INTERPRETER_PYTHON_DISTRO_MAP: Treat oracle same as rhel/centos (#73498)
* treat oracle same as rhel/centos

* fix misunderstood use of yaml anchors

* add changelog fragment

* Update changelogs/fragments/73498-INTERPRETER_PYTHON_DISTRO_MAP-Treat-oracle-same-as-rhel-centos.yml
2021-02-09 10:40:39 -05:00
Matt Martz
d3441bc313
Add new split filter (#73532)
* Add version added
2021-02-09 10:37:15 -05:00
Matt Clay
1a2da990a4
Temporary fix for cryptography issues. (#73530) 2021-02-08 14:35:00 -08:00
Rick Elrod
aca5b0e43b
[setup_rpm_repo test] Ensure rpm-build is present (#73516)
Change:
- Other targets might remove rpm-build as they clean up after
  themselves. Ensure that it's present in setup_rpm_repo because
  rpmfluff needs it.

Test Plan:
- Local experimentation with yum_repository and mysql_db (the latter of
  which depends on a handler which was removing rpm-build) on
  stable-2.9.

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-05 15:58:02 -05:00
Matt Martz
125c220343
Make non-lockstep callback tests more deterministic (#73511) 2021-02-05 14:11:40 -06:00
Shane McDonald
67f5bb39c7
galaxy - add format options for collection list (#73474)
* Include all collections in single json object / yaml document
* Add tests
* For galaxy list yaml/json output, use dictionary of dictionaries instead of list
* Add tests for listing single collection in yaml / output format
* --output -> --format
* Add explicit test for listing collection in human format
* Fix bug where empty json object was emitted + add test
2021-02-05 13:24:59 -05:00
Matt Martz
d3f3784b86
Don't treat host_pinned as lockstep (#73484)
* Don't treat host_pinned as lockstep. Fixes #73364

* Add intg tests
2021-02-05 09:06:19 -06:00
Florian Heiderich
8d6136eab9
uri - add ca_path parameter (#71979)
* add changelog fragment for #71979 (ca_path for uri)
* add integration tests for ca_path in the uri module
* return path of ca cert instead of its content
* connect to port 444 on self_signed_host
  and use quay.io/ansible/http-test-container:1.3.0
* state that the certificate in ca_path is used for validation
2021-02-05 09:40:52 -05:00
Felix Fontein
82cdd7e735 Bump antsibull-changelog version for changelog sanity test. 2021-02-04 16:42:26 -08:00
jakegatsby
f271d02a9f
bugfix: use to_text(stdout) in psrp.Connection.put_file method (#73491)
Co-authored-by: jakegatsby <jakegatsby@example.com>
2021-02-05 10:01:07 +10:00
Jordan Borean
0cdc410dce
no_log mask suboption fallback values and defaults CVE-2021-20228 (#73487)
* no_log mask suboption fallback values and defaults

* Added changelog

* Remove lambda expression
2021-02-05 09:11:55 +10:00
Michael Shnit
4315e18807
Fix regression introduced in b77abd0491 causing bug in inventory modu… (#73429)
* Fix regression introduced in b77abd0491 causing bug in inventory modules which break functionality in user setting  use_contrib_script_compatible_sanitization  parameter.

* Add changelog

Co-authored-by: s-hertel <19572925+s-hertel@users.noreply.github.com>
2021-02-04 17:25:08 -05:00
Jill R
7cef930aa4
Update kubernetes collection routing (#73485)
The community.kubernetes collection will be migrated / renamed to
kubernetes.core in the collection's 2.0 dev cycle. This should coincide
with the ansible-core 2.11 cycle.  The collection is being released to
galaxy under both the community.k8s and k8s.core names today.
2021-02-05 07:58:42 +10:00
Jill R
7fb7d138fb
Update working group links for cloud collections (#73471) 2021-02-05 05:15:32 +10:00
Florian Heiderich
8f887b7adb
add self-signed https endpoint for ansible-test (#73463)
* introduce self-signed.ansible.http.tests

* forwarding of port 444

* forward port 8444 to port 444 on http test container

* Fix port forwarding for Windows under docker

* add changelog fragment

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2021-02-05 05:12:22 +10:00
Sloane Hertel
bd18be6c0c
Fix ansible-galaxy collection subdir searching and update documentation (#73406)
* Ensure there is a single source of collection metadata
* Allow collection subdirs to be detected by a galaxy.yml or MANIFEST.json
* Add documentation about installing and downloading collection directories
* Add an example for downloading a git repository
* Update documented valid metadata sources for installing git repositories

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
2021-02-04 13:42:45 -05:00
Sam Doran
997b2d2a19
Make yum_repository test more reliable (#73467)
* Improve setup_rpm_repo

- add handlers to remove repos
- add variable to control whethere or not repos are created

* Use local repo for all distros
* Change repo creation script to module
2021-02-03 17:34:12 -08:00
Sloane Hertel
9759e0ca49
add option to upgrade collections (#73336)
* Add a flag to ansible-galaxy to update collections


Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-02-04 07:08:02 +10:00
Akira Yokochi
fce22529c4
fix typo (#73434) 2021-02-03 09:17:02 -06:00
Toshio Kuratomi
564a0fce4a
Add the porting guide for Ansible-3.x (#73455) 2021-02-02 17:03:40 -05:00
Gonéri Le Bouder
7cf80f50d1
validate_modules: fails with .id attribute not found (#73322)
* validate_modules: fails with .id attribute not found

This patch addresses a problem in the `found_try_except_import` test.

This module tries to identify lines like:

`HAS_FOO = True`

In this case, the target (`HAS_FOO`) is of type `ast.Name` and has a
`id` attribute which provide the name.

In my case, I've a line that set a module attribute`. In this case, the
target (`module.var`) has the type `ast.Attribute` and no `id`
attribute. The code trigger an `AttributeError` exception.

This patch ensures we compare a `ast.Name`.

* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py
2021-02-02 13:29:36 -08:00
Felix Fontein
a78a416e85
hostname: fix references in docs (#73454) 2021-02-02 16:11:50 -05:00
Jose Angel Munoz
69631da889
lineinfile - add search_string parameter for non-regexp searching (#70647)
* Add tests for search_string
* Improve examples
* Add changelog
2021-02-02 15:37:06 -05:00
Rick Elrod
9a9272305a
Correct pkg_mgr for Fedora-derived OSTree distros (#73445)
Change:
- Remove check that states that only Fedora can be an OSTree
  distribution.
- This allows us to correctly return "atomic_container" as the pkg_mgr
  fact for RHEL for Edge, Fedora/RHEL/CentOS Atomic Host, etc.

Test Plan:
- Created local RHEL for Edge image and tested against it.
- Tested against regular RHEL 8 and still got `dnf` as expected.
- Tested against RHEL 7 Atomic Host and got `atomic_container` now.
- New unit tests.

Tickets:
- Fixes #73084

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-02 15:09:30 -05:00
Alexander Sowitzki
1c83672532
Allow $ & ~ inside paths in galaxy roles (#72966)
ansible-galaxy currently behaves bad then a role to be installed
contains ~ or $ at any place in the path of a file in that role.
It extracts the parent directory of the offending path level as an
empty file. This explodes if that directory contains anything else.

Change this behaviour. `~` is now allowed allowed when it is
not a full level (Yes: `some~thing/`, no: `~/`). The code should
get refactoring in an other PR.
2021-02-02 18:10:05 +01:00
Matt Clay
a9b5bebab3
Overhaul ansible-test SSH key management. (#73451)
* Pass remote.sh to shell over stdin.
* Pass docker.sh to shell over stdin.
* Standardize SSH key management.
* Update docker containers.
2021-02-02 08:43:54 -08:00
Alexander Sowitzki
218f5c3648
user - Prevent user fact lookup failure if LOGNAME is set (#17029) (#73439)
The `UserFactCollector` queries the user login name via
`getpass.getuser()` and looks up the corresponding entry
in the password database.
The login name may differ from the actual user name,
eg. if the `LOGNAME` env variable is set. The lookup
fails in this case. Added a fallback in this case that
tries to get the entry via the user ID.
2021-02-02 10:36:02 -05:00
Rick Elrod
bce01655cc
[ansible-test] Bump CentOS 6 image (yum repos) (#73446)
Change:
- Bump centos6 image version to one which includes multiple fallbacks
  for vault.centos.org content.

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-01 16:24:21 -06:00
Sloane Hertel
840bdc1e10
Fix warning for nonexistent inventory cache (#72840)
* Fix inventory cache warning by checking if the key exists before loading it
* changelog
2021-02-01 14:29:36 -05:00
Matt Clay
81cd8e46f2 Update ansible-test distro containers to 1.29.0. 2021-02-01 09:24:28 -08:00
Sloane Hertel
728dafb6eb
Make ansible-galaxy collection verify error out on no MANIFEST.json
PR #73403

Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
2021-02-01 09:55:21 +01:00
Sviatoslav Sydorenko
6f4b4c345b
Match user-requested transitive pre-releases in collection dependency resolver (#73416)
* Add direct+transitive pre-release regression test

* Match user-requested transitive pre-releases

This change makes sure that in scenario when a user requests
collectionA-pre and collectionB that depends on collectionA,
collectionA-pre would actually match that collectionA requirement
while `--pre` is unset.

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2021-02-01 14:26:49 +10:00
Matt Clay
c7cb944315 Always use python exec wrapper in ansible-test. 2021-01-28 15:59:55 -08:00
Matt Martz
c986cbb996
Add Python 3.8 and Python 3.9 to the fallback list (#73405) 2021-01-28 11:14:45 -08:00
Matt Clay
902c402b6f Update ansible-test default containers. 2021-01-28 09:58:54 -08:00
Matt Clay
ded2d0b90e Use sh for ansible-test raw remote shell. 2021-01-28 09:00:55 -08:00
Matt Clay
b1344a1847 Avoid sphinx usage in changelog sanity test. 2021-01-27 23:52:13 -08:00
Matt Clay
418b9b781d No longer warn about missing pip/python pair.
This warning was intended to help improve test environment configuration.
However, it has ended up producing mostly non-actionable warning noise instead.

In most situations a missing pip or python should result in test failures.
Reporting a missing pip also implies that it should be used by tests, which is not the case.
Tests should be invoking pip as a python module with the appropriate python interpreter instead.
2021-01-27 23:51:48 -08:00
Matt Clay
672941fb0d
Add test support for Python 3.8 on RHEL and FreeBSD. (#73393)
* Support Python 3.8 on RHEL 8.2+ in ansible-test.

* Support Python 3.8 on FreeBSD in ansible-test.

* Use libyaml with PyYAML on FreeBSD.
2021-01-27 21:35:29 -08:00
Matt Clay
1b157ef42f
Limit rstcheck sanity test to ansible-core. (#73391) 2021-01-27 21:06:00 -08:00
Matt Clay
607af05d36 Fix syntax errors detected by rstcheck.
The errors occur when using sphinx >= 2.0.0.
2021-01-27 16:32:11 -08:00
Brian Coca
5aa4295d74
change detection and check mode fixes for apt_key (#73334)
Change detection and check mode fixes for apt_key

* allow apt-key to use proxies
* add note about deprecation of apt-key itself
* expanded error msgs
* show all keys
* fix short_format parsing
* added more return info and documented it
2021-01-27 18:40:58 -05:00
Sviatoslav Sydorenko
595413d113
Replace the inhouse collection dependency resolver with resolvelib
PR #72591

This change:

  * Adds an artifacts manager that abstracts away extracting the
    metadata from artifacts, downloading and caching them in a
    temporary location.

  * Adds `resolvelib` to direct ansible-core dependencies[0].

  * Implements a `resolvelib`-based dependency resolver for
    `collection` subcommands that replaces the legacy
    in-house code.

    This is a dependency resolution library that pip 20.3+ uses
    by default. It's now integrated for use for the collection
    dependency resolution in ansible-galaxy CLI.

  * Refactors of the `ansible-galaxy collection` CLI.
    In particular, it:

      - reimplements most of the `download`, `install`, `list` and
        `verify` subcommands from scratch;

      - reuses helper bits previously moved out into external modules;

      - replaces the old in-house resolver with a more clear
        implementation based on the resolvelib library[0][1][2].

  * Adds a multi Galaxy API proxy layer that abstracts accessing the
    version and dependencies via API or local artifacts manager.

  * Makes `GalaxyAPI` instances sortable.

  * Adds string representation methods to `GalaxyAPI`.

  * Adds dev representation to `GalaxyAPI`.

  * Removes unnecessary integration and unit tests.

  * Aligns the tests with the new expectations.

  * Adds more tests, integration ones in particular.

[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing

Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>

Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
2021-01-27 22:23:22 +01:00
Abhijeet Kasurde
82b74f7fd7
iptables: Fixed get_chain_policy API (#69152)
While getting policy name in get_chain_policy API,
module does not require any additional parameters except chain
Enabling flag in get_chain_policy API call fixes this.

Fixes: #68612

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-01-27 15:24:53 -05:00
Thomas Sjögren
11398aac09
update faq link (#73353)
Fix faq link.
2021-01-27 14:58:23 -05:00
Brian Coca
c7d4acc12f
allow env to override unspecified unsafe_writes (#73282)
* allow env var for fallback value for unspecified unsafe_writes
2021-01-27 14:16:10 -05:00
Brian Coca
2b0cd2c13f
use correct executable and options in all cases (#73323)
Use correct ssh executable and options in all cases on connection plugin

  * Also nicer naming/comments

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-01-26 20:21:24 -05:00
David Moreau Simard
1265c6def3 Update porting guide for 2.10.6 2021-01-26 16:29:40 -08:00