Commit graph

51357 commits

Author SHA1 Message Date
Sloane Hertel
bd18be6c0c
Fix ansible-galaxy collection subdir searching and update documentation (#73406)
* Ensure there is a single source of collection metadata
* Allow collection subdirs to be detected by a galaxy.yml or MANIFEST.json
* Add documentation about installing and downloading collection directories
* Add an example for downloading a git repository
* Update documented valid metadata sources for installing git repositories

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
2021-02-04 13:42:45 -05:00
Sam Doran
997b2d2a19
Make yum_repository test more reliable (#73467)
* Improve setup_rpm_repo

- add handlers to remove repos
- add variable to control whethere or not repos are created

* Use local repo for all distros
* Change repo creation script to module
2021-02-03 17:34:12 -08:00
Sloane Hertel
9759e0ca49
add option to upgrade collections (#73336)
* Add a flag to ansible-galaxy to update collections


Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-02-04 07:08:02 +10:00
Akira Yokochi
fce22529c4
fix typo (#73434) 2021-02-03 09:17:02 -06:00
Toshio Kuratomi
564a0fce4a
Add the porting guide for Ansible-3.x (#73455) 2021-02-02 17:03:40 -05:00
Gonéri Le Bouder
7cf80f50d1
validate_modules: fails with .id attribute not found (#73322)
* validate_modules: fails with .id attribute not found

This patch addresses a problem in the `found_try_except_import` test.

This module tries to identify lines like:

`HAS_FOO = True`

In this case, the target (`HAS_FOO`) is of type `ast.Name` and has a
`id` attribute which provide the name.

In my case, I've a line that set a module attribute`. In this case, the
target (`module.var`) has the type `ast.Attribute` and no `id`
attribute. The code trigger an `AttributeError` exception.

This patch ensures we compare a `ast.Name`.

* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py
2021-02-02 13:29:36 -08:00
Felix Fontein
a78a416e85
hostname: fix references in docs (#73454) 2021-02-02 16:11:50 -05:00
Jose Angel Munoz
69631da889
lineinfile - add search_string parameter for non-regexp searching (#70647)
* Add tests for search_string
* Improve examples
* Add changelog
2021-02-02 15:37:06 -05:00
Rick Elrod
9a9272305a
Correct pkg_mgr for Fedora-derived OSTree distros (#73445)
Change:
- Remove check that states that only Fedora can be an OSTree
  distribution.
- This allows us to correctly return "atomic_container" as the pkg_mgr
  fact for RHEL for Edge, Fedora/RHEL/CentOS Atomic Host, etc.

Test Plan:
- Created local RHEL for Edge image and tested against it.
- Tested against regular RHEL 8 and still got `dnf` as expected.
- Tested against RHEL 7 Atomic Host and got `atomic_container` now.
- New unit tests.

Tickets:
- Fixes #73084

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-02 15:09:30 -05:00
Alexander Sowitzki
1c83672532
Allow $ & ~ inside paths in galaxy roles (#72966)
ansible-galaxy currently behaves bad then a role to be installed
contains ~ or $ at any place in the path of a file in that role.
It extracts the parent directory of the offending path level as an
empty file. This explodes if that directory contains anything else.

Change this behaviour. `~` is now allowed allowed when it is
not a full level (Yes: `some~thing/`, no: `~/`). The code should
get refactoring in an other PR.
2021-02-02 18:10:05 +01:00
Matt Clay
a9b5bebab3
Overhaul ansible-test SSH key management. (#73451)
* Pass remote.sh to shell over stdin.
* Pass docker.sh to shell over stdin.
* Standardize SSH key management.
* Update docker containers.
2021-02-02 08:43:54 -08:00
Alexander Sowitzki
218f5c3648
user - Prevent user fact lookup failure if LOGNAME is set (#17029) (#73439)
The `UserFactCollector` queries the user login name via
`getpass.getuser()` and looks up the corresponding entry
in the password database.
The login name may differ from the actual user name,
eg. if the `LOGNAME` env variable is set. The lookup
fails in this case. Added a fallback in this case that
tries to get the entry via the user ID.
2021-02-02 10:36:02 -05:00
Rick Elrod
bce01655cc
[ansible-test] Bump CentOS 6 image (yum repos) (#73446)
Change:
- Bump centos6 image version to one which includes multiple fallbacks
  for vault.centos.org content.

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-01 16:24:21 -06:00
Sloane Hertel
840bdc1e10
Fix warning for nonexistent inventory cache (#72840)
* Fix inventory cache warning by checking if the key exists before loading it
* changelog
2021-02-01 14:29:36 -05:00
Matt Clay
81cd8e46f2 Update ansible-test distro containers to 1.29.0. 2021-02-01 09:24:28 -08:00
Sloane Hertel
728dafb6eb
Make ansible-galaxy collection verify error out on no MANIFEST.json
PR #73403

Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
2021-02-01 09:55:21 +01:00
Sviatoslav Sydorenko
6f4b4c345b
Match user-requested transitive pre-releases in collection dependency resolver (#73416)
* Add direct+transitive pre-release regression test

* Match user-requested transitive pre-releases

This change makes sure that in scenario when a user requests
collectionA-pre and collectionB that depends on collectionA,
collectionA-pre would actually match that collectionA requirement
while `--pre` is unset.

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2021-02-01 14:26:49 +10:00
Matt Clay
c7cb944315 Always use python exec wrapper in ansible-test. 2021-01-28 15:59:55 -08:00
Matt Martz
c986cbb996
Add Python 3.8 and Python 3.9 to the fallback list (#73405) 2021-01-28 11:14:45 -08:00
Matt Clay
902c402b6f Update ansible-test default containers. 2021-01-28 09:58:54 -08:00
Matt Clay
ded2d0b90e Use sh for ansible-test raw remote shell. 2021-01-28 09:00:55 -08:00
Matt Clay
b1344a1847 Avoid sphinx usage in changelog sanity test. 2021-01-27 23:52:13 -08:00
Matt Clay
418b9b781d No longer warn about missing pip/python pair.
This warning was intended to help improve test environment configuration.
However, it has ended up producing mostly non-actionable warning noise instead.

In most situations a missing pip or python should result in test failures.
Reporting a missing pip also implies that it should be used by tests, which is not the case.
Tests should be invoking pip as a python module with the appropriate python interpreter instead.
2021-01-27 23:51:48 -08:00
Matt Clay
672941fb0d
Add test support for Python 3.8 on RHEL and FreeBSD. (#73393)
* Support Python 3.8 on RHEL 8.2+ in ansible-test.

* Support Python 3.8 on FreeBSD in ansible-test.

* Use libyaml with PyYAML on FreeBSD.
2021-01-27 21:35:29 -08:00
Matt Clay
1b157ef42f
Limit rstcheck sanity test to ansible-core. (#73391) 2021-01-27 21:06:00 -08:00
Matt Clay
607af05d36 Fix syntax errors detected by rstcheck.
The errors occur when using sphinx >= 2.0.0.
2021-01-27 16:32:11 -08:00
Brian Coca
5aa4295d74
change detection and check mode fixes for apt_key (#73334)
Change detection and check mode fixes for apt_key

* allow apt-key to use proxies
* add note about deprecation of apt-key itself
* expanded error msgs
* show all keys
* fix short_format parsing
* added more return info and documented it
2021-01-27 18:40:58 -05:00
Sviatoslav Sydorenko
595413d113
Replace the inhouse collection dependency resolver with resolvelib
PR #72591

This change:

  * Adds an artifacts manager that abstracts away extracting the
    metadata from artifacts, downloading and caching them in a
    temporary location.

  * Adds `resolvelib` to direct ansible-core dependencies[0].

  * Implements a `resolvelib`-based dependency resolver for
    `collection` subcommands that replaces the legacy
    in-house code.

    This is a dependency resolution library that pip 20.3+ uses
    by default. It's now integrated for use for the collection
    dependency resolution in ansible-galaxy CLI.

  * Refactors of the `ansible-galaxy collection` CLI.
    In particular, it:

      - reimplements most of the `download`, `install`, `list` and
        `verify` subcommands from scratch;

      - reuses helper bits previously moved out into external modules;

      - replaces the old in-house resolver with a more clear
        implementation based on the resolvelib library[0][1][2].

  * Adds a multi Galaxy API proxy layer that abstracts accessing the
    version and dependencies via API or local artifacts manager.

  * Makes `GalaxyAPI` instances sortable.

  * Adds string representation methods to `GalaxyAPI`.

  * Adds dev representation to `GalaxyAPI`.

  * Removes unnecessary integration and unit tests.

  * Aligns the tests with the new expectations.

  * Adds more tests, integration ones in particular.

[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing

Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>

Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
2021-01-27 22:23:22 +01:00
Abhijeet Kasurde
82b74f7fd7
iptables: Fixed get_chain_policy API (#69152)
While getting policy name in get_chain_policy API,
module does not require any additional parameters except chain
Enabling flag in get_chain_policy API call fixes this.

Fixes: #68612

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-01-27 15:24:53 -05:00
Thomas Sjögren
11398aac09
update faq link (#73353)
Fix faq link.
2021-01-27 14:58:23 -05:00
Brian Coca
c7d4acc12f
allow env to override unspecified unsafe_writes (#73282)
* allow env var for fallback value for unspecified unsafe_writes
2021-01-27 14:16:10 -05:00
Brian Coca
2b0cd2c13f
use correct executable and options in all cases (#73323)
Use correct ssh executable and options in all cases on connection plugin

  * Also nicer naming/comments

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-01-26 20:21:24 -05:00
David Moreau Simard
1265c6def3 Update porting guide for 2.10.6 2021-01-26 16:29:40 -08:00
Matt Clay
13d08d232c Rename ansible-test functions to match constants. 2021-01-26 14:24:16 -08:00
Matt Clay
73fadc5e97
Clean up code in ansible-test. (#73379)
* Relocate code to fix type dependencies.

* Fix missing and unused imports.

* Fix type hints.

* Suppress PyCharm false positives.

* Avoid shadowing `file` built-in.

* Use json.JSONEncoder directly instead of super().

This matches the recommended usage and avoids a PyCharm warning.

* Remove redundant regex escape.

* Remove redundant find_python call.

* Use tarfile.open directly.

* Add changelog fragment.
2021-01-26 14:02:08 -08:00
Felix Fontein
76604397cb
Mention that C(...) should be used for inline code. (#73312) 2021-01-26 10:48:47 -06:00
Martin Krizek
a2af8432f3
Local vars should have highest precedence in AnsibleJ2Vars (#72830)
Ability to add local variables into AnsibleJ2Vars was added in
18a9eff11f to fix #6653. Local variables
are added using ``AnsibleJ2Vars.add_locals()`` method when creating a
new context - typically when including/importing a template with
context. For that use case local template variables created using
``set`` should override variables from higher contexts - either from the
play or any parent template, or both; Jinja behaves the same way.

Also removes AnsibleJ2Vars.extras instance variable which is not used.

Also adds missing test for #6653.

Fixes #72262
Fixes #72615

ci_complete
2021-01-26 11:37:56 +01:00
Matt Clay
fc590aeb21 Use a versioned pip bootstrapper in ansible-test. 2021-01-25 12:31:11 -08:00
Sviatoslav Sydorenko
ab667200cc
Mention renaming ansible-base to -core @ roadmap (#73354)
* Mention renaming ansible-base to -core @ roadmap

* Replace ansible-base with ansible-core @ roadmap
2021-01-25 12:34:40 -05:00
Sagar Paul
f3ce7cfe53
tiny grammatical edit (#73339) 2021-01-25 10:28:55 -05:00
Sviatoslav Sydorenko
6abeabaff1 Fix versionadded syntax @ galaxy user guide 2021-01-25 12:08:52 +01:00
Matt Clay
484e4af4d0 Handle get-pip.py breaking change on Python 2.7. 2021-01-23 11:45:53 -08:00
Abhijeet Kasurde
7f6fcc3407
json_query: Update documentation (#73302) 2021-01-22 15:41:24 -05:00
Brian Coca
e0c9f285ff
undeprecated hash_merge setting (#73328)
added updates from sloan's PR
 feedback from others

Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>

Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>
2021-01-22 15:00:19 -05:00
Brian Coca
dec443e3a5
why no mfa? (#73324)
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Rick Elrod <rick@elrod.me>
2021-01-22 14:58:35 -05:00
Alexander Sowitzki
e6da544310 Let command always return stdout & stderr (#73004)
The `command` module does not return stdout & sterr when calling
a non existing executable or an unknown exception arises. This fix
lets the module return empty byte strings in those cases.
2021-01-22 08:40:53 +01:00
Rick Elrod
1934ca9a55
Add Ubuntu 20.04 to CI and ansible-test (#69161)
Change:
- Add Ubuntu 20.04 to CI now that venv is default instead of virtualenv in ansible-test.

Test Plan:
- CI

Tickets:
- Fixes #69203

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-21 22:58:31 -06:00
Rick Elrod
4357a78130
[ansible-test] First attempt at freebsd/11.4 (#72655)
Change:
- Try CI against freebsd 11.4

Test Plan:
- ci_complete

Tickets:
- Fixes #48782

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-21 17:12:18 -06:00
Alexander Sowitzki
fe792fdcd2
Vanquish rare container bootstrap failure (#73288)
The init script for the test container writes additional lines to
the .bashrc of the user. This was done via a `cat` multiline
instruction, which is implemented internally by writing a
temporary file to TMPDIR (/tmp in this case) first. Docker fails
to provide /tmp just after creation, which results in a race
condition that rarely makes the init fail. Changed the `cat`
statement to multiple `echo`s.

Co-authored-by: Matt Clay <matt@mystile.com>
2021-01-21 12:22:58 -05:00
Nilashish Chakraborty
45240c1871
Update NX-OS platform guide with MDS information (#73315)
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
2021-01-21 12:03:50 -05:00