Commit graph

51080 commits

Author SHA1 Message Date
Sviatoslav Sydorenko 595413d113
Replace the inhouse collection dependency resolver with resolvelib
PR #72591

This change:

  * Adds an artifacts manager that abstracts away extracting the
    metadata from artifacts, downloading and caching them in a
    temporary location.

  * Adds `resolvelib` to direct ansible-core dependencies[0].

  * Implements a `resolvelib`-based dependency resolver for
    `collection` subcommands that replaces the legacy
    in-house code.

    This is a dependency resolution library that pip 20.3+ uses
    by default. It's now integrated for use for the collection
    dependency resolution in ansible-galaxy CLI.

  * Refactors of the `ansible-galaxy collection` CLI.
    In particular, it:

      - reimplements most of the `download`, `install`, `list` and
        `verify` subcommands from scratch;

      - reuses helper bits previously moved out into external modules;

      - replaces the old in-house resolver with a more clear
        implementation based on the resolvelib library[0][1][2].

  * Adds a multi Galaxy API proxy layer that abstracts accessing the
    version and dependencies via API or local artifacts manager.

  * Makes `GalaxyAPI` instances sortable.

  * Adds string representation methods to `GalaxyAPI`.

  * Adds dev representation to `GalaxyAPI`.

  * Removes unnecessary integration and unit tests.

  * Aligns the tests with the new expectations.

  * Adds more tests, integration ones in particular.

[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing

Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>

Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
2021-01-27 22:23:22 +01:00
Abhijeet Kasurde 82b74f7fd7
iptables: Fixed get_chain_policy API (#69152)
While getting policy name in get_chain_policy API,
module does not require any additional parameters except chain
Enabling flag in get_chain_policy API call fixes this.

Fixes: #68612

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-01-27 15:24:53 -05:00
Thomas Sjögren 11398aac09
update faq link (#73353)
Fix faq link.
2021-01-27 14:58:23 -05:00
Brian Coca c7d4acc12f
allow env to override unspecified unsafe_writes (#73282)
* allow env var for fallback value for unspecified unsafe_writes
2021-01-27 14:16:10 -05:00
Brian Coca 2b0cd2c13f
use correct executable and options in all cases (#73323)
Use correct ssh executable and options in all cases on connection plugin

  * Also nicer naming/comments

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-01-26 20:21:24 -05:00
David Moreau Simard 1265c6def3 Update porting guide for 2.10.6 2021-01-26 16:29:40 -08:00
Matt Clay 13d08d232c Rename ansible-test functions to match constants. 2021-01-26 14:24:16 -08:00
Matt Clay 73fadc5e97
Clean up code in ansible-test. (#73379)
* Relocate code to fix type dependencies.

* Fix missing and unused imports.

* Fix type hints.

* Suppress PyCharm false positives.

* Avoid shadowing `file` built-in.

* Use json.JSONEncoder directly instead of super().

This matches the recommended usage and avoids a PyCharm warning.

* Remove redundant regex escape.

* Remove redundant find_python call.

* Use tarfile.open directly.

* Add changelog fragment.
2021-01-26 14:02:08 -08:00
Felix Fontein 76604397cb
Mention that C(...) should be used for inline code. (#73312) 2021-01-26 10:48:47 -06:00
Martin Krizek a2af8432f3
Local vars should have highest precedence in AnsibleJ2Vars (#72830)
Ability to add local variables into AnsibleJ2Vars was added in
18a9eff11f to fix #6653. Local variables
are added using ``AnsibleJ2Vars.add_locals()`` method when creating a
new context - typically when including/importing a template with
context. For that use case local template variables created using
``set`` should override variables from higher contexts - either from the
play or any parent template, or both; Jinja behaves the same way.

Also removes AnsibleJ2Vars.extras instance variable which is not used.

Also adds missing test for #6653.

Fixes #72262
Fixes #72615

ci_complete
2021-01-26 11:37:56 +01:00
Matt Clay fc590aeb21 Use a versioned pip bootstrapper in ansible-test. 2021-01-25 12:31:11 -08:00
Sviatoslav Sydorenko ab667200cc
Mention renaming ansible-base to -core @ roadmap (#73354)
* Mention renaming ansible-base to -core @ roadmap

* Replace ansible-base with ansible-core @ roadmap
2021-01-25 12:34:40 -05:00
Sagar Paul f3ce7cfe53
tiny grammatical edit (#73339) 2021-01-25 10:28:55 -05:00
Sviatoslav Sydorenko 6abeabaff1 Fix versionadded syntax @ galaxy user guide 2021-01-25 12:08:52 +01:00
Matt Clay 484e4af4d0 Handle get-pip.py breaking change on Python 2.7. 2021-01-23 11:45:53 -08:00
Abhijeet Kasurde 7f6fcc3407
json_query: Update documentation (#73302) 2021-01-22 15:41:24 -05:00
Brian Coca e0c9f285ff
undeprecated hash_merge setting (#73328)
added updates from sloan's PR
 feedback from others

Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>

Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Sandra McCann <samccann@redhat.com>
2021-01-22 15:00:19 -05:00
Brian Coca dec443e3a5
why no mfa? (#73324)
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Rick Elrod <rick@elrod.me>
2021-01-22 14:58:35 -05:00
Alexander Sowitzki e6da544310 Let command always return stdout & stderr (#73004)
The `command` module does not return stdout & sterr when calling
a non existing executable or an unknown exception arises. This fix
lets the module return empty byte strings in those cases.
2021-01-22 08:40:53 +01:00
Rick Elrod 1934ca9a55
Add Ubuntu 20.04 to CI and ansible-test (#69161)
Change:
- Add Ubuntu 20.04 to CI now that venv is default instead of virtualenv in ansible-test.

Test Plan:
- CI

Tickets:
- Fixes #69203

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-21 22:58:31 -06:00
Rick Elrod 4357a78130
[ansible-test] First attempt at freebsd/11.4 (#72655)
Change:
- Try CI against freebsd 11.4

Test Plan:
- ci_complete

Tickets:
- Fixes #48782

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-21 17:12:18 -06:00
Alexander Sowitzki fe792fdcd2
Vanquish rare container bootstrap failure (#73288)
The init script for the test container writes additional lines to
the .bashrc of the user. This was done via a `cat` multiline
instruction, which is implemented internally by writing a
temporary file to TMPDIR (/tmp in this case) first. Docker fails
to provide /tmp just after creation, which results in a race
condition that rarely makes the init fail. Changed the `cat`
statement to multiple `echo`s.

Co-authored-by: Matt Clay <matt@mystile.com>
2021-01-21 12:22:58 -05:00
Nilashish Chakraborty 45240c1871
Update NX-OS platform guide with MDS information (#73315)
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
2021-01-21 12:03:50 -05:00
Martin Krizek 7f9ac0f364
Consolidate filters/tests handling into JinjaPluginIntercept (#71463)
* Consolidate filters/tests handling into JinjaPluginIntercept

ci_complete

* Postpone loading all ansible plugins

* Do we need to create an overlay?

ci_complete

* Typo

ci_complete

* Add FIXME

* conditional.py: use public Environment.parse() method

* Remove remaining occurrences of shared_loader_obj being passed to Templar

* __UNROLLED__ not needed with this change anymore

* Incorrect rebase at some point?
2021-01-21 11:22:33 +01:00
Joshua Bayfield 823c72bcb5
Shadow input for encrypt_string by default unless asked (fixes #71618) (#73263)
* Shadow input for encrypt_string by default unless asked (fixes #71618)
2021-01-20 15:50:24 -05:00
Jordan Borean bc60d8ccda
Galaxy - make versions list consistent across versions (#72932)
* Galaxy - make versions list consistent across versions

* Fix up unit tests
2021-01-20 21:19:29 +01:00
luto 51e00c5e86
Fix documentation for slaac filter (#73294)
##### SUMMARY

One of the parameters to the slaac filter was unquoted causing a crash:

```
template error while templating string: expected token 'end of print statement', got ':'. String: {{ fdcf:1894:23b5:d38c:0000:0000:0000:0000 | slaac('c2:31:b3:83:bf:2b') }}
```

This adds the missing quotes.

##### ISSUE TYPE
- Docs Pull Request

+label: docsite_pr
2021-01-20 13:10:24 -05:00
Brian Coca 13ea312cc8
added pb fqcn docs (#72602)
* added pb fqcn docs
2021-01-20 11:34:33 -05:00
Oldřich Jedlička ec55a221f6
Do not pretend expression is filename in compile() Python built-in call (#73113)
When calling compile(), the filename argument should be either a real file
name or a string. According to Python docs, suggested one is '<string>'.
 Keep the current behaviour (encapsulate the actual expression), 
but enclose it into angle brackets.

Signed-off-by: Oldřich Jedlička <oldium.pro@gmail.com>
2021-01-19 12:20:26 -05:00
Sam Doran e8d4b62b41
Fix YAML error message when error is at the end of the file (#73241)
* Fix YAML error message when error is at the end of the file

If a YAML file fails to load due to a syntax error in a file, or there is an error in the last line of a
file, PyYAML reports the last line number of the file as the index where the error occurred.

When reading the file lines, we use that index to the get the relevant line.  If the index value is out
of range, the relevant line is lost for error reporting.

Subtract one from the index value to avoid the IndexError in this specific scenario. It is possible
to still get an IndexError, which will be handled as it is currently.

* Update existing tests and add new tests
2021-01-19 10:07:36 -05:00
Brian Coca 30d93995dd
Use a pty for local connections (#73023)
* Use a pty for local connections

Fixes #38696

Co-authored-by: James Cammarata <jimi@sngx.net>
2021-01-18 16:02:04 -05:00
Koki Kato d500e6ec45
doc: fix ansible_play_hosts description about failure (#73190)
The variable ansible_play_hosts actually does _not_ includes failed or unreachable hosts.

+label: docsite_pr
2021-01-18 15:24:29 -05:00
Jan aae82ff560
Update documentation for "contains" parameter (#73107)
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-01-18 15:21:45 -05:00
Felix Fontein 73aa571305
Make sure to mention collection version for version_added as well. (#73270) 2021-01-18 14:53:10 -05:00
Brian Coca 1e27d4052a
fix inventory source parse error handling (#73160)
fixes #51025

added test cases
2021-01-18 09:33:51 -05:00
Sandra McCann 9d71bbd13e
add changelog fragment for 73176 (#73259) 2021-01-15 16:26:53 -06:00
Martijn Suijlen aa21804165
Added TLS security protocol for Powershell downloads (#73226)
##### SUMMARY
Added `[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12` to the Powershell scripts that use a file download. If not specified the script won't be allowed to download data from the mentioned URL's.


##### ISSUE TYPE
- Docs Pull Request

+label: docsite_pr
2021-01-14 14:23:19 -08:00
Brian Coca 1202dd000f
Allow restricting config values to enumerated list (#73162)
* Allow restricting config values to enumerated list
* dont document internal entries
* also ignore private defs for ansible-config
* remove invalid value from tests
* added porting entry
2021-01-14 15:11:30 -05:00
Sandra McCann 88763c7347
update rstcheck and sphinx versions in contraints (#73176)
* update rstceck and sphinx versions in contraints
* fix test error
2021-01-14 13:50:44 -06:00
Adam Miller e9a1ff296e
Fix block example line emphasis (#73238)
Signed-off-by: Adam Miller <admiller@redhat.com>
2021-01-14 14:40:31 -05:00
Gregor Riepl 403a3b90b5
Fix list to dict example (#73213)
##### SUMMARY
<!--- Your description here -->
single_list must be transformed to a list of pairs before feeding it into dict()

##### ISSUE TYPE
- Docs Pull Request

+label: docsite_pr
2021-01-14 14:15:55 -05:00
Sam Doran e2308ed6e3
ansible-galaxy - correct collections-path command line arg (#73193)
It should be plural to match other options and environment variables.
2021-01-14 14:13:01 -05:00
Sloane Hertel 0c4de6839b
Add integration tests to ansible-galaxy-collection for 'ansible-galaxy collection verify' (#73229) 2021-01-14 11:49:53 -05:00
Felix Fontein df9cf368c0
Always mention the name of the deprecated plugin in routing deprecation messages (#73059) 2021-01-14 17:45:02 +01:00
Sam Doran df451636e7
facts - properly report virtualization facts for Linux guests on bhyve (#73204) 2021-01-14 10:53:03 -05:00
Sam Doran 0e6c334115
pause - adjust warning when run in background (#73182)
When the pause module is run in the background and seconds parameter is provided,
do not warn.

* Add tests
* Fix existing tests
  The test wasn't failing when it should have.
2021-01-14 09:35:39 -05:00
Rick Elrod 7f0eb7ad79
[facts] Differentiate CentOS vs CentOS Stream (#73034)
Change:
- On CentOS Stream, make distribution_release be "Stream"
- On CentOS Core, it continues to be "Core"
- Implement custom distribution file parser for CentOS, so we can look
  for "CentOS Linux" and "CentOS Stream"
- Two new fixtures introduced (CentOS Linux 8.1 and CentOS Stream 8)
- Removed two dicts from `Distribution` class that were seemingly not
  used anywhere.

Test Plan:
- ci_complete
- New test fixtures

Tickets:
- Fixes #73027

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-13 17:54:04 -05:00
Matt Clay 4164cb26f0 Remove obsolete tower plugin from ansible-test. 2021-01-13 12:37:21 -08:00
Alexander Sowitzki 3e8c8cd536 Make test_adhoc succeed from within a git checkout
The test currently only expects the literal `ansible` followed
by a semver in the first output line of `ansible --version`.
When running from within a git checkout, additional information
like the currently checkout branch are attached, which lets
the test fail. This commit allows arbitrary information to
follow the semver.
2021-01-13 21:22:21 +01:00
Alexander Sowitzki 6459fbb7bc Make test_prepare_multipart succeed on non Debian distros
`test_prepare_multipart` fails in non Debian environments since
Debian installations map the file ending `.key` to the MIME type
`application/pgp-keys`, which is not IANA conformant. This commit
explicitly sets the corresponding file type to
`application/octet-stream` and adjusts the expected serialized
result.
2021-01-13 21:21:33 +01:00