Commit graph

10870 commits

Author SHA1 Message Date
Matt Martz
11f1177e6c
Operate pexpect with bytes to limit encoding issues (#73255)
* Operate pexpect with bytes to limit encoding issues

* Update tests to ensure no pepxect encoding issues

* Add changelog fragment

* Add multiline note

* Use rst formatting directly
2021-02-23 18:57:25 +01:00
Sam Doran
3ab52f2965
Disable junit callback on roles_arg_spec_tests (#73663)
This prevents the role arg spec validation tasks from showing up as failure
in the CI output.
2021-02-23 10:21:28 -05:00
Rick Elrod
fa046d302c
[InventoryManager] Fix two unhandled exceptions (#73667)
Change:
- Fix regression: unhandled exception when given inventory directory
  is empty or contains empty subdirectories.
- Fix unhandled exception when limit file is actually a directory
  instead of a file.
- Fix inventory tests which previously could never fail due to missing
  `set -e`. Fixed up tests that failed after `set -e` was added. Added
  several tests.

Test Plan:
- New tests
- Fixed existing tests which previously could never fail

Tickets:
- Fixes #73658

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-22 16:18:23 -05:00
Felix Fontein
b6566d26df
pylint sanity test: stop ignoring 'used-before-assignment' (#73639)
* Remove pylint ignore 'used-before-assignment'.

* Adjust changelog fragment.
2021-02-18 12:36:28 -08:00
Sandra McCann
ccbfdec334
Split Ansible docs from core docs (#73616)
* excludes scenario guides from core docs, splits porting guides and roadmaps, symlinks indices to create index.html pages, and adds .gitignore entries for conf.py and the toplevel index.rst files generated by the docs build

This solution builds three types of docs:
* ansible-2.10 and earlier: all the docs.  Handle this via `make webdocs
  ANSIBLE_VERSION=2.10`
* ansible-3 and later: a subset of the docs for the ansible package.
  Handle this via `make webdocs ANSIBLE_VERSION=3` (change the
  ANSIBLE_VERSION to match the version being built for.
* ansible-core: a subset of the docs for the ansible-core package.
  Handle this via `make coredocs`.

* `make webdocs` now always builds all the collection docs
*  Use `make coredocs` to limit it to core plugins only
*  The user specifies the desired version. If no ANSIBLE_VERSION is specified, build plugins for the latest release of ansible
 
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored-by: Matt Clay <matt@mystile.com>
2021-02-17 09:57:05 -06:00
Sloane Hertel
00bd0b893d
ansible-galaxy - set the cache file after getting all collection versions (#73557)
* Manage the in-memory cache in _call_galaxy but let the caller set the file cache after getting paginated results

* Add a test for caching successful and not caching unsuccessful paginated results

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-02-15 09:45:01 -05:00
Felix Fontein
29aef842d7
ansible-test: make sure tests are also run for the new plugin types (#73599)
* Make sure tests are also run for the new plugin types.
* New plugin types are not documentable.
2021-02-13 14:05:16 -08:00
Matt Clay
d092356fc5 Fix ansible-test handling of egg-info.
Resolves https://github.com/ansible/ansible/issues/67990
2021-02-13 01:05:00 -08:00
Matt Clay
133a29acb4 Add test to verify pkg_resources imports work. 2021-02-12 23:51:37 -08:00
Sam Doran
18f7282ccf
ansible-doc - account for empty meta/main.yml (#73590)
* ansible-doc - account for empty meta/main.yml

 from_yaml() will return None when encrounting an empty file.
2021-02-12 17:37:00 -05:00
David Shrewsbury
d1d9406066
Add rolespec_validate to import/include_role (#73589)
* Add rolespec_validate to import/include_role

* Add changelog

* fix sanity, not private
2021-02-12 16:26:33 -06:00
Felix Fontein
1f3a90270b
Import sanity test for plugins (#72497) 2021-02-12 14:09:50 -08:00
David Shrewsbury
f0ec10dbc3
Role arg spec validation implementation (#73152)
* Initial import of modified version of alikins' code
* Add unit testing for new Role methods
* Fix validate_arg_spec module for sanity test. Add test_include_role_fails.yml integration test from orig PR.
* Add testing of suboptions
* Use new ArgumentSpecValidator class instead of AnsibleModule
* fix for roles with no tasks, use FQ name of new plugin
* Add role dep warning
2021-02-12 16:00:17 -05:00
Brian Coca
ea2f37d253
allow constructed to use vars plugin (#73418)
Allow constructed to optionally use vars plugin data

* mostly for those looking to leverage group_vars/ and host_vars/
* limited to already processed sources
2021-02-12 11:14:50 -05:00
Matt Clay
30a4ef4414 Update ansible-test default test containers. 2021-02-11 19:43:02 -08:00
Gonéri Le Bouder
0a8d5c0983 ansible-test: yamllint, check the assigment
Ensure `yamllint`'s `check_assignment()` correctly ignore the
attribute assignment. Those don't have any `.id` attribute and will
trigger an `AttributeError` exception.

See: https://github.com/ansible/ansible/pull/73322
2021-02-11 19:42:27 -08:00
Sviatoslav Sydorenko
f327e65d11
Use iskeyword and str.isidentifier for "is FQCN" (#73279)
* Use valid FQCN in test_verbosity_arguments

* Use iskeyword and str.isidentifier for "is FQCN"
2021-02-11 18:23:57 -08:00
Matt Clay
f533d46572
Fix ansible-test constraints issues. (#73578)
* Silence Python 3.5 EOL notice in pip.
* Skip import test on compat/selinux.py utils.
* Improve Python version skip warning.
* Use Python 3.6 as minimum Python for sanity tests.
* Improve min Python handling for code-smell tests.
* Overhaul test-constraints sanity test.
* Merge sanity test constraints with requirements.
* Remove legacy content specific constraints.
* Add changelog fragment.
2021-02-11 16:45:53 -08:00
Sam Doran
b6811dfb61
Add argument spec validator (#73335)
Add argument spec validator class
2021-02-11 19:17:14 -05:00
Alexander Sowitzki
d0fda3e901
Let vault lookup output unicode string. (#73571)
Until now, the lookup plugin returned a byte string.
Changed this to output a unicode string instead.
2021-02-11 14:27:47 -05:00
Brian Coca
ec8a556538
only add data when there is data to add (#54559)
Only add data when there is data to add

  also avoid clobbering existing data with empty file
  fixes #45843

* remove redundant code, update comments
* fix mock dataloader, original does not return None
* added test
2021-02-11 10:07:20 -05:00
Matt Davis
4c5ce5a1a9
module compat for py3.8+ controller (#73423)
* module compat for py3.8+ controller

* replaced internal usages of selinux bindings with internal ctypes binding (allows basic selinux operations from any Python interpreter), plus tests

* added new respawn_module API to allow modules to import Python packages that are only available under a well-known interpreter, plus tests

* added respawn logic to modules that need Python libs from a specific system interpreter (apt, apt_repository, dnf, yum)

minimize internal HAVE_SELINUX usage

spurious junk

pep8

* pylint fixes

* add RHEL8 Python 3.8 testing

* more pylint

* import sanity

* unit tests

* changelog update

* fix a bunch of stuff

* tweak changelog

* fix setup_rpm_repo on EL8

* misc sanity/test fixes

* misc feedback tweaks

* fix import fallback in test module

* fix selinux MU test

* fix dnf tests to avoid python-dependent test packages

* add trailing LFs to aliases

* fix yum tests to avoid test package with Python deps

* hack create_repo for EL6 to create noarch package
2021-02-10 21:32:59 -08:00
Matt Clay
8a175f59c9
Upgrade pylint used by ansible-test. (#70155)
* Upgrade pylint and deps in ansible-test.
* Enable pylint on Python 3.9.
* Update pylint config.
* Add ignore for vendored six.
* Add ignores for support plugins.
* Fix issue reported by pylint.
2021-02-10 15:52:41 -08:00
Jeff Squyres
6fd0a74601
unarchive: add support for .tar.zst (zstd compression) (#73265) 2021-02-09 23:14:41 -06:00
pushkarkumar15
4344607d7d
user - add password expiration parameters (#69531)
* Add integration test

Co-authored-by: Sam Doran <sdoran@redhat.com>
2021-02-09 16:41:15 -05:00
Sam Doran
2f5c83dfb1
Add AlmaLinux to the family of Red Hat-like operating systems (#73541)
* Add changelog and fixtures for AlmaLinux support

Co-authored-by: Christoph Schug <com+github@schug.net>
2021-02-09 13:04:39 -05:00
Sam Doran
7d18ea5e93
default callback - add "show_task_path_on_failure" option (#73260)
When running in verbosity <2, display the file and line number for tasks that fail. This provides
useful information without having to run at increased verbosity.

* Move _print_task_path to CallbackBase class
* Add integration tests
* Add color parameter to _print_task_path()

* Keep color output consistent for now
  Currently the path is display with COLOR_DEBUG formatting with verbosity >= 2.
  Instead of the color of the path changing based on verbosity level, just keep it at the
  currently behavior of COLOR_DEBUG. Having the color of the same information change
  based on verbosity level seems incorrect and makes visual parsing of the information
  more difficult.

Co-authored-by: tahar.jegham <jeghamseifeddine@gmail.com>
2021-02-09 12:43:59 -05:00
Matt Martz
d3441bc313
Add new split filter (#73532)
* Add version added
2021-02-09 10:37:15 -05:00
Matt Clay
1a2da990a4
Temporary fix for cryptography issues. (#73530) 2021-02-08 14:35:00 -08:00
Rick Elrod
aca5b0e43b
[setup_rpm_repo test] Ensure rpm-build is present (#73516)
Change:
- Other targets might remove rpm-build as they clean up after
  themselves. Ensure that it's present in setup_rpm_repo because
  rpmfluff needs it.

Test Plan:
- Local experimentation with yum_repository and mysql_db (the latter of
  which depends on a handler which was removing rpm-build) on
  stable-2.9.

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-05 15:58:02 -05:00
Matt Martz
125c220343
Make non-lockstep callback tests more deterministic (#73511) 2021-02-05 14:11:40 -06:00
Shane McDonald
67f5bb39c7
galaxy - add format options for collection list (#73474)
* Include all collections in single json object / yaml document
* Add tests
* For galaxy list yaml/json output, use dictionary of dictionaries instead of list
* Add tests for listing single collection in yaml / output format
* --output -> --format
* Add explicit test for listing collection in human format
* Fix bug where empty json object was emitted + add test
2021-02-05 13:24:59 -05:00
Matt Martz
d3f3784b86
Don't treat host_pinned as lockstep (#73484)
* Don't treat host_pinned as lockstep. Fixes #73364

* Add intg tests
2021-02-05 09:06:19 -06:00
Florian Heiderich
8d6136eab9
uri - add ca_path parameter (#71979)
* add changelog fragment for #71979 (ca_path for uri)
* add integration tests for ca_path in the uri module
* return path of ca cert instead of its content
* connect to port 444 on self_signed_host
  and use quay.io/ansible/http-test-container:1.3.0
* state that the certificate in ca_path is used for validation
2021-02-05 09:40:52 -05:00
Felix Fontein
82cdd7e735 Bump antsibull-changelog version for changelog sanity test. 2021-02-04 16:42:26 -08:00
Jordan Borean
0cdc410dce
no_log mask suboption fallback values and defaults CVE-2021-20228 (#73487)
* no_log mask suboption fallback values and defaults

* Added changelog

* Remove lambda expression
2021-02-05 09:11:55 +10:00
Florian Heiderich
8f887b7adb
add self-signed https endpoint for ansible-test (#73463)
* introduce self-signed.ansible.http.tests

* forwarding of port 444

* forward port 8444 to port 444 on http test container

* Fix port forwarding for Windows under docker

* add changelog fragment

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2021-02-05 05:12:22 +10:00
Sam Doran
997b2d2a19
Make yum_repository test more reliable (#73467)
* Improve setup_rpm_repo

- add handlers to remove repos
- add variable to control whethere or not repos are created

* Use local repo for all distros
* Change repo creation script to module
2021-02-03 17:34:12 -08:00
Sloane Hertel
9759e0ca49
add option to upgrade collections (#73336)
* Add a flag to ansible-galaxy to update collections


Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-02-04 07:08:02 +10:00
Gonéri Le Bouder
7cf80f50d1
validate_modules: fails with .id attribute not found (#73322)
* validate_modules: fails with .id attribute not found

This patch addresses a problem in the `found_try_except_import` test.

This module tries to identify lines like:

`HAS_FOO = True`

In this case, the target (`HAS_FOO`) is of type `ast.Name` and has a
`id` attribute which provide the name.

In my case, I've a line that set a module attribute`. In this case, the
target (`module.var`) has the type `ast.Attribute` and no `id`
attribute. The code trigger an `AttributeError` exception.

This patch ensures we compare a `ast.Name`.

* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py
2021-02-02 13:29:36 -08:00
Jose Angel Munoz
69631da889
lineinfile - add search_string parameter for non-regexp searching (#70647)
* Add tests for search_string
* Improve examples
* Add changelog
2021-02-02 15:37:06 -05:00
Rick Elrod
9a9272305a
Correct pkg_mgr for Fedora-derived OSTree distros (#73445)
Change:
- Remove check that states that only Fedora can be an OSTree
  distribution.
- This allows us to correctly return "atomic_container" as the pkg_mgr
  fact for RHEL for Edge, Fedora/RHEL/CentOS Atomic Host, etc.

Test Plan:
- Created local RHEL for Edge image and tested against it.
- Tested against regular RHEL 8 and still got `dnf` as expected.
- Tested against RHEL 7 Atomic Host and got `atomic_container` now.
- New unit tests.

Tickets:
- Fixes #73084

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-02 15:09:30 -05:00
Alexander Sowitzki
1c83672532
Allow $ & ~ inside paths in galaxy roles (#72966)
ansible-galaxy currently behaves bad then a role to be installed
contains ~ or $ at any place in the path of a file in that role.
It extracts the parent directory of the offending path level as an
empty file. This explodes if that directory contains anything else.

Change this behaviour. `~` is now allowed allowed when it is
not a full level (Yes: `some~thing/`, no: `~/`). The code should
get refactoring in an other PR.
2021-02-02 18:10:05 +01:00
Matt Clay
a9b5bebab3
Overhaul ansible-test SSH key management. (#73451)
* Pass remote.sh to shell over stdin.
* Pass docker.sh to shell over stdin.
* Standardize SSH key management.
* Update docker containers.
2021-02-02 08:43:54 -08:00
Alexander Sowitzki
218f5c3648
user - Prevent user fact lookup failure if LOGNAME is set (#17029) (#73439)
The `UserFactCollector` queries the user login name via
`getpass.getuser()` and looks up the corresponding entry
in the password database.
The login name may differ from the actual user name,
eg. if the `LOGNAME` env variable is set. The lookup
fails in this case. Added a fallback in this case that
tries to get the entry via the user ID.
2021-02-02 10:36:02 -05:00
Rick Elrod
bce01655cc
[ansible-test] Bump CentOS 6 image (yum repos) (#73446)
Change:
- Bump centos6 image version to one which includes multiple fallbacks
  for vault.centos.org content.

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-01 16:24:21 -06:00
Sloane Hertel
840bdc1e10
Fix warning for nonexistent inventory cache (#72840)
* Fix inventory cache warning by checking if the key exists before loading it
* changelog
2021-02-01 14:29:36 -05:00
Matt Clay
81cd8e46f2 Update ansible-test distro containers to 1.29.0. 2021-02-01 09:24:28 -08:00
Sloane Hertel
728dafb6eb
Make ansible-galaxy collection verify error out on no MANIFEST.json
PR #73403

Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
2021-02-01 09:55:21 +01:00
Sviatoslav Sydorenko
6f4b4c345b
Match user-requested transitive pre-releases in collection dependency resolver (#73416)
* Add direct+transitive pre-release regression test

* Match user-requested transitive pre-releases

This change makes sure that in scenario when a user requests
collectionA-pre and collectionB that depends on collectionA,
collectionA-pre would actually match that collectionA requirement
while `--pre` is unset.

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2021-02-01 14:26:49 +10:00