Commit graph

10896 commits

Author SHA1 Message Date
David Shrewsbury
1e5ccb326f
Allow for searching handler subdir for included tasks (#73809)
* Allow for searching handler subdir for included tasks
2021-03-15 15:39:59 -04:00
Paul Belanger
9ec4e08534
Don't match passive_interface for validate-modules (#73880)
This is a follow up to:

  https://github.com/ansible/ansible/pull/73508

To avoid adding no_log statements to passive_interface args.

Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2021-03-13 08:28:57 -06:00
Felix Fontein
7c0af58449
validate-modules no_log test: extend search range, add more known false-positives (#73882)
* validate-modules no_log test: extend search range, add more known false-positives.

* Mark false-positives.
2021-03-13 08:28:33 -06:00
Mark Chappell
ed18fcac3b
Ensure unit test paths for connection and inventory plugins are based on the context (#73877) 2021-03-12 12:46:40 -08:00
Matt Clay
3a8c9242e1 Use virtual host style S3 bucket names in tests. 2021-03-12 11:34:25 -08:00
Felix Fontein
b2015c98e2
validate-modules: make sure that options that potentially contain secret data have no_log set (#73508)
* Catch more potential errors (and increase false-positive rate).
* Flag some false-positives in lib/ansible/modules/ with no_log=False.

Co-authored-by: John Barker <john@johnrbarker.com>
2021-03-11 12:57:11 -05:00
Matt Davis
3e1f6484d7
add optional module_utils import support (#73832)
* add optional module_utils import support

Treat core and collections module_utils imports nested within any Python block statement (eg, `try`, `if`) as optional. This allows Ansible modules to implement runtime fallback behavior for missing module_utils (eg from a newer version of ansible-core), where previously, the module payload builder would always fail when unable to locate a module_util (regardless of any runtime behavior the module may implement).

* sanity test fixes

ci_complete
2021-03-10 11:08:13 -08:00
Matt Clay
15064c7a42 Update resource_prefix syntax for ansible-test. 2021-03-10 09:00:12 -08:00
Brian Coca
9db557e431
Nonfatal facts (#73804)
continue with local facts vs at script error
 actually capture execution errors
 better error messages in general
 add more local facts tests

 fixes #52427
2021-03-08 16:20:37 -05:00
Abhijeet Kasurde
fb66b4ffbc
distribution: Add Amazon Linux distribution facts (#73767)
* Update Amazon Linux Distribution facts gathering logic
* Update tests

Fixes: #73742

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-03-05 09:16:10 -05:00
David Shrewsbury
78d3810fdf
Auto cleanup of async cache file (#73760)
* Auto cleanup of async cache file

* Add changelog
2021-03-04 11:06:27 -08:00
Brian Coca
ce1de28061
Fix notify on import_tasks (#73572)
* Fix notify on import_tasks

  also able to apply to blocks now.
2021-03-03 16:11:09 -05:00
Brian Coca
935528e22e
finish migrating ssh plugin to config system (#73708)
* finish migrating ssh plugin to config system

  fixes #72739
  fixes #57220

* fix connection detection in reset
* correct options for connection meta reset

Co-authored-by: David Shrewsbury <Shrews@users.noreply.github.com>
2021-03-03 15:25:16 -05:00
Brian Coca
43300e2279
module output is only json objects (#73765)
* module output is only json objects

 remove json lists as they are not valid from modules
 fixes #73744
2021-03-03 15:11:18 -05:00
Brian Coca
527bff6b79
Don't fail for mixed typed keys (#73726)
* Don't fail for mixed typed keys

  but warn that content cound not be sorted because of this

* added tests
2021-03-03 14:24:50 -05:00
Matt Clay
f40dde44f4 Remove old ansible-test windows completion entries. 2021-03-02 10:48:03 -08:00
Matt Clay
b6add939ff Remove outdated --docker and --remote entries. 2021-03-02 10:05:39 -08:00
Felix Fontein
3cc693b92c
Import test: improve docs, add more tests (#73600) 2021-03-01 16:10:46 -08:00
Abhijeet Kasurde
6cb324bb0e
selinux: return selinux_getpolicytype facts (#73609)
* selinux: return selinux_getpolicytype facts

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

* add basic selinux facts tests

* fix selinux facts test when selinux missing

Co-authored-by: Matt Davis <mrd@redhat.com>
2021-03-01 11:11:09 -08:00
Matt Davis
d0e991e892
Clarify CLI version number as core version (#72287)
* clarify CLI version number as core version

* reduce confusion with `ansible` PyPI package >= 2.10 drifting from core version

* fix units
2021-03-01 11:14:56 -06:00
Alexei Znamensky
920b68f5f2
Fixed/improved regular expresssion for collection names (#73577)
* added changelog fragment
* added a couple of tests to coll name validation
2021-03-01 10:40:38 -05:00
Alexander Sowitzki
e804fccf1c Let file module not change link to absolute when src not given (#65448)
The file module changes existing sym links from relative to absolute
if the src is not stated in the tasks since it uses `os.path.realpath`
to fetch the link source and not `os.readlink`. Changed that.
2021-03-01 15:14:03 +01:00
Sam Doran
2377a0a776
Improve ArgumentSpecValidator unit tests (#73642)
* Add more scenarios to basic valid testing
* Update invalid tests
* Fix test for Python 2
* Condense data
* Add tests for missing required and invalid-elements
* Update aliases tests
* Add invalid scenarios for aliases
* Add tests for _add_error() method
* Fix sanity test failure
2021-02-26 09:30:16 -05:00
Sloane Hertel
8628c12f30
find module - stop traversing directories with os.walk when depth is already exceeded (#73718) 2021-02-26 05:32:49 +10:00
Matt Martz
950ab74758
Normalize ConfigParser between Python2 and Python3 (#73715)
* Normalize config parser between py2 and py3

* Add tests and changelog

* Use different config entry, since we supply certain env vars
2021-02-25 11:03:03 -06:00
manas-init
eb72c36a71
galaxy: Handle ignored directory names in role skeleton (#72035)
* galaxy: restore left hand slicing in assignment

Fix 'ansible-galaxy role init --role-skeleton=role-skeleton' when the role skeleton
contains an ignored directory.

The issue was because the 'dirs' variable was changed to reference a different list,
but needs to be mutated instead to stop os.walk from traversing ignored directories.

Fixes: #71977

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2021-02-24 18:52:24 -05:00
Matt Martz
11f1177e6c
Operate pexpect with bytes to limit encoding issues (#73255)
* Operate pexpect with bytes to limit encoding issues

* Update tests to ensure no pepxect encoding issues

* Add changelog fragment

* Add multiline note

* Use rst formatting directly
2021-02-23 18:57:25 +01:00
Sam Doran
3ab52f2965
Disable junit callback on roles_arg_spec_tests (#73663)
This prevents the role arg spec validation tasks from showing up as failure
in the CI output.
2021-02-23 10:21:28 -05:00
Rick Elrod
fa046d302c
[InventoryManager] Fix two unhandled exceptions (#73667)
Change:
- Fix regression: unhandled exception when given inventory directory
  is empty or contains empty subdirectories.
- Fix unhandled exception when limit file is actually a directory
  instead of a file.
- Fix inventory tests which previously could never fail due to missing
  `set -e`. Fixed up tests that failed after `set -e` was added. Added
  several tests.

Test Plan:
- New tests
- Fixed existing tests which previously could never fail

Tickets:
- Fixes #73658

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-02-22 16:18:23 -05:00
Felix Fontein
b6566d26df
pylint sanity test: stop ignoring 'used-before-assignment' (#73639)
* Remove pylint ignore 'used-before-assignment'.

* Adjust changelog fragment.
2021-02-18 12:36:28 -08:00
Sandra McCann
ccbfdec334
Split Ansible docs from core docs (#73616)
* excludes scenario guides from core docs, splits porting guides and roadmaps, symlinks indices to create index.html pages, and adds .gitignore entries for conf.py and the toplevel index.rst files generated by the docs build

This solution builds three types of docs:
* ansible-2.10 and earlier: all the docs.  Handle this via `make webdocs
  ANSIBLE_VERSION=2.10`
* ansible-3 and later: a subset of the docs for the ansible package.
  Handle this via `make webdocs ANSIBLE_VERSION=3` (change the
  ANSIBLE_VERSION to match the version being built for.
* ansible-core: a subset of the docs for the ansible-core package.
  Handle this via `make coredocs`.

* `make webdocs` now always builds all the collection docs
*  Use `make coredocs` to limit it to core plugins only
*  The user specifies the desired version. If no ANSIBLE_VERSION is specified, build plugins for the latest release of ansible
 
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored-by: Matt Clay <matt@mystile.com>
2021-02-17 09:57:05 -06:00
Sloane Hertel
00bd0b893d
ansible-galaxy - set the cache file after getting all collection versions (#73557)
* Manage the in-memory cache in _call_galaxy but let the caller set the file cache after getting paginated results

* Add a test for caching successful and not caching unsuccessful paginated results

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2021-02-15 09:45:01 -05:00
Felix Fontein
29aef842d7
ansible-test: make sure tests are also run for the new plugin types (#73599)
* Make sure tests are also run for the new plugin types.
* New plugin types are not documentable.
2021-02-13 14:05:16 -08:00
Matt Clay
d092356fc5 Fix ansible-test handling of egg-info.
Resolves https://github.com/ansible/ansible/issues/67990
2021-02-13 01:05:00 -08:00
Matt Clay
133a29acb4 Add test to verify pkg_resources imports work. 2021-02-12 23:51:37 -08:00
Sam Doran
18f7282ccf
ansible-doc - account for empty meta/main.yml (#73590)
* ansible-doc - account for empty meta/main.yml

 from_yaml() will return None when encrounting an empty file.
2021-02-12 17:37:00 -05:00
David Shrewsbury
d1d9406066
Add rolespec_validate to import/include_role (#73589)
* Add rolespec_validate to import/include_role

* Add changelog

* fix sanity, not private
2021-02-12 16:26:33 -06:00
Felix Fontein
1f3a90270b
Import sanity test for plugins (#72497) 2021-02-12 14:09:50 -08:00
David Shrewsbury
f0ec10dbc3
Role arg spec validation implementation (#73152)
* Initial import of modified version of alikins' code
* Add unit testing for new Role methods
* Fix validate_arg_spec module for sanity test. Add test_include_role_fails.yml integration test from orig PR.
* Add testing of suboptions
* Use new ArgumentSpecValidator class instead of AnsibleModule
* fix for roles with no tasks, use FQ name of new plugin
* Add role dep warning
2021-02-12 16:00:17 -05:00
Brian Coca
ea2f37d253
allow constructed to use vars plugin (#73418)
Allow constructed to optionally use vars plugin data

* mostly for those looking to leverage group_vars/ and host_vars/
* limited to already processed sources
2021-02-12 11:14:50 -05:00
Matt Clay
30a4ef4414 Update ansible-test default test containers. 2021-02-11 19:43:02 -08:00
Gonéri Le Bouder
0a8d5c0983 ansible-test: yamllint, check the assigment
Ensure `yamllint`'s `check_assignment()` correctly ignore the
attribute assignment. Those don't have any `.id` attribute and will
trigger an `AttributeError` exception.

See: https://github.com/ansible/ansible/pull/73322
2021-02-11 19:42:27 -08:00
Sviatoslav Sydorenko
f327e65d11
Use iskeyword and str.isidentifier for "is FQCN" (#73279)
* Use valid FQCN in test_verbosity_arguments

* Use iskeyword and str.isidentifier for "is FQCN"
2021-02-11 18:23:57 -08:00
Matt Clay
f533d46572
Fix ansible-test constraints issues. (#73578)
* Silence Python 3.5 EOL notice in pip.
* Skip import test on compat/selinux.py utils.
* Improve Python version skip warning.
* Use Python 3.6 as minimum Python for sanity tests.
* Improve min Python handling for code-smell tests.
* Overhaul test-constraints sanity test.
* Merge sanity test constraints with requirements.
* Remove legacy content specific constraints.
* Add changelog fragment.
2021-02-11 16:45:53 -08:00
Sam Doran
b6811dfb61
Add argument spec validator (#73335)
Add argument spec validator class
2021-02-11 19:17:14 -05:00
Alexander Sowitzki
d0fda3e901
Let vault lookup output unicode string. (#73571)
Until now, the lookup plugin returned a byte string.
Changed this to output a unicode string instead.
2021-02-11 14:27:47 -05:00
Brian Coca
ec8a556538
only add data when there is data to add (#54559)
Only add data when there is data to add

  also avoid clobbering existing data with empty file
  fixes #45843

* remove redundant code, update comments
* fix mock dataloader, original does not return None
* added test
2021-02-11 10:07:20 -05:00
Matt Davis
4c5ce5a1a9
module compat for py3.8+ controller (#73423)
* module compat for py3.8+ controller

* replaced internal usages of selinux bindings with internal ctypes binding (allows basic selinux operations from any Python interpreter), plus tests

* added new respawn_module API to allow modules to import Python packages that are only available under a well-known interpreter, plus tests

* added respawn logic to modules that need Python libs from a specific system interpreter (apt, apt_repository, dnf, yum)

minimize internal HAVE_SELINUX usage

spurious junk

pep8

* pylint fixes

* add RHEL8 Python 3.8 testing

* more pylint

* import sanity

* unit tests

* changelog update

* fix a bunch of stuff

* tweak changelog

* fix setup_rpm_repo on EL8

* misc sanity/test fixes

* misc feedback tweaks

* fix import fallback in test module

* fix selinux MU test

* fix dnf tests to avoid python-dependent test packages

* add trailing LFs to aliases

* fix yum tests to avoid test package with Python deps

* hack create_repo for EL6 to create noarch package
2021-02-10 21:32:59 -08:00
Matt Clay
8a175f59c9
Upgrade pylint used by ansible-test. (#70155)
* Upgrade pylint and deps in ansible-test.
* Enable pylint on Python 3.9.
* Update pylint config.
* Add ignore for vendored six.
* Add ignores for support plugins.
* Fix issue reported by pylint.
2021-02-10 15:52:41 -08:00
Jeff Squyres
6fd0a74601
unarchive: add support for .tar.zst (zstd compression) (#73265) 2021-02-09 23:14:41 -06:00