Commit graph

2451 commits

Author SHA1 Message Date
Sloane Hertel
8b07d46166
Add a toggle for the leading separator for keyed_groups (#60882)
* Add a toggle for the leading separator for keyed_groups if no prefix is given

* changelog

* Add tests for backward compatibility and the new toggle
2020-10-23 15:57:44 -04:00
Felix Fontein
0b2b82c0b6
Add changelog fragment for #71736. (#72323) 2020-10-23 14:34:31 -05:00
Matt Davis
83909bfa22
Remove ansible-galaxy login (#72288)
* GitHub is removing the underlying API used to implement the `login` command. Since the general consensus seems to be that relatively nobody currently uses this command (in favor of explicit token passing), support was simply removed for interactive login. If a future need arises, this command should be reimplemented via OAuth Device Auth Grants.
* login or role login commands now produce a fatal error with a descriptive message
* updated 2.10 and 2.11 porting guide entries

* remove dead code/config, update messages and porting guides
2020-10-23 09:11:45 -07:00
Matt Clay
3c2e8b99be
Fix ansible-test handling of user-defined docker networks. (#72256)
* Fix ansible-test docker container detection.

* Attach test containers to the correct network.

* Do not assume `localhost` for accesing Docker.

* Look for containers on current network.

* Always map /var/run/docker.sock into containers.

This fixes issues when using a remote Docker host.

* Support container IP lookup from networks list.

* Fix container network attachment.

* Remove redundant container detection messages.

* Limit DOCKER_HOST parsing to TCP.

* Restore docker socket existence check.

The check is skipped if the docker hostname is not localhost.

* Correct changelog entry.
2020-10-22 18:45:03 -07:00
Ruediger Pluem
e64ba97e8a
Fix the reversed order of return values (#72088)
* Fix the reversed order of return values

AnsibleModule.run_command returns a tuple of return code, stdout and stderr.
The module main function of the user module expects user.create_user to
return a tuple of return code, stdout and stderr.
Fix the locations where stdout and stderr got reversed.

* Fix another missed occurance

* Add changelog

* Fix further reversed order of return values occurrences

Followup to 72c3ba62c8:
Fix further reversed order of return values occurrences.
2020-10-22 20:02:53 +02:00
Rick Elrod
69e510e767
[virt facts] recognize containerd as a container (#72210)
Change:
- containerd is now recognized as container tech

Test Plan:
- Tested with Docker inside Github Actions in a private repo:

        "ansible_virtualization_role": "guest",
        "ansible_virtualization_tech_guest": [
            "containerd",
            "container",
            "VirtualPC"
        ],
        "ansible_virtualization_tech_host": [],
        "ansible_virtualization_type": "containerd",

Tickets:
- Fixes #66304 because this is what docker containers show up as
  in Github Actions.

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-10-20 10:39:13 -05:00
Jordan Borean
c9fa1d0e7e
Fix race condition when creating async dir (#72069)
* Fix race condition when creating async dir

* Simplify exception wrapper

* Remove var used for testing
2020-10-19 15:05:19 -04:00
Matt Martz
be5fc4e642
Ensure we call action_loader.get with collection_list (#72206)
* Ensure we call action_loader.get with collection_list. Fixes #72170

* Add tests and changelog

* Remove grep, do assertion in playbook. ci_complete

* Skip old jinja2 versions

* ci_complete

* dedupe
2020-10-19 09:54:24 -05:00
Brian Coca
5ec53f9db8
make collection callbacks follow normal flow (#59932)
make collections whitelist follow normal flow

* fixes missing set_options call and adhoc and stdout processing rules
* avoid dupes
* fixed to handle redirects
* also updated tests with new and more accurate skip message
* fix callback tests for envs with cowsay installed
* lots MOAR comments on why the code is as it is, some todos to refactor in future

Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
2020-10-15 15:31:18 -04:00
Matt Clay
2ef4b7e07e Fix ansible-test Azure Pipelines container auth. 2020-10-14 12:35:11 -07:00
Rick Elrod
3b40c6f3b7
[config] coerce more to string when 'type: str' (#72172)
Change:
- When a plugin defines `type: str` on a parameter, treat more kinds of
  input as a string instead of whatever it is parsed as.

Test Plan:
- New unit tests
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-10-14 12:53:08 -04:00
dberg1
cdf62edc65
service_facts: return service state information on AIX (#72073)
* service_facts: return service state information on AIX

AIX uses the System Resource Controller (SRC) to manage services.
See https://www.ibm.com/support/knowledgecenter/ssw_aix_72/osmanagement/sysrescon.htm
Use lssrc command on AIX to return service state information.


Co-authored-by: dberg1 <dberg1@github.com>
2020-10-13 11:28:59 -04:00
Jordan Borean
caba47dd3f
Add support for GSSAPI/Kerberos to urls.py (#72113)
* Add support for GSSAPI/Kerberos to urls.py

* Test out changes with the latest test container

* Get remote hosts working

* Fix up httptester_krb5_password reader

* Fix tests for opensuse and macOS

* Hopefully last lot of testing changes

* Dont do CBT on macOS

* Fixes from review
2020-10-13 14:16:07 +10:00
Matt Martz
c4acd41d6e
Ensure delegate vars calculation has correct loop context. Fixes #37132 (#71477) 2020-10-12 14:21:15 -05:00
Matt Clay
5f76bd2af7 Support collection constraints in ansible-test.
This allows collections to specify requirements and constraints for packages that ansible-test has requirements or constraints for.
2020-10-08 12:20:19 -07:00
Abhijeet Kasurde
9ffa84cc1c
basic: Handle exception in default selector (#72101)
In Python 2.7, default selector raises IOError
instead of OSError. Fix handles this exception.

Fixes: #71704

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-10-06 11:25:51 -04:00
Rick Elrod
ebc91a9b93
[unarchive] work on older RHEL with group: <gid> (#72098)
Change:
- No longer fail due to old Fedora/RHEL and our failure to try to cast
  gids to integers before trying to pass them to getgrgid() before
  trying to use them.
- Add tests for user/mode for various unarchive formats.

Test Plan:
- New integration tests, ran against centos6 container

Tickets:
- Fixes #71903
2020-10-06 11:25:03 -04:00
Rick Elrod
a90e37d017
[virt facts] identify KubeVirt properly (#72092)
Change:
- Make KubeVirt be identified as such, and with a guest role.
- This is a breaking change, since playbooks could be relying on the
  incorrect behavior previously.

Test Plan:
- Issue author

Tickets:
- Fixes #72001

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-10-06 11:22:22 -04:00
Rick Elrod
0332046699
[ansible-test] attempt to work around podman (#72096)
Change:
- podman > 2 && < 2.2 does not support "images --format {{json .}}"
- podman also now outputs images JSON differently than docker
- Work around both of the above.

Test Plan:
- Tested with podman 2.0.6 in Fedora 31.

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2020-10-05 20:13:52 -05:00
Evan Van Dam
3db08adbb1
Add optional attribute arg for min and max filters (#50909)
* Pass **kwargs to min and max filters

* Use the jinja2 filters if available

* Add unit tests

* Add examples to docs passing attribute
2020-10-05 10:40:37 -04:00
Rick Elrod
709484969c
sysctl/openbsd fact fixes (#72070)
Change:
- Use `sysctl -n` for openbsd uptime information
- Allow `get_sysctl()` to account for multi-line sysctl settings
- Add unit tests for `get_sysctl()`

Test Plan:
- New unit tests

Tickets:
- Fixes #71968
- Refs #72025
- Refs #72067

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Brian Coca <brian.coca+git@gmail.com>
Co-authored-by: Brian Coca <bcoca@ansible.com>
2020-10-02 18:36:31 -05:00
Ruediger Pluem
a7170da851
user - allow local users with an expiry date to be created (#72022)
The luseradd / lusermod commands do not support the -e option. Set
the expiry time in this case via lchage after the user was
created / modified.

Fixes: #71942

In Python3 math.floor returns an integer whereas Python2 returns a float.
Hence always convert the result of math.floor to an int to ensure that
lexpires is an integer.

Move local expires tests in a separate file and import the tasks to the
main.yml to keep main.yml smaller.
2020-10-02 15:38:27 -04:00
Sloane Hertel
cceba07114
VariableManager - add the 'vars' key before getting delegated variables (#71214)
* VariableManager - add the 'vars' key before getting delegated variables
2020-10-02 15:13:56 -04:00
Matt Martz
fb03ac7019
Add libyaml info to version output, and restore git info (#71948)
* Add libyaml info to version, restore git functionality

* Add clog fragment

* Update tests to reflect new version output
2020-10-02 14:04:49 -05:00
Matt Martz
0e480d38dd
uri status_code elements are type int (#72062) 2020-10-02 13:36:14 -05:00
Felix Fontein
1107aace1b
Add deprecation collection name to plugin options (#71735)
* Add changelog fragment.

* Use correct field that's expected by lib/ansible/cli/__init__.py..

* Add basic unit tests.
2020-10-02 09:43:23 -04:00
Amin Vakil
e68a638e7c
WIP: OpenBSD: add uptime to gather_facts (#72025)
* OpenBSD: add uptime to gather_facts
2020-10-01 15:34:40 -04:00
Bernhard Dick
f1c4a954d5
Close file descriptor of temporary file after building certificate chain (#71825)
* Close file descriptor of temporary file after building certificate chain.

* Add changelog fragment for PR71825
2020-09-30 20:43:24 +02:00
Christian Loos
ea119d3089
fix distribution fact for SLES4SAP (#71559)
b6b238a fixed the SLES4SAP detection, which was at this time ok.
Sadly Suse changed with SLES 15 the /etc/os-release file, so the above
change will no longer work.

This commit updates the SLES4SAP detection regarding
https://www.suse.com/support/kb/doc/?id=000019341.

The symlink realpath is matched with endswith, because in SLES 12+ the
link target is SLES_SAP.prod, but in SLES 11 the link target is
SUSE_SLES_SAP.prod.
2020-09-30 10:52:41 -04:00
Martin Krizek
4197666179
Only apply the unroll wrapper once (#72003)
Co-authored-by: Matt Martz <matt@sivel.net>

Fixes #71920
2020-09-30 09:15:28 +02:00
Felix Fontein
8893a244b9
Validate plugin option type 'dict' / 'dictionary' (#71928)
* Validate option type 'dict' / 'dictionary'.

* Add changelog fragment.

* Change type of 'environment' to list.
2020-09-29 17:16:14 -04:00
Felix Fontein
a077bca5d5
ansible-test: improve version number validation, validate some semantic versioning properties (#71679)
* Validate removal versions.
* Validate that removal collection versions and version_added collection versions conform to semver spec.
* Validate removal version numbers in meta/runtime.yml.
* Stricter validation for isodates (f.ex. YYYY-M-D is not allowed).
* Improve error reporting.
* Validate removal collection versions.

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
2020-09-29 13:33:45 -07:00
Jordan Borean
3c33618cf6
powershell - remove env var (#72010) 2020-09-30 06:24:31 +10:00
Carlos Matos
cbd3df369f
Adding check to ensure 'container' get's added to guest_tech when appropriate (#71385) 2020-09-28 14:21:44 -05:00
Felix Fontein
0c3a9c7ae6
ansible-doc: show correct plugin name (#71966) 2020-09-28 10:40:33 -05:00
Alicia Cozine
91b0dfb659
adds changelog for already-merged PR (#71947)
* adds changelog for already-merged PR

* Update changelogs/71921-raise-bytes-for-binary-test

Co-authored-by: Matt Clay <matt@mystile.com>

* adds yaml extension, corrects placement

Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Co-authored-by: Matt Clay <matt@mystile.com>
2020-09-25 14:14:17 -04:00
Matt Clay
92b66e3e31
CI provider fixes for ansible-test. (#71929)
* Make Azure Pipelines resource_prefix lowercase.

* Make classification of CI files consistent.

* Update package-data sanity test for AZP.
2020-09-24 17:34:07 -07:00
Brian Coca
9a39452a31
update the definition of changed (#71865)
* add nuance to the definition of changed, add notes to modules that return "changed" without changing state
2020-09-24 14:33:22 -05:00
René Moser
5348e7bf61
test: cloudstack: Use test-container 1.4.0 (#71905) 2020-09-24 11:58:37 -07:00
Rick Elrod
5cd489af06
Let get_file_attributes() work without lsattr -v (#71845)
* Let get_file_attributes() work without `lsattr -v`

Change:
- module_utils's get_file_attributes() expects `lsattr -v` to work, but
  in some cases, it may not.
- The function now takes an optional include_version bool parameter,
  which removes this expectation.
- Places where we call get_file_attributes() without using the 'version'
  it returns, we now call it with include_version=False.

Test Plan:
- New unit tests

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-09-22 16:15:47 -05:00
Amin Vakil
3e9943bc5e
Reduce ignored module sanity tests (#70319)
* Reorder test/sanity/ignore.txt to better see changes
* Remove extra empty line causing pass fail
* Apply suggestions made by Akasurde
* Minor fix in package_facts and remove two unnecessary ignores
* Fix subversion based on a suggestion made by felixfontein
* Apply suggestions made by felixfontein and Andersson007
* Fix subversion.py as suggested by felixfontein
* Minor reformatting in yum_repository description
* Reformat changelog
* Add key to apt_key, add deprecated changelog
* Add PR url to changelog
* Ignore paramater-type-not-in-doc in favour of adding key back to apt_key
* Fix apt_key
* Remove undocumented-paramater from apt_key ignore
* Ignore doc-choices-do-not-match-spec in package_facts
* Fix package_facts
* Fix filter option in setup module
2020-09-22 16:07:29 -04:00
Rick Elrod
abfb7919dc
[default callback] Show include_tasks task banner (#71821)
Change:
- In some cases (always with free strategy, sometimes with linear), the
  default callback would not show the task banner for include_tasks.
- This only affects the include_tasks task itself, not the tasks in the
  included file.

Test Plan:
- Updated default callback tests

Tickets:
- Fixes #71277

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-09-22 11:40:12 -04:00
Martin Krizek
00b22ab55e
Provide more information in AnsibleUndefinedVariable (#71666)
* Provide more information in AnsibleUndefinedVariable

Fixes #55152
2020-09-22 09:38:53 -04:00
Rick Elrod
fdf80690e4
[dnf] accumulate update filters (#71726)
Change:
- Previously when `security: true` and `bugfix: true` were both given,
  only security updates would get applied. Filters now accumulate so
  that both get applied in this case.

Test Plan:
- New integration tests for both check_mode and not. These tests make
  use of a contrived yum repository which is stored in S3.

Tickets:
- Fixes #70854

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Matt Martz <matt@sivel.net>

Co-authored-by: Matt Martz <matt@sivel.net>
2020-09-15 11:36:18 -04:00
Sloane Hertel
7048542199
Fix passing the connection timeout to connection plugins (#71722)
* Fix passing the connection timeout to connection plugins
2020-09-15 11:34:11 -04:00
Abhijeet Kasurde
173091e2e3
distro: Add support for Pardus distribution (#71663)
Fixes: #71636

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-09-15 09:47:24 -04:00
Jordan Borean
fc08c1f3c5
runas - create new SYSTEM token on become (#71714) 2020-09-15 05:39:23 +10:00
Matt Davis
e813b0151c
fix coverage output from synthetic packages (#71727)
* fix coverage output from synthetic packages

* synthetic packages (eg, implicit collection packages without `__init__.py`) were always created at runtime with empty string source, which was compiled to a code object and exec'd during the package load. When run with code coverage, it created a bogus coverage entry (since the `__synthetic__`-suffixed `__file__` entry didn't exist on disk).
* modified collection loader `get_code` to preserve the distinction between `None` (eg synthetic package) and empty string (eg empty `__init__.py`) values from `get_source`, and to return `None` when the source is `None`. This allows the package loader to skip `exec`ing things that truly have no source file on disk, thus not creating bogus coverage entries, while preserving behavior and coverage reporting for empty package inits that actually exist.

* add unit test
2020-09-14 09:14:23 -07:00
Tadej Borovšak
74a103d655 Remove em dash from the pytest configuration file
On some systems (tested with official Debian Buster-based Python 3.6
docker image), setting the LC_ALL environment variable to en_US.UTF-8
will trigger Python into switching its preferred encoding to ASCII.
If any python process tries to read a non-ASCII file in this scenario,
it will terminate with an error.

And this is exactly what happens to pytest when it tries to load its
configuration that ansible-test supplies because the configuration
contains an em dash.

In order to bypass this issue, we replaced the em dash with a regular
dash and things started working again.

An alternative solution would be to replace the en_US.UTF-8 locale with
something safer, but unfortunately, the en_US.UTF-8 is probably as safe
as it gets.
2020-09-14 09:02:48 -07:00
Jérôme Loyet
daca3ade99
Allow list of filters for the setup module (#68551)
The setup module can now filter out multiple pattern by providing a list
to the filter parameter instead of just a string. Single string sill
works. Previous behaviour remains.

(cherry picked from commit b5c36dac483fdd74d6c570d77cc8f3e396720366)
2020-09-09 20:40:48 +02:00