Commit graph

2491 commits

Author SHA1 Message Date
Sam Doran
bbef250c2b
Fix super annoying Python 2.6 multiprocessing.Queue stack trace in CI (#72604)
* Fix super annoying Python 2.6 multiprocessing.Queue stack trace in CI

A bug exists in Python 2.6 that sometimes raises an exception during interpreter shutdown. We
encounter this frequently in our CI since we run tests on CentOS 6 as the control node, which
has Python 2.6.6 with this bug.

This PR adds a very minor sleep only on Python 2.6 which gets around this issue. I did lot of testing
using a standalon script I found that easily duplicated the issue to find the minimum sleep value
needed to avoid this issue.

CPython issue: https://bugs.python.org/issue4106
Fix in CPython: https://hg.python.org/cpython/rev/d316315a8781

* Use correct attribute
2020-11-12 14:39:08 -05:00
Sam Doran
4b8cb6582b
pause - do not hang if run in the background (#72065)
* Consolidate logic for determining whether or not session is interactive
  into a single function, is_interactive()
* Increase test coverage

I wasn't able to find a good way of simulating running a backgrounded test with CI since the
whole test is essentially run not in a TTY, which is similar enough to cause the new is_interactive()
function to always return false.
2020-11-12 12:22:57 -05:00
Paul
44a38c9f33
add return documentation for unarchive. (#72546)
* add return documentation for unarchive, add fragment for changelog, apply suggestions from code review

Co-authored-by: Amin Vakil <info@aminvakil.com>
2020-11-10 16:27:39 -06:00
Jordan Borean
bdd0c48837
Added changelog for RHEL 7.9 (#72572) 2020-11-11 05:30:11 +10:00
Brian Coca
4f0e2fff95
avoid fatal exception on invalid collection name (#72296)
* avoid fatal exception on invalid collection name used in ansible-doc filter
2020-11-10 11:46:15 -05:00
Brian Coca
48c08f410c
allow any type of unsafe data (#72547)
* allow any type of unsafe data

  dont limit to strings
2020-11-10 10:48:20 -05:00
Egor Margineanu
e879f12fb9
Fix AIX networks facts when nestat is either missing or has incorrect permissions (#72516)
* Added check for none on netstat_path variable

* Added changelog
2020-11-10 10:07:28 -05:00
Matt Clay
fa2be89cd4 Add coverage exporting to ansible-test
A new `--export` option for `ansible-test coverage combine` allows multi-step aggregation of code coverage for CI pipelines.
2020-11-09 22:34:05 -08:00
Matt Clay
f022dedd0a Fix container discovery for the acme test plugin. 2020-11-09 14:11:12 -08:00
Brian Coca
aa4d53ccdf
ensure local exposes correct user (#72543)
* ensure local exposes correct user

  avoid corner case in which delegation relied on
  playcontext fallback which was removed

  fixes #72541
2020-11-09 16:21:17 -05:00
Jordan Borean
de5858f48d
Added caching mechanism for Galaxy API requests (#71904)
* Added caching mechanism for Galaxy API requests

* Add cache options and split up code

* Added unit tests

* Fix sanity test

* Use modified date and fix up caching for explicit servers

* Make sure credentials are not in cached server name

* Added test for getting updated cache version

* Changes from review
2020-11-10 06:50:32 +10:00
Brian Coca
a1730af91f
Ensure blockinfile correctly returns backupfile (#72544)
* Ensure blockinfile correctly returns backupfile

  Fixes #27626
  based on #27859

Co-authored-by: Giovanni Sciortino (@giovannisciortino)
2020-11-09 14:45:09 -05:00
Rick Elrod
e7bf0696ef
[ansible-test] use newer container images (#72126)
Change:
- Bump default, ansible-base, distro containers
- We do NOT add fedora33 yet, because it doesn't work right on Shippable
  due to an old kernel. This will be added post-AZP.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-11-09 13:04:44 -06:00
Amin Vakil
c1da427a5e
iptables: Reorder comment postition (#71496)
* Reorder comment postition

* Add comment unit test

* Fix unit test

* Fix unit test

* Add changelog

* Add paramaters which would be problematic without this fix

* Fix typo

* Fix unit test

* Fix unit test
2020-11-09 19:40:55 +01:00
Rick Elrod
3b379f372e [ansible-test] first attempt at freebsd 12.2
Change:
- Add freebsd/12.2 remote target

Test Plan:
- ci_complete

Tickets:
- Fixes #72366

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-11-09 12:09:53 -06:00
Brian Coca
8eab113cb1
show keyword documentation in ansible-doc (#72476)
* show keyword documentation in ansible-doc

Co-authored-by: Felix Fontein <felix@fontein.de>
2020-11-09 09:55:17 -05:00
Matt Martz
96ad5b799e
Emit warning when running on the controller with a Python older than 3.8 (#72467)
* Emit warning when running on the controller with a Python older than 3.8

* Add spaces

Co-authored-by: Matt Clay <matt@mystile.com>

* and more spaces

Co-authored-by: Matt Clay <matt@mystile.com>

* s/Py/Python/

Co-authored-by: Matt Clay <matt@mystile.com>

* Add note to Control node requirements about Py3.8 requirement

* Add collection_name to deprecated call

* more spaces

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

* Clarify that we are only packaging for py3.8+

Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2020-11-09 08:53:52 -06:00
Sloane Hertel
0ed7bfc694
Fix task get_name to always prepend the role name (#72511)
* Fix 'role_name : tast_name' notation if task contains role name

* Add tests for notifying handler names which contain the role name

Co-authored-by: Thomas Wouters <thomaswouters@gmail.com>
2020-11-06 11:46:58 -05:00
Martin Krizek
a2593b5e27 CLI - Specify jinja version in --version output 2020-11-05 11:37:55 -08:00
Brian Coca
488b9d6c35 remove redundant remote_user for local setting
local action plugin already does and this also should fix
  fork/thread issue by removing use of pwd library

  fixes #59642
2020-11-05 11:17:46 -08:00
Sam Doran
b33d7e2e29
Add changelog (#72502) 2020-11-05 12:50:44 -05:00
Felix Fontein
c7a4b39633
Adjust action groups to moved modules (#72428)
* Support docker and k8s action groups for moved modules in community.docker and community.kubevirt.

* Also support k8s action group for community.okd.

* Also add kubernetes.core.

* Adjust PR #.

* Fix changelog fragment.

* Remove community.okd.

* Revert "Remove community.okd."

This reverts commit 812b5aa6e2.
2020-11-05 10:13:51 +00:00
Alexei Znamensky
5654de6fce
Return error if cwd directory does not exist (#72390)
* Return warning or error if cwd directory does not exist, in AnsibleModule.run_command()
2020-11-04 14:33:39 -05:00
Felix Fontein
da60525610
Fix missing ansible.builtin FQCNs in hardcoded action names (#71824)
* Make sure hard-coded action names also check for FQCN.
* Use _add_internal_fqcn() to avoid hardcoded lists and typoes.
2020-11-03 08:51:31 -05:00
Felix Fontein
64a809d2b6
Improve ansible-test classifications for collections (#72353) 2020-11-02 10:44:51 -08:00
Jordan Borean
809d5fc398
ansible-test - skip installing PowerShell sanity test reqs if they are already installed (#72423) 2020-10-30 21:52:27 -04:00
Felix Fontein
4fb336cef1
ansible-doc: export has_action when --json is used (#72359)
* ansible-doc: export has_action when --json is used.
* Remove docuri and now_data, which were not used resp. ignored in format_plugin_doc and the functions it calls anyway.
* Add function _combine_plugin_doc.
2020-10-30 10:51:54 -04:00
Brian Coca
cb94c0cc55
added timeout options to adhoc and console (#71230)
* added timeout options to adhoc and console

* added test

* fix typosesz

* fix conflict

* task_timeout

* fix timeout option, added extra vars to console

* actually use right cli switch .. DUH!

* added timeout to include 'valid' but ignored keys

* fix default

* fixes per review
2020-10-30 10:09:30 -04:00
Brian Coca
96c1972439
find will give more specific reasons on skip (#68823)
* find will give more specific reasons on skip

 - added more error handling
 - use random path for test
2020-10-30 10:08:19 -04:00
Sam Doran
fb09fd2a23
wait_for - ignore psutil related errors (#72401)
When enumerating connections with psutil, catch and ignore errors to avoid returning a stack trace.

Co-authored-by:  Matt Martz <matt@sivel.net>
2020-10-29 21:40:31 -04:00
Matt Martz
e73a0b2460
Explicitly get the include task, and not assume it is the parent (#72378)
* Explicitly get the include task, and not assume it is the parent. Fixes #6571
2020-10-29 15:15:18 -04:00
Sam Doran
a51a6f4a25
reboot - add reboot_command parameter (#69847)
Fixes #51359
* Update default search paths
* Fix returns for args and command, don't allow conversion
* Reorganize tests
* Fix test for Azure Pipelines
2020-10-28 12:10:59 -04:00
Sam Doran
8f9cf456b0
url lookup - set default user agent (#72324)
* Add unit tests
* Add note about when default changed
2020-10-28 09:34:06 -04:00
Sam Doran
d6115887fa
systemd - use list-unit-files rather than list-units (#72363)
list-unit-files will return all files on the system. list-units omits those
that are disabled.

Co-authored-by: Ken Dreyer <ktdreyer@ktdreyer.com>
2020-10-27 17:43:36 -04:00
Sam Doran
7352457e7b
hostname - add macOS (#54439)
* Add DarwinStrategy class and integration tests

macOS has three seprate hostname params that need to be set. One of those params, LocalHostName, has more stringent requirements than the other two, which accept special characters and spaces. Create a method to scrub the hostname to ensure it works well with the system requirements.

* Update documentation

* Account for virtualization type returned on Azure Pipelines

* Do not be dependent on order of self.name_types

Use the scrubbed name when the name type is LocalHostName
2020-10-27 17:30:54 -04:00
Sam Doran
c51438312a
blockinfile - properly insert block when no trailing new line exists (#72350) 2020-10-27 10:34:09 -05:00
Felix Fontein
233e7beb5b
Fix processing of add_file_common_args=True when argument_spec is not specified as kwarg. (#72334) 2020-10-27 10:25:18 -05:00
Sloane Hertel
dee8a4a8f1
Use type 'path' for fact_caching_connection in jsonfile cache plugin (#72317) 2020-10-27 10:10:13 -05:00
Sam Doran
db84e2c989
systemd - fix issue with capbpf and newer kernel (#72337)
A bug existed in systemd 245 that did not properly handle unknown kernel
capabilities gracefully. This resulted in incomplete output when querying
for the service status. It is possible to get service status by other means.
This PR works around this issue by getting service status using other commands
in the event of a failure due to this bug.
2020-10-26 13:53:31 -04:00
Rick Elrod
2293b327c0
[ansible-test] Limit cryptography to <3.2 (#72342)
Change:
- Cryptography 3.2 drops support for OpenSSL 1.0.2. Some of our CI
  infrastructure still uses this version (FreeBSD, namely). For now,
  just add a constraint to use old cryptography.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-10-26 11:39:37 -05:00
Sloane Hertel
8b07d46166
Add a toggle for the leading separator for keyed_groups (#60882)
* Add a toggle for the leading separator for keyed_groups if no prefix is given

* changelog

* Add tests for backward compatibility and the new toggle
2020-10-23 15:57:44 -04:00
Felix Fontein
0b2b82c0b6
Add changelog fragment for #71736. (#72323) 2020-10-23 14:34:31 -05:00
Matt Davis
83909bfa22
Remove ansible-galaxy login (#72288)
* GitHub is removing the underlying API used to implement the `login` command. Since the general consensus seems to be that relatively nobody currently uses this command (in favor of explicit token passing), support was simply removed for interactive login. If a future need arises, this command should be reimplemented via OAuth Device Auth Grants.
* login or role login commands now produce a fatal error with a descriptive message
* updated 2.10 and 2.11 porting guide entries

* remove dead code/config, update messages and porting guides
2020-10-23 09:11:45 -07:00
Matt Clay
3c2e8b99be
Fix ansible-test handling of user-defined docker networks. (#72256)
* Fix ansible-test docker container detection.

* Attach test containers to the correct network.

* Do not assume `localhost` for accesing Docker.

* Look for containers on current network.

* Always map /var/run/docker.sock into containers.

This fixes issues when using a remote Docker host.

* Support container IP lookup from networks list.

* Fix container network attachment.

* Remove redundant container detection messages.

* Limit DOCKER_HOST parsing to TCP.

* Restore docker socket existence check.

The check is skipped if the docker hostname is not localhost.

* Correct changelog entry.
2020-10-22 18:45:03 -07:00
Ruediger Pluem
e64ba97e8a
Fix the reversed order of return values (#72088)
* Fix the reversed order of return values

AnsibleModule.run_command returns a tuple of return code, stdout and stderr.
The module main function of the user module expects user.create_user to
return a tuple of return code, stdout and stderr.
Fix the locations where stdout and stderr got reversed.

* Fix another missed occurance

* Add changelog

* Fix further reversed order of return values occurrences

Followup to 72c3ba62c8:
Fix further reversed order of return values occurrences.
2020-10-22 20:02:53 +02:00
Rick Elrod
69e510e767
[virt facts] recognize containerd as a container (#72210)
Change:
- containerd is now recognized as container tech

Test Plan:
- Tested with Docker inside Github Actions in a private repo:

        "ansible_virtualization_role": "guest",
        "ansible_virtualization_tech_guest": [
            "containerd",
            "container",
            "VirtualPC"
        ],
        "ansible_virtualization_tech_host": [],
        "ansible_virtualization_type": "containerd",

Tickets:
- Fixes #66304 because this is what docker containers show up as
  in Github Actions.

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-10-20 10:39:13 -05:00
Jordan Borean
c9fa1d0e7e
Fix race condition when creating async dir (#72069)
* Fix race condition when creating async dir

* Simplify exception wrapper

* Remove var used for testing
2020-10-19 15:05:19 -04:00
Matt Martz
be5fc4e642
Ensure we call action_loader.get with collection_list (#72206)
* Ensure we call action_loader.get with collection_list. Fixes #72170

* Add tests and changelog

* Remove grep, do assertion in playbook. ci_complete

* Skip old jinja2 versions

* ci_complete

* dedupe
2020-10-19 09:54:24 -05:00
Brian Coca
5ec53f9db8
make collection callbacks follow normal flow (#59932)
make collections whitelist follow normal flow

* fixes missing set_options call and adhoc and stdout processing rules
* avoid dupes
* fixed to handle redirects
* also updated tests with new and more accurate skip message
* fix callback tests for envs with cowsay installed
* lots MOAR comments on why the code is as it is, some todos to refactor in future

Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
2020-10-15 15:31:18 -04:00
Matt Clay
2ef4b7e07e Fix ansible-test Azure Pipelines container auth. 2020-10-14 12:35:11 -07:00