Commit graph

10523 commits

Author SHA1 Message Date
Sam Doran
775d6e6752
[stable-2.10] Changes required for macOS 10.15 (#71841) (#71954)
* Use default group of staff on macos to prevent sudo issues
* Install gnu-tar for macos in git and unarchive tests
* Enable timezone module to support py3 on macos
* If the virtualenv command is missing, try python -m virtualenv
* Install passlib for filter_core on macos
* Install paramiko via pip on macos for paramiko tests
* Normalize discovered python interpreter on macos
* Get pip tests passing, by ensuring we have wheel installed
* Create /etc/ansible for ca certs on mac, list lookup_url as destructive
* Fixups for CA certs
* Include macos
* Dynamically get cafile instead of hardcoding the path
(cherry picked from commit 35b0fef536)

Co-authored-by: Matt Martz <matt@sivel.net>
2020-10-23 15:18:24 -05:00
Rick Elrod
83c34eb7de
[dnf] accumulate update filters (#71726) (#72181)
Change:
- Previously when `security: true` and `bugfix: true` were both given,
  only security updates would get applied. Filters now accumulate so
  that both get applied in this case.

Test Plan:
- New integration tests for both check_mode and not. These tests make
  use of a contrived yum repository which is stored in S3.

Tickets:
- Fixes #70854

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Matt Martz <matt@sivel.net>

Co-authored-by: Matt Martz <matt@sivel.net>
(cherry picked from commit fdf80690e4)
2020-10-23 14:15:47 -05:00
Matt Clay
3bd6daa4a4 [stable-2.10] Fix ansible-test handling of user-defined docker networks. (#72256)
* Fix ansible-test docker container detection.

* Attach test containers to the correct network.

* Do not assume `localhost` for accesing Docker.

* Look for containers on current network.

* Always map /var/run/docker.sock into containers.

This fixes issues when using a remote Docker host.

* Support container IP lookup from networks list.

* Fix container network attachment.

* Remove redundant container detection messages.

* Limit DOCKER_HOST parsing to TCP.

* Restore docker socket existence check.

The check is skipped if the docker hostname is not localhost.

* Correct changelog entry..
(cherry picked from commit 3c2e8b99be)

Co-authored-by: Matt Clay <mclay@redhat.com>
2020-10-22 21:40:23 -07:00
Abhijeet Kasurde
98cc9cb834
[2.10] AnsibleVaultEncryptedUnicode should be considered a string (#72216)
* AnsibleVaultEncryptedUnicode should be considered a string
* linting fix
* clog frag

(cherry picked from commit 48f12c14e9)

Co-authored-by: Matt Martz <matt@sivel.net>
2020-10-21 18:08:18 -05:00
Ruediger Pluem
7499848cb1
user - allow local users with an expiry date to be created (#72022) (#72085)
The luseradd / lusermod commands do not support the -e option. Set
the expiry time in this case via lchage after the user was
created / modified.

Fixes: #71942

In Python3 math.floor returns an integer whereas Python2 returns a float.
Hence always convert the result of math.floor to an int to ensure that
lexpires is an integer.

Move local expires tests in a separate file and import the tasks to the
main.yml to keep main.yml smaller.

(cherry picked from commit a7170da851)
2020-10-21 17:04:35 -05:00
Matt Clay
6c8d6a3182 [stable-2.10] Fix ansible-test Azure Pipelines container auth.
(cherry picked from commit 2ef4b7e07e)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-10-21 14:15:39 -07:00
Felix Fontein
961ead55c6
Add deprecation collection name to plugin options (#71735) (#72081)
* Add changelog fragment.

* Use correct field that's expected by lib/ansible/cli/__init__.py..

* Add basic unit tests.

(cherry picked from commit 1107aace1b)
2020-10-21 15:58:51 -05:00
Christian Loos
505df0d564
fix distribution fact for SLES4SAP (#71559) (#72026)
b6b238a fixed the SLES4SAP detection, which was at this time ok.
Sadly Suse changed with SLES 15 the /etc/os-release file, so the above
change will no longer work.

This commit updates the SLES4SAP detection regarding
https://www.suse.com/support/kb/doc/?id=000019341.

The symlink realpath is matched with endswith, because in SLES 12+ the
link target is SLES_SAP.prod, but in SLES 11 the link target is
SUSE_SLES_SAP.prod.

(cherry picked from commit ea119d3089)
2020-10-21 15:35:36 -05:00
Rick Elrod
e6a4585807
[dnf] show installations/removals in check_mode (#70892) (#72180)
Change:
- Previously, we only showed that something would have changed, not what
  would have changed. This allows us to show what will chang as well.

Test Plan:
- Local RHEL8 VM
- New integration tests

Tickets:
- Fixes #66132

Signed-off-by: Rick Elrod <rick@elrod.me>
(cherry picked from commit 7d32129efb)
2020-10-21 14:42:26 -05:00
Rick Elrod
2c5e4b5a88
[tests/dnf] work around dnf packaging issue (#72249) (#72251)
Change:
- In this test we end up upgrading dnf (and python3-dnf) so that we can
  test its new logging behavior. However, the latest Fedora 32 dnf had a
  packaging issue which caused it to not pull in the latest
  python3-libdnf. This is fixed, but not synced out to mirrors yet.
  Fixing it in this test will get CI passing again in the meanwhile.

Test Plan:
- CI

Tickets:
- https://bugzilla.redhat.com/show_bug.cgi?id=1887502

Signed-off-by: Rick Elrod <rick@elrod.me>
(cherry picked from commit 9ddb1d76af)
2020-10-19 11:04:32 -05:00
Matt Clay
49504da1ed
[stable-2.10] Support collection constraints in ansible-test. (#72157)
This allows collections to specify requirements and constraints for packages that ansible-test has requirements or constraints for.
(cherry picked from commit 5f76bd2af7)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-10-08 16:17:28 -05:00
Rick Elrod
f57b149ec8
Intentional coverage, incidental_azure_rm_resource (#71052) (#72159)
Change:
- Adds some intentional coverage around PluginLoader for cases that
  incidental_azure_rm_resource covered.
- Specifically, modules starting with an underscore, and starting with
  an underscore but a symlink.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
(cherry picked from commit cdcf0aa42a)
2020-10-08 16:16:53 -05:00
Rick Elrod
70172dde27
Add intentional unit tests for basic._set_cwd and common.dict_merge (#70283) (#72160)
* Add unit tests for basic._set_cwd

* incidental coverage for dict_merge

* add test for async stderr inclusion

(cherry picked from commit b019029bf3)

Co-authored-by: jctanner <tanner.jc@gmail.com>
2020-10-08 16:16:34 -05:00
Martin Krizek
4df129c6ae
Provide more information in AnsibleUndefinedVariable (#71666) (#71876)
* Provide more information in AnsibleUndefinedVariable

Fixes #55152

(cherry picked from commit 00b22ab55e)
2020-09-28 00:34:38 -05:00
Matt Clay
c4e3552c8d
[stable-2.10] CI provider fixes for ansible-test. (#71929) (#71932)
* Make Azure Pipelines resource_prefix lowercase.

* Make classification of CI files consistent.

* Update package-data sanity test for AZP.
(cherry picked from commit 92b66e3e31)

Co-authored-by: Matt Clay <mclay@redhat.com>
2020-09-28 00:34:12 -05:00
Sam Doran
988d410e17
[stable-2.10] Test changes to allow using "macos" test image (#71849) (#71861)
* Add link to remote.sh from macos.sh
* Add skip/macos to tests that have skip/osx.
(cherry picked from commit 6984081111)

Co-authored-by: Sam Doran <sdoran@redhat.com>
2020-09-28 00:33:11 -05:00
Matt Davis
07a9de1247
fix coverage output from synthetic packages (#71727) (#71748)
* fix coverage output from synthetic packages

* synthetic packages (eg, implicit collection packages without `__init__.py`) were always created at runtime with empty string source, which was compiled to a code object and exec'd during the package load. When run with code coverage, it created a bogus coverage entry (since the `__synthetic__`-suffixed `__file__` entry didn't exist on disk).
* modified collection loader `get_code` to preserve the distinction between `None` (eg synthetic package) and empty string (eg empty `__init__.py`) values from `get_source`, and to return `None` when the source is `None`. This allows the package loader to skip `exec`ing things that truly have no source file on disk, thus not creating bogus coverage entries, while preserving behavior and coverage reporting for empty package inits that actually exist.

* add unit test

(cherry picked from commit e813b0151c)
2020-09-28 00:29:24 -05:00
Jordan Borean
68278f36fd
psrp - fix hang when copying an empty file (#71649) (#71651)
(cherry picked from commit b615789fcc)
2020-09-28 00:28:01 -05:00
Rick Elrod
ab100f8ee5
[tests] Make setup_docker clean up better (#71949) (#71959)
Change:
- pip packages should get removed after, not try to add them again
- Try removing containerd.io package too

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
(cherry picked from commit 6a7e637c5f)
2020-09-26 03:41:03 -05:00
Alicia Cozine
12874bbdea
[2.10] updates intersphinx references for docs links (#71921) (#71945)
* updates intersphinx references for docs links (#71921)

* DOCS: updates intersphinx references for docs links
* TESTS: Raise the number of bytes scanned to determine if a file is binary. The newest ansible-2.10.inv file has its first null byte at position 2261. 4096 is still a cheap chunksize to read so it still makes sense to raise this.

Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
(cherry picked from commit 27826827e9)

* adds changelog for already-merged PR (#71947)
* adds changelogs/fragments/71921-raise-bytes-for-binary-test.yml

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit 91b0dfb659)
2020-09-25 13:53:44 -05:00
Rick Elrod
d7f6d8c8f0
[incidental_setup_docker] changes to upstream repo (#71897) (#71899)
Change:
- The docker-ce.repo file for centos does not work on RHEL since it uses
  $releasever and on RHEL that is, e.g., "7Server".
- Instead, set up the repo manually.
- Additionally, the docker centos8 repo no longer has old versions, so
  we use the (only) version in the repo instead.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
(cherry picked from commit 31ddca4c0d)
2020-09-23 21:36:53 -05:00
Matt Clay
df150a2780 [stable-2.10] Fix attribute testing in file integration test. (#71843)
The file test will no longer attempt to test attributes if `lsattr -vd` does not work on the system under test.
(cherry picked from commit 17765cd4e8)

Co-authored-by: Matt Clay <mclay@redhat.com>
2020-09-21 15:06:26 -07:00
Sam Doran
b6637c0afc [stable-2.10] incidental_setup_openssl - Pin version of Python packages
A recent update to cffi that was yanked is still being installed on our
Mac OS X 10.11 test image since the version of pip there is very old and
does not ignore yanked packages.

Pin the version of pyOpenSSL and its dependencies to fix this and avoid
future spontaneous failures.
(cherry picked from commit 65cdb86c8a)

Co-authored-by: Sam Doran <sdoran@redhat.com>
2020-09-17 08:44:41 -07:00
Matt Clay
fd27c4db71 [stable-2.10] Add pause to avoid same mtime in test.
(cherry picked from commit 3d769f3a76)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-09-02 15:49:08 -07:00
Matt Clay
ec8878ced4 [stable-2.10] Use new endpoint for Parallels based instances.
(cherry picked from commit 98febab975)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-09-02 15:08:32 -07:00
Rick Elrod
333f7bc088
[tests] Bump container versions (#71518) (#71598)
Change:
- This pulls in python 3.9.0rc1

Test Plan:
- CI, hopefully

Signed-off-by: Rick Elrod <rick@elrod.me>
(cherry picked from commit db519bc17c)
2020-09-02 14:44:23 -05:00
Matt Clay
4273443686 [stable-2.10] Support macOS 10.15 for ansible-test --remote.
Use of this new version is experimental, so it is not enabled in CI yet.
(cherry picked from commit 2bbcbe99fd)

Co-authored-by: Matt Clay <matt@mystile.com>
2020-09-01 19:07:10 -07:00
Matt Clay
356ea0b8d0
[stable-2.10] Fix ansible-test coverage traceback. (#71446) (#71578)
* [stable-2.10] Fix ansible-test coverage traceback. (#71446)

* Add integration test for ansible-test coverage.

* Fix ansible-test coverage traceback.

* Fix coverage reporting on Python 2.6.
(cherry picked from commit f5b6df14ab)

Co-authored-by: Matt Clay <mclay@redhat.com>

* Add empty ignore.txt file for tests.
2020-09-01 17:05:19 -07:00
Matt Clay
d699d38dd5 [stable-2.10] Update ansible-test remote endpoint handling. (#71413)
* Request ansible-core-ci resources by provider.
* Remove obsolete us-east-2 CI endpoint.
* Add new --remote-endpoint option.
* Add warning for --remote-aws-region option.
* Update service endpoints.
* Allow non-standard remote stages.
* Add changelog fragment.
(cherry picked from commit d099591964)

Co-authored-by: Matt Clay <mclay@redhat.com>
2020-09-01 14:54:17 -07:00
Rick Elrod
9a48ffd61b
Attempt at reverting CVE-2020-1736 changes [2.10] (#71514)
* Revert atomic_move changes
* add note about mode reverts in porting guide

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-09-01 01:00:26 -05:00
Rick Elrod
1678577681
[tests] fix fallout from dnf gpg fix (#71552)
Change:
- Missed needed gpg ignores

Test Plan:
- CI
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-08-31 18:24:00 -05:00
Rick Elrod
dc97027453
[dnf] ensure packages are gpg-verified (#71539)
Change:
- By default the dnf API does not gpg-verify packages. This is a feature
  that is executed in its CLI code. It never made it into Ansible's
  usage of the API, so packages were previously not verified.
- This fixes CVE-2020-14365.

Test Plan:
- New integration tests

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-08-31 10:05:30 -05:00
David Shrewsbury
393412dc64
Fix play stats when rescue block is a child block (#70922) (#71334)
* check run state of current block only

* Add changelog and test

* Add test for issue 29047

(cherry picked from commit f2f6c34632)
2020-08-28 12:23:30 -05:00
Jordan Borean
7131c75d93
powershell - fix quoting values (#71411) (#71449)
* powershell - fix quoting values

* Add ignore for smart quote skip

(cherry picked from commit 72a7cb4a2c)
2020-08-28 12:22:22 -05:00
Jordan Borean
b936539ae1
powershell - fix nested CLIXML parser (#71412) (#71451)
(cherry picked from commit 8897d7e2ff)
2020-08-28 09:54:14 -07:00
Martin Krizek
ce7b95499f
native types: properly handle Undefined in nested data (#68432) (#71105)
(cherry picked from commit 5ca3aec3c4)
2020-08-28 07:57:17 -05:00
Sam Doran
c39753d7ad
[stable-2.10] linux facts - return proper broadcast address (#64528) (#71064)
Check that the value being returned is actually a broadcast address

(cherry picked from commit e6bf202738)
2020-08-27 13:48:50 -05:00
Martin Krizek
0c32a4f793
Emit proper error for x in y when y is undefined (#70990) (#71011)
Fixes #70984

(cherry picked from commit bf7276a4e8)
2020-08-27 13:46:52 -05:00
Sam Doran
0c6edb34f9
[stable-2.10] unarchive - Check 'fut_gid' against 'run_gid' in addition to supplemental groups (#65666) (#71001)
* [stable-2.10] unarchive - Check 'fut_gid' against 'run_gid' in addition to supplemental groups (#65666)

Add integration tests for unarchiving as unprivileged user
Break tasks into separate files for easier reading and maintenance

Create a user by specifying a default group of 'staff' for macOS.

The user module does not actually remove the user directory on macOS,
so explicitly remove it.

Put the removal tasks in an always block to ensure they always run

Co-authored-by: Philip Douglass <philip.douglass@amadeus.com>
Co-authored-by: Sam Doran <sdoran@redhat.com>
(cherry picked from commit ac5f3f8bef)

Co-authored-by: Philip Douglass <philip@philipdouglass.com>

* [stable-2.10] Fix unstable unarchive test (#71004)

* Add mode to copy tasks
* Fix unreliable test by ignoring errors

(cherry picked from commit f99f96ceb6)

Co-authored-by: Philip Douglass <philip@philipdouglass.com>
2020-08-27 13:45:19 -05:00
Sloane Hertel
714cd2ad2e
copy - redact 'content' from invocation in check mode (#71033) (#71067)
* sanitize copy module invocation secrets in check mode

(cherry picked from commit 991714b9d1)
2020-08-27 13:41:14 -05:00
Jordan Borean
f41ff33ca6
ansible-galaxy - fix download for subdirs in SCM (#71005) (#71093)
(cherry picked from commit f6b3b4b430)
2020-08-27 13:39:54 -05:00
Brian Coca
00caeff928
dont clobber facts in loop (#71032) (#71095)
(cherry picked from commit f9af27c631)
2020-08-27 13:39:19 -05:00
Toshio Kuratomi
6b639f147d
[stable-2.10] Update ansible doc formats (#71070) (#71111)
* Fix tty_ify bugs and refactor

* Move tty_ify() and supporting attributes to the DocCLI class as that's
  the only thing using it.
* Add unittest for the code.
* Fix a bug where the substitution macros can be detected when they are
  a part of another word.
* Add support for L(), R(), and HORIZONTALLINE which were added to the
  website docs many years ago.

* Update test/units/cli/test_doc.py

Co-authored-by: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit fb144c4)

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
2020-08-27 13:33:24 -05:00
Felix Fontein
e33fff922a
ansible-test: bump acme test container version to 2.0.0 (#71097) (#71165)
(cherry picked from commit 050841324c)
2020-08-27 12:58:06 -05:00
Jordan Borean
092ec680e6
Ensure -k is set to delegated hosts without a pass (#71136) (#71168)
* Ensure -k is set to delegated hosts without a pass

* Fix up some broken tests

* Update task_executor.py

one possible fix, the other is updating winrm to normalize on 'password' like the other connection plugins

* Add alias for winrm and fix incorrect assumption

* Make sure aliases are used for keyword options

* Conditionally run test if sshpass is present, fix sanity

Co-authored-by: Brian Coca <bcoca@users.noreply.github.com>
(cherry picked from commit 3f22f79e73)
2020-08-27 12:57:24 -05:00
GomathiselviS
e95b45c5ec
Update network integration auth timeout (#71057) (#71238)
*  Incase of network integration test for connection local
   test the paramiko_ssh auth_timeout is the value of timeout
   under defaults section which is 10 seconds.
*  For slower connection 10sec timout value result in authentication
   timeout error hence increase the timeout value to 90 seconds

(cherry picked from commit 6160e82bf2)

Co-authored-by: Ganesh Nalawade <ganesh634@gmail.com>
2020-08-27 12:55:04 -05:00
Rick Elrod
7b3d2a00a0
Fix cron file regression (#71207) (#71243)
Co-authored-by: Florent PIGOUT <toopy@users.noreply.github.com>
2020-08-27 12:54:15 -05:00
Toshio Kuratomi
7662568bd8
Antsibull build format backport 71080 (#71285)
* Updates to docs build for new antsibull CLI

* --ansible-base-cache renamed to --ansible-base-source
* _acd_version in the .deps file has been renamed to _ansible_version

(cherry picked from commit 22d2c97b81)

* Fix expr regex for MacOSX compat

MacOSX seems to want bare `+` whereas GNU expr wants escaped `+` (`\+`)
to mean match one or more.  Use `\{1,\}` instead which will match one or
more on both MaxOSX and GNU-using systems.

Fixes #71053

(cherry picked from commit 99cac0b135)

* Force an upgrade to a newer version of antsibull

(cherry picked from commit d816a5966e)
2020-08-27 12:53:29 -05:00
Martin Krizek
02f4fc1a14
Skip literal_eval for string filters results in native jinja. (#70988) (#71313)
Fixes #70831

(cherry picked from commit b66d66027e)
2020-08-27 12:47:02 -05:00
Matt Martz
6d1cc0dede
[stable-2.10] epoch can be a float with strftime filter. Fixes #71257 (#71314) (#71319)
(cherry picked from commit 6289570)

Co-authored-by: Matt Martz <matt@sivel.net>
2020-08-27 12:35:21 -05:00