Commit graph

10351 commits

Author SHA1 Message Date
Matt Davis
d79b23910a
fix sys.modules cleanup and blacklist behavior (#69898)
* fix sys.modules cleanup and blacklist behavior

* fix map-as-generator py2/py3 issue
* clear path_importer_cache between runs

* sanity fix

* don't be stupid with moving target generators
2020-06-04 19:02:23 -07:00
Rick Elrod
efe103cdff
Make add_host be idempotent/show changed status (#69897)
Change:
- The `add_host` action now shows an accurate change status.

Test Plan:
- Added a plethora of integration tests.

Tickets:
Fixes #69881

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-06-04 21:01:56 -05:00
Brian Coca
062e780a68
starting metadata sunset (#69454)
* starting metadata sunset

 - purged metadata from any requirements
 - fix indent in generic handler for yaml content (whey metadata display was off)
 - make more resilient against bad formed docs
 - removed all metadata from docs template
 - remove metadata from schemas
 - removed mdata tests and from unrelated tests

Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Rick Elrod <rick@elrod.me>
2020-06-04 21:01:46 -04:00
Felix Fontein
2dbd5dc2ae
ansible-test: do not accept empty string as valid version number (#69816)
* Work around strange behavior of StrictVersion and SemanticVersion constructors that they accept an falsy value.

* Do not accept empty strings as versions.
2020-06-04 13:25:05 -07:00
Rick Elrod
3b00c161dc
Re-add documentation for yum "install_repoquery" (#69871)
Change:
- This was removed in 2014 in 122a7021bc.
- The option still exists and is enabled by default and can lead to user
  confusion when people aren't expecting packages (or updated
  dependencies for it) to get installed and they do.
- Add the option documentation back with a few notes to make it clear what
  is happening.

Test Plan:
N/A, no code change, just documentation

Tickets:
- Refs #69497

Signed-off-by: Rick Elrod <rick@elrod.me>

* Fix sanity errors

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-06-04 15:45:22 -04:00
Alex Schultz
247e43b252
Fix IncludedFile equality check (#69524)
In the case of a free style strategy, it is possible to end up with
multiple hosts trying to include from the same role, however the tasks
being included may be different with the use of tasks_from.  Previously
if you had two hosts that were included the same role when the
process_include_results function tries to determine if a included needs
to be run on a specific host, it would end up merging two different
tasks into which ever one was processed first.

This change updates the equality check to also check if the task uuid
associated with the IncludedFile is the same. The previous check only
checked if the task's parent uuid was the same. This breaks down when
both includes have the same parent.

    - hosts: all
      strategy: free
      gather_facts: false
      tasks:
        - include_role:
            name: random_sleep
        - block:
          - name: set a fact (1)
            include_role:
              name: set_a_fact
              tasks_from: fact1.yml
          - name: set a fact (2)
            include_role:
              name: set_a_fact
              tasks_from: fact2.yml
        - name: include didn't run
          fail:
            msg: >
              set_a_fact didn't run
              fact1: {{ fact1 | default('not defined')}}
              fact2: {{ fact2 | default('not defined') }}"
          when: (fact1 is not defined or fact2 is not defined)

Closes #69521
2020-06-04 10:41:46 -05:00
Brian Coca
22da903e9c
return changed for group_by (#32057)
* return changed for group_by

* added tests and fixed 'early registeration'
2020-06-03 12:46:40 -04:00
Andrew Klychkov
00ead98595
systemd: should fail in check_mode when service not found on host (#68136)
* systemd: should fail in check_mode when service not found on host
2020-06-02 11:52:12 -04:00
Dongsu Park
598e3392a9
Discover Flatcar Linux properly for hostname (#69627)
To avoid issues with Flatcar Container Linux being unable to be found,
detect Flatcar distro name especially for hostname, just like CoreOS
Container Linux was supported.

See also https://github.com/ansible/ansible/issues/69516
2020-06-02 18:41:53 +05:30
Jack
80f09efd03
do not return the body even if it failed (#69706)
* do not return the body even if it failed

* add some tests for this and rebase

* import test task

* ignore_errors when fails

Co-authored-by: Jack Zhang <jack.zhang@aspiraconnect.com>
2020-06-01 09:55:38 -04:00
Abhijeet Kasurde
723a904f4e
Handle disabled service units (#69349)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-06-01 09:40:52 -04:00
Hideki Saito
79ab798427 Enable logging using setup_loggers() API in dnf-4.2.17-6 or later
- Fixed issue #28061
- https://bugzilla.redhat.com/show_bug.cgi?id=1788212

Signed-off-by: Hideki Saito <saito@fgrep.org>
2020-05-30 20:37:27 -05:00
Rick Elrod
61a1fe1d7d
Fix network_cli test to use a trap for cleanup (#69762)
Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-29 15:45:33 -05:00
Martin Krizek
3606dcfe65
Ensure --version works with non-ascii project path (#66624)
Fixes #66617
2020-05-29 14:42:44 -04:00
Sloane Hertel
e40889e711
Add support to install collections from git repositories (#69154)
* Enable installing collections from git repositories

* Add tests for installing individual and multiple collections from git repositories

* Test to make sure recursive dependencies with different syntax are deduplicated

* Add documentation

* add a changelog

* Skip Python 2.6

* Only fail if no collections are located in a git repository

Add support for a 'type' key for collections in requirement.yml files.
Update the changelog and document the supported keys and allowed values for the type.

Add a note that the collection(s) in the repo must contain a galaxy.yml

* Add a warning about embedding credentials in SCM URLs

* Update with review suggestions

* suppress sanity compile failure for Python 2.6
2020-05-29 13:33:32 -04:00
Karl Fogel
225ae65b0f
Make Python path warning say what it means to say (#69669) 2020-05-29 13:15:11 -04:00
Felix Fontein
0b82d4499e
ansible-test ansible-doc sanity test: also run with --json parameter (#69288) 2020-05-29 08:52:29 -07:00
Felix Fontein
4794b98f2a
Fix version_added test (#69756)
* Fix version_added test for changes in #69680.

* Fix error message.
2020-05-29 08:25:14 -07:00
Rick Elrod
8e19ab178a
connection plugins: try config, then play_context (#69751)
Change:
Rather than only using config, have base connection plugins fall back to
play_context.

Test Plan:
- Tested ansible-connection logic against an IOS device
- Tested -k against a VM
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-29 10:00:21 -05:00
Abhijeet Kasurde
7b3d9cbcb8
distribution: Added support for openEuler OS (#69324)
Detect os_family for openEuler OS as 'RedHat', instead of 'openEuler'.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-05-29 12:43:16 +05:30
Jordan Borean
f81f5da20e
Ansible.Basic added generic fragment merger for module options (#69719) 2020-05-29 16:11:38 +10:00
Felix Fontein
40f21dfd3c
Version source tagging (automatic and manual) for version_added and deprecation versions (#69680)
* Track collection for version_added.
Validate *all* version numbers in validate-modules.
For tagged version numbers (i.e. version_added), consider source collection to chose validation.

* Make tagging/untagging functions more flexible.

* Tag all versions in doc fragments.

* Tag all deprecation versions issued by code.

* Make Display.deprecated() understand tagged versions.

* Extend validation to enforce tagged version numbers.

* Tag versions in tests.

* Lint and fix test.

* Mention collection name in collection loader's deprecation/removal messages.

* Fix error IDs.

* Handle tagged dates in Display.deprecated().

* Also require that removed_at_date and deprecated_aliases.date are tagged.

* Also automatically tag/untag removed_at_date; fix sanity module removal version check.

* Improve error message when invalid version number is used (like '2.14' in collections).
2020-05-28 22:46:16 -07:00
Jordan Borean
f5f3ba7ab5
ansible-test - fix up relative util import for powershell validate-modules (#69753)
* ansible-test - fix up relative util import for powershell validate-modules

* Use different tactic for generic group

* Use python 2 and 3
2020-05-29 14:31:59 +10:00
Felix Fontein
31bf3a5622
Deprecate module in collection: allow removal date in documentation, make validate-modules ensure version and date match (#69727)
* Allow to deprecate module by date in documentation.

* Make sure deprecation date/version match between module docs and meta/runtime.yml.

* Unrelated fix: don't compare deprecated module version to Ansible's version in collection.

* Allow documentation's removal version to be something else than fixed list of Ansible versions for collections.

* Linting.

* Allow to deprecate plugin options by date.

* Add changelog fragment for deprecation by date (also covers #68177).
2020-05-28 14:20:52 -07:00
Toshio Kuratomi
412af6ae91
Fix docs return formatting. (#69632)
Gets rid of the unknown field names in the return data.
Allows the plugin return docs to format under the new docs pipeline.
* Expect that package_facts will pass return-syntax-error now.
2020-05-28 15:23:30 -05:00
Sam Doran
d7618712fd
blockinfile - fix line end regression (#69734)
PR #66461 introduced a regression that resulted in an in correct block in the file
if the block to be inserted did not end with a line separator. Fix this bug and add
tests to cover this scenario.
2020-05-27 16:54:38 -04:00
Matt Clay
65c57a7f5b Update default-test-container to version 1.14. 2020-05-27 11:31:38 -07:00
Riyad Preukschas
e5cc12a64f
blockinfile - Preserve line endings (#66461)
Fixes #64966
* Fix "TypeError: splitlines() takes no keyword arguments" on Python2.7
* Add changelog fragment
* Don't use `grep -P` for BSD/macOS compatibility
* Fix sanity checks complaining about test fixtures with mixed line endings
* Update changelogs/fragments/66461-blockinfile_preserve_line_endings.yaml
2020-05-27 11:05:07 -04:00
Abhijeet Kasurde
564907d8ac
distribution: Refactor test_distribution_version testcases (#69300) 2020-05-27 14:01:31 +05:30
Rick Elrod
304c3e57e8
[ssh] Add new sshpass_prompt option (#68874)
Change:
Allows the user to configure sshpass (1.06+) to look for a different
substring than the default "assword" that it comes with.

Test Plan:
Set a custom ssh password prompt on a VM with PAM and tried connecting to
it. Without `ansible_sshpass_prompt` set in inventory: experienced hang.
With `ansible_sshpass_prompt` in inventory: connected successfully.

Tried setting `ansible_sshpass_prompt` with an older `sshpass` in PATH
and got a loud error, as expected.

Tickets:
Fixes #34722, fixes #54743, refs #11565.

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-26 19:58:31 -05:00
Felix Fontein
ea04e0048d
Allow to deprecate options and aliases by date (#68177)
* Allow to deprecate options and aliases by date instead of only by version.

* Update display.deprecate().

* Adjust behavior to conform to tested behavior, extend tests, and improve C# style.

* Parse date and fail on invalid date.

This is mainly to make sure that people start using invalid dates, and we eventually have a mess to clean up.

* C# code: improve validation and update/extend tests.

* Make sure that deprecate() is not called with both date and version.

* Forgot to remove no longer necessary formatting.

* Adjust order of warnings in C# code.

* Adjust unrelated test.

* Fix grammar (and make that test pass).

* Don't parse date, and adjust message to be same as in #67684.

* Sanity tests: disable date in past test.

* Validate-modules: validate ISO 8601 date format.

* Validate-modules: switch schema declaration for deprecated_aliases to improve error messages for invalid dates.

* Use DateTime instead of string for date deprecation.

* Validate that date in deprecated_aliases is actually a DateTime.

* Fix tests.

* Fix rebasing error.

* Adjust error codes for pylint, and add removed_at_date and deprecated_aliases.date checks to validate-modules.

* Make deprecation date in the past error codes optional.

* Make sure not both version and date are specified for AnsibleModule.deprecate() calls.

* Stop using Python 3.7+ API.

* Make sure errors are actually reported. Re-add 'ansible-' prefix.

* Avoid crashing when 'name' isn't there.

* Linting.

* Update lib/ansible/module_utils/csharp/Ansible.Basic.cs

Co-authored-by: Jordan Borean <jborean93@gmail.com>

* Adjust test to latest change.

* Prefer date over version if both end up in Display.deprecated().

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2020-05-26 20:23:56 -04:00
Matt Davis
fa81cc6a0c
fix delegated loading when path is not a directory (#69713)
* find_module can't pop ImportError- we need to just translate to `None` since this is a normal condition with files on sys.path (eg `/usr/lib/python36.zip`)
* added test
2020-05-26 15:55:17 -07:00
Matt Clay
5fc01c0a1d
Use a custom build container on Shippable. (#69660)
* Use a custom build container on Shippable.

This supports key generation before git_sync, to avoid issues with pre-migration PRs.

If the node pool is switched to another version, a matching build container should be built and used.

* Don't kill our own custom container.
2020-05-26 14:05:46 -07:00
Rick Elrod
4aff87770e
[yum] Make package removal confirmation strict (#69592)
* [yum] Make package removal confirmation strict

Change:
After removing packages, the yum module does a final check to ensure the
packages are really installed. The check would include packages that
were RPM `Provides:` values of another package.

This means that, for example, if a third-party kernel RPM spec had
`Provides: kernel` in it, removing the stock kernel would be successful
but the check to see if it was really removed would fail and cause
Ansible to report a failure.

Test Plan:
Tested on local CentOS 7 VM with kernel from elrepo which is known to
`Provides: kernel`.

Tickets:
Fixes #69237
Refs #35672
Refs #40723

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-26 14:47:39 -04:00
Matt Davis
f7dfa817ae
collection routing (#67684)
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
2020-05-26 09:42:06 -07:00
Nilashish Chakraborty
dae3ba71a8
Add FQCN mappings for network facts modules (#69601)
* Add FQCN mappings for network facts modules

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Add unit tests

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* update entry for ironware

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
2020-05-22 23:20:21 +05:30
Rick Elrod
fe9696be52
Fix ansible -K become_pass regression (#69629)
* Fix `ansible -K` become_pass regression

Change:
- This fixes a breaking change introduced in
  2165f9ac40

Test Plan:
- Local VM for now, with plans to add an integration test for -K going
  forward.

Tickets:
Refs #69244
2020-05-22 09:34:26 -04:00
Brian Coca
de3f7c7739
fix delegated interpreter discovery (#69604)
* fix delegated interpeter
* allow returning fact if it is 'the right host'
* added note for future fix/efficiency
 as it stands we rerun discovery for the delegated host
unless its saving facts to itself
 * fixed test lacking delegate_to mock
2020-05-22 09:31:34 -04:00
James Cammarata
a4072ad0e9
Split regular and handler results into their own queues (#69498)
When mixed with the free strategy (or any custom strategy that does not behave in
a lock-step manner), the linear methodology of _wait_on_handler_results may cause
race conditions with regular task result processing if the strategy uses
_process_pending_results directly. This patch addresses that by splitting the queues
used for results and adding a flag to _process_pending_results to determine which
queue to check.

Fixes #69457
2020-05-21 15:55:08 -05:00
Sam Doran
cd8920af99
file - return 'state': 'absent' when a file does not exist (#66503)
This was changed in ansible/ansible#51350.
2020-05-21 16:35:45 -04:00
Matt Martz
e0f25a2b1f
Strip no log values from module response keys. Fixes #68400 (#69653) 2020-05-21 16:17:57 -04:00
Martin Krizek
1ee70fc272
ansible.utils.vars.isidentifier improvements (#58278)
ci_complete
2020-05-21 19:54:40 +02:00
Brian Coca
9281148b62
correctly merge multiple facts results (#68987)
* correctly merge multiple facts results

  fixes #68532
2020-05-20 18:53:37 -04:00
Sam Doran
fe941a4045 Pin Docker version at 19.03.1 2020-05-20 15:10:41 -07:00
Matt Clay
8fe812b212
Remove conflicting plugin from test/support/ dir. (#69628) 2020-05-20 14:52:26 -07:00
Matt Martz
b748edea45
Add multipart/form-data functionality (#69376)
* Add multipart/form-data functionality

* Fix some linting issues

* Fix error message

* Allow filename to be provided with content

* Add integration test

* Update examples

* General improvements to multipart handling

* Use prepare_multipart for galaxy collection publish

* Properly account for py2 vs py3, ensuring no max header length

* Address test assumptions

* Add unit tests

* Add changelog

* Ensure to use CRLF instead of NL

* Ignore line-endings in fixture

* Consolidate code, add comment

* Bump fallaxy container version

* ci_complete
2020-05-20 15:44:01 -05:00
Martin Krizek
ecd986006e
Properly handle unicode in safe_eval (#68576)
* Properly handle unicode in safe_eval

Fixes #66943

* Update lib/ansible/template/safe_eval.py

Co-authored-by: Sam Doran <sdoran@redhat.com>
2020-05-20 12:08:50 -04:00
Sam Doran
889da811d7
Pin docker-ce-cli version in tests (#69620)
Installing docker-ce has a dependency of docker-ce-cli. If the version of docker-ci-cli is not specified, it installs the latest version.
2020-05-20 11:38:01 -04:00
Brian Coca
06ecdaa7b1
comment heavy playbook (#68981)
* create examples/play.yml


Co-authored-by: flowerysong <junk+github@flowerysong.com>
2020-05-19 17:27:08 -04:00
Brian Coca
df4e83deda
added 'task timeout' feature (#69284)
* added 'task timeout' feature


Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-05-19 17:20:08 -04:00