Commit graph

30812 commits

Author SHA1 Message Date
Ganesh Nalawade
646536643f
Use fqcr from command module invocation using shell module (#69790)
* Use fqcr from command module invocation using shell module

Fixes https://github.com/ansible/ansible/issues/69788

Use fully qualified collection reference while invoking
command module from shell module

* Fox review comment
2020-06-04 10:00:11 -07:00
Alex Schultz
247e43b252
Fix IncludedFile equality check (#69524)
In the case of a free style strategy, it is possible to end up with
multiple hosts trying to include from the same role, however the tasks
being included may be different with the use of tasks_from.  Previously
if you had two hosts that were included the same role when the
process_include_results function tries to determine if a included needs
to be run on a specific host, it would end up merging two different
tasks into which ever one was processed first.

This change updates the equality check to also check if the task uuid
associated with the IncludedFile is the same. The previous check only
checked if the task's parent uuid was the same. This breaks down when
both includes have the same parent.

    - hosts: all
      strategy: free
      gather_facts: false
      tasks:
        - include_role:
            name: random_sleep
        - block:
          - name: set a fact (1)
            include_role:
              name: set_a_fact
              tasks_from: fact1.yml
          - name: set a fact (2)
            include_role:
              name: set_a_fact
              tasks_from: fact2.yml
        - name: include didn't run
          fail:
            msg: >
              set_a_fact didn't run
              fact1: {{ fact1 | default('not defined')}}
              fact2: {{ fact2 | default('not defined') }}"
          when: (fact1 is not defined or fact2 is not defined)

Closes #69521
2020-06-04 10:41:46 -05:00
Toshio Kuratomi
53b95b865f __eq__ should be redefined if __hash__ is defined. 2020-06-03 21:57:21 -07:00
Matt Clay
4ecf22db9a Revert "fixes f5 modules specific redirects (#69865)"
This reverts commit 16f5dbf985.
2020-06-03 19:41:04 -07:00
Wojciech Wypior
16f5dbf985
fixes f5 modules specific redirects (#69865)
* fixes f5 modules specific redirects

* adds removed entry
2020-06-03 21:58:27 +01:00
Brian Coca
22da903e9c
return changed for group_by (#32057)
* return changed for group_by

* added tests and fixed 'early registeration'
2020-06-03 12:46:40 -04:00
Andrew Klychkov
00ead98595
systemd: should fail in check_mode when service not found on host (#68136)
* systemd: should fail in check_mode when service not found on host
2020-06-02 11:52:12 -04:00
Abhijeet Kasurde
d757995bb6
service_facts: Update documentation about service status (#69757)
Update documentation about undocumented service status provided
by systemd

Fixes: #69752

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-06-02 11:07:02 -04:00
Dongsu Park
598e3392a9
Discover Flatcar Linux properly for hostname (#69627)
To avoid issues with Flatcar Container Linux being unable to be found,
detect Flatcar distro name especially for hostname, just like CoreOS
Container Linux was supported.

See also https://github.com/ansible/ansible/issues/69516
2020-06-02 18:41:53 +05:30
Evgeni Golov
e10e5fc4ad
only ever catch ImportError (#69792)
ModuleNotFoundError is a subclass of ImportError but only exists in
Python 3.6 or newer. Instead of doing hacks to be able to catch that on
older Pythons, just always only catch ImportError, which will also catch
ModuleNotFoundError on Python 3.6 or later
2020-06-01 09:01:38 -07:00
Jack
80f09efd03
do not return the body even if it failed (#69706)
* do not return the body even if it failed

* add some tests for this and rebase

* import test task

* ignore_errors when fails

Co-authored-by: Jack Zhang <jack.zhang@aspiraconnect.com>
2020-06-01 09:55:38 -04:00
Abhijeet Kasurde
723a904f4e
Handle disabled service units (#69349)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-06-01 09:40:52 -04:00
Rick Elrod
2abaf320d7
[ansiballz] ensure that '' is not in sys.path (#69342)
Change:
On OpenBSD when using pipelining, we do not set cwd which results in a
permissions fatal. Ensure that `''` - cwd - is not in `sys.path`.

Test Plan:
Tested against local OpenBSD VM

Tickets:
Fixes #69320

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-06-01 03:43:20 -05:00
Hideki Saito
79ab798427 Enable logging using setup_loggers() API in dnf-4.2.17-6 or later
- Fixed issue #28061
- https://bugzilla.redhat.com/show_bug.cgi?id=1788212

Signed-off-by: Hideki Saito <saito@fgrep.org>
2020-05-30 20:37:27 -05:00
Felix Fontein
b1b79be08b
Fix return value interpretation of PluginLoader.find_plugin_with_context(). (#69772) 2020-05-29 17:17:28 -07:00
Evgeni Golov
53f9822e75
make plugins loader Python 3.5 compatible again (#69754)
ModuleNotFoundError was only introduced in Python 3.6
2020-05-29 12:37:52 -07:00
Martin Krizek
3606dcfe65
Ensure --version works with non-ascii project path (#66624)
Fixes #66617
2020-05-29 14:42:44 -04:00
Sloane Hertel
e40889e711
Add support to install collections from git repositories (#69154)
* Enable installing collections from git repositories

* Add tests for installing individual and multiple collections from git repositories

* Test to make sure recursive dependencies with different syntax are deduplicated

* Add documentation

* add a changelog

* Skip Python 2.6

* Only fail if no collections are located in a git repository

Add support for a 'type' key for collections in requirement.yml files.
Update the changelog and document the supported keys and allowed values for the type.

Add a note that the collection(s) in the repo must contain a galaxy.yml

* Add a warning about embedding credentials in SCM URLs

* Update with review suggestions

* suppress sanity compile failure for Python 2.6
2020-05-29 13:33:32 -04:00
Karl Fogel
225ae65b0f
Make Python path warning say what it means to say (#69669) 2020-05-29 13:15:11 -04:00
Nilashish Chakraborty
c3dff34c15
Add entries for missing network modules (#69760)
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
2020-05-29 10:11:45 -07:00
Rick Elrod
8e19ab178a
connection plugins: try config, then play_context (#69751)
Change:
Rather than only using config, have base connection plugins fall back to
play_context.

Test Plan:
- Tested ansible-connection logic against an IOS device
- Tested -k against a VM
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-29 10:00:21 -05:00
John R Barker
eea84b6fdd
routing.yml: update collections (#69699)
* routing.yml: update collections

routing.yml had gotten out of sync since `migrate.py` was run.
Update based on where files are today (more updates are needed)
Also validate routing.yml's schema

* Update lib/ansible/config/routing.yml

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>

* rename scripts

* docker_service

* updated ansible_builtin_runtime schema

* Correct schema for plugin_router and import_redirect

* Correct schema for plugin_router and import_redirect

* validation moved to #69742

* netap, slxos, checkpoint

* test moved to separate PR

* even more

Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru>
2020-05-29 14:25:39 +01:00
Abhijeet Kasurde
7b3d9cbcb8
distribution: Added support for openEuler OS (#69324)
Detect os_family for openEuler OS as 'RedHat', instead of 'openEuler'.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-05-29 12:43:16 +05:30
Jordan Borean
f81f5da20e
Ansible.Basic added generic fragment merger for module options (#69719) 2020-05-29 16:11:38 +10:00
Felix Fontein
40f21dfd3c
Version source tagging (automatic and manual) for version_added and deprecation versions (#69680)
* Track collection for version_added.
Validate *all* version numbers in validate-modules.
For tagged version numbers (i.e. version_added), consider source collection to chose validation.

* Make tagging/untagging functions more flexible.

* Tag all versions in doc fragments.

* Tag all deprecation versions issued by code.

* Make Display.deprecated() understand tagged versions.

* Extend validation to enforce tagged version numbers.

* Tag versions in tests.

* Lint and fix test.

* Mention collection name in collection loader's deprecation/removal messages.

* Fix error IDs.

* Handle tagged dates in Display.deprecated().

* Also require that removed_at_date and deprecated_aliases.date are tagged.

* Also automatically tag/untag removed_at_date; fix sanity module removal version check.

* Improve error message when invalid version number is used (like '2.14' in collections).
2020-05-28 22:46:16 -07:00
Jordan Borean
f5f3ba7ab5
ansible-test - fix up relative util import for powershell validate-modules (#69753)
* ansible-test - fix up relative util import for powershell validate-modules

* Use different tactic for generic group

* Use python 2 and 3
2020-05-29 14:31:59 +10:00
Felix Fontein
31bf3a5622
Deprecate module in collection: allow removal date in documentation, make validate-modules ensure version and date match (#69727)
* Allow to deprecate module by date in documentation.

* Make sure deprecation date/version match between module docs and meta/runtime.yml.

* Unrelated fix: don't compare deprecated module version to Ansible's version in collection.

* Allow documentation's removal version to be something else than fixed list of Ansible versions for collections.

* Linting.

* Allow to deprecate plugin options by date.

* Add changelog fragment for deprecation by date (also covers #68177).
2020-05-28 14:20:52 -07:00
Martin Krizek
061c6c7c6f
Prevent losing error info by including both loop and cond error msgs (#68485)
Fixes #66529
2020-05-28 17:13:39 -04:00
Nick Busey
0894ea1b1d
Adding nested variable example (#69518)
* Adding nested variable example to lib/ansible/plugins/lookup/vars.py
Co-authored-by: Matt Martz <matt@sivel.net>
2020-05-28 15:48:52 -05:00
Andrew Klychkov
323d2adfcc
Fix example format in Ansible modules (#69509)
* Fix example format in several Ansible modules
* Update lib/ansible/modules/fail.py
* Update lib/ansible/modules/pip.py

Co-authored-by: Felix Fontein <felix@fontein.de>
2020-05-28 15:42:26 -05:00
Toshio Kuratomi
412af6ae91
Fix docs return formatting. (#69632)
Gets rid of the unknown field names in the return data.
Allows the plugin return docs to format under the new docs pipeline.
* Expect that package_facts will pass return-syntax-error now.
2020-05-28 15:23:30 -05:00
Ganesh Biradar
187de7a8aa
Updated CLI help for 'ansible-galaxy' for 'collection' subcommand (#69458)
* Added YAML header line and link to existing github issue
2020-05-28 10:38:48 -04:00
Sam Doran
d7618712fd
blockinfile - fix line end regression (#69734)
PR #66461 introduced a regression that resulted in an in correct block in the file
if the block to be inserted did not end with a line separator. Fix this bug and add
tests to cover this scenario.
2020-05-27 16:54:38 -04:00
Riyad Preukschas
e5cc12a64f
blockinfile - Preserve line endings (#66461)
Fixes #64966
* Fix "TypeError: splitlines() takes no keyword arguments" on Python2.7
* Add changelog fragment
* Don't use `grep -P` for BSD/macOS compatibility
* Fix sanity checks complaining about test fixtures with mixed line endings
* Update changelogs/fragments/66461-blockinfile_preserve_line_endings.yaml
2020-05-27 11:05:07 -04:00
Rick Elrod
8b6c02fc69
Make connection plugins use password from config (#69670)
Change:
Rather than connection plugins directly accessing play_context and pulling
the password from there, have them pull it from the config system, and
have TaskExecutor store it there for now.

Internally, it still routes through play_context for now, but this is
the first step away from that.

Test Plan:
- Local test with `ansible -c ssh`
- grep -R play_context.pass lib/ansible/plugins/connection/
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-27 09:59:55 -05:00
Felix Fontein
cb8f645e0f
Allow to deprecate / remove plugins by version. (#69712) 2020-05-26 18:43:40 -07:00
Rick Elrod
304c3e57e8
[ssh] Add new sshpass_prompt option (#68874)
Change:
Allows the user to configure sshpass (1.06+) to look for a different
substring than the default "assword" that it comes with.

Test Plan:
Set a custom ssh password prompt on a VM with PAM and tried connecting to
it. Without `ansible_sshpass_prompt` set in inventory: experienced hang.
With `ansible_sshpass_prompt` in inventory: connected successfully.

Tried setting `ansible_sshpass_prompt` with an older `sshpass` in PATH
and got a loud error, as expected.

Tickets:
Fixes #34722, fixes #54743, refs #11565.

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-26 19:58:31 -05:00
Felix Fontein
ea04e0048d
Allow to deprecate options and aliases by date (#68177)
* Allow to deprecate options and aliases by date instead of only by version.

* Update display.deprecate().

* Adjust behavior to conform to tested behavior, extend tests, and improve C# style.

* Parse date and fail on invalid date.

This is mainly to make sure that people start using invalid dates, and we eventually have a mess to clean up.

* C# code: improve validation and update/extend tests.

* Make sure that deprecate() is not called with both date and version.

* Forgot to remove no longer necessary formatting.

* Adjust order of warnings in C# code.

* Adjust unrelated test.

* Fix grammar (and make that test pass).

* Don't parse date, and adjust message to be same as in #67684.

* Sanity tests: disable date in past test.

* Validate-modules: validate ISO 8601 date format.

* Validate-modules: switch schema declaration for deprecated_aliases to improve error messages for invalid dates.

* Use DateTime instead of string for date deprecation.

* Validate that date in deprecated_aliases is actually a DateTime.

* Fix tests.

* Fix rebasing error.

* Adjust error codes for pylint, and add removed_at_date and deprecated_aliases.date checks to validate-modules.

* Make deprecation date in the past error codes optional.

* Make sure not both version and date are specified for AnsibleModule.deprecate() calls.

* Stop using Python 3.7+ API.

* Make sure errors are actually reported. Re-add 'ansible-' prefix.

* Avoid crashing when 'name' isn't there.

* Linting.

* Update lib/ansible/module_utils/csharp/Ansible.Basic.cs

Co-authored-by: Jordan Borean <jborean93@gmail.com>

* Adjust test to latest change.

* Prefer date over version if both end up in Display.deprecated().

Co-authored-by: Jordan Borean <jborean93@gmail.com>
2020-05-26 20:23:56 -04:00
Matt Davis
fa81cc6a0c
fix delegated loading when path is not a directory (#69713)
* find_module can't pop ImportError- we need to just translate to `None` since this is a normal condition with files on sys.path (eg `/usr/lib/python36.zip`)
* added test
2020-05-26 15:55:17 -07:00
Brian Coca
c987c0c8a1
provides per host/plugin world readable temp opt (#69393)
fixes #46174
2020-05-26 15:29:37 -04:00
Rick Elrod
4aff87770e
[yum] Make package removal confirmation strict (#69592)
* [yum] Make package removal confirmation strict

Change:
After removing packages, the yum module does a final check to ensure the
packages are really installed. The check would include packages that
were RPM `Provides:` values of another package.

This means that, for example, if a third-party kernel RPM spec had
`Provides: kernel` in it, removing the stock kernel would be successful
but the check to see if it was really removed would fail and cause
Ansible to report a failure.

Test Plan:
Tested on local CentOS 7 VM with kernel from elrepo which is known to
`Provides: kernel`.

Tickets:
Fixes #69237
Refs #35672
Refs #40723

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-26 14:47:39 -04:00
Matt Davis
f7dfa817ae
collection routing (#67684)
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
2020-05-26 09:42:06 -07:00
Helder Garcia
fdfa6fec75
Module Find returns empty list when setting path to "/" and depth to 1 (#69650)
* Fix bug on find module when path is / and depth 1.
2020-05-26 12:30:59 -04:00
Nilashish Chakraborty
dae3ba71a8
Add FQCN mappings for network facts modules (#69601)
* Add FQCN mappings for network facts modules

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* Add unit tests

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>

* update entry for ironware

Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
2020-05-22 23:20:21 +05:30
Rick Elrod
fe9696be52
Fix ansible -K become_pass regression (#69629)
* Fix `ansible -K` become_pass regression

Change:
- This fixes a breaking change introduced in
  2165f9ac40

Test Plan:
- Local VM for now, with plans to add an integration test for -K going
  forward.

Tickets:
Refs #69244
2020-05-22 09:34:26 -04:00
Brian Coca
de3f7c7739
fix delegated interpreter discovery (#69604)
* fix delegated interpeter
* allow returning fact if it is 'the right host'
* added note for future fix/efficiency
 as it stands we rerun discovery for the delegated host
unless its saving facts to itself
 * fixed test lacking delegate_to mock
2020-05-22 09:31:34 -04:00
Rick Elrod
dc63b36501
Spelling: dependant -> dependent (#69661) 2020-05-22 11:24:25 +05:30
James Cammarata
a4072ad0e9
Split regular and handler results into their own queues (#69498)
When mixed with the free strategy (or any custom strategy that does not behave in
a lock-step manner), the linear methodology of _wait_on_handler_results may cause
race conditions with regular task result processing if the strategy uses
_process_pending_results directly. This patch addresses that by splitting the queues
used for results and adding a flag to _process_pending_results to determine which
queue to check.

Fixes #69457
2020-05-21 15:55:08 -05:00
Sam Doran
cd8920af99
file - return 'state': 'absent' when a file does not exist (#66503)
This was changed in ansible/ansible#51350.
2020-05-21 16:35:45 -04:00
Matt Martz
e0f25a2b1f
Strip no log values from module response keys. Fixes #68400 (#69653) 2020-05-21 16:17:57 -04:00
David Soper
eb5147e57b
intersight_info and intersight_facts routed to cisco.intersight (#69648)
Co-authored-by: “dsoper2” <“dsoper@cisco.com”>
2020-05-21 15:28:22 -04:00
Martin Krizek
1ee70fc272
ansible.utils.vars.isidentifier improvements (#58278)
ci_complete
2020-05-21 19:54:40 +02:00
Brian Coca
9281148b62
correctly merge multiple facts results (#68987)
* correctly merge multiple facts results

  fixes #68532
2020-05-20 18:53:37 -04:00
Matt Martz
b748edea45
Add multipart/form-data functionality (#69376)
* Add multipart/form-data functionality

* Fix some linting issues

* Fix error message

* Allow filename to be provided with content

* Add integration test

* Update examples

* General improvements to multipart handling

* Use prepare_multipart for galaxy collection publish

* Properly account for py2 vs py3, ensuring no max header length

* Address test assumptions

* Add unit tests

* Add changelog

* Ensure to use CRLF instead of NL

* Ignore line-endings in fixture

* Consolidate code, add comment

* Bump fallaxy container version

* ci_complete
2020-05-20 15:44:01 -05:00
Martin Krizek
ecd986006e
Properly handle unicode in safe_eval (#68576)
* Properly handle unicode in safe_eval

Fixes #66943

* Update lib/ansible/template/safe_eval.py

Co-authored-by: Sam Doran <sdoran@redhat.com>
2020-05-20 12:08:50 -04:00
Toshio Kuratomi
9780d3db18 Remove "keywords" field in DOCUMENTATION.
This doesn't seem to be used by config although it came in with the
config PR.  It's not displayed in docs.  Removing it as this is the only
place where it's present.
2020-05-19 22:36:53 -07:00
Brian Coca
df4e83deda
added 'task timeout' feature (#69284)
* added 'task timeout' feature


Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-05-19 17:20:08 -04:00
Felix Fontein
ac533403e3
Google gcp_*_facts modules were symlinks to gcp_*_info modules in 2.9. (#69566)
Since it doesn't make sense to have extra copies of the _info modules in community.general only so that the _facts
symlinks are not in google.cloud, let's simply redirect the _facts modules to their google.cloud _info counterpart.

See also ansible-collections/community.general#351.
2020-05-19 10:40:48 -05:00
Sam Doran
f01de15d25
ansible-galaxy - remove warning during collection install (#69541)
* ansible-galaxy - remove warning during collection install

If existing collections do not contain a MANIFEST.json, which is common
for collections under development that were not installed from Ansible
Galaxy, fall back to inspecting galaxy.yml rather than displaying a
warning.

A warning will still be displayed in neither a MANIFEST.json nor
galaxy.yml are present.

* Update unit tests
2020-05-19 10:31:49 -05:00
Monty Taylor
dd4219184b
Update openstack module redirections (#69562)
The openstack related modules moved to the openstack cloud
collection. Update the target pointers.
2020-05-19 10:00:04 -04:00
Jordan Borean
ecea15c508
Unify ansible-galaxy install -r (#67843)
* Unify ansible-galaxy install -r

* Minor nit fixes for docs

* Re-align warnings

* Fix up integration test

* Fix up test where no roles/collections were in file
2020-05-19 05:09:42 +10:00
Matt Clay
86743292ec Remove incorrect function args in unarchive. 2020-05-15 18:31:43 -07:00
Matt Clay
ed4fd9be67 Code cleanup in ansible-inventory. 2020-05-15 18:02:18 -07:00
Sam Doran
b8469d5c7a
lineinfile - Use ANSIBLE_REMOTE_TMP for temporary file (#69543) 2020-05-15 15:52:17 -04:00
Lukas Pirl
34db57a47f
introduce fact "ansible_processor_nproc": number of usable vcpus (#66569)
This fact reflects the number of usable vcpus (which might be different
from ansible_processor_vcpus, e.g., in containers with limits). See
also #51504.

* Add fixture data and update unit tests

Co-authored-by: Sam Doran <sdoran@redhat.com>
2020-05-15 09:38:56 -04:00
Jeff Geerling
d63a71e3f8
typofix - globbing instead of globing. (#69515)
Fix typo introduced in PR #69087
2020-05-15 13:23:35 +05:30
Matt Clay
d0484fbf74 Cleanup for Python 3.9 and pycodestyle compat. 2020-05-14 21:06:16 -07:00
Richard Lau
9d8cf1ed21
Add path for yum on IBM i to PKG_MGRS (#69484)
* Add path for yum on IBM i to PKG_MGRS

On IBM i, yum is installed under the `/QOpenSys/pkgs` prefix, see:
https://bitbucket.org/ibmi/opensource/src/master/docs/yum/#markdown-header-must-know-usage-notes-read-this-after-you-install

* Add changelog fragment
2020-05-14 19:09:09 -04:00
Brian Coca
2165f9ac40
fix delegation vars usage (debug still shows inventory_hostname) (#69244)
* fix delegation vars usage and reporting

 - just pass delegated host vars + task vars to plugins
   and avoid poluting with original host vars
 - updated tests
2020-05-14 16:36:13 -04:00
Chris Holland
dfa2863dee
Remove FIXME where no change is needed (#69466)
* Added exception

* Added changelog fragment

* Remove FIXME comment

* Edit changelog fragment
2020-05-14 15:44:33 -04:00
Felix Fontein
9767d592a8
Add resource_from_fqcr utility function. (#68474) 2020-05-14 13:50:29 -04:00
Sloane Hertel
55e29a1464
ansible-galaxy - fall back to galaxy.yml when listing collections (#68925)
Fallback to galaxy.yml for listing collections that don't have a FILES.json or MANIFEST.json

Co-authored-by: Sam Doran <sdoran@redhat.com>
2020-05-14 12:28:08 -04:00
Sloane Hertel
343ffaa18b
better error for "ansible-galaxy collection verify" if there's no MANIFEST.json (#67498)
* Add a better error for "ansible-galaxy verify" if the MANIFEST.json has been deleted from the installed collection or if the collection hasn't been installed via normal means

* Fix unit tests for the remote collection

If there's something wrong with the local collection's version it will fail before the remote collection is sought

* Add a test for the new error msg

* Prevent the duplicate warning

Mock the new isfile call where needed

* Update lib/ansible/galaxy/collection.py

Co-Authored-By: Martin Krizek <martin.krizek@gmail.com>

Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
2020-05-14 12:25:19 -04:00
Bob Weinand
f200487414
Fix filedescriptor out of range in select() when running commands (#65058)
* Fix filedescriptor out of range in select() when running commands

* Simplify the run_command() code

Now that we're using selectors in run_command(), we can simplify some of
the code.

* Use fileobj.read() instead of os.read()
* No longer use get_buffer_size() as we can just slurp all of the data
  instead.

Also use a simpler conditional check of whether the selector map is
empty

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
2020-05-14 11:46:34 -04:00
Brian Coca
87d9b49de2
Fix listing of colleciton plugins with symlinks (#69305)
* Fix listing of colleciton plugins with symlinks
2020-05-14 11:45:02 -04:00
Ted Pearson
c13b040e67
systemd: set scope default to "system" (#67006)
- Make "system" default scope for systemd
- Remove extra None checks since there is now a default
2020-05-14 11:36:47 -04:00
Brian Coca
cedfe34619
added unvault lookup plugin (#69087)
* added unvault lookup plugin
2020-05-14 11:19:19 -04:00
Matt Martz
4c4406b2df
Flatten the directory hierarchy of modules (#68966)
* Flatten the directory hierarchy of modules

* Update ignore.txt, flatten units

* Update imports

* Completely flatten the modules directory

* Update sanity ignore

* Fix some sanity test ignores

* Fix relative import

* Fix docs builds without category

* ci_complete

* Clean up docs. ci_complete

* Adjust needs/file alias

* ci_complete

* fix hardcoded ping module paths

Co-authored-by: Matt Davis <mrd@redhat.com>
2020-05-13 19:14:53 -07:00
Brian Coca
c43f275c7e
added missing headers + note (#69500) 2020-05-13 17:12:48 -04:00
Brian Coca
f0b6f76bc6
only show_vars when showing vars (#69365)
* only show_vars when showing vars

avoid processing function params that can be very expensive
and might not be used at all in called function.

fixes #69357

* Update changelogs/fragments/69357_optimize_inventory_graph_wo_vars.yml

Co-authored-by: Sloane Hertel <shertel@redhat.com>

Co-authored-by: Sloane Hertel <shertel@redhat.com>
2020-05-13 15:03:30 -04:00
Toshio Kuratomi
3b3c811be5
Separate the galaxy lib from the cli (#69473)
The galaxy lib knew about the cli args in context.  This shouldn't be
the case as it makes it hard to use the lib in other contexts.  Moved
the context knowledge into cli/galaxy.py.
2020-05-13 13:13:30 -04:00
Brian Coca
0aa76503dc
avoid fatal tb on bad fqcn callback name (#69440) 2020-05-13 12:02:31 -04:00
Toshio Kuratomi
eb3e4b3a7b cyberark appears to have renamed their collection
Trying to build docs for Ansible-2.10 and ran into error #69475
in ansible-doc.  To work around that I need to get a newer cyberark
collection which fixes the issues.  Unfortunately, it looks like the
cyberark team has renamed their collection from bizdev to pas is not
responding to issues on github.  I'm putting in this PR for them to
get this updated before 2.10 so that the modules will work out of the
box.
2020-05-13 07:54:02 -07:00
Martin Krizek
6086ea62ee
Remove deprecation for TRANSFORM_INVALID_GROUP_CHARS (#66650)
Fixes #61889
2020-05-13 10:16:32 -04:00
Mark Goddard
aa36b02ede
Fix fileglob plugin with non-existent subdirectory (#69451)
Since Ansible 2.9.8, if the fileglob plugin is passed a path containing
a subdirectory of a non-existent directory, it will fail. For example:

lookup('fileglob', '/'): ok
lookup('fileglob', '/foo'): (non-existent): ok
lookup('fileglob', '/foo/bar'): (non-existent): FAIL

The exact error depends on Python 2 or 3, but here is the error on
Python 2:

    AttributeError: 'NoneType' object has no attribute 'endswith'

And on Python 3:

    TypeError: expected str, bytes or os.PathLike object, not NoneType

This change fixes the issue by skipping paths that are falsey before
passing them to os.path.join().

Fixes: #69450
2020-05-12 09:12:21 -04:00
Toby Foster
01258580b9
Fix formatting in docs for --become-method (#68152)
The default value isn't displayed currently: <https://docs.ansible.com/ansible/latest/cli/ansible.html#cmdoption-ansible-become-method>

+label: docsite_pr
2020-05-12 04:06:24 -04:00
Martin Krizek
9645304da3
Validate args for includes in handlers too (#57537) 2020-05-12 09:35:58 +02:00
Rylan Polster
eb40ecc843
Check for correct version of systemd Python library (#60692)
Fix issue where some versions of systemd don't have journal.sendv
2020-05-12 11:01:08 +05:30
Adrian Likins
cd8dd4a272
Fix galaxy publish sha256 value format. (#67942)
* Fix galaxy publish sha256 value format.

The multipart/form content used for the body
of the POST to /api/automation-hub/v3/collections
was missing a newline before the line with the value
of the sha256.

automation-hub/galaxy/django skips the field entirely in
that case and automation-hub code will use None for default
to indicate that no sha256 is provided (an available option).

Fixes ansible/galaxy-dev#246

* Add changelog fragment

Co-authored-by: Matt Martz <matt@sivel.net>
2020-05-11 12:52:38 -05:00
Felix Fontein
5b9418c06c
Fix SemanticVersion comparison (#69395)
* Fix SemanticVersion comparison.

* Complete tests for _Alpha and _Numeric comparators.

* Linting, and add comment.
2020-05-11 11:58:15 -05:00
Rick Elrod
9579113941 Remove some no-op code from InventoryManager
Change:
- Remove some no-op code
- Split up a somewhat complex line into two lines
- Nuke an incorrect comment

Test Plan:
CI

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-07 21:42:48 -05:00
Gonéri Le Bouder
c62ff861d3
git: don't import get_module_path (#69378)
The module don't use the `get_module_path()` function. No need to import
it.
2020-05-07 16:07:41 -04:00
rchincholkar
bc1364952b
changing $HOME to $HOSTNAME in Synopsis (#69273)
$HOME variable does work with command module. This may be a bug. $HOSTNAME never works. Changing the example to use $HOSTNAME in the Synopsis of command module documentation page.
2020-05-07 14:39:16 -05:00
Abhijeet Kasurde
e28e86a428
hostname: PopOS support (#69295)
Added support for PopOS in hostname module.

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-05-07 15:36:14 -04:00
Toshio Kuratomi
39b3942048 Remove left hand side slicing
Left hand side slicing is confusing and slower but maybe more memory
efficient in some circumstances.  There is one case where it adds to
code safety: when it's used to substitute a different list in place of a
slice of the original list and the original list could have been bound
to a different variable in some other code.  (The most likely case of
this is when it's a global variable and some other code might import
that variable name).

Because of the confusion factor we think it should only be used for the
safety case or where it's been benchmarked and shown to have some sort
of documentatble improvement.  At the moment, only one piece of code
falls into those categories so this PR removes all the other instances
of left hand side slicing.
2020-05-06 09:56:40 -07:00
Eric L
852906fe7f
Make description of removes option of command module clearer (hopefully) (#68097)
* Improves description of removes option in command module

Co-authored-by: Eric L <android@lavar.de>
2020-05-06 11:47:52 -05:00
Brian Coca
4e72ce805d
Order my dicts (#58000)
* use orderdict for yaml dictionaries

* clog

* SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS

* allow user to toggle odict off

* removed config, since requored to load config

* remove unused import

* Update changelogs/fragments/yaml_orderd_mappings.yml

Co-authored-by: Matt Clay <matt@mystile.com>

* Update lib/ansible/parsing/yaml/objects.py

Co-authored-by: Matt Clay <matt@mystile.com>

* Update lib/ansible/parsing/yaml/objects.py

Co-authored-by: Matt Clay <matt@mystile.com>

Co-authored-by: Matt Clay <matt@mystile.com>
2020-05-06 11:05:37 -04:00
Kenyon Ralph
f6860a7a89
generator inventory plugin: remove "strict" from example (#69178)
This "strict" option appears to be unused. It's probably a copy-paste error from the constructed plugin.
2020-05-05 15:09:31 -05:00
Matt Davis
47da0f7a4f
restore jsonfile cache plugin (#69337)
* restore jsonfile

* update botmeta for jsonfile restoration
2020-05-05 10:50:39 -07:00
flowerysong
d446a4f70d
yum: avoid running non-yum modules via the action (#69296)
If the system's detected package manager is not yum or dnf it should
refuse to run without an explicit backend, instead of running the
non-yum module.
2020-05-05 11:48:54 -05:00