Commit graph

2314 commits

Author SHA1 Message Date
Rick Elrod
21ac550828
virtualization facts: check /dev/kvm as a fallback (#70829)
Change:
- On Linux, there are situations where a host might be a KVM host but
  not have the kernel module enabled (it might be compiled in instead).
  In these cases, /dev/kvm will still exist, and rather than reporting
  NA, we should report that the host is a KVM host.

Test Plan:
- Local

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-23 19:50:04 -05:00
Jordan Borean
5e1a968983
Do not add connection vars to the output results (#70853)
* Do not add connection vars to the output results

* Also revert the delgated scenario JIC

* Added regression test
2020-07-24 09:53:08 +10:00
Matt Davis
4c0af6c808
fix internal cases of actions calling unqualified module names (#70818)
* fix internal cases of actions calling unqualified module names

* add porting_guide entry
* misc other fixes around action/module resolution broken by redirection

ci_complete

* Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst

Co-authored-by: Rick Elrod <rick@elrod.me>

* Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst

Co-authored-by: Rick Elrod <rick@elrod.me>

* address review feedback

* pep8

* unit test fixes

* win fixes

* gather_facts fix module args ignores

* docs sanity

* pep8

* fix timeout test

* fix win name rewrites

Co-authored-by: Rick Elrod <rick@elrod.me>
2020-07-23 09:02:57 -07:00
Abhijeet Kasurde
e439194c8c
basic: Add name of aliases in error message (#69427)
With this patch, user will be notified with available aliases
of arg parameter.

Fixes: #58752

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-23 16:02:18 +05:30
Sam Doran
5260527c4a
Change default file permissions so they are not world readable (#70221)
* Change default file permissions so they are not world readable

CVE-2020-1736

Set the default permissions for files we create with atomic_move() to 0o0660. Track
which files we create that did not exist and warn if the module supports 'mode'
and it was not specified and the module did not call set_mode_if_different(). This allows the user to take action and specify a mode rather than using the defaults.

A code audit is needed to find all instances of modules that call atomic_move()
but do not call set_mode_if_different(). The findings need to be documented in
a changelog since we are not warning. Warning in those instances would be frustrating
to the user since they have no way to change the module code.

- use a set for storing list of created files
- just check the argument spac and params rather than using another property
- improve the warning message to include the default permissions
2020-07-22 17:05:38 -04:00
David Shrewsbury
bf98f031f3
Sanitize URI module keys with no_log values (#70762)
* Add sanitize_keys() to module_utils.

* More robust tests

* Revert 69653 change

* Allow list or dict

* fix pep8

* Sanitize lists within dict values

* words

* First pass at uri module

* Fix insane sanity tests

* fix integration tests

* Add changelog

* Remove unit test introduced in 69653

* Add ignore_keys param

* Sanitize all-the-things

* Ignore '_ansible*' keys

* cleanup

* Use module.no_log_values

* Avoid deep recursion issues by using deferred removal structure.

* Nit cleanups

* Add doc blurb

* spelling

* ci_complete
2020-07-22 14:49:37 -05:00
Brian Coca
84adaba6f5
Allow hostvars delegation (#70331)
* ensure hostvars are available on delegation
* also inventory_hostname must point to current host and not delegated one
* fix get_connection since it was still mixing original host vars and delegated ones
* also return connection vars for delegation and non delegation alike
* add test to ensure we have expected usage when directly assigning for non delegated host
2020-07-22 11:13:57 -04:00
Nathaniel Case
06a4fc2833
Document existing ansi_re sequences and add ESC[m (#70683)
* Document existing ansi_re sequences and add `ESC[m`

* Add changelog
2020-07-22 08:56:21 -04:00
Abhijeet Kasurde
8cccede0d4
basic: use PollSelector implementation (#70352)
Some platform such as ESXi does not implement EpollSelector,
which is selected by DefaultSelector. Use SelectSelector which is
based upon 'select' implementation. This works perfectly with
a platform like VMware ESXi.

Fixes: #70238

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-22 09:18:21 +05:30
Matt Martz
96b74d3e0b
Ensure single vaulted values aren't counted as sequences. Fixes #70784 (#70786) 2020-07-21 16:48:35 -05:00
Baptiste Mille-Mathias
0ead4306a8
user - Create home and parent directories only when requested (#70600)
The home user and the parents directories should only be created when
create_home == True
2020-07-21 15:50:44 -04:00
psi / Ryo Hirafuji
5ce47646ad
cron - Allow non-ascii (UTF-8) chars in cron file paths and jobs (#70426)
* Encode/Decode files in UTF-8
* Use helper function in ansible
* Add an integration test
* Use emoji in test data.
* add changelog
* Also support non-ascii chars in filepath and add tests about this.
* Also use non-ascii chars in replaced text and ensure not to break cron syntax.
* rename self.existing to self.n_existing
* rename crontab.existing to crontab.n_existing
2020-07-21 15:00:21 -04:00
Abhijeet Kasurde
4f96f9826c
distribution: Add support for DragonFly (#70748)
partially fixes #43739

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-21 13:47:11 -04:00
Rick Elrod
69472a5f8d
Refactor _fixup_perms2 to remove way-nested logic (#70701)
Change:
- Refactoring to make it harder to get wrong and easier to read.
- Generalize become_unprivileged tests and fix some that never worked
  but also never failed.

Test Plan:
- CI, new units/integration tests

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-20 18:46:47 -05:00
Matt Martz
eb06488e1d
Remove preview status from TOML inventory (#70770) 2020-07-20 17:47:29 -04:00
Brian Kohles
77d0effcc5
Fix missing quoting for remote_tmp in second mkdir of shell module. Issue #69577 (#69578)
* Fix missing quoting for remote_tmp in second mkdir of shell module. Issue #69577

* adding changelog

* fixing typo in changelog entry

* adding test case

Adding test case written by bmillemayhias.

* using $HOME instead of ~

* fixing commit measage

* Update 69578-shell-remote_tmp-quoting.yaml

Co-authored-by: Brian Kohles <me@briankohles.com>
2020-07-20 11:25:29 -05:00
Abhijeet Kasurde
84b4387702
debconf: add a note about no_log usage (#70738)
debconf module exposes sensitive information to logs, console.
Add a note to user about using no_log=True to hide such
information from console.

Fixes: #32386

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-20 21:08:13 +05:30
Sloane Hertel
2811d9486f
Update default from True to False for CONDITIONAL_BARE_VARS (#70709)
ci_complete
2020-07-20 09:29:31 -05:00
Matt Martz
f90aa5599f
Don't treat empty excludes as a match. Fixes #70640 (#70710) 2020-07-17 17:34:24 -04:00
Eduard Rozenberg
566c5e6ce1
Handle Slackware OS version strings containing a plus (“+”) (#68142)
A couple of years ago Slackware -current began using a plus (“+”) at the end of the distribution version string to indicate a future version work-in-progress.

Rearrange distribution_files unit tests to easily support more tests
  - add conftest with common fixtures 
  - use parametrize for testing multiple scenarios

* Add changelog
* Add unit tests for Slackware distribution parsing
* Use correct fixtures for Slackware
Data comes from /etc/slackware-version

Co-authored-by: Sam Doran <sdoran@redhat.com>
Co-authored-by: <Eduard Rozenberg <eduardr@pobox.com>>
2020-07-17 16:39:26 -04:00
Jordan Borean
154efd97f2
win async - use full path for powershell (#70703) 2020-07-18 06:08:29 +10:00
Matt Davis
c616e54a6e
refactor Python module_utils locator (#70610)
* refactor Python module_utils locator

* no longer recursive
* embed special-case module code internally
* share common code between collections/not cases
* fixes #70134
* properly support subpackage redirection
* adds support for FQCN redirect targets used by migration (expands to FQ Python name)
* add tests

* add changelog
2020-07-16 17:57:47 -07:00
Abhijeet Kasurde
055871cbb8
api: time.clock compatible code (#70650)
time.clock is removed in Python 3.8. Add time.clock
compatible code.

Fixes: #70649

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-16 12:24:22 +05:30
Rick Elrod
4a735adc21
fix typo in changelog (#70673)
Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-15 18:45:44 -05:00
Sam Doran
9d27d7c8b1
ansible_runner test - Add constraints (#70667)
A recent updated to psutil, which is a dependency of ansible-runner, fails
to install on older versions of pip.

Commit with the breaking change:

  135628639b
2020-07-15 18:22:44 -04:00
Brian Coca
112aa574f5
dont recommend copy content in template (#70658)
* remove content ref in template

  fixes #63484
2020-07-15 13:16:56 -05:00
Julien
57b548598c
add alpine apk package manager to package_facts [wip] (#70587)
* add alpine apk package manager to package_facts
2020-07-15 12:40:25 -04:00
Mykola Grygoriev
fe86a93482
Add a new date_time fact to provide DST timezone
PR #70449
2020-07-14 18:22:51 +02:00
Jordan Borean
e22e103cdf
winrm - Added kinit_args to control the args for kinit calls (#70624) 2020-07-14 12:05:11 -04:00
Matt Martz
a77dbf0866
Allow single vault encrypted values to be used directly as module parameters. Fixes #68275 (#70607) 2020-07-14 11:56:26 -04:00
jabdr
0690b68bd3
Support datetime.date object in module result (#70595)
* Support datetime.date object in module result

Fixes #70583

* change blank lines for pep8 sanity test
2020-07-14 11:42:40 -04:00
Matt Clay
1e02a201a6 Fix ansible-test virtualenv management. 2020-07-14 01:26:06 -07:00
Matt Clay
09f02980a4 Update ansible-test change detection for plugins. 2020-07-13 18:28:59 -07:00
Sviatoslav Sydorenko
918388b85f
Fix building Ansible dist w/ setuptools>=48,<49.1 (#70525)
* Fix building Ansible dist w/ setuptools>=48,<49.1

This change addresses the deprecation of the use of stdlib
`distutils`. It's a short-term hotfix for the problem and we'll
need to consider dropping the use of `distutils` from our `setup.py`.

Refs:
* https://github.com/ansible/ansible/issues/70456
* https://github.com/pypa/setuptools/issues/2230
* https://github.com/pypa/setuptools/commit/bd110264

Co-Authored-By: Jason R. Coombs <jaraco@jaraco.com>

* Add a change note for PR #70525

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
2020-07-13 17:16:26 -04:00
Abhijeet Kasurde
e5649ca3e8
pipe: update docs for Popen with shell=True usage (#70596)
pipe lookup plugin uses Popen with shell=True intentionally.
This is considered a security issue if user input is not validated.
Updated docs to reflect this information for the user. Also, added
Bandit B602 documentation link for further reading.

Fixes: #70159

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-13 21:40:52 +05:30
Rick Elrod
d5480572c8
varnames: add tests, fix exception grammar (#70573)
Change:
- Add integration tests for various cases
- Fix wrong use of "its" in an exception thrown in varnames when it
  throws an AnsibleError, given a term of the wrong type.

Test Plan:
- new tests, CI

Tickets:
- Fixes #70546

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-10 21:04:02 -05:00
Brian Coca
cf89ca8a03
Make filter type errors 'loop friendly' (#70417)
- ensure we preserve the typeerror part of the exception so loop defereed error handling
 can postpone those caused by undefined variables until the when check is done.
 - fix tests to comply with the 'new normal'

 - human_to_bytes and others can issue TypeError not only on 'non string'
 but also bad string that is not convertable.

Co-authored-by: Sloane Hertel <shertel@redhat.com>

Co-authored-by: Sloane Hertel <shertel@redhat.com>
2020-07-10 18:49:57 -04:00
Felix Fontein
24dcaf8974
plugin loader: return collection name; ansible-doc: handle ansible.builtin correctly (#70026)
* Determine collection in plugin loader.

* Fix test.

* Use PluginPathContext objects in PluginLoader._plugin_path_cache instead of tuples.
2020-07-11 00:24:08 +02:00
Rick Elrod
1b4fd23ba6
csvfile: use parse_kv() for args, add tests (#70550)
Change:
- Use parse_kv() for parsing in the csvfile lookup plugin. This allows
  us to handle multi-word search keys and filenames. Previously, the
  plugin split on space and so none of these things worked as expected.
- Add integration tests for csvfile, testing a plethora of weird cases.

Test Plan:
- New integration tests, CI

Tickets:
- Fixes #70545

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-10 16:21:03 -05:00
Felix Fontein
f4c89eab23
ansible-doc: include collection name in text output (#70401)
* ansible-doc: include collection name in text output

* Be more careful to not accidentally pass ansible.builtin for user-supplied modules.
2020-07-10 22:52:47 +02:00
Brian Coca
8789d7968d
try to capture better winrm/put_file error (#70508)
* try to capture better winrm/put_file error

fixes #70361

* Update lib/ansible/plugins/connection/winrm.py

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
2020-07-10 15:37:52 -04:00
Amin Vakil
9c40b1b2ff
Replace filename with file in apt_key (#70492)
* Replace filename with file from apt_key check

  one is internal variable, the other is actual parameter used and required for parameter check.
2020-07-10 12:42:49 -04:00
Mykola Grygoriev
71c378e139
Fix decrypt argument in assemble module (#70465)
* Do not pass decrypt parameter to assemble module

* Add integration tests where decrypt=True

* Add changelog #70465
2020-07-09 15:24:12 -04:00
Rick Elrod
28fda23284
command warnings: don't send the param from action (#70531)
Change:
- Followup to #70504. We need to not pass the 'warn' parameter from the
  action plugin either, unless it's True. Otherwise, even though it
  defaults to false, we always show the deprecation.

Test Plan:
- Local

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-09 14:13:10 -04:00
Matt Martz
c4fd5bee00
Speedup modify module (#70475)
* speed up modify_module

* Remove debugging

* ci_complete

* Simplify generic_visit, alias to visit, eliminate some attr lookups

* ci_complete

* Add changelog fragment
2020-07-09 10:22:26 -05:00
Alan Rominger
b0d9deeae3
Fix ansible-test error in community.aws (#70507)
* Fix ansible-test error in community.aws

* Add changelog entry for fix

* Change check from None to string_types

* Update changelogs/fragments/70507-validate-null-author.yaml

clarify wording "or a list of strings"

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/schema.py

clarify wording - single string or not specified valid

Co-authored-by: Felix Fontein <felix@fontein.de>

* Do not fail but return None when given outside list

Co-authored-by: Felix Fontein <felix@fontein.de>
2020-07-08 15:17:12 -05:00
David Shrewsbury
8aca464b8b
Make sure ansible_become treated as a boolean (#70484)
* Make sure ansible_become treated as a boolean
2020-07-08 14:53:38 -04:00
Rick Elrod
f3ef4ed076
Remove some constants.py deprecated items (#70466)
Change:
- Remove mk_boolean
- Remove BECOME_METHODS
- Remove get_config

Test Plan:
- CI, removed mk_boolean unit tests

Tickets:
- Fixes #69678

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-07 18:41:19 -05:00
Rick Elrod
41414ed475
Deprecate command warnings feature (#70504)
Change:
- The command warnings feature which suggests that users use modules
  instead of certain commands is now deprecated. Its `warn` paramater
  and `COMMAND_WARNINGS` configuration options are also deprecated.
  Their use will become an error in version 2.13.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sam Doran <sdoran@redhat.com>
2020-07-07 18:40:38 -05:00
Rick Elrod
707458cc8c
Make netbsd virtualization facts more specific (#70467)
Change:
Our handling of NetBSD virtualization facts led to facts that were just
plain incorrect. One example is reporting Xen even when the system is
running on something completely different (like KVM).

As stated by the reporter of #69352, NetBSD has a better sysctl setting
to use for this information, machdep.hypervisor.

This PR does the following:

- Try to use machdep.hypervisor sysctl value if the other sysctl values
  we check don't end up with enough information to be useful
- Only look for /dev/xencons and assume Xen if nothing else works
  (Really this should probably return 'unknown' since the file exists on
  non-Xen systems and is not very useful).
- Add a few more patterns (Xen matches and also Hyper-V) to
  VirtualSysctlDetectionMixin#detect_virt_product.

This change is slightly breaking:
- If the first two attempts at using sysctl worked before,
  (machdep.dmi.system-product and machdep.dmi.system-vendor), they will
  continue to work.
- For cases when those values didn't work, previously the existence of
  /dev/xencons was checked, and if found, we reported 'xen' (even on
  non-Xen systems when the file existed). After this PR, we try the
  machdep.hypervisor sysctl key before still falling back to
  /dev/xencons. This means that in some cases, we might go from
  (wrongly) saying "xen" to giving a more accurate value such as "kvm"
  or "Hyper-V".

Test Plan:
- Tested with local NetBSD VM and got 'kvm' instead of 'xen' back.

Tickets:
- Fixes #69352

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-07 17:28:13 -05:00