Commit graph

1186 commits

Author SHA1 Message Date
Brian Coca
8fe9618c84
preserve underlying block building error (#56465)
fixes #56462
2019-05-22 11:28:06 -04:00
Marcin
afc678e2c7 Raise error in case of empty hosts list in playbook (#56354)
Fixes cases of playbook contaiing:
- hosts:
  -
ERROR! Unexpected Exception, this is probably a bug: sequence item 0: expected string, NoneType found
2019-05-21 11:11:15 -04:00
Martin Krizek
34e9d6781b Templar: encapsulate _available_variables (#55435)
Ensure variables are reset between iterations
2019-05-20 11:49:54 -04:00
Matt Martz
576593e175
Whitelist listen as a valid keyword on TaskInclude (#56586)
* Whitelist listen as a valid keyword on TaskInclude. Fixes #56580

* Move 'listen' to HandlerTaskInclude

* Remove trailing newline
2019-05-17 12:52:39 -05:00
Brian Coca
183ba93ba3
removed previouslly deprecated settings (#55662)
* removed previouslly deprecated settings

  fixes #55304

* also removed from play context
2019-05-15 16:36:30 -04:00
Brian Coca
5d4c73e197 unite poll defaults (#55470)
* unite poll defaults and update docs
2019-05-01 13:47:50 -05:00
Brian Coca
ea57377ddf correct method docs (#55961) 2019-05-01 09:13:57 -05:00
Brian Coca
815a375177
allow empty plays again, for now (#55878) 2019-04-29 17:59:10 -04:00
Xaroth
a9f24e097f Add ansible_parent_role_names magic variable (#46687)
-Add: Test cases for ansible_parent_role_names and ansible_parent_role_paths
-Add: ansible_parent_role_names/paths variables for when a role is being included by another role.
2019-04-23 11:55:05 -04:00
Brian Coca
e40832df84
become mixin is no more (#54002)
* become mixin is no more
  since sudo/su keywords are removed in 2.9 .. no need to keep this code around
 * also don't need test for code that is removed
 * made preprocess_data on base noop
   its not used by anything anymore, but kept for backwards compat since other methods of same name are used
2019-04-12 12:13:55 -04:00
Matt Martz
fbf2d5d2f4
Don't pollute include_variables (#54687)
* Don't pollute include_variables. Fixes #51667. Fixes #54618.

* Rename include_variables to include_args, so we can make the distinction about what they are

* Track args and vars separately

* oops

* oops again

* linting fix

* Add test
2019-04-09 10:14:42 -05:00
Sam Doran
6ce9cf7741
Change default smart connection to ssh on macOS and remove paramiko from requirements.txt (#54738)
* Remove default use of paramiko connection plugin on macOS
    This fix was originally to work around a bug that caused a kernel panic on macOS
    that has since been fixed.
* Remove paramiko from requirements.txt
* Move paramiko checking to common place
* Drop the warnings obfiscation code
* Update pip installation instructions to reflect upstream instructions
* Fix tests on CentOS 6 (Python 2.6) that now show Python deprecation warnings
* Add changelog fragment
2019-04-03 22:35:59 -04:00
Matt Davis
f86345f777
Collection content loading (#52194)
* basic plugin loading working (with many hacks)

* task collections working

* play/block-level collection module/action working

* implement PEP302 loader

* implicit package support (no need for __init.py__ in collections)
* provides future options for secure loading of content that shouldn't execute inside controller (eg, actively ignore __init__.py on content/module paths)
* provide hook for synthetic collection setup (eg ansible.core pseudo-collection for specifying built-in plugins without legacy path, etc)

* synthetic package support

* ansible.core.plugins mapping works, others don't

* synthetic collections working for modules/actions

* fix direct-load legacy

* change base package name to ansible_collections

* note

* collection role loading

* expand paths from installed content root vars

* feature complete?

* rename ansible.core to ansible.builtin

* and various sanity fixes

* sanity tweaks

* unittest fixes

* less grabby error handler on has_plugin

* probably need to replace with a or harden callers

* fix win_ping test

* disable module test with explicit file extension; might be able to support in some scenarios, but can't see any other tests that verify that behavior...

* fix unicode conversion issues on py2

* attempt to keep things working-ish on py2.6

* python2.6 test fun round 2

* rename dirs/configs to "collections"

* add wrapper dir for content-adjacent

* fix pythoncheck to use localhost

* unicode tweaks, native/bytes string prefixing

* rename COLLECTION_PATHS to COLLECTIONS_PATHS

* switch to pathspec

* path handling cleanup

* change expensive `all` back to or chain

* unused import cleanup

* quotes tweak

* use wrapped iter/len in Jinja proxy

* var name expansion

* comment seemingly overcomplicated playbook_paths resolution

* drop unnecessary conditional nesting

* eliminate extraneous local

* zap superfluous validation function

* use slice for rolespec NS assembly

* misc naming/unicode fixes

* collection callback loader asks if valid FQ name instead of just '.'
* switch collection role resolution behavior to be internally `text` as much as possible

* misc fixmes

* to_native in exception constructor
* (slightly) detangle tuple accumulation mess in module_utils __init__ walker

* more misc fixmes

* tighten up action dispatch, add unqualified action test

* rename Collection mixin to CollectionSearch

* (attempt to) avoid potential confusion/conflict with builtin collections, etc

* stale fixmes

* tighten up pluginloader collections determination

* sanity test fixes

* ditch regex escape

* clarify comment

* update default collections paths config entry

* use PATH format instead of list

* skip integration tests on Python 2.6

ci_complete
2019-03-28 10:41:39 -07:00
Abhijeet Kasurde
83be129923
helper: raise Exception when ds is not dict type (#53936)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-03-25 11:18:33 +05:30
René Moser
c5609c51bf
include_tasks: fix traceback if no file specified (#54044) 2019-03-20 19:17:20 +01:00
Pilou
dbb782a594 Conditional bare: fix typos (#53807) 2019-03-19 21:29:59 -04:00
Brian Coca
42e6700a71 Undeprecate force handlers (#53705)
* removed deprecation notice, made comment instead

* remove noisy deprecation

* space
2019-03-13 15:45:52 -05:00
Brian Coca
f73e03632f
remove obsolete set_play on play_context (#47934)
* remove redundant assignments from play

 - fix unit tests
2019-03-11 18:25:48 -04:00
Veaceslav Mindru
6f91bccf78 handle non strings in requirements version for ansible-galaxy (#52109) 2019-02-20 14:28:04 -05:00
Matt Martz
445ff39f94
Become plugins (#50991)
* [WIP] become plugins

Move from hardcoded method to plugins for ease of use, expansion and overrides
  - load into connection as it is going to be the main consumer
  - play_context will also use to keep backwards compat API
  - ensure shell is used to construct commands when needed
  - migrate settings remove from base config in favor of plugin specific configs
  - cleanup ansible-doc
  - add become plugin docs
  - remove deprecated sudo/su code and keywords
  - adjust become options for cli
  - set plugin options from context
  - ensure config defs are avaialbe before instance
  - refactored getting the shell plugin, fixed tests
     - changed into regex as they were string matching, which does not work with random string generation
     - explicitly set flags for play context tests
 - moved plugin loading up front
 - now loads for basedir also
 - allow pyc/o for non m modules
 - fixes to tests and some plugins
 - migrate to play objects fro play_context
 - simiplify gathering
 -  added utf8 headers
 - moved option setting
 - add fail msg to dzdo
 - use tuple for multiple options on fail/missing
 - fix relative plugin paths
 - shift from play context to play
 - all tasks already inherit this from play directly
 - remove obsolete 'set play'
 - correct environment handling
 - add wrap_exe option to pfexec
 - fix runas to noop
 - fixed setting play context
 - added password configs
 - removed required false
 - remove from doc building till they are ready

future development:
  - deal with 'enable' and 'runas' which are not 'command wrappers' but 'state flags' and currently hardcoded in diff subsystems

* cleanup

  remove callers to removed func
  removed --sudo cli doc refs
  remove runas become_exe
  ensure keyerorr on plugin
  also fix backwards compat, missing method is attributeerror, not ansible error
  get remote_user consistently
  ignore missing system_tmpdirs on plugin load
  correct config precedence
  add deprecation
  fix networking imports
  backwards compat for plugins using BECOME_METHODS

* Port become_plugins to context.CLIARGS

This is a work in progress:
* Stop passing options around everywhere as we can use context.CLIARGS
  instead

* Refactor make_become_commands as asked for by alikins

* Typo in comment fix

* Stop loading values from the cli in more than one place

Both play and play_context were saving default values from the cli
arguments directly.  This changes things so that the default values are
loaded into the play and then play_context takes them from there.

* Rename BECOME_PLUGIN_PATH to DEFAULT_BECOME_PLUGIN_PATH

As alikins said, all other plugin paths are named
DEFAULT_plugintype_PLUGIN_PATH.  If we're going to rename these, that
should be done all at one time rather than piecemeal.

* One to throw away

This is a set of hacks to get setting FieldAttribute defaults to command
line args to work.  It's not fully done yet.

After talking it over with sivel and jimi-c this should be done by
fixing FieldAttributeBase and _get_parent_attribute() calls to do the
right thing when there is a non-None default.

What we want to be able to do ideally is something like this:

class Base(FieldAttributeBase):
    _check_mode = FieldAttribute([..] default=lambda: context.CLIARGS['check'])

class Play(Base):
    # lambda so that we have a chance to parse the command line args
    # before we get here.  In the future we might be able to restructure
    # this so that the cli parsing code runs before these classes are
    # defined.

class Task(Base):
    pass

And still have a playbook like this function:

---
- hosts:
  tasks:
  - command: whoami
    check_mode: True

(The check_mode test that is added as a separate commit in this PR will
let you test variations on this case).

There's a few separate reasons that the code doesn't let us do this or
a non-ugly workaround for this as written right now.  The fix that
jimi-c, sivel, and I talked about may let us do this or it may still
require a workaround (but less ugly) (having one class that has the
FieldAttributes with default values and one class that inherits from
that but just overrides the FieldAttributes which now have defaults)

* Revert "One to throw away"

This reverts commit 23aa883cbed11429ef1be2a2d0ed18f83a3b8064.

* Set FieldAttr defaults directly from CLIARGS

* Remove dead code

* Move timeout directly to PlayContext, it's never needed on Play

* just for backwards compat, add a static version of BECOME_METHODS to constants

* Make the become attr on the connection public, since it's used outside of the connection

* Logic fix

* Nuke connection testing if it supports specific become methods

* Remove unused vars

* Address rebase issues

* Fix path encoding issue

* Remove unused import

* Various cleanups

* Restore network_cli check in _low_level_execute_command

* type improvements for cliargs_deferred_get and swap shallowcopy to default to False

* minor cleanups

* Allow the su plugin to work, since it doesn't define a prompt the same way

* Fix up ksu become plugin

* Only set prompt if build_become_command was called

* Add helper to assist connection plugins in knowing they need to wait for a prompt

* Fix tests and code expectations

* Doc updates

* Various additional minor cleanups

* Make doas functional

* Don't change connection signature, load become plugin from TaskExecutor

* Remove unused imports

* Add comment about setting the become plugin on the playcontext

* Fix up tests for recent changes

* Support 'Password:' natively for the doas plugin

* Make default prompts raw

* wording cleanups. ci_complete

* Remove unrelated changes

* Address spelling mistake

* Restore removed test, and udpate to use new functionality

* Add changelog fragment

* Don't hard fail in set_attributes_from_cli on missing CLI keys

* Remove unrelated change to loader

* Remove internal deprecated FieldAttributes now

* Emit deprecation warnings now
2019-02-11 11:27:44 -06:00
Brian Coca
4a0fceaa3b remove bare var handling in conditionals (#51030)
* remove bare var handling in conditionals

  this makes top level and multilevel vars (dicts keys) behave the same
  it will require adding |bool for 'string comparissons' in indirect templates

  - added new tests to ensure uniform handling
  - switched to 'is' testing for status
  - changed warning to 'conditional' as 'when:' is not only place it gets triggered

* updated to include toggle and deprecation

* fix deprecated

* updated tests to handle toggle

* fixed typo and added note about the future
2019-01-30 15:00:24 -05:00
Brian Coca
be776daefe
add static facility and apply to register (#49737)
* add static facility and apply to register

* added warning

* added test for templated register

* test register 'static' status

* rely on subshell to deal with quote context

* use corrects pb for test

* bring constants back cause new code in devel
2019-01-24 11:51:52 -05:00
Matt Martz
8c08d03989
Fieldattribute inheritance with defaults (#50891)
* Add tests for check_mode at play and task level

These test inheritance of check_mode from the various levels (command
line, as a play attribute and as a task attribute) so they will be
useful for checking that the change to fieldattribute inheritance with
defaults works

* Add a sentinel object

The Sentinel object can be used in place of None when we need to mark an
entry as being special (usually used to mark something as not having
been set)

* Start of using a Sentinel object instead of None.

* Handle edge cases around use of Sentinel

* _get_parent_attribute needs to deal in Sentinel not None

* No need to special case any_errors_fatal in task.py any longer

* Handle more edge cases around Sentinel

* Use Sentinel instead of None in TaskInclude

* Update code to clarify the vars we are copying are class attrs

* Add changelog fragment

* Use a default of Sentinel for delegate_to, this also allows 'delegate_to: ~' now to unset inherited delegate_to

* Explain Sentinel stripping in _extend_value

* Fix ModuleArgsParser tests to compare with Sentinel

* Fixes for tasks inside of roles inheriting from play

* Remove incorrect note. ci_complete

* Remove commented code
2019-01-23 11:40:07 -06:00
Brian Coca
30da7a363c
allows loop pause for subseconds (#50461)
* allows loop pause for subseconds
* skip test on os x and bsd since date commands lacks nano

fixes #49951
2019-01-18 22:41:29 -05:00
Brian Coca
eca7c3c8c7 Prevent duplicate role insertion into roles: (#50552)
* Corner case in which import_role would add another instance of a role with the same signature into roles: when it already existed there.

  roles:
	- name: a
  tasks:
       - import_role: name=a

  would execute role 'a' 3 times instead of the intended 2 (x2 in roles: phase +1 in tasks:)

* added tests
2019-01-16 13:39:03 -05:00
Abhijeet Kasurde
4817dcd0fc Minor typos (#50371)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-01-15 11:37:00 +00:00
Matt Clay
465df0ef8d
Fix encoding issues with file paths. (#50830)
* Fix encoding issues with file paths.

Discovered while testing with ANSIBLE_CONFIG env var set to a path
that contained unicode characters while LC_ALL=C.

* Fix unit tests.

* Fix another path encoding issue.
2019-01-14 13:06:47 -08:00
Toshio Kuratomi
afdbb0d9d5 Save the command line arguments into a global context
* Once cli args are parsed, they're constant.  So, save the parsed args
  into the global context for everyone else to use them from now on.
* Port cli scripts to use the CLIARGS in the context
* Refactor call to parse cli args into the run() method
* Fix unittests for changes to the internals of CLI arg parsing
* Port callback plugins to use context.CLIARGS
  * Got rid of the private self._options attribute
  * Use context.CLIARGS in the individual callback plugins instead.
  * Also output positional arguments in default and unixy plugins
  * Code has been simplified since we're now dealing with a dict rather
    than Optparse.Value
2019-01-03 18:12:23 -08:00
Matt Martz
3b49bbcfde Give IncludedFile more context via ansible_search_path (#50045)
* Give IncludedFile more context via ansible_search_path to template lookups. Fixes #49969

* Update units
2019-01-03 16:13:02 -05:00
Brian Coca
88c87a3583
added handlers_from (#49220)
* added handlers_from

fixes #46769
2018-12-18 11:28:24 -05:00
Matt Martz
7eb1ab45a7
Allow tags to be templated from a variable (#49833)
* Allow tags to be templated from a variable. Fixes #49825

* Restore _load_tags to ensure we do csv tag splitting

* Add tests for csv tags and templated tags

* evaluate_tags doesn't need to accept strings, because _load_tags handles this
2018-12-17 15:40:26 -06:00
Matt Martz
2a469fd959 Consolidate handler tracking (#49338)
* Consolidate handler tracking
  -  Remove unused code. ci_complete
  - unit test fixes. ci_complete
  - Restore previous behavior of matching a single handler
  - when notifying a host for a handler, return True if it was added, False otherwise, to reduce copied logic
  - rename funcitons for clarity. ci_complete
  - Remove handler logic for static includes which was disabled previously
2018-12-11 10:58:19 -05:00
Matt Martz
9007dbec2f
Add info about loop based on jinja2 loop var (#42134)
* Add info about loop based on jinja2 loop var

* ansible_loop

* Update test count

* Add extended loop_control that defines whether ansible_loop should be added

* Extended needs to be defaulted

* Revert "Update test count"

This reverts commit f1e93ee469825f4cdcd90fb28667d29aa088275c.

* Add docs about loop_control.extended

* Add revindex and revindex0

* Document ansible_loop in special vars

* Add changelog fragment

* Add tests, change items to allitems so that dot notation works, fix logic error with previtem
2018-12-07 13:49:50 -06:00
James Cassell
2bd6b1415b become_method: make dzdo more like sudo (#47946)
dzdo is basically a drop-in replacement for sudo and supports the same
command line options.

There is no become_flags set for dzdo like there is for sudo, so users
will have to set that separately to have exactly the same functionality.
2018-12-04 18:24:32 -05:00
Adam Miller
81540bd754 require git when trying to use it in ansible-galaxy cli (#49212)
* require git when trying to use it in ansible-galaxy cli

Previously we weren't setting `required=True` when calling
`get_bin_path` and the path would return `None`, this would cause
a traceback when attempting to ' '.join() to create a string
representation of the failed command for error output

Fixes #49200

Signed-off-by: Adam Miller <admiller@redhat.com>

* add changelog

Signed-off-by: Adam Miller <admiller@redhat.com>
2018-11-28 13:28:21 -05:00
Martin Krizek
48ffd8789f
Properly handle FieldAttribute.default if callable (#48992)
* Properly handle FieldAttribute.default if callable

Fixes #48673

* Add changelog...

* Add integration test

* Add aliases file
2018-11-26 18:14:46 +01:00
Matt Martz
9773a1f289
Add a Singleton metaclass, use it with Display (#48935)
* Add a Singleton class, use it with Display

* update six import

* Move remaining failes to display singleton

* Fix rebase issues

* Singleton improvements

* Add code-smell for 'from __main__ import display'. ci_complete

* s/self/cls/g

* Add docs for no-main-display

* Address linting issues

* Add changelog fragment. ci_complete

* Implement reentrant lock for class instantiation in Singleton

* Add Display singleton porting guide
2018-11-20 17:06:51 -06:00
Martin Krizek
a80c25cbd9
Do not use str() on exceptions (#46950) 2018-11-09 07:59:30 +01:00
Jimmy McCrory
d2969884b4 Add run_once as a valid TaskInclude keyword (#48068)
* Add run_once as a valid TaskInclude keyword

* Add changelog fragment

* Add integration test that documents run_once behavior
2018-11-08 10:29:24 -05:00
Martin Krizek
421d67f1ee
conditional: fix direct boolean "shortcut" (#47941)
* conditional: fix direct boolean "shortcut"

* Add unit tests
2018-11-07 21:33:54 +01:00
Matt Martz
d5e4f37ca0
Ensure we don't overwrite roles from include/import_role when loading the play (#47512)
* Ensure we don't overwrite roles from include/import_role when loading the play. Fixes #47454

* Add changelog fragment
2018-10-23 11:08:48 -05:00
Matt Martz
a06a5ded61
Do not use mutable defaults in FieldAttribute, instead allow supplying a callable for defaults of mutable types. Fixes #46824 (#46833) 2018-10-12 10:43:09 -05:00
Brian Coca
8743e6ae2e remove default from delegate_facts to inherit (#45492)
* remove default from delegate_facts to inherit

fixes #45456

* test delegate_facts

* added note about inheritance and defaults

* yamllint
2018-10-10 15:10:23 -04:00
Matt Clay
a11f631ee4 Python 3.8 collections compatibility fixes.
Includes a new pylint blacklist plugin to prevent regressions.
2018-10-08 11:26:37 -07:00
Brian Coca
2d01966e87 better error on empty playbook
fixes #45480
2018-09-28 15:21:57 -04:00
Matt Martz
698c8059e9 Fail on attrs that don't work for includes (#46177)
* Fail on attrs that don't work for includes

* Prefix with VALID_ to make it more clear

* Sort keywords

* Remove unnecessary list

* Adjust dynamic/static checking
2018-09-27 10:06:30 -04:00
Matt Martz
0015d4cef3
2.8 Core Deprecation Removal (#45232)
* Remove deprecated ansible.vars.unsafe_proxy. Fixes #45040

* Remove deprecated validate_md5 alias from fetch module. Fixes #45039

* Remove deprecated private arg from import/include_role. Fixes #45038

* All include deprecations bumped to 2.12. Fixes #45037

* Add changelog for deprecated removals
2018-09-10 15:05:52 -05:00
Matt Martz
a0d7d4b82f Change insertion order of apply block to not affect the include_X task itself (#44912) 2018-08-30 16:56:28 -04:00
Matt Martz
d5ed818e7c
Make sure we skip handlers from includes in compile_roles_handlers. Fixes #44848 (#44852) 2018-08-29 15:05:55 -05:00
Matt Martz
cd2f66f731
Don't allow import_tasks to transition to dynamic when file is missing. See #44822 (#44836) 2018-08-29 11:43:26 -05:00