Commit graph

13 commits

Author SHA1 Message Date
Sam Doran
6f76a48f59
Make sorting in collection_loader match plugin loader (#65776)
* Simply sorting of Windows files below other plugin types
    Using the sort method with a custom key function uses less memory than creating multiple lists then joining them.

    This seemed to be an acceptable use of a lamdba, even though I geneally try to avoid them.

* Fix sorting of plugins inside of collections
    Explicitly sort Windows files below others, mimicking what we do in plugin/loader.py

* Add documentation about ansible.builtin and ansible.legacy
    Also document to the two different methods used for searching based on the candidate type.

* Add changelog
* Add integration test
* Update comment with expected sort order
2019-12-16 11:28:24 -05:00
Sloane Hertel
c1f1b2029c
Support vars plugins in collections (#61078)
* Move var plugins handling to a separate file

* Allow var plugins to require whitelisting

* Add global configuration ('demand', 'start') for users to control when they execute

* Add 'stage' configuration ('all', 'task', 'inventory') for users to control on a per-plugin basis when they execute

* Update ansible-inventory and InventoryManager to the global and stage configuration

* Update host_group_vars to use stage configuration and whitelisting

* Add documentation for using new options and to the developer's guide

* Add integration tests to exercise whitelisting and the new configuration options, using vars plugins in collections, and maintain backward compatibility

* Changelog

Co-Authored-By: Brian Coca <brian.coca+git@gmail.com>
Co-Authored-By: Sandra McCann <samccann@redhat.com>
2019-11-04 11:41:14 -05:00
Matt Davis
7f4328ad12
allow external collections under the ansible NS (#61908)
* fixes #59988
2019-09-05 18:50:22 -07:00
Matt Davis
d81ae27a4a
Collection role relative deps (#61517)
* default collection support

* playbooks run from inside a registered collection will set that collection as the first item in the search order (as will all non-collection roles)
* this allows easy migration of runme.sh style playbook/role integration tests to collections without the playbooks/roles needing to know the name of their enclosing collection

* disable default collection test under Windows

* enable collection search for role dependencies

* unqualified role deps in collection-hosted roles will first search the containing collection
* if the calling role has specified a collections search list in metadata, it will be appended to the search order for unqualified role deps

* disable cycle detection unit test

* failing on 3.7+, needs proper cycle detection
* see #61527
2019-08-29 02:25:44 -07:00
Matt Davis
7d1a981b61
default collection support (#61415)
* default collection support

* playbooks run from inside a registered collection will set that collection as the first item in the search order (as will all non-collection roles)
* this allows easy migration of runme.sh style playbook/role integration tests to collections without the playbooks/roles needing to know the name of their enclosing collection

* ignore bogus sanity error

* filed #61460

* fixed task unit test failure

* don't append an empty collections list to the ds

* ignore leftover local_action in mod_args ds action parsing

* fix async_extra_data test to not require ssh and bogus locale

* disable default collection test under Windows

* ensure collection location FS code is always bytes

* add changelog
2019-08-28 16:31:40 -07:00
Jordan Borean
66f52b74b1 Change collection PS util import pattern (#61307)
* Change collection PS util import pattern

* Add changes for py2 compat

* fix up regex and doc errors

* fix up import analysis

* Sanity fix for 2.6 CI workers

* Get collection util path for coverage collection
2019-08-27 16:02:27 -07:00
Matt Davis
bfa004930a
add subdir support to collection loading (#60682)
* add subdir support to collection loading

* collections may now load plugins from subdirs under a plugin type or roles dir, eg `ns.coll.subdir1.subdir2.myrole`->ns.coll's roles/subdir1/subdir2/myrole, `ns.coll.subdir1.mymodule`->ns.coll's plugins/modules/subdir1/mymodule.py
* centralize parsing/validation in AnsibleCollectionRef class
* fix issues loading Jinja2 plugins from multiple sources 
* resolves #59462, #59890,

* sanity test fixes

* string fixes

* add changelog entry
2019-08-21 10:45:04 +01:00
Matt Clay
60f8483603
Add missing boilerplate to collections test. (#60507)
* Add missing boilerplate to collections test.
* Remove redundant Python version check.
2019-08-13 12:20:14 -07:00
Matt Clay
d4dec59cb0 Fix collection loader builtin handling. (#58897)
- Support more import statements:

  from ansible_collections.ansible.builtin.plugins.module_utils import basic
  from ansible_collections.ansible.builtin.plugins.module_utils.basic import AnsibleModule

- Add unit tests for more import statements.
- Raise ImportError instead of returning None if load_module fails.
2019-07-16 13:40:22 -07:00
Matt Clay
d5542afd97 Properly enforce shebang on collections. 2019-06-25 19:21:05 -07:00
flowerysong
2ef8b297ff Fix loading namespaced doc_fragments from collections (#55249)
* Fix loading namespaced doc_fragments

The syntax for specifying a different fragment name was already
using '.' as a separator, so the code needed to be tweaked to
avoid choking on names like `testns.testcoll.fragname` and
`testns.testcoll.fragname.altvar`.

`get_plugin_class()` returns 'docfragment' for the fragment loader;
mangling `subdir` provides consistent alignment with the normal plugin
directory names and avoids needing special handling of plugin types
with 'module' in the name.

* Add changelog entry
2019-05-02 21:15:57 -04:00
Matt Davis
1dc8436ed9
module_utils fixes in collections (#55118)
* module_utils fixes in collections

* fixed Windows module_utils in collections
* fixed more Python module_utils cases (from X import module)
* "medium style" Ansiballz modules now work properly with collections (ie, non-replacer but also not using basic.py)
* added more tests
* split Windows/POSIX exec

* sanity
2019-04-10 22:59:53 -07:00
Matt Davis
f86345f777
Collection content loading (#52194)
* basic plugin loading working (with many hacks)

* task collections working

* play/block-level collection module/action working

* implement PEP302 loader

* implicit package support (no need for __init.py__ in collections)
* provides future options for secure loading of content that shouldn't execute inside controller (eg, actively ignore __init__.py on content/module paths)
* provide hook for synthetic collection setup (eg ansible.core pseudo-collection for specifying built-in plugins without legacy path, etc)

* synthetic package support

* ansible.core.plugins mapping works, others don't

* synthetic collections working for modules/actions

* fix direct-load legacy

* change base package name to ansible_collections

* note

* collection role loading

* expand paths from installed content root vars

* feature complete?

* rename ansible.core to ansible.builtin

* and various sanity fixes

* sanity tweaks

* unittest fixes

* less grabby error handler on has_plugin

* probably need to replace with a or harden callers

* fix win_ping test

* disable module test with explicit file extension; might be able to support in some scenarios, but can't see any other tests that verify that behavior...

* fix unicode conversion issues on py2

* attempt to keep things working-ish on py2.6

* python2.6 test fun round 2

* rename dirs/configs to "collections"

* add wrapper dir for content-adjacent

* fix pythoncheck to use localhost

* unicode tweaks, native/bytes string prefixing

* rename COLLECTION_PATHS to COLLECTIONS_PATHS

* switch to pathspec

* path handling cleanup

* change expensive `all` back to or chain

* unused import cleanup

* quotes tweak

* use wrapped iter/len in Jinja proxy

* var name expansion

* comment seemingly overcomplicated playbook_paths resolution

* drop unnecessary conditional nesting

* eliminate extraneous local

* zap superfluous validation function

* use slice for rolespec NS assembly

* misc naming/unicode fixes

* collection callback loader asks if valid FQ name instead of just '.'
* switch collection role resolution behavior to be internally `text` as much as possible

* misc fixmes

* to_native in exception constructor
* (slightly) detangle tuple accumulation mess in module_utils __init__ walker

* more misc fixmes

* tighten up action dispatch, add unqualified action test

* rename Collection mixin to CollectionSearch

* (attempt to) avoid potential confusion/conflict with builtin collections, etc

* stale fixmes

* tighten up pluginloader collections determination

* sanity test fixes

* ditch regex escape

* clarify comment

* update default collections paths config entry

* use PATH format instead of list

* skip integration tests on Python 2.6

ci_complete
2019-03-28 10:41:39 -07:00