Doc fragments to plugins (#50172)

* promote doc_fragments into actual plugins

  change tests hardcoded path to doc fragments
  avoid sanity in fragments
  avoid improper testing of doc_fragments
  also change runner paths
 fix botmeta
 updated comment for fragments
 updated docs

.github/BOTMETA.yml

@@ -1025,6 +1025,28 @@ files:
     labels:
       - windows
 ###############################
+# plugins/doc_fragments
+  $plugins/doc_fragments/:
+    support: community
+  $plugins/doc_fragments/__init__.py:
+    support: core
+  $plugins/doc_fragments/aci.py: *aci
+  $plugins/doc_fragments/acme.py:
+    maintainers: resmo felixfontein
+  $plugins/doc_fragments/cloudstack.py:
+    maintainers: $team_cloudstack
+    labels: cloudstack
+  $plugins/doc_fragments/docker.py:
+    support: community
+    maintainers: $team_docker
+    supershipit: felixfontein
+  $plugins/doc_fragments/mso.py: *aci
+  $plugins/doc_fragments/vultr.py:
+    maintainers: $team_vultr
+    labels: cloud
+  $plugins/doc_fragments/xenserver.py:
+    maintainers: bvitnik
+###############################
 # plugins/filter
   $plugins/filter/:
     support: community
@@ -1211,24 +1233,6 @@ files:
     keywords:
       - jinja
       - jinja2
-  lib/ansible/utils/module_docs_fragments/:
-    support: community
-  lib/ansible/utils/module_docs_fragments/aci.py: *aci
-  lib/ansible/utils/module_docs_fragments/acme.py:
-    maintainers: resmo felixfontein
-  lib/ansible/utils/module_docs_fragments/docker.py:
-    support: community
-    maintainers: $team_docker
-    supershipit: felixfontein
-  lib/ansible/utils/module_docs_fragments/mso.py: *aci
-  lib/ansible/utils/module_docs_fragments/xenserver.py:
-    maintainers: bvitnik
-  lib/ansible/utils/module_docs_fragments/cloudstack.py:
-    maintainers: $team_cloudstack
-    labels: cloudstack
-  lib/ansible/utils/module_docs_fragments/vultr.py:
-    maintainers: $team_vultr
-    labels: cloud
   test/sanity/validate-modules:
     notified:
       - mattclay

docs/docsite/rst/dev_guide/developing_modules_documenting.rst

@@ -272,7 +272,14 @@ You can link from your module documentation to other module docs, other resource
 Documentation fragments
 -----------------------
 
-If you're writing multiple related modules, they may share common documentation, such as authentication details or file mode settings. Rather than duplicate that information in each module's ``DOCUMENTATION`` block, you can save it once as a fragment and use it in each module's documentation. Shared documentation fragments are contained in a ``ModuleDocFragment`` class in `lib/ansible/utils/module_docs_fragments/ <https://github.com/ansible/ansible/tree/devel/lib/ansible/utils/module_docs_fragments>`_. To include a documentation fragment, add ``extends_documentation_fragment: FRAGMENT_NAME`` in your module's documentation.
+If you're writing multiple related modules, they may share common documentation, such as authentication details or file mode settings. Rather than duplicate that information in each module's ``DOCUMENTATION`` block, you can save it once as a doc_fragment plugin and use it in each module's documentation. In Ansible, shared documentation fragments are contained in a ``ModuleDocFragment`` class in `lib/ansible/plugins/doc_fragments/ <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/doc_fragments>`_. To include a documentation fragment, add ``extends_documentation_fragment: FRAGMENT_NAME`` in your module's documentation.
+
+.. _note:
+  * in 2.8 the Ansible directories for doc fragments changed, see documentation of previous versions to find the old locations.
+
+.. versionadded:: 2.8
+
+Since version 2.8, you can have user supplied doc_fragments by using a ``doc_fragments`` directory adjacent to play or role, just like any other plugin.
 
 For example, all AWS modules should include::
 

docs/docsite/rst/dev_guide/developing_modules_in_groups.rst

@@ -28,7 +28,7 @@ Although it's tempting to get straight into coding, there are a few things to be
 * Starting with Ansible version 2.7, all new modules must :ref:`support Python 2.7+ and Python 3.5+ <developing_python_3>`. If this is an issue, please contact us (see the "Speak to us" section later in this document to learn how).
 * Have a look at the existing modules and how they've been named in the :ref:`all_modules`, especially in the same functional area (such as cloud, networking, databases).
 * Shared code can be placed into ``lib/ansible/module_utils/``
-* Shared documentation (for example describing common arguments) can be placed in ``lib/ansible/utils/module_docs_fragments/``.
+* Shared documentation (for example describing common arguments) can be placed in ``lib/ansible/plugins/doc_fragments/``.
 * With great power comes great responsibility: Ansible module maintainers have a duty to help keep modules up to date. As with all successful community projects, module maintainers should keep a watchful eye for reported issues and contributions.
 * Although not required, unit and/or integration tests are strongly recommended. Unit tests are especially valuable when external resources (such as cloud or network devices) are required. For more information see :doc:`testing` and the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
   * Starting with Ansible 2.4 all :ref:`network_modules` MUST have unit tests.
@@ -81,7 +81,7 @@ The first PR is slightly different to the rest because it:
 
 * defines the namespace
 * provides a basis for detailed review that will help shape your future PRs
-* may include shared documentation (`docs_fragments`) that multiple modules require
+* may include shared documentation (`doc_fragments`) that multiple modules require
 * may include shared code (`module_utils`) that multiple modules require
 
 
@@ -89,7 +89,7 @@ The first PR should include the following files:
 
 * ``lib/ansible/modules/$category/$topic/__init__.py`` - An empty file to initialize namespace and allow Python to import the files. *Required new file*
 * ``lib/ansible/modules/$category/$topic/$yourfirstmodule.py`` - A single module. *Required new file*
-* ``lib/ansible/utils/module_docs_fragments/$topic.py`` - Code documentation, such as details regarding common arguments. *Optional new file*
+* ``lib/ansible/plugins/doc_fragments/$topic.py`` - Code documentation, such as details regarding common arguments. *Optional new file*
 * ``lib/ansible/module_utils/$topic.py`` - Code shared between more than one module, such as common arguments. *Optional new file*
 *  ``docs/docsite/rst/dev_guide/developing_module_utilities.rst`` - Document your new `module_utils` file. *Optional update to existing file*
 

lib/ansible/config/base.yml

@@ -352,6 +352,14 @@ LOCALHOST_WARNING:
   - {key: localhost_warning, section: defaults}
   type: boolean
   version_added: "2.6"
+DOC_FRAGMENT_PLUGIN_PATH:
+  name: documentation fragment plugins path
+  default: ~/.ansible/plugins/doc_fragments:/usr/share/ansible/plugins/doc_fragments
+  description: Colon separated paths in which Ansible will search for Documentation Fragments Plugins.
+  env: [{name: ANSIBLE_DOC_FRAGMENT_PLUGINS}]
+  ini:
+  - {key: doc_fragment_plugins, section: defaults}
+  type: pathspec
 DEFAULT_ACTION_PLUGIN_PATH:
   name: Action plugins path
   default: ~/.ansible/plugins/action:/usr/share/ansible/plugins/action

lib/ansible/plugins/loader.py

@@ -633,12 +633,11 @@ _PLUGIN_FILTERS = _load_plugin_filter()
 # doc fragments first
 fragment_loader = PluginLoader(
     'ModuleDocFragment',
-    'ansible.utils.module_docs_fragments',
+    'ansible.plugins.doc_fragments',
-    os.path.join(os.path.dirname(__file__), 'module_docs_fragments'),
+    C.DOC_FRAGMENT_PLUGIN_PATH,
-    '',
+    'doc_fragments',
 )
 
-
 action_loader = PluginLoader(
     'ActionModule',
     'ansible.plugins.action',