cmueller/ansible
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Developer documentation update involving module invocation (#55747)

Browse source 
* Update docs for the 2.7 change to AnsiballZ which invokes modules with one
  less Python interpreter

* Add a section on how module results are returned and on trust between modules, action plugins, and the executor.

* Update docs/docsite/rst/dev_guide/developing_program_flow_modules.rst

Co-Authored-By: abadger <a.badger@gmail.com>
This commit is contained in:
Toshio Kuratomi 2019-05-01 18:52:23 -05:00 committed by Alicia Cozine
parent c455635500
commit edafa71f42
1 changed files with 180 additions and 181 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

361
docs/docsite/rst/dev_guide/developing_program_flow_modules.rst
View file

@ -5,13 +5,9 @@
Ansible module architecture Ansible module architecture
*************************** ***************************
This in-depth dive helps you understand Ansible's program flow to execute If you're working on Ansible's Core code, writing an Ansible module, or developing an action plugin, this deep dive helps you understand how Ansible's program flow executes. If you're just using Ansible Modules in playbooks, you can skip this section.
modules. It is written for people working on the portions of the Core Ansible
Engine that execute a module. Those writing Ansible Modules may also find this
in-depth dive to be of interest, but individuals simply using Ansible Modules
will not likely find this to be helpful.
.. contents:: Topics .. contents::
:local: :local:
.. _flow_types_of_modules: .. _flow_types_of_modules:
@ -19,7 +15,7 @@ will not likely find this to be helpful.
Types of modules Types of modules
================ ================
Ansible supports several different types of modules in its code base. Some of Ansible supports several different types of modules in its code base. Some of
these are for backwards compatibility and others are to enable flexibility. these are for backwards compatibility and others are to enable flexibility.
.. _flow_action_plugins: .. _flow_action_plugins:
@ -27,20 +23,18 @@ these are for backwards compatibility and others are to enable flexibility.
Action plugins Action plugins
-------------- --------------
Action Plugins look like modules to end users who are writing :term:`playbooks` but Action plugins look like modules to anyone writing a playbook. Usage documentation for most action plugins lives inside a module of the same name. Some action plugins do all the work, with the module providing only documentation. Some action plugins execute modules. The ``normal`` action plugin executes modules that don't have special action plugins. Action plugins always execute on the controller.
they're distinct entities for the purposes of this document. Action Plugins
always execute on the controller and are sometimes able to do all work there
(for instance, the ``debug`` Action Plugin which prints some text for the user to
see or the ``assert`` Action Plugin which can test whether several values in
a playbook satisfy certain criteria.)
More often, Action Plugins set up some values on the controller, then invoke an Some action plugins do all their work on the controller. For
actual module on the managed node that does something with these values. An example, the :ref:`debug <debug_module>` action plugin (which prints text for
easy to understand version of this is the :ref:`template Action Plugin the user to see) and the :ref:`assert <assert_module>` action plugin (which
<template_module>`. The :ref:`template Action Plugin <template_module>` takes values from tests whether values in a playbook satisfy certain criteria) execute entirely on the controller.
Most action plugins set up some values on the controller, then invoke an
actual module on the managed node that does something with these values. For example, the :ref:`template <template_module>` action plugin takes values from
the user to construct a file in a temporary location on the controller using the user to construct a file in a temporary location on the controller using
variables from the playbook environment. It then transfers the temporary file variables from the playbook environment. It then transfers the temporary file
to a temporary file on the remote system. After that, it invokes the to a temporary file on the remote system. After that, it invokes the
:ref:`copy module <copy_module>` which operates on the remote system to move the file :ref:`copy module <copy_module>` which operates on the remote system to move the file
into its final location, sets file permissions, and so on. into its final location, sets file permissions, and so on.
@ -49,23 +43,20 @@ into its final location, sets file permissions, and so on.
New-style modules New-style modules
----------------- -----------------
All of the modules that ship with Ansible fall into this category. All of the modules that ship with Ansible fall into this category. While you can write modules in any language, all official modules (shipped with Ansible) use either Python or PowerShell.
New-style modules have the arguments to the module embedded inside of them in New-style modules have the arguments to the module embedded inside of them in
some manner. Non-new-style modules must copy a separate file over to the some manner. Old-style modules must copy a separate file over to the
managed node, which is less efficient as it requires two over-the-wire managed node, which is less efficient as it requires two over-the-wire
connections instead of only one. connections instead of only one.
.. _flow_python_modules: .. _flow_python_modules:
Python Python
------ ^^^^^^
New-style Python modules use the :ref:`Ansiballz` framework for constructing New-style Python modules use the :ref:`Ansiballz` framework for constructing
modules. All official modules (shipped with Ansible) use either this or the modules. These modules use imports from :code:`ansible.module_utils` to pull in
:ref:`powershell module framework <flow_powershell_modules>`.
These modules use imports from :code:`ansible.module_utils` in order to pull in
boilerplate module code, such as argument parsing, formatting of return boilerplate module code, such as argument parsing, formatting of return
values as :term:`JSON`, and various file operations. values as :term:`JSON`, and various file operations.
@ -76,21 +67,21 @@ values as :term:`JSON`, and various file operations.
.. _flow_powershell_modules: .. _flow_powershell_modules:
Powershell PowerShell
---------- ^^^^^^^^^^
New-style powershell modules use the :ref:`module_replacer` framework for New-style PowerShell modules use the :ref:`module_replacer` framework for
constructing modules. These modules get a library of powershell code embedded constructing modules. These modules get a library of PowerShell code embedded
in them before being sent to the managed node. in them before being sent to the managed node.
.. _flow_jsonargs_modules: .. _flow_jsonargs_modules:
JSONARGS JSONARGS modules
-------- ----------------
Scripts can arrange for an argument string to be placed within them by placing These modules are scripts that include the string
the string ``<<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>`` somewhere inside of the ``<<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>`` in their body.
file. The module typically sets a variable to that value like this: This string is replaced with the JSON-formatted argument string. These modules typically set a variable to that value like this:
.. code-block:: python .. code-block:: python
@ -102,20 +93,20 @@ Which is expanded as:
json_arguments = """{"param1": "test's quotes", "param2": "\"To be or not to be\" - Hamlet"}""" json_arguments = """{"param1": "test's quotes", "param2": "\"To be or not to be\" - Hamlet"}"""
.. note:: Ansible outputs a :term:`JSON` string with bare quotes. Double quotes are .. note:: Ansible outputs a :term:`JSON` string with bare quotes. Double quotes are
used to quote string values, double quotes inside of string values are used to quote string values, double quotes inside of string values are
backslash escaped, and single quotes may appear unescaped inside of backslash escaped, and single quotes may appear unescaped inside of
a string value. To use JSONARGS, your scripting language must have a way a string value. To use JSONARGS, your scripting language must have a way
to handle this type of string. The example uses Python's triple quoted to handle this type of string. The example uses Python's triple quoted
strings to do this. Other scripting languages may have a similar quote strings to do this. Other scripting languages may have a similar quote
character that won't be confused by any quotes in the JSON or it may character that won't be confused by any quotes in the JSON or it may
allow you to define your own start-of-quote and end-of-quote characters. allow you to define your own start-of-quote and end-of-quote characters.
If the language doesn't give you any of these then you'll need to write If the language doesn't give you any of these then you'll need to write
a :ref:`non-native JSON module <flow_want_json_modules>` or a :ref:`non-native JSON module <flow_want_json_modules>` or
:ref:`Old-style module <flow_old_style_modules>` instead. :ref:`Old-style module <flow_old_style_modules>` instead.
The module typically parses the contents of ``json_arguments`` using a JSON These modules typically parse the contents of ``json_arguments`` using a JSON
library and then use them as native variables throughout the rest of its code. library and then use them as native variables throughout the code.
.. _flow_want_json_modules: .. _flow_want_json_modules:
@ -124,12 +115,12 @@ Non-native want JSON modules
If a module has the string ``WANT_JSON`` in it anywhere, Ansible treats If a module has the string ``WANT_JSON`` in it anywhere, Ansible treats
it as a non-native module that accepts a filename as its only command line it as a non-native module that accepts a filename as its only command line
parameter. The filename is for a temporary file containing a :term:`JSON` parameter. The filename is for a temporary file containing a :term:`JSON`
string containing the module's parameters. The module needs to open the file, string containing the module's parameters. The module needs to open the file,
read and parse the parameters, operate on the data, and print its return data read and parse the parameters, operate on the data, and print its return data
as a JSON encoded dictionary to stdout before exiting. as a JSON encoded dictionary to stdout before exiting.
These types of modules are self-contained entities. As of Ansible 2.1, Ansible These types of modules are self-contained entities. As of Ansible 2.1, Ansible
only modifies them to change a shebang line if present. only modifies them to change a shebang line if present.
.. seealso:: Examples of Non-native modules written in ruby are in the `Ansible .. seealso:: Examples of Non-native modules written in ruby are in the `Ansible
@ -140,14 +131,14 @@ only modifies them to change a shebang line if present.
Binary modules Binary modules
-------------- --------------
From Ansible 2.2 onwards, modules may also be small binary programs. Ansible From Ansible 2.2 onwards, modules may also be small binary programs. Ansible
doesn't perform any magic to make these portable to different systems so they doesn't perform any magic to make these portable to different systems so they
may be specific to the system on which they were compiled or require other may be specific to the system on which they were compiled or require other
binary runtime dependencies. Despite these drawbacks, a site may sometimes binary runtime dependencies. Despite these drawbacks, you may have
have no choice but to compile a custom module against a specific binary to compile a custom module against a specific binary
library if that's the only way they have to get access to certain resources. library if that's the only way to get access to certain resources.
Binary modules take their arguments and will return data to Ansible in the same Binary modules take their arguments and return data to Ansible in the same
way as :ref:`want JSON modules <flow_want_json_modules>`. way as :ref:`want JSON modules <flow_want_json_modules>`.
.. seealso:: One example of a `binary module .. seealso:: One example of a `binary module
@ -162,10 +153,8 @@ Old-style modules
Old-style modules are similar to Old-style modules are similar to
:ref:`want JSON modules <flow_want_json_modules>`, except that the file that :ref:`want JSON modules <flow_want_json_modules>`, except that the file that
they take contains ``key=value`` pairs for their parameters instead of they take contains ``key=value`` pairs for their parameters instead of
:term:`JSON`. :term:`JSON`. Ansible decides that a module is old-style when it doesn't have
any of the markers that would show that it is one of the other types.
Ansible decides that a module is old-style when it doesn't have any of the
markers that would show that it is one of the other types.
.. _flow_how_modules_are_executed: .. _flow_how_modules_are_executed:
@ -173,8 +162,8 @@ How modules are executed
======================== ========================
When a user uses :program:`ansible` or :program:`ansible-playbook`, they When a user uses :program:`ansible` or :program:`ansible-playbook`, they
specify a task to execute. The task is usually the name of a module along specify a task to execute. The task is usually the name of a module along
with several parameters to be passed to the module. Ansible takes these with several parameters to be passed to the module. Ansible takes these
values and processes them in various ways before they are finally executed on values and processes them in various ways before they are finally executed on
the remote machine. the remote machine.
@ -185,44 +174,43 @@ Executor/task_executor
The TaskExecutor receives the module name and parameters that were parsed from The TaskExecutor receives the module name and parameters that were parsed from
the :term:`playbook <playbooks>` (or from the command line in the case of the :term:`playbook <playbooks>` (or from the command line in the case of
:command:`/usr/bin/ansible`). It uses the name to decide whether it's looking :command:`/usr/bin/ansible`). It uses the name to decide whether it's looking
at a module or an :ref:`Action Plugin <flow_action_plugins>`. If it's at a module or an :ref:`Action Plugin <flow_action_plugins>`. If it's
a module, it loads the :ref:`Normal Action Plugin <flow_normal_action_plugin>` a module, it loads the :ref:`Normal Action Plugin <flow_normal_action_plugin>`
and passes the name, variables, and other information about the task and play and passes the name, variables, and other information about the task and play
to that Action Plugin for further processing. to that Action Plugin for further processing.
.. _flow_normal_action_plugin: .. _flow_normal_action_plugin:
Normal action plugin The ``normal`` action plugin
-------------------- ----------------------------
The ``normal`` action plugin executes the module on the remote host. It is The ``normal`` action plugin executes the module on the remote host. It is
the primary coordinator of much of the work to actually execute the module on the primary coordinator of much of the work to actually execute the module on
the managed machine. the managed machine.
* It takes care of creating a connection to the managed machine by * It loads the appropriate connection plugin for the task, which then transfers
instantiating a ``Connection`` class according to the inventory or executes as needed to create a connection to that host.
configuration for that host. * It adds any internal Ansible properties to the module's parameters (for
* It adds any internal Ansible variables to the module's parameters (for
instance, the ones that pass along ``no_log`` to the module). instance, the ones that pass along ``no_log`` to the module).
* It takes care of creating any temporary files on the remote machine and * It works with other plugins (connection, shell, become, other action plugins)
to create any temporary files on the remote machine and
cleans up afterwards. cleans up afterwards.
* It does the actual work of pushing the module and module parameters to the * It pushes the module and module parameters to the
remote host, although the :ref:`module_common <flow_executor_module_common>` remote host, although the :ref:`module_common <flow_executor_module_common>`
code described in the next section does the work of deciding which format code described in the next section decides which format
those will take. those will take.
* It handles any special cases regarding modules (for instance, various * It handles any special cases regarding modules (for instance, async
complications around Windows modules that must have the same names as Python execution, or complications around Windows modules that must have the same names as Python modules, so that internal calling of modules from other Action Plugins work.)
modules, so that internal calling of modules from other Action Plugins work.)
Much of this functionality comes from the `BaseAction` class, Much of this functionality comes from the `BaseAction` class,
which lives in :file:`plugins/action/__init__.py`. It makes use of which lives in :file:`plugins/action/__init__.py`. It uses the
``Connection`` and ``Shell`` objects to do its work. ``Connection`` and ``Shell`` objects to do its work.
.. note:: .. note::
When :term:`tasks <tasks>` are run with the ``async:`` parameter, Ansible When :term:`tasks <tasks>` are run with the ``async:`` parameter, Ansible
uses the ``async`` Action Plugin instead of the ``normal`` Action Plugin uses the ``async`` Action Plugin instead of the ``normal`` Action Plugin
to invoke it. That program flow is currently not documented. Read the to invoke it. That program flow is currently not documented. Read the
source for information on how that works. source for information on how that works.
.. _flow_executor_module_common: .. _flow_executor_module_common:
@ -230,25 +218,27 @@ which lives in :file:`plugins/action/__init__.py`. It makes use of
Executor/module_common.py Executor/module_common.py
------------------------- -------------------------
Code in :file:`executor/module_common.py` takes care of assembling the module Code in :file:`executor/module_common.py` assembles the module
to be shipped to the managed node. The module is first read in, then examined to be shipped to the managed node. The module is first read in, then examined
to determine its type. :ref:`PowerShell <flow_powershell_modules>` and to determine its type:
:ref:`JSON-args modules <flow_jsonargs_modules>` are passed through
:ref:`Module Replacer <module_replacer>`. New-style * :ref:`PowerShell <flow_powershell_modules>` and :ref:`JSON-args modules <flow_jsonargs_modules>` are passed through :ref:`Module Replacer <module_replacer>`.
:ref:`Python modules <flow_python_modules>` are assembled by :ref:`Ansiballz`. * New-style :ref:`Python modules <flow_python_modules>` are assembled by :ref:`Ansiballz`.
:ref:`Non-native-want-JSON <flow_want_json_modules>`, * :ref:`Non-native-want-JSON <flow_want_json_modules>`, :ref:`Binary modules <flow_binary_modules>`, and :ref:`Old-Style modules <flow_old_style_modules>` aren't touched by either of these and pass through unchanged.
:ref:`Binary modules <flow_binary_modules>`, and
:ref:`Old-Style modules <flow_old_style_modules>` aren't touched by either of After the assembling step, one final
these and pass through unchanged. After the assembling step, one final modification is made to all modules that have a shebang line. Ansible checks
modification is made to all modules that have a shebang line. Ansible checks
whether the interpreter in the shebang line has a specific path configured via whether the interpreter in the shebang line has a specific path configured via
an ``ansible_$X_interpreter`` inventory variable. If it does, Ansible an ``ansible_$X_interpreter`` inventory variable. If it does, Ansible
substitutes that path for the interpreter path given in the module. After substitutes that path for the interpreter path given in the module. After
this, Ansible returns the complete module data and the module type to the this, Ansible returns the complete module data and the module type to the
:ref:`Normal Action <flow_normal_action_plugin>` which continues execution of :ref:`Normal Action <flow_normal_action_plugin>` which continues execution of
the module. the module.
Next we'll go into some details of the two assembler frameworks. Assembler frameworks
--------------------
Ansible supports two assembler frameworks: Ansiballz and the older Module Replacer.
.. _module_replacer: .. _module_replacer:
@ -256,12 +246,12 @@ Module Replacer framework
^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^
The Module Replacer framework is the original framework implementing new-style The Module Replacer framework is the original framework implementing new-style
modules. It is essentially a preprocessor (like the C Preprocessor for those modules, and is still used for PowerShell modules. It is essentially a preprocessor (like the C Preprocessor for those
familiar with that programming language). It does straight substitutions of familiar with that programming language). It does straight substitutions of
specific substring patterns in the module file. There are two types of specific substring patterns in the module file. There are two types of
substitutions: substitutions:
* Replacements that only happen in the module file. These are public * Replacements that only happen in the module file. These are public
replacement strings that modules can utilize to get helpful boilerplate or replacement strings that modules can utilize to get helpful boilerplate or
access to arguments. access to arguments.
@ -272,12 +262,10 @@ substitutions:
:code:`from ansible.module_utils.basic import *` and should also only apply :code:`from ansible.module_utils.basic import *` and should also only apply
to new-style Python modules. to new-style Python modules.
- :code:`# POWERSHELL_COMMON` substitutes the contents of - :code:`# POWERSHELL_COMMON` substitutes the contents of
:file:`ansible/module_utils/powershell.ps1`. It should only be used with :file:`ansible/module_utils/powershell.ps1`. It should only be used with
:ref:`new-style Powershell modules <flow_powershell_modules>`. :ref:`new-style Powershell modules <flow_powershell_modules>`.
* Replacements that are used by ``ansible.module_utils`` code. These are internal * Replacements that are used by ``ansible.module_utils`` code. These are internal replacement patterns. They may be used internally, in the above public replacements, but shouldn't be used directly by modules.
replacement patterns. They may be used internally, in the above public
replacements, but shouldn't be used directly by modules.
- :code:`"<<ANSIBLE_VERSION>>"` is substituted with the Ansible version. In - :code:`"<<ANSIBLE_VERSION>>"` is substituted with the Ansible version. In
:ref:`new-style Python modules <flow_python_modules>` under the :ref:`new-style Python modules <flow_python_modules>` under the
@ -286,29 +274,29 @@ substitutions:
:attr:``AnsibleModule.ansible_version``. :attr:``AnsibleModule.ansible_version``.
- :code:`"<<INCLUDE_ANSIBLE_MODULE_COMPLEX_ARGS>>"` is substituted with - :code:`"<<INCLUDE_ANSIBLE_MODULE_COMPLEX_ARGS>>"` is substituted with
a string which is the Python ``repr`` of the :term:`JSON` encoded module a string which is the Python ``repr`` of the :term:`JSON` encoded module
parameters. Using ``repr`` on the JSON string makes it safe to embed in parameters. Using ``repr`` on the JSON string makes it safe to embed in
a Python file. In new-style Python modules under the Ansiballz framework a Python file. In new-style Python modules under the Ansiballz framework
this is better accessed by instantiating an `AnsibleModule` and this is better accessed by instantiating an `AnsibleModule` and
then using :attr:`AnsibleModule.params`. then using :attr:`AnsibleModule.params`.
- :code:`<<SELINUX_SPECIAL_FILESYSTEMS>>` substitutes a string which is - :code:`<<SELINUX_SPECIAL_FILESYSTEMS>>` substitutes a string which is
a comma separated list of file systems which have a file system dependent a comma separated list of file systems which have a file system dependent
security context in SELinux. In new-style Python modules, if you really security context in SELinux. In new-style Python modules, if you really
need this you should instantiate an `AnsibleModule` and then use need this you should instantiate an `AnsibleModule` and then use
:attr:`AnsibleModule._selinux_special_fs`. The variable has also changed :attr:`AnsibleModule._selinux_special_fs`. The variable has also changed
from a comma separated string of file system names to an actual python from a comma separated string of file system names to an actual python
list of filesystem names. list of filesystem names.
- :code:`<<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>` substitutes the module - :code:`<<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>` substitutes the module
parameters as a JSON string. Care must be taken to properly quote the parameters as a JSON string. Care must be taken to properly quote the
string as JSON data may contain quotes. This pattern is not substituted string as JSON data may contain quotes. This pattern is not substituted
in new-style Python modules as they can get the module parameters another in new-style Python modules as they can get the module parameters another
way. way.
- The string :code:`syslog.LOG_USER` is replaced wherever it occurs with the - The string :code:`syslog.LOG_USER` is replaced wherever it occurs with the
``syslog_facility`` which was named in :file:`ansible.cfg` or any ``syslog_facility`` which was named in :file:`ansible.cfg` or any
``ansible_syslog_facility`` inventory variable that applies to this host. In ``ansible_syslog_facility`` inventory variable that applies to this host. In
new-style Python modules this has changed slightly. If you really need to new-style Python modules this has changed slightly. If you really need to
access it, you should instantiate an `AnsibleModule` and then use access it, you should instantiate an `AnsibleModule` and then use
:attr:`AnsibleModule._syslog_facility` to access it. It is no longer the :attr:`AnsibleModule._syslog_facility` to access it. It is no longer the
actual syslog facility and is now the name of the syslog facility. See actual syslog facility and is now the name of the syslog facility. See
the :ref:`documentation on internal arguments <flow_internal_arguments>` the :ref:`documentation on internal arguments <flow_internal_arguments>`
for details. for details.
@ -317,32 +305,36 @@ substitutions:
Ansiballz framework Ansiballz framework
^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
Ansible 2.1 switched from the :ref:`module_replacer` framework to the The Ansiballz framework was adopted in Ansible 2.1 and is used for all new-style Python modules. Unlike the Module Replacer, Ansiballz uses real Python imports of things in
Ansiballz framework for assembling modules. The Ansiballz framework differs :file:`ansible/module_utils` instead of merely preprocessing the module. It
from module replacer in that it uses real Python imports of things in
:file:`ansible/module_utils` instead of merely preprocessing the module. It
does this by constructing a zipfile -- which includes the module file, files does this by constructing a zipfile -- which includes the module file, files
in :file:`ansible/module_utils` that are imported by the module, and some in :file:`ansible/module_utils` that are imported by the module, and some
boilerplate to pass in the module's parameters. The zipfile is then Base64 boilerplate to pass in the module's parameters. The zipfile is then Base64
encoded and wrapped in a small Python script which decodes the Base64 encoding encoded and wrapped in a small Python script which decodes the Base64 encoding
and places the zipfile into a temp directory on the managed node. It then and places the zipfile into a temp directory on the managed node. It then
extracts just the ansible module script from the zip file and places that in extracts just the Ansible module script from the zip file and places that in
the temporary directory as well. Then it sets the PYTHONPATH to find python the temporary directory as well. Then it sets the PYTHONPATH to find Python
modules inside of the zip file and invokes :command:`python` on the extracted modules inside of the zip file and imports the Ansible module as the special name, ``__main__``.
ansible module. Importing it as ``__main__`` causes Python to think that it is executing a script rather than simply
importing a module. This lets Ansible run both the wrapper script and the module code in a single copy of Python on the remote machine.
.. note:: .. note::
Ansible wraps the zipfile in the Python script for two reasons: * Ansible wraps the zipfile in the Python script for two reasons:
* for compatibility with Python 2.6 which has a less * for compatibility with Python 2.6 which has a less
functional version of Python's ``-m`` command line switch. functional version of Python's ``-m`` command line switch.
* so that pipelining will function properly. Pipelining needs to pipe the
Python module into the Python interpreter on the remote node. Python * so that pipelining will function properly. Pipelining needs to pipe the
understands scripts on stdin but does not understand zip files. Python module into the Python interpreter on the remote node. Python
understands scripts on stdin but does not understand zip files.
* Prior to Ansible 2.7, the module was executed via a second Python interpreter instead of being
executed inside of the same process. This change was made once Python-2.4 support was dropped
to speed up module execution.
In Ansiballz, any imports of Python modules from the In Ansiballz, any imports of Python modules from the
:py:mod:`ansible.module_utils` package trigger inclusion of that Python file :py:mod:`ansible.module_utils` package trigger inclusion of that Python file
into the zipfile. Instances of :code:`#<<INCLUDE_ANSIBLE_MODULE_COMMON>>` in into the zipfile. Instances of :code:`#<<INCLUDE_ANSIBLE_MODULE_COMMON>>` in
the module are turned into :code:`from ansible.module_utils.basic import *` the module are turned into :code:`from ansible.module_utils.basic import *`
and :file:`ansible/module-utils/basic.py` is then included in the zipfile. and :file:`ansible/module-utils/basic.py` is then included in the zipfile.
Files that are included from :file:`module_utils` are themselves scanned for Files that are included from :file:`module_utils` are themselves scanned for
@ -351,34 +343,34 @@ the zipfile as well.
.. warning:: .. warning::
At present, the Ansiballz Framework cannot determine whether an import At present, the Ansiballz Framework cannot determine whether an import
should be included if it is a relative import. Always use an absolute should be included if it is a relative import. Always use an absolute
import that has :py:mod:`ansible.module_utils` in it to allow Ansiballz to import that has :py:mod:`ansible.module_utils` in it to allow Ansiballz to
determine that the file should be included. determine that the file should be included.
.. _flow_passing_module_args: .. _flow_passing_module_args:
Passing args Passing args
------------ ------------
In :ref:`module_replacer`, module arguments are turned into a JSON-ified Arguments are passed differently by the two frameworks:
string and substituted into the combined module file. In :ref:`Ansiballz`,
the JSON-ified string is passed into the module via stdin. When * In :ref:`module_replacer`, module arguments are turned into a JSON-ified string and substituted into the combined module file.
a :class:`ansible.module_utils.basic.AnsibleModule` is instantiated, * In :ref:`Ansiballz`, the JSON-ified string is part of the script which wraps the zipfile. Just before the wrapper script imports the Ansible module as ``__main__``, it monkey-patches the private, ``_ANSIBLE_ARGS`` variable in ``basic.py`` with the variable values. When a :class:`ansible.module_utils.basic.AnsibleModule` is instantiated, it parses this string and places the args into :attr:`AnsibleModule.params` where it can be accessed by the module's other code.
it parses this string and places the args into
:attr:`AnsibleModule.params` where it can be accessed by the module's .. warning::
other code. If you are writing modules, remember that the way we pass arguments is an internal implementation detail: it has changed in the past and will change again as soon as changes to the common module_utils
code allow Ansible modules to forgo using :class:`ansible.module_utils.basic.AnsibleModule`. Do not rely on the internal global ``_ANSIBLE_ARGS`` variable.
Very dynamic custom modules which need to parse arguments before they
instantiate an ``AnsibleModule`` may use ``_load_params`` to retrieve those parameters.
Although ``_load_params`` may change in breaking ways if necessary to support
changes in the code, it is likely to be more stable than either the way we pass parameters or the internal global variable.
.. note:: .. note::
Internally, the `AnsibleModule` uses the helper function, Prior to Ansible 2.7, the Ansible module was invoked in a second Python interpreter and the
:py:func:`ansible.module_utils.basic._load_params`, to load the parameters arguments were then passed to the script over the script's stdin.
from stdin and save them into an internal global variable. Very dynamic
custom modules which need to parse the parameters prior to instantiating
an ``AnsibleModule`` may use ``_load_params`` to retrieve the
parameters. Be aware that ``_load_params`` is an internal function and
may change in breaking ways if necessary to support changes in the code.
However, we'll do our best not to break it gratuitously, which is not
something that can be said for either the way parameters are passed or
the internal global variable.
.. _flow_internal_arguments: .. _flow_internal_arguments:
@ -386,93 +378,85 @@ Internal arguments
------------------ ------------------
Both :ref:`module_replacer` and :ref:`Ansiballz` send additional arguments to Both :ref:`module_replacer` and :ref:`Ansiballz` send additional arguments to
the module beyond those which the user specified in the playbook. These the module beyond those which the user specified in the playbook. These
additional arguments are internal parameters that help implement global additional arguments are internal parameters that help implement global
Ansible features. Modules often do not need to know about these explicitly as Ansible features. Modules often do not need to know about these explicitly as
the features are implemented in :py:mod:`ansible.module_utils.basic` but certain the features are implemented in :py:mod:`ansible.module_utils.basic` but certain
features need support from the module so it's good to know about them. features need support from the module so it's good to know about them.
The internal arguments listed here are global. If you need to add a local internal argument to a custom module, create an action plugin for that specific module - see ``_original_basename`` in the `copy action plugin <https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/copy.py#L329>`_ for an example.
_ansible_no_log _ansible_no_log
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
This is a boolean. If it's True then the playbook specified ``no_log`` (in Boolean. Set to True whenever a parameter in a task or play specifies ``no_log``. Any module that calls :py:meth:`AnsibleModule.log` handles this automatically. If a module implements its own logging then
a task's parameters or as a play parameter). This automatically affects calls it needs to check this value. To access in a module, instantiate an
to :py:meth:`AnsibleModule.log`. If a module implements its own logging then ``AnsibleModule`` and then check the value of :attr:`AnsibleModule.no_log`.
it needs to check this value. The best way to look at this is for the module
to instantiate an `AnsibleModule` and then check the value of
:attr:`AnsibleModule.no_log`.
.. note:: .. note::
``no_log`` specified in a module's argument_spec are handled by a different mechanism. ``no_log`` specified in a module's argument_spec is handled by a different mechanism.
_ansible_debug _ansible_debug
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
This is a boolean that turns on more verbose logging. If a module uses Boolean. Turns more verbose logging on or off and turns on logging of
external commands that the module executes. If a module uses
:py:meth:`AnsibleModule.debug` rather than :py:meth:`AnsibleModule.log` then :py:meth:`AnsibleModule.debug` rather than :py:meth:`AnsibleModule.log` then
the messages are only logged if this is True. This also turns on logging of the messages are only logged if ``_ansible_debug`` is set to ``True``.
external commands that the module executes. This can be changed via To set, add ``debug: True`` to :file:`ansible.cfg` or set the environment
the ``debug`` setting in :file:`ansible.cfg` or the environment variable variable :envvar:`ANSIBLE_DEBUG`. To access in a module, instantiate an
:envvar:`ANSIBLE_DEBUG`. If, for some reason, a module must access this, it ``AnsibleModule`` and access :attr:`AnsibleModule._debug`.
should do so by instantiating an `AnsibleModule` and accessing
:attr:`AnsibleModule._debug`.
_ansible_diff _ansible_diff
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
This boolean is turned on via the ``--diff`` command line option. If a module Boolean. If a module supports it, tells the module to show a unified diff of
supports it, it will tell the module to show a unified diff of changes to be changes to be made to templated files. To set, pass the ``--diff`` command line
made to templated files. The proper way for a module to access this is by option. To access in a module, instantiate an `AnsibleModule` and access
instantiating an `AnsibleModule` and accessing
:attr:`AnsibleModule._diff`. :attr:`AnsibleModule._diff`.
_ansible_verbosity _ansible_verbosity
^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^
This value could be used for finer grained control over logging. However, it Unused. This value could be used for finer grained control over logging.
is currently unused.
_ansible_selinux_special_fs _ansible_selinux_special_fs
^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
This is a list of names of filesystems which should have a special selinux List. Names of filesystems which should have a special SELinux
context. They are used by the `AnsibleModule` methods which operate on context. They are used by the `AnsibleModule` methods which operate on
files (changing attributes, moving, and copying). The list of names is set files (changing attributes, moving, and copying). To set, add a comma separated string of filesystem names in :file:`ansible.cfg`::
via a comma separated string of filesystem names from :file:`ansible.cfg`::
# ansible.cfg # ansible.cfg
[selinux] [selinux]
special_context_filesystems=nfs,vboxsf,fuse,ramfs special_context_filesystems=nfs,vboxsf,fuse,ramfs
If a module cannot use the builtin ``AnsibleModule`` methods to manipulate Most modules can use the built-in ``AnsibleModule`` methods to manipulate
files and needs to know about these special context filesystems, it should files. To access in a module that needs to know about these special context filesystems, instantiate an ``AnsibleModule`` and examine the list in
instantiate an ``AnsibleModule`` and then examine the list in
:attr:`AnsibleModule._selinux_special_fs`. :attr:`AnsibleModule._selinux_special_fs`.
This replaces :attr:`ansible.module_utils.basic.SELINUX_SPECIAL_FS` from This replaces :attr:`ansible.module_utils.basic.SELINUX_SPECIAL_FS` from
:ref:`module_replacer`. In module replacer it was a comma separated string of :ref:`module_replacer`. In module replacer it was a comma separated string of
filesystem names. Under Ansiballz it's an actual list. filesystem names. Under Ansiballz it's an actual list.
.. versionadded:: 2.1 .. versionadded:: 2.1
_ansible_syslog_facility _ansible_syslog_facility
^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^
This parameter controls which syslog facility ansible module logs to. It may This parameter controls which syslog facility Ansible module logs to. To set, change the ``syslog_facility`` value in :file:`ansible.cfg`. Most
be set by changing the ``syslog_facility`` value in :file:`ansible.cfg`. Most
modules should just use :meth:`AnsibleModule.log` which will then make use of modules should just use :meth:`AnsibleModule.log` which will then make use of
this. If a module has to use this on its own, it should instantiate an this. If a module has to use this on its own, it should instantiate an
`AnsibleModule` and then retrieve the name of the syslog facility from `AnsibleModule` and then retrieve the name of the syslog facility from
:attr:`AnsibleModule._syslog_facility`. The code will look slightly different :attr:`AnsibleModule._syslog_facility`. The Ansiballz code is less hacky than the old :ref:`module_replacer` code:
than it did under :ref:`module_replacer` due to how hacky the old way was
.. code-block:: python .. code-block:: python
# Old way # Old module_replacer way
import syslog import syslog
syslog.openlog(NAME, 0, syslog.LOG_USER) syslog.openlog(NAME, 0, syslog.LOG_USER)
# New way # New Ansiballz way
import syslog import syslog
facility_name = module._syslog_facility facility_name = module._syslog_facility
facility = getattr(syslog, facility_name, syslog.LOG_USER) facility = getattr(syslog, facility_name, syslog.LOG_USER)
@ -483,14 +467,29 @@ than it did under :ref:`module_replacer` due to how hacky the old way was
_ansible_version _ansible_version
^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^
This parameter passes the version of ansible that runs the module. To access This parameter passes the version of Ansible that runs the module. To access
it, a module should instantiate an `AnsibleModule` and then retrieve it it, a module should instantiate an `AnsibleModule` and then retrieve it
from :attr:`AnsibleModule.ansible_version`. This replaces from :attr:`AnsibleModule.ansible_version`. This replaces
:attr:`ansible.module_utils.basic.ANSIBLE_VERSION` from :attr:`ansible.module_utils.basic.ANSIBLE_VERSION` from
:ref:`module_replacer`. :ref:`module_replacer`.
.. versionadded:: 2.1 .. versionadded:: 2.1
.. _flow_module_return_values:
Module return values & Unsafe strings
-------------------------------------
At the end of a module's execution, it formats the data that it wants to return as a JSON string and prints the string to its stdout. The normal action plugin receives the JSON string, parses it into a Python dictionary, and returns it to the executor.
If Ansible templated every string return value, it would be vulnerable to an attack from users with access to managed nodes. If an unscrupulous user disguised malicious code as Ansible return value strings, and if those strings were then templated on the controller, Ansible could execute arbitrary code. To prevent this scenario, Ansible marks all strings inside returned data as ``Unsafe``, emitting any Jinja2 templates in the strings verbatim, not expanded by Jinja2.
Strings returned by invoking a module through ``ActionPlugin._execute_module()`` are automatically marked as ``Unsafe`` by the normal action plugin. If another action plugin retrieves information from a module through some other means, it must mark its return data as ``Unsafe`` on its own.
In case a poorly-coded action plugin fails to mark its results as "Unsafe," Ansible audits the results again when they are returned to the executor,
marking all strings as ``Unsafe``. The normal action plugin protects itself and any other code that it calls with the result data as a parameter. The check inside the executor protects the output of all other action plugins, ensuring that subsequent tasks run by Ansible will not template anything from those results either.
.. _flow_special_considerations: .. _flow_special_considerations:
Special considerations Special considerations
@ -510,7 +509,7 @@ Ansible can transfer a module to a remote machine in one of two ways:
into the remote interpreter's stdin. into the remote interpreter's stdin.
Pipelining only works with modules written in Python at this time because Pipelining only works with modules written in Python at this time because
Ansible only knows that Python supports this mode of operation. Supporting Ansible only knows that Python supports this mode of operation. Supporting
pipelining means that whatever format the module payload takes before being pipelining means that whatever format the module payload takes before being
sent over the wire must be executable by Python via stdin. sent over the wire must be executable by Python via stdin.
@ -522,13 +521,13 @@ Why pass args over stdin?
Passing arguments via stdin was chosen for the following reasons: Passing arguments via stdin was chosen for the following reasons:
* When combined with :ref:`ANSIBLE_PIPELINING`, this keeps the module's arguments from * When combined with :ref:`ANSIBLE_PIPELINING`, this keeps the module's arguments from
temporarily being saved onto disk on the remote machine. This makes it temporarily being saved onto disk on the remote machine. This makes it
harder (but not impossible) for a malicious user on the remote machine to harder (but not impossible) for a malicious user on the remote machine to
steal any sensitive information that may be present in the arguments. steal any sensitive information that may be present in the arguments.
* Command line arguments would be insecure as most systems allow unprivileged * Command line arguments would be insecure as most systems allow unprivileged
users to read the full commandline of a process. users to read the full commandline of a process.
* Environment variables are usually more secure than the commandline but some * Environment variables are usually more secure than the commandline but some
systems limit the total size of the environment. This could lead to systems limit the total size of the environment. This could lead to
truncation of the parameters if we hit that limit. truncation of the parameters if we hit that limit.