Start a document for developing modules for python3
This commit is contained in:
Toshio Kuratomi
parent
56bb3ec680
commit
9a69354b6a
1 changed files with 135 additions and 0 deletions
135
docsite/rst/developing_modules_python3.rst
Normal file
135
docsite/rst/developing_modules_python3.rst
Normal file
|
|
@ -0,0 +1,135 @@
|
|||
===========================
|
||||
Porting Modules to Python 3
|
||||
===========================
|
||||
|
||||
Ansible modules are not the usual Python-3 porting exercise. There are two
|
||||
factors that make it harder to port them than most code:
|
||||
|
||||
1. Many modules need to run on Python-2.4 in addition to Python-3.
|
||||
2. A lot of mocking has to go into unittesting a Python-3 module. So it's
|
||||
harder to test that your porting has fixed everything or to make sure that
|
||||
later commits haven't regressed.
|
||||
|
||||
Which version of Python-3.x and which version of Python-2.x are our minimums?
|
||||
=============================================================================
|
||||
|
||||
The short answer is Python-3.4 and Python-2.4 but please read on for more
|
||||
information.
|
||||
|
||||
For Python-3 we are currently using Python-3.4 as a minimum. However, no long
|
||||
term supported Linux distributions currently ship with Python-3. When that
|
||||
occurs, we will probably take that as our minimum Python-3 version rather than
|
||||
Python-3.4. Thus far, Python-3 has been adding small changes that make it
|
||||
more compatible with Python-2 in its newer versions (For instance, Python-3.5
|
||||
added the ability to use percent-formatted byte strings.) so it should be more
|
||||
pleasant to use a newer version of Python-3 if it's available. At some point
|
||||
this will change but we'll just have to cross that bridge when we get to it.
|
||||
|
||||
For Python-2 the default is for modules to run on Python-2.4. This allows
|
||||
users with older distributions that are stuck on Python-2.4 to manage their
|
||||
machines. Modules are allowed to drop support for Python-2.4 when one of
|
||||
their dependent libraries require a higher version of python. This is not an
|
||||
invitation to add unnecessary dependent libraries in order to force your
|
||||
module to be usable only with a newer version of Python. Instead it is an
|
||||
acknowledgment that some libraries (for instance, boto3 and docker-py) will
|
||||
only function with newer Python.
|
||||
|
||||
.. note:: When will we drop support for Python-2.4?
|
||||
|
||||
The only long term supported distro that we know of with Python-2.4 is
|
||||
RHEL5 (and its rebuilds like CentOS5) which is supported until April of
|
||||
2017. We will likely end our support for Python-2.4 in modules in an
|
||||
Ansible release around that time. We know of no long term supported
|
||||
distributions with Python-2.5 so the new minimum Python-2 version will
|
||||
likely be Python-2.6. This will let us take advantage of the
|
||||
forwards-compat features of Python-2.6 so porting and maintainance of
|
||||
Python-2/Python-3 code will be easier after that.
|
||||
|
||||
Supporting only Python-2 or only Python-3
|
||||
=========================================
|
||||
|
||||
Sometimes a module's dependent libraries only run on Python-2 or only run on
|
||||
Python-3. We do not yet have a strategy for these modules but we'll need to
|
||||
come up with one. I see three possibilities:
|
||||
|
||||
1. We treat these libraries like any other libraries that may not be installed
|
||||
on the system. When we import them we check if the import was successful.
|
||||
If so, then we continue. If not we return an error about the library being
|
||||
missing. Users will have to find out that the library is unavailable on
|
||||
their version of Python either by searching for the library on their own or
|
||||
reading the requirements section in :command:`ansible-doc`.
|
||||
|
||||
2. The shebang line is the only metadata that Ansible extracts from a module
|
||||
so we may end up using that to specify what we mean. Something like
|
||||
``#!/usr/bin/python`` means the module will run on both Python-2 and
|
||||
Python-3, ``#!/usr/bin/python2`` means the module will only run on
|
||||
Python-2, and ``#!/usr/bin/python3`` means the module will only run on
|
||||
Python-3. Ansible's code will need to be modified to accommodate this.
|
||||
For :command:`python2`, if ``ansible_python2_interpreter`` is not set, it
|
||||
will have to fallback to `` ansible_python_interpreter`` and if that's not
|
||||
set, fallback to ``/usr/bin/python``. For :command:`python3`, Ansible
|
||||
will have to first try ``ansible_python3_interpreter`` and then fallback to
|
||||
``/usr/bin/python3`` as normal.
|
||||
|
||||
3. We add a way for Ansible to retrieve metadata about modules. The metadata
|
||||
will include the version of Python that is required.
|
||||
|
||||
Methods 2 and 3 will both require that we modify modules or otherwise add this
|
||||
additional information somewhere.
|
||||
|
||||
Tips, tricks, and idioms to adopt
|
||||
=================================
|
||||
|
||||
Exceptions
|
||||
----------
|
||||
|
||||
In code which already needs Python-2.6+ (For instance, because a library it
|
||||
depends on only runs on Python >= 2.6) it is okay to port directly to the new
|
||||
exception catching syntax::
|
||||
|
||||
try:
|
||||
a = 2/0
|
||||
except ValueError as e:
|
||||
module.fail_json(msg="Tried to divide by zero!")
|
||||
|
||||
For modules which also run on Python-2.4, we have to use an uglier
|
||||
construction to make this work under both Python-2.4 and Python-3::
|
||||
|
||||
from ansible.module_utils.pycompat import get_exception
|
||||
[...]
|
||||
|
||||
try:
|
||||
a = 2/0
|
||||
except ValueError:
|
||||
e = get_exception()
|
||||
module.fail_json(msg="Tried to divide by zero!")
|
||||
|
||||
Octal numbers
|
||||
-------------
|
||||
|
||||
In Python-2.4, octal literals are specified as ``0755``. In Python-3, that is
|
||||
invalid and octals must be specified as ``0o755``. To bridge this gap,
|
||||
modules should create their octals like this::
|
||||
|
||||
# Can't use 0755 on Python-3 and can't use 0o755 on Python-2.4
|
||||
EXECUTABLE_PERMS = int('0755', 8)
|
||||
|
||||
|
||||
Compile Test
|
||||
------------
|
||||
|
||||
We have travis compiling all modules with various versions of Python to check
|
||||
that the modules conform to the syntax at those versions. When you've
|
||||
ported a module so that its syntax works with Python-3, we need to modify
|
||||
.travis.yml so that the module is included in the syntax check. Here's the
|
||||
relevant section of .travis.yml::
|
||||
|
||||
script:
|
||||
[...]
|
||||
- python3.4 -m compileall -fq system/ping.py
|
||||
- python3.5 -m compileall -fq system/ping.py
|
||||
|
||||
At the moment this is a whitelist. Just add your newly ported module to that
|
||||
line. Eventually, not compiling on Python-3 will be the exception. When that
|
||||
occurs, we will move to a blacklist for listing which modules do not compile
|
||||
under Python-3.
|
||||
Loading…
Reference in a new issue