ansible/docs/docsite/rst/dev_guide/testing_units.rst
Sandra McCann ceb474bb9e [WIP] Backport/2.7/batch port (#45859)
Batch of docs backports:

* docs: Clarify include_task v import_tasks with conditionals (#43856)
(cherry picked from commit 6be42a2a0e)

* Add single quotes around package name (#45152)
(cherry picked from commit 0d81386144)

* prefer ansible_facts namespace and dict notation (#44980)
(cherry picked from commit 44510448b0)

* fix cherrypick conflict - scenario_guides

* Update implicit_localhost.rst (#45455)
(cherry picked from commit f68cd1acc6)

* updated fbsd install instructions (#45309)
(cherry picked from commit e9c2695ce7)

* Change "Defaulting Undefined Variables" (#41379)
(cherry picked from commit e35c4be1c1)

* adds license details to dev guide pages (#45574)
(cherry picked from commit 6e68d77f6d)

* FAQ: fix a typo, add link to 'vars' lookup (#42412)
(cherry picked from commit 95649dc793)

* Fix link and toctree (#45595)
(cherry picked from commit 6999bf318f)

* Improve the local toctree (and title) (#45590)
(cherry picked from commit afea00fa9f)

* Add undocumented configuration parameter and explain in porting guide (#36059)
(cherry picked from commit a892a6ef03)

* Simplify PPA installation for Ubuntu (#45690)
(cherry picked from commit 78e9f452a5)

* adding git+ssh uri scheme (#36025)
(cherry picked from commit 84a4257774)

* Add workaround for non-standard kerberos environments (#41465)
(cherry picked from commit 4e532e0ad9)

* Restore license agreement (#45809)
(cherry picked from commit f430f60541)

* partial cherry-pick - lenovo doc update PR 45483
2018-09-19 22:02:01 -05:00

211 lines
6.7 KiB
ReStructuredText

:orphan:
.. _testing_units:
**********
Unit Tests
**********
Unit tests are small isolated tests that target a specific library or module. Unit tests
in Ansible are currently the only way of driving tests from python within Ansible's
continuous integration process. This means that in some circumstances the tests may be a
bit wider than just units.
.. contents:: Topics
Available Tests
===============
Unit tests can be found in `test/units
<https://github.com/ansible/ansible/tree/devel/test/units>`_. Notice that the directory
structure of the tests matches that of ``lib/ansible/``.
Running Tests
=============
The Ansible unit tests can be run across the whole code base by doing:
.. code:: shell
cd /path/to/ansible/source
source hacking/env-setup
ansible-test units --tox
Against a single file by doing:
.. code:: shell
ansible-test units --tox apt
Or against a specific Python version by doing:
.. code:: shell
ansible-test units --tox --python 2.7 apt
For advanced usage see the online help::
ansible-test units --help
You can also run tests in Ansible's continuous integration system by opening a pull
request. This will automatically determine which tests to run based on the changes made
in your pull request.
Installing dependencies
=======================
``ansible-test`` has a number of dependencies. For ``units`` tests we suggest using ``tox``.
The dependencies can be installed using the ``--requirements`` argument, which will
install all the required dependencies needed for unit tests. For example:
.. code:: shell
ansible-test units --tox --python 2.7 --requirements apache2_module
.. note:: tox version requirement
When using ``ansible-test`` with ``--tox`` requires tox >= 2.5.0
The full list of requirements can be found at `test/runner/requirements
<https://github.com/ansible/ansible/tree/devel/test/runner/requirements>`_. Requirements
files are named after their respective commands. See also the `constraints
<https://github.com/ansible/ansible/blob/devel/test/runner/requirements/constraints.txt>`_
applicable to all commands.
Extending unit tests
====================
.. warning:: What a unit test isn't
If you start writing a test that requires external services then
you may be writing an integration test, rather than a unit test.
Structuring Unit Tests
``````````````````````
Ansible drives unit tests through `pytest <https://docs.pytest.org/en/latest/>`_. This
means that tests can either be written a simple functions which are included in any file
name like ``test_<something>.py`` or as classes.
Here is an example of a function::
#this function will be called simply because it is called test_*()
def test_add()
a = 10
b = 23
c = 33
assert a + b = c
Here is an example of a class::
import unittest
class AddTester(unittest.TestCase)
def SetUp()
self.a = 10
self.b = 23
# this function will
def test_add()
c = 33
assert self.a + self.b = c
# this function will
def test_subtract()
c = -13
assert self.a - self.b = c
Both methods work fine in most circumstances; the function-based interface is simpler and
quicker and so that's probably where you should start when you are just trying to add a
few basic tests for a module. The class-based test allows more tidy set up and tear down
of pre-requisites, so if you have many test cases for your module you may want to refactor
to use that.
Assertions using the simple ``assert`` function inside the tests will give full
information on the cause of the failure with a trace-back of functions called during the
assertion. This means that plain asserts are recommended over other external assertion
libraries.
A number of the unit test suites include functions that are shared between several
modules, especially in the networking arena. In these cases a file is created in the same
directory, which is then included directly.
Module test case common code
````````````````````````````
Keep common code as specific as possible within the `test/units/` directory structure. For
example, if it's specific to testing Amazon modules, it should be in
`test/units/modules/cloud/amazon/`. Don't import common unit test code from directories
outside the current or parent directories.
Don't import other unit tests from a unit test. Any common code should be in dedicated
files that aren't themselves tests.
Fixtures files
``````````````
To mock out fetching results from devices, or provide other complex datastructures that
come from external libraries, you can use ``fixtures`` to read in pre-generated data.
Text files live in ``test/units/modules/network/PLATFORM/fixtures/``
Data is loaded using the ``load_fixture`` method
See `eos_banner test
<https://github.com/ansible/ansible/blob/devel/test/units/modules/network/eos/test_eos_banner.py>`_
for a practical example.
If you are simulating APIs you may find that python placebo is useful. See
:ref:`testing_units_modules` for more information.
Code Coverage For New or Updated Unit Tests
```````````````````````````````````````````
New code will be missing from the codecov.io coverage reports (see :doc:`testing`), so
local reporting is needed. Most ``ansible-test`` commands allow you to collect code
coverage; this is particularly useful when to indicate where to extend testing.
To collect coverage data add the ``--coverage`` argument to your ``ansible-test`` command line:
.. code:: shell
ansible-test units --coverage apt
ansible-test coverage html
Results will be written to ``test/results/reports/coverage/index.html``
Reports can be generated in several different formats:
* ``ansible-test coverage report`` - Console report.
* ``ansible-test coverage html`` - HTML report.
* ``ansible-test coverage xml`` - XML report.
To clear data between test runs, use the ``ansible-test coverage erase`` command. See
:ref:`testing_running_locally` for more information about generating coverage
reports.
.. seealso::
:ref:`testing_units_modules`
Special considerations for unit testing modules
:doc:`testing_running_locally`
Running tests locally including gathering and reporting coverage data
`Python 3 documentation - 26.4. unittest — Unit testing framework <https://docs.python.org/3/library/unittest.html>`_
The documentation of the unittest framework in python 3
`Python 2 documentation - 25.3. unittest — Unit testing framework <https://docs.python.org/3/library/unittest.html>`_
The documentation of the earliest supported unittest framework - from Python 2.6
`pytest: helps you write better programs <https://docs.pytest.org/en/latest/>`_
The documentation of pytest - the framework actually used to run Ansible unit tests