diff --git a/docs/docsite/rst/dev_guide/developing_collections.rst b/docs/docsite/rst/dev_guide/developing_collections.rst index 4253088fd73..b1cc89e54e1 100644 --- a/docs/docsite/rst/dev_guide/developing_collections.rst +++ b/docs/docsite/rst/dev_guide/developing_collections.rst @@ -21,8 +21,7 @@ You can publish and use collections through `Ansible Galaxy BOTMETA.yml ----------- -The `BOTMETA.yml `_ is the source of truth for: +The `BOTMETA.yml `_ in the ansible/ansible GitHub repository is the source of truth for: + * ansibullbot * the docs build for collections-based modules @@ -504,6 +507,93 @@ The build process for docs.ansible.com will know where to find the module docs. * ReStructured Text docs (anything under ``docs/docsite/rst/``) * Files that never existed in ``ansible/ansible:devel`` +.. _testing_collections: + +Testing collections +=================== + +The main tool for testing collections is ``ansible-test``, Ansible's testing tool described in :ref:`developing_testing`. You can run several compile and sanity checks, as well as run unit and integration tests for plugins using ``ansible-test``. + +You must always execute ``ansible-test`` from the root directory of a collection. You can run ``ansible-test`` in Docker containers without installing any special requirements. The Ansible team uses this approach in Shippable both in the ansible/ansible GitHub repository and in the large community collections such as `community.general `_ and `community.network `_. The examples below demonstrate running tests in Docker containers. + +Compile and sanity tests +------------------------ + +To run all compile and sanity tests:: + + ansible-test sanity --docker default -v + +See :ref:`testing_compile` and :ref:`testing_sanity` for more information. See the :ref:`full list of sanity tests ` for details on the sanity tests and how to fix identified issues. + +Unit tests +---------- + +You must place unit tests in the appropriate``tests/unit/plugins/`` directory. For example, you would place tests for ``plugins/module_utils/foo/bar.py`` in ``tests/unit/plugins/module_utils/foo/test_bar.py`` or ``tests/unit/plugins/module_utils/foo/bar/test_bar.py``. For examples, see the `unit tests in community.general `_. + +To run all unit tests for all supported Python versions:: + + ansible-test units --docker default -v + +To run all unit tests only for a specific Python version:: + + ansible-test units --docker default -v --python 3.6 + +To run only a specific unit test:: + + ansible-test units --docker default -v --python 3.6 tests/unit/plugins/module_utils/foo/test_bar.py + +You can specify Python requirements in the ``tests/unit/requirements.txt`` file. See :ref:`testing_units` for more information, especially on fixture files. + +Integration tests +----------------- + +You must place integration tests in the appropriate ``tests/integration/targets/`` directory. For module integration tests, you can use the module name alone. For example, you would place integration tests for ``plugins/modules/foo.py`` in a directory called ``tests/integration/targets/foo/``. For non-module plugin integration tests, you must add the plugin type to the directory name. For example, you would place integration tests for ``plugins/connections/bar.py`` in a directory called ``tests/integration/targets/connection_bar/``. For lookup plugins, the directory must be called ``lookup_foo``, for inventory plugins, ``inventory_foo``, and so on. + +You can write two different kinds of integration tests: + +* Ansible role tests run with ``ansible-playbook`` and validate various aspects of the module. They can depend on other integration tests (usually named ``prepare_bar`` or ``setup_bar``, which prepare a service or install a requirement named ``bar`` in order to test module ``foo``) to set-up required resources, such as installing required libraries or setting up server services. +* ``runme.sh`` tests run directly as scripts. They can set up inventory files, and execute ``ansible-playbook`` or ``ansible-inventory`` with various settings. + +For examples, see the `integration tests in community.general `_. See also :ref:`testing_integration` for more details. + +Since integration tests can install requirements, and set-up, start and stop services, we recommended running them in docker containers or otherwise restricted environments whenever possible. By default, ``ansible-test`` supports Docker images for several operating systems. See the `list of supported docker images `_ for all options. Use the ``default`` image mainly for platform-independent integration tests, such as those for cloud modules. The following examples use the ``centos8`` image. + +To execute all integration tests for a collection:: + + ansible-test integration --docker centos8 -v + +If you want more detailed output, run the command with ``-vvv`` instead of ``-v``. Alternatively, specify ``--retry-on-error`` to automatically re-run failed tests with higher verbosity levels. + +To execute only the integration tests in a specific directory:: + + ansible-test integration --docker centos8 -v connection_bar + +You can specify multiple target names. Each target name is the name of a directory in ``tests/integration/targets/``. + +.. _hacking_collections: + +Contributing to collections +=========================== + +If you want to add functionality to an existing collection, modify a collection you are using to fix a bug, or change the behavior of a module in a collection, clone the git repository for that collection and make changes on a branch. You can combine changes to a collection with a local checkout of Ansible (``source hacking/env-setup``). + +This section describes the process for `community.general `_. To contribute to other collections, replace the folder names ``community`` and ``general`` with the namespace and collection name of a different collection. + +We assume that you have included ``~/dev/ansible/collections/`` in :ref:`COLLECTIONS_PATHS`, and if that path mentions multiple directories, that you made sure that no other directory earlier in the search path contains a copy of ``community.general``. Create the directory ``~/dev/ansible/collections/ansible_collections/community``, and in it clone `the community.general Git repository `_ or a fork of it into the folder ``general``:: + + mkdir -p ~/dev/ansible/collections/ansible_collections/community + cd ~/dev/ansible/collections/ansible_collections/community + git clone git@github.com:ansible-collections/community.general.git general + +If you clone a fork, add the original repository as a remote ``upstream``:: + + cd ~/dev/ansible/collections/ansible_collections/community/general + git remote add upstream git@github.com:ansible-collections/community.general.git + +Now you can use this checkout of ``community.general`` in playbooks and roles with whichever version of Ansible you have installed locally, including a local checkout of the ``devel`` branch. + +For collections hosted in the ``ansible_collections`` GitHub org, create a branch and commit your changes on the branch. When you are done (remember to add tests, see :ref:`testing_collections`), push your changes to your fork of the collection and create a Pull Request. For other collections, especially for collections not hosted on GitHub, check the ``README.md`` of the collection for information on contributing to it. + .. seealso:: diff --git a/docs/docsite/rst/dev_guide/testing.rst b/docs/docsite/rst/dev_guide/testing.rst index b0701128a5d..480d2c3c56e 100644 --- a/docs/docsite/rst/dev_guide/testing.rst +++ b/docs/docsite/rst/dev_guide/testing.rst @@ -17,6 +17,8 @@ Ansible users who understand how to write playbooks and roles should be able to Read on to learn how Ansible is tested, how to test your contributions locally, and how to extend testing capabilities. +If you want to learn on how testing collections, you should read :ref:`testing_collections` + Types of tests @@ -91,7 +93,8 @@ Occasionally you may find your PR fails due to a reason unrelated to your change If either of these issues appear to be the case, you can rerun the Shippable test by: -* closing and re-opening the PR +* adding a comment with ``/rebuild`` (full rebuild) or ``/rebuild_failed`` (rebuild only failed CI nodes) to the PR +* closing and re-opening the PR (full rebuild) * making another change to the PR and pushing to GitHub If the issue persists, please contact us in ``#ansible-devel`` on Freenode IRC. diff --git a/docs/docsite/rst/dev_guide/testing_compile.rst b/docs/docsite/rst/dev_guide/testing_compile.rst index 9781966c1b4..5c22194d521 100644 --- a/docs/docsite/rst/dev_guide/testing_compile.rst +++ b/docs/docsite/rst/dev_guide/testing_compile.rst @@ -18,6 +18,9 @@ Compile tests check source files for valid syntax on all supported python versio - 2.7 - 3.5 - 3.6 +- 3.7 +- 3.8 +- 3.9 NOTE: In Ansible 2.4 and earlier the compile test was provided by a dedicated sub-command ``ansible-test compile`` instead of a sanity test using ``ansible-test sanity --test compile``.