c936b8b638
* Suggest ansible ad-hoc command while developing module (#70308) If a local module has no documentation, the doc command will fail without any hints of what is wrong. Add another way to confirm the presence of a local module. * Update docs/docsite/rst/dev_guide/developing_locally.rst Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com> (cherry picked from commit82e5d03bdb
) * Update AWS Integration test docmentation (#70454) (cherry picked from commite1ba7dc52a
) * Update hacking/shippable docs. The `--all` option downloads more than is needed for analyzing code coverage. (cherry picked from commitfb7740ae3b
) * Document that subversion module requires subversion (#70537) (cherry picked from commit64c2cb273f
) * update documentation link to python 3 (#70509) update the docs link for strftime on the filters page to point to the python3 docs (cherry picked from commitf7db428375
) * Update windows_winrm.rst (#70306) gcc also needed to be installed alongside python-devel, krb5-devel, krb5-libs, and krb5-workstation. (cherry picked from commitf4ea43c4a0
) * updated requirements file for docs build (#70609) (cherry picked from commit38ccfb4a3e
) * Clarify that index_var is 0 indexed (#70548) A little further down the page is another index, ansible_loop.index, which shares a similar description but is 1 indexed. Its zero indexed twin has a 0 suffix. ``ansible_loop.index`` The current iteration of the loop. (1 indexed) ``ansible_loop.index0`` The current iteration of the loop. (0 indexed) To remove ambiguity around the usage of index_var, explicitly mention that this variable is 0 indexed. (cherry picked from commitc410311f55
) * docs: update module development docs (#70594) Update module development docs for flattened modules directory. Fixes: #70261 (at least partially) Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com> (cherry picked from commit20209c508f
) * Doc: fix examples of changelog entries. (#70551) (cherry picked from commitedcd1a1a70
) * Document tags are not supported with task meta. (#70590) fixes #70338 (cherry picked from commit40591d5fbb
) * docs: update date format in removed_at_date (#70597) removed_at_date requires YYYY-MM-DD format. Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com> (cherry picked from commit375c6b4ae4
) * partial update of community docs to reflect collections transition (#70488) (cherry picked from commitf1f782fc37
) Co-authored-by: Michael Ritsema <michaelritsema@users.noreply.github.com> Co-authored-by: Mark Chappell <mchappel@redhat.com> Co-authored-by: Matt Clay <matt@mystile.com> Co-authored-by: Alan Rominger <arominge@redhat.com> Co-authored-by: FloMiau <37121807+FloMiau@users.noreply.github.com> Co-authored-by: mahadelmi <mahadelmi@cmail.carleton.ca> Co-authored-by: Sayee <57951841+sayee-jadhav@users.noreply.github.com> Co-authored-by: Karl Goetz <goetzk@users.noreply.github.com> Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com> Co-authored-by: Andrew Klychkov <aaklychkov@mail.ru> Co-authored-by: Baptiste Mille-Mathias <baptiste.millemathias@gmail.com> Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
228 lines
7.9 KiB
ReStructuredText
228 lines
7.9 KiB
ReStructuredText
:orphan:
|
|
|
|
.. _testing_integration:
|
|
|
|
*****************
|
|
Integration tests
|
|
*****************
|
|
|
|
.. contents:: Topics
|
|
|
|
The Ansible integration Test system.
|
|
|
|
Tests for playbooks, by playbooks.
|
|
|
|
Some tests may require credentials. Credentials may be specified with `credentials.yml`.
|
|
|
|
Some tests may require root.
|
|
|
|
Quick Start
|
|
===========
|
|
|
|
It is highly recommended that you install and activate the ``argcomplete`` python package.
|
|
It provides tab completion in ``bash`` for the ``ansible-test`` test runner.
|
|
|
|
Configuration
|
|
=============
|
|
|
|
ansible-test command
|
|
--------------------
|
|
|
|
The example below assumes ``bin/`` is in your ``$PATH``. An easy way to achieve that
|
|
is to initialize your environment with the ``env-setup`` command::
|
|
|
|
source hacking/env-setup
|
|
ansible-test --help
|
|
|
|
You can also call ``ansible-test`` with the full path::
|
|
|
|
bin/ansible-test --help
|
|
|
|
integration_config.yml
|
|
----------------------
|
|
|
|
Making your own version of ``integration_config.yml`` can allow for setting some
|
|
tunable parameters to help run the tests better in your environment. Some
|
|
tests (e.g. cloud) will only run when access credentials are provided. For more
|
|
information about supported credentials, refer to the various ``cloud-config-*.template``
|
|
files in the ``test/integration/`` directory.
|
|
|
|
Prerequisites
|
|
=============
|
|
|
|
Some tests assume things like hg, svn, and git are installed, and in path. Some tests
|
|
(such as those for Amazon Web Services) need separate definitions, which will be covered
|
|
later in this document.
|
|
|
|
(Complete list pending)
|
|
|
|
Non-destructive Tests
|
|
=====================
|
|
|
|
These tests will modify files in subdirectories, but will not do things that install or remove packages or things
|
|
outside of those test subdirectories. They will also not reconfigure or bounce system services.
|
|
|
|
.. note:: Running integration tests within Docker
|
|
|
|
To protect your system from any potential changes caused by integration tests, and to ensure a sensible set of dependencies are available we recommend that you always run integration tests with the ``--docker`` option, for example ``--docker centos8``. See the `list of supported docker images <https://github.com/ansible/ansible/blob/devel/test/lib/ansible_test/_data/completion/docker.txt>`_ for options (the ``default`` image is used for sanity and unit tests, as well as for platform independent integration tests such as those for cloud modules).
|
|
|
|
.. note:: Avoiding pulling new Docker images
|
|
|
|
Use the ``--docker-no-pull`` option to avoid pulling the latest container image. This is required when using custom local images that are not available for download.
|
|
|
|
Run as follows for all POSIX platform tests executed by our CI system::
|
|
|
|
ansible-test integration --docker fedora29 -v shippable/
|
|
|
|
You can target a specific tests as well, such as for individual modules::
|
|
|
|
ansible-test integration -v ping
|
|
|
|
Use the following command to list all the available targets::
|
|
|
|
ansible-test integration --list-targets
|
|
|
|
.. note:: Bash users
|
|
|
|
If you use ``bash`` with ``argcomplete``, obtain a full list by doing: ``ansible-test integration <tab><tab>``
|
|
|
|
Destructive Tests
|
|
=================
|
|
|
|
These tests are allowed to install and remove some trivial packages. You will likely want to devote these
|
|
to a virtual environment, such as Docker. They won't reformat your filesystem::
|
|
|
|
ansible-test integration --docker fedora29 -v destructive/
|
|
|
|
Windows Tests
|
|
=============
|
|
|
|
These tests exercise the ``winrm`` connection plugin and Windows modules. You'll
|
|
need to define an inventory with a remote Windows 2008 or 2012 Server to use
|
|
for testing, and enable PowerShell Remoting to continue.
|
|
|
|
Running these tests may result in changes to your Windows host, so don't run
|
|
them against a production/critical Windows environment.
|
|
|
|
Enable PowerShell Remoting (run on the Windows host via Remote Desktop)::
|
|
|
|
Enable-PSRemoting -Force
|
|
|
|
Define Windows inventory::
|
|
|
|
cp inventory.winrm.template inventory.winrm
|
|
${EDITOR:-vi} inventory.winrm
|
|
|
|
Run the Windows tests executed by our CI system::
|
|
|
|
ansible-test windows-integration -v shippable/
|
|
|
|
Tests in Docker containers
|
|
==========================
|
|
|
|
If you have a Linux system with Docker installed, running integration tests using the same Docker containers used by
|
|
the Ansible continuous integration (CI) system is recommended.
|
|
|
|
.. note:: Docker on non-Linux
|
|
|
|
Using Docker Engine to run Docker on a non-Linux host (such as macOS) is not recommended.
|
|
Some tests may fail, depending on the image used for testing.
|
|
Using the ``--docker-privileged`` option when running ``integration`` (not ``network-integration`` or ``windows-integration``) may resolve the issue.
|
|
|
|
Running Integration Tests
|
|
-------------------------
|
|
|
|
To run all CI integration test targets for POSIX platforms in a Ubuntu 16.04 container::
|
|
|
|
ansible-test integration --docker ubuntu1604 -v shippable/
|
|
|
|
You can also run specific tests or select a different Linux distribution.
|
|
For example, to run tests for the ``ping`` module on a Ubuntu 14.04 container::
|
|
|
|
ansible-test integration -v ping --docker ubuntu1404
|
|
|
|
Container Images
|
|
----------------
|
|
|
|
Python 2
|
|
````````
|
|
|
|
Most container images are for testing with Python 2:
|
|
|
|
- centos6
|
|
- centos7
|
|
- fedora28
|
|
- opensuse15py2
|
|
- ubuntu1404
|
|
- ubuntu1604
|
|
|
|
Python 3
|
|
````````
|
|
|
|
To test with Python 3 use the following images:
|
|
|
|
- fedora29
|
|
- opensuse15
|
|
- ubuntu1604py3
|
|
- ubuntu1804
|
|
|
|
|
|
Legacy Cloud Tests
|
|
==================
|
|
|
|
Some of the cloud tests run as normal integration tests, and others run as legacy tests; see the
|
|
:ref:`testing_integration_legacy` page for more information.
|
|
|
|
|
|
Other configuration for Cloud Tests
|
|
===================================
|
|
|
|
In order to run some tests, you must provide access credentials in a file named
|
|
``cloud-config-aws.yml`` or ``cloud-config-cs.ini`` in the test/integration
|
|
directory. Corresponding .template files are available for for syntax help. The newer AWS
|
|
tests now use the file test/integration/cloud-config-aws.yml
|
|
|
|
IAM policies for AWS
|
|
====================
|
|
|
|
Ansible needs fairly wide ranging powers to run the tests in an AWS account. This rights can be provided to a dedicated user. These need to be configured before running the test.
|
|
|
|
testing-policies
|
|
----------------
|
|
|
|
The GitHub repository `mattclay/aws-terminator <https://github.com/mattclay/aws-terminator/>`_
|
|
contains two sets of policies used for all existing AWS module integratoin tests.
|
|
The `hacking/aws_config/setup_iam.yml` playbook can be used to setup two groups:
|
|
|
|
- `ansible-integration-ci` will have the policies applied necessary to run any
|
|
integration tests not marked as `unsupported` and are designed to mirror those
|
|
used by Ansible's CI.
|
|
- `ansible-integration-unsupported` will have the additional policies applied
|
|
necessary to run the integraion tests marked as `unsupported` including tests
|
|
for managing IAM roles, users and groups.
|
|
|
|
Once the groups have been created, you'll need to create a user and make the user a member of these
|
|
groups. The policies are designed to minimize the rights of that user. Please note that while this policy does limit
|
|
the user to one region, this does not fully restrict the user (primarily due to the limitations of the Amazon ARN
|
|
notation). The user will still have wide privileges for viewing account definitions, and will also able to manage
|
|
some resources that are not related to testing (for example, AWS lambdas with different names). Tests should not
|
|
be run in a primary production account in any case.
|
|
|
|
Other Definitions required
|
|
--------------------------
|
|
|
|
Apart from installing the policy and giving it to the user identity running the tests, a
|
|
lambda role `ansible_integration_tests` has to be created which has lambda basic execution
|
|
privileges.
|
|
|
|
|
|
Network Tests
|
|
=============
|
|
|
|
For guidance on writing network test see :ref:`testing_resource_modules`.
|
|
|
|
|
|
Where to find out more
|
|
======================
|
|
|
|
If you'd like to know more about the plans for improving testing Ansible, join the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
|