cmueller/ansible
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

How to contribute to Ansible maintained collections (#69202)

Browse source 
contributor info with HTML table
This commit is contained in:
Sandra McCann 2020-06-15 16:38:42 -04:00 committed by GitHub
parent 80410f292b
commit 2609482975
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
5 changed files with 650 additions and 95 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

269
docs/docsite/rst/community/contributing_maintained_collections.rst Normal file
View file

@ -0,0 +1,269 @@
.. _contributing_maintained_collections:
***********************************************
Contributing to Ansible-maintained Collections
***********************************************
The Ansible team welcomes community contributions to the collections maintained by Red Hat Ansible Engineering. This section describes how you can open issues and create PRs with the required testing before your PR can be merged.
.. contents::
:local:
Ansible-maintained collections
=================================
The following table shows:
* **Ansible-maintained collection** - Click the link to the collection on Galaxy, then click the ``repo`` button in Galaxy to find the GitHub repository for this collection.
* **Related community collection** - Collection that holds community-created content (modules, roles, and so on) that may also be of interest to a user of the Ansible-maintained collection. You can, for example, add new modules to the community collection as a technical preview before the content is moved to the Ansible-maintained collection.
* **Sponsor** - Working group that manages the collections. You can join the meetings to discuss important proposed changes and enhancements to the collections.
* **Test requirements** - Testing required for any new or changed content for the Ansible-maintained collection.
* **Developer details** - Describes whether the Ansible-maintained collection accepts direct community issues and PRs for existing collection content, as well as more specific developer guidelines based on the collection type.
.. _ansible-collection-table:
.. raw:: html
<style>
/* Style for this single table. Add delimiters between header columns */
table#ansible-collection-table th {
border-width: 1px;
border-color: #dddddd /*rgb(225, 228, 229)*/;
border-style: solid;
text-align: center;
padding: 5px;
background-color: #eeeeee;
}
tr, td {
border-width: 1px;
border-color: rgb(225, 228, 229);
border-style: solid;
text-align: center;
padding: 5px;
}
</style>
<table id="ansible-collection-table">
<tr>
<th colspan="3">Collection details</th>
<th colspan="4">Test requirements: Ansible collections</th>
<th colspan="2">Developer details</th>
</tr>
<tr>
<th>Ansible collection</th>
<th>Related community collection</th>
<th>Sponsor</th>
<th>Sanity</th>
<th>Unit</th>
<th>Integration</th>
<th>CI Platform</th>
<th>Open to PRs*</th>
<th>Guidelines</th>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/amazon/aws">amazon.aws</a></td>
<td><a href="https://galaxy.ansible.com/community/aws">community.aws</a></td>
<td><a href="https://github.com/ansible/community/tree/master/group-aws">Cloud</a></td>
<td>✓</td>
<td>✓</td>
<td>✓</td>
<td>Shippable</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/devel/dev_guide/platforms/aws_guidelines.html">AWS guide</a></td>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/ansible/netcommon">ansible.netcommon**</a></td>
<td><a href="https://galaxy.ansible.com/community/network">community.network</a></td>
<td><a href="https://github.com/ansible/community/wiki/Network">Network</a></td>
<td>✓</td>
<td>✓</td>
<td>✓</td>
<td>Zuul</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/devel/network/dev_guide/index.html">Network guide</a></td>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/ansible/posix">ansible.posix</a></td>
<td><a href="https://galaxy.ansible.com/community/general">community.general</a></td>
<td>Linux</a></td>
<td>✓</td>
<td></td>
<td></td>
<td>Shippable</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/latest/dev_guide/index.html">Developer guide</a></td>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/ansible/windows">ansible.windows</a></td>
<td><a href="https://galaxy.ansible.com/community/windows">community.windows</a></td>
<td><a href="https://github.com/ansible/community/wiki/Windows">Windows</a></td>
<td>✓</td>
<td>✓***</td>
<td>✓</td>
<td>Shippable</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/devel/dev_guide/developing_modules_general_windows.html#developing-modules-general-windows">Windows guide</a></td>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/arista/eos">arista.eos</a></td>
<td><a href="https://galaxy.ansible.com/community/network">community.network</a></td>
<td><a href="https://github.com/ansible/community/wiki/Network">Network</a></td>
<td>✓</td>
<td>✓</td>
<td>✓</td>
<td>Zuul</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/devel/network/dev_guide/index.html">Network guide</a></td>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/cisco/asa">cisco.asa</a></td>
<td>community.asa</td>
<td><a href="https://github.com/ansible/community/wiki/Security-Automation">Security</a></td>
<td>✓</td>
<td>✓</td>
<td>✓</td>
<td>Zuul</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/latest/dev_guide/index.html">Developer guide</a></td>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/cisco/ios">cisco.ios</a></td>
<td><a href="https://galaxy.ansible.com/community/network">community.network</a></td>
<td><a href="https://github.com/ansible/community/wiki/Network">Network</a></td>
<td>✓</td>
<td>✓</td>
<td>✓</td>
<td>Zuul</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/devel/network/dev_guide/index.html">Network guide</a></td>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/cisco/iosxr">cisco.iosxr</a></td>
<td><a href="https://galaxy.ansible.com/community/network">community.network</a></td>
<td><a href="https://github.com/ansible/community/wiki/Network">Network</a></td>
<td>✓</td>
<td>✓</td>
<td>✓</td>
<td>Zuul</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/devel/network/dev_guide/index.html">Network guide</a></td>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/cisco/nxos">cisco.nxos</a></td>
<td><a href="https://galaxy.ansible.com/community/network">community.network</a></td>
<td><a href="https://github.com/ansible/community/wiki/Network">Network</a></td>
<td>✓</td>
<td>✓</td>
<td>✓</td>
<td>Zuul</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/devel/network/dev_guide/index.html">Network guide</a></td>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/ibm/qradar">ibm.qradar</a></td>
<td>community.qradar</td>
<td><a href="https://github.com/ansible/community/wiki/Security-Automation">Security</a></td>
<td>✓</td>
<td></td>
<td>✓</td>
<td>Zuul</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/latest/dev_guide/index.html">Developer guide</a></td>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/junipernetworks/junos">junipernetworks.junos</a></td>
<td><a href="https://galaxy.ansible.com/community/network">community.network</a></td>
<td><a href="https://github.com/ansible/community/wiki/Network">Network</a></td>
<td>✓</td>
<td>✓</td>
<td>✓</td>
<td>Zuul</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/devel/network/dev_guide/index.html">Network guide</a></td>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/openvswitch/openvswitch">openvswitch.openvswitch</a></td>
<td><a href="https://galaxy.ansible.com/community/network">community.network</a></td>
<td><a href="https://github.com/ansible/community/wiki/Network">Network</a></td>
<td>✓</td>
<td>✓</td>
<td>✓</td>
<td>Zuul</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/devel/network/dev_guide/index.html">Network guide</a></td>
</tr>
<tr>
<td><a href="https://github.com/ansible-collections/splunk.es">splunk.es</a></td>
<td>community.es</td>
<td><a href="https://github.com/ansible/community/wiki/Security-Automation">Security</a></td>
<td>✓</td>
<td></td>
<td>✓</td>
<td>Zuul</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/latest/dev_guide/index.html">Developer guide</a></td>
</tr>
<tr>
<td><a href="https://galaxy.ansible.com/vyos/vyos">vyos.vyos</a></td>
<td><a href="https://galaxy.ansible.com/community/network">community.network</a></td>
<td><a href="https://github.com/ansible/community/wiki/Network">Network</a></td>
<td>✓</td>
<td>✓</td>
<td>✓</td>
<td>Zuul</td>
<td>✓</td>
<td><a href="https://docs.ansible.com/ansible/devel/network/dev_guide/index.html">Network guide</a></td>
</tr>
</table>
.. note::
\* A ✓ under **Open to PRs** means the collection welcomes GitHub issues and PRs for any changes to existing collection content (plugins, roles, and so on).
\*\* ``ansible.netcommon`` contains all foundational components for enabling many network and security :ref:`platform <platform_options>` collections. It contains all connection and filter plugins required, and installs as a dependency when you install the the platform collection.
\*\*\* Unit tests for Windows PowerShell modules are an exception to testing, but unit tests are valid and required for the remainder of the collection, including Ansible-side plugins.
.. _which_collection:
Deciding where your contribution belongs
=========================================
We welcome contributions to Ansible-maintained collections. Because these collections are part of a downstream supported Red Hat product, the criteria for contribution, testing, and release may be higher than other community collections. The related community collections (such as ``community.general`` and ``community.network``) have less-stringent requirements and are a great place for new functionality that may become part of the Ansible-maintained collection in a future release.
The following scenarios use the ``arista.eos`` to help explain when to contribute to the Ansible-maintained collection, and when to propose your change or idea to the related community collection:
1. You want to fix a problem in the ``arista.eos`` Ansible-maintained collection. Create the PR directly in the `arista.eos collection GitHub repository <https://github.com/ansible-collections/arista.eos>`_. Apply all the :ref:`merge requirements <ansible_collection_merge_requirements>`.
2. You want to add a new Ansible module for Arista. Your options are one of the following:
* Propose a new module in the ``arista.eos`` collection (requires approval from Arista and Red Hat).
* Propose a new collection in the ``arista`` namespace (requires approval from Arista and Red Hat).
* Propose a new module in the ``community.network`` collection (requires network community approval).
* Place your new module in a collection in your own namespace (no approvals required).
Most new content should go into either a related community collection or your own collection first so that is well established in the community before you can propose adding it to the ``arista`` namespace, where inclusion and maintenance criteria are much higher.
.. _ansible_collection_merge_requirements:
Requirements to merge your PR
==============================
Your PR must meet the following requirements before it can merge into an Ansible-maintained collection:
#. The PR is in the intended scope of the collection. Communicate with the appropriate Ansible sponsor listed in the :ref:`Ansible-maintained collection table <ansible-collection-table>` for help.
#. For network and security domains, the PR follows the :ref:`resource module development principles <developing_resource_modules>`.
#. Passes :ref:`sanity tests and tox <tox_resource_modules>`.
#. Passes unit, and integration tests, as listed in the :ref:`Ansible-maintained collection table <ansible-collection-table>` and described in :ref:`testing_resource_modules`.
#. Follows Ansible guidelines. See :ref:`developing_modules` and :ref:`developing_collections`.
#. Addresses all review comments.
#. Includes an appropriate :ref:`changelog <community_changelogs>`.

2
docs/docsite/rst/community/index.rst
View file

@ -35,6 +35,7 @@ Going deeper
* I think Ansible is broken. How do I :ref:`report a bug <reporting_bugs>`?
* I need functionality that Ansible doesn't offer. How do I :ref:`request a feature <request_features>`?
* How do I :ref:`contribute to an Ansible-maintained collection <contributing_maintained_collections>`?
* I'm waiting for a particular feature. How do I see what's :ref:`planned for future Ansible Releases <roadmaps>`?
* I have a specific Ansible interest or expertise (for example, VMware, Linode, and so on.). How do I get involved in a :ref:`working group <working_group_list>`?
* I'd like to participate in conversations about features and fixes. How do I review GitHub issues and pull requests?
@ -69,6 +70,7 @@ If you prefer to read the entire Community Guide, here's a list of the pages in
documentation_contributions
communication
development_process
contributing_maintained_collections
contributor_license_agreement
triage_process
other_tools_and_programs

2
docs/docsite/rst/dev_guide/index.rst
View file

@ -17,6 +17,7 @@ Find the task that best describes what you want to do:
* I want to :ref:`add a custom plugin or module locally <developing_locally>`.
* I want to figure out if :ref:`developing a module is the right approach <module_dev_should_you>` for my use case.
* I want to :ref:`develop a collection <developing_collections>`.
* I want to :ref:`contribute to an Ansible-maintained collection <contributing_maintained_collections>`.
* I want to :ref:`migrate a role to a collection <migrating_roles>`.
* I've read the info above, and I'm sure I want to develop a module:
@ -84,6 +85,7 @@ If you prefer to read the entire guide, here's a list of the pages in order.
developing_rebasing
developing_module_utilities
developing_collections
migrating_roles
collections_galaxy_meta
migrating_roles
overview_architecture

56
docs/docsite/rst/dev_guide/testing_integration.rst
View file

@ -210,61 +210,7 @@ privileges.
Network Tests
=============
Starting with Ansible 2.4, all network modules MUST include unit tests that cover all functionality. You must add unit tests for each new network module and for each added feature. Please submit the unit tests and the code in a single PR. Integration tests are also strongly encouraged.
Writing network integration tests
---------------------------------
For guidance on writing network test see the `adding tests for Network modules guide <https://github.com/ansible/community/blob/master/group-network/network_test.rst>`_.
Running network integration tests locally
-----------------------------------------
Ansible uses Shippable to run an integration test suite on every PR, including new tests introduced by that PR. To find and fix problems in network modules, run the network integration test locally before you submit a PR.
To run the network integration tests, use a command in the form::
ansible-test network-integration --inventory /path/to/inventory tests_to_run
First, define a network inventory file::
cd test/integration
cp inventory.network.template inventory.networking
${EDITOR:-vi} inventory.networking
# Add in machines for the platform(s) you wish to test
To run all Network tests for a particular platform::
ansible-test network-integration --inventory /path/to/ansible/test/integration/inventory.networking vyos_.*
This example will run against all vyos modules. Note that ``vyos_.*`` is a regex match, not a bash wildcard - include the `.` if you modify this example.
To run integration tests for a specific module::
ansible-test network-integration --inventory /path/to/ansible/test/integration/inventory.networking vyos_vlan
To run a single test case on a specific module::
# Only run vyos_vlan/tests/cli/basic.yaml
ansible-test network-integration --inventory /path/to/ansible/test/integration/inventory.networking vyos_vlan --testcase basic
To run integration tests for a specific transport::
# Only run nxapi test
ansible-test network-integration --inventory /path/to/ansible/test/integration/inventory.networking --tags="nxapi" nxos_.*
# Skip any cli tests
ansible-test network-integration --inventory /path/to/ansible/test/integration/inventory.networking --skip-tags="cli" nxos_.*
See `test/integration/targets/nxos_bgp/tasks/main.yaml <https://github.com/ansible/ansible/blob/devel/test/integration/targets/nxos_bgp/tasks/main.yaml>`_ for how this is implemented in the tests.
For more options::
ansible-test network-integration --help
If you need additional help or feedback, reach out in ``#ansible-network`` on Freenode.
For guidance on writing network test see :ref:`testing_resource_modules`.
Where to find out more

416
docs/docsite/rst/network/dev_guide/developing_resource_modules_network.rst
View file

@ -6,9 +6,83 @@ Developing network resource modules
***********************************
.. contents::
:local:
:local:
:depth: 2
The resource module builder is an Ansible Playbook that helps developers scaffold and maintain an Ansible network resource module.
Understanding network and security resource modules
===================================================
Network and security devices separate configuration into sections (such as interfaces, VLANs, and so on) that apply to a network or security service. Ansible resource modules take advantage of this to allow users to configure subsections or resources within the device configuration. Resource modules provide a consistent experience across different network and security devices. For example, a network resource module may only update the configuration for a specific portion of the network interfaces, VLANs, ACLs, and so on, for a network device. The resource module:
#. Fetches a piece of the configuration (fact gathering), for example, the interfaces configuration.
#. Converts the returned configuration into key-value pairs.
#. Places those key-value pairs into an internal agnostic structured data format.
Now that the configuration data is normalized, the user can update and modify the data and then use the resource module to send the configuration data back to the device. This results in a full round-trip configuration update without the need for manual parsing, data manipulation, and data model management.
The resource module has two top-level keys - ``config`` and ``state``:
* ``config`` defines the resource configuration data model as key-value pairs. The type of the ``config`` option can be ``dict`` or ``list of dict`` based on the resource managed. That is, if the device has a single global configuration, it should be a ``dict`` (for example, a global LLDP configuration). If the device has multiple instances of configuration, it should be of type ``list`` with each element in the list of type ``dict`` (for example, interfaces configuration).
* ``state`` defines the action the resource module takes on the end device.
The ``state`` for a new resource module should support the following values (as applicable for the devices that support them):
merged
Ansible merges the on-device configuration with the provided configuration in the task.
replaced
Ansible replaces the on-device configuration subsection with the provided configuration subsection in the task.
overridden
Ansible overrides the on-device configuration for the resource with the provided configuration in the task. Use caution with this state as you could remove your access to the device (for example, by overriding the management interface configuration).
deleted
Ansible deletes the on-device configuration subsection and restores any default settings.
gathered
Ansible displays the resource details gathered from the network device and accessed with the ``gathered`` key in the result.
rendered
Ansible renders the provided configuration in the task in the device-native format (for example, Cisco IOS CLI). Ansible returns this rendered configuration in the ``rendered`` key in the result. Note this state does not communicate with the network device and can be used offline.
parsed
Ansible parses the configuration from the ``running_configuration`` option into Ansible structured data in the ``parsed`` key in the result. Note this does not gather the configuration from the network device so this state can be used offline.
Modules in Ansible-maintained collections must support these state values. If you develop a module with only "present" and "absent" for state, you may submit it to a community collection.
.. note::
The states ``rendered``, ``gathered``, and ``parsed`` do not perform any change on the device.
.. seealso::
`Deep Dive on VLANs Resource Modules for Network Automation <https://www.ansible.com/blog/deep-dive-on-vlans-resource-modules-for-network-automation>`_
Walkthrough of how state values are implemented for VLANs.
Developing network and security resource modules
=================================================
The Ansible Engineering team ensures the module design and code pattern within Ansible-maintained collections is uniform across resources and across platforms to give a vendor-agnostic feel and deliver good quality code. We recommend you use the `resource module builder <https://github.com/ansible-network/resource_module_builder>`_ to develop a resource module.
The highlevel process for developing a resource module is:
#. Create and share a resource model design in the `resource module models repository <https://github.com/ansible-network/resource_module_models>`_ as a PR for review.
#. Download the latest version of the `resource module builder <https://github.com/ansible-network/resource_module_builder>`_.
#. Run the ``resource module builder`` to create a collection scaffold from your approved resource model.
#. Write the code to implement your resource module.
#. Develop integration and unit tests to verify your resource module.
#. Create a PR to the appropriate collection that you want to add your new resource module to. See :ref:`contributing_maintained_collections` for details on determining the correct collection for your module.
Understanding the model and resource module builder
-----------------------------------------------------
The resource module builder is an Ansible Playbook that helps developers scaffold and maintain an Ansible resource module. It uses a model as the single source of truth for the module. This model is a ``yaml`` file that is used for the module DOCUMENTATION section and the argument spec.
The resource module builder has the following capabilities:
@ -20,7 +94,7 @@ The resource module builder has the following capabilities:
- Generates working sample modules for both ``<network_os>_<resource>`` and ``<network_os>_facts``.
Accessing the resource module builder
=====================================
-------------------------------------
To access the resource module builder:
@ -37,15 +111,15 @@ To access the resource module builder:
pip install -r requirements.txt
Creating a model
================
-----------------
You must create a model for your new resource. The resource module builder uses this model to create:
You must create a model for your new resource. The model is the single source of truth for both the argspec and docstring, keeping them in sync. Once your model is approved, you can use the resource module builder to generate three items based on the model:
* The scaffold for a new module
* The argspec for the new module
* The docstring for the new module
The model is then the single source of truth for both the argspec and docstring, keeping them in sync. Use the resource module builder to generate this scaffolding. For any subsequent updates to the module, update the model first and use the resource module builder to update the module argspec and docstring.
For any subsequent changes to the functionality, update the model first and use the resource module builder to update the module argspec and docstring.
For example, the resource model builder includes the ``myos_interfaces.yml`` sample in the :file:`models` directory, as seen below:
@ -53,6 +127,7 @@ For example, the resource model builder includes the ``myos_interfaces.yml`` sam
---
GENERATOR_VERSION: '1.0'
NETWORK_OS: myos
RESOURCE: interfaces
COPYRIGHT: Copyright 2019 Red Hat
@ -60,7 +135,7 @@ For example, the resource model builder includes the ``myos_interfaces.yml`` sam
DOCUMENTATION: |
module: myos_interfaces
version_added: 2.9
version_added: 1.0.0
short_description: 'Manages <xxxx> attributes of <network_os> <resource>'
description: 'Manages <xxxx> attributes of <network_os> <resource>.'
author: Ansible Network Engineer
@ -120,12 +195,13 @@ For example, the resource model builder includes the ``myos_interfaces.yml`` sam
Notice that you should include examples for each of the states that the resource supports. The resource module builder also includes these in the sample model.
See `Ansible network resource models <https://github.com/ansible-network/resource_module_models>`_ for more examples.
Share this model as a PR for review at `resource module models repository <https://github.com/ansible-network/resource_module_models>`_. You can also see more model examples at that location.
Using the resource module builder
=================================
To use the resource module builder to create a collection scaffold from your resource model:
Creating a collection scaffold from a resource model
----------------------------------------------------
To use the resource module builder to create a collection scaffold from your approved resource model:
.. code-block:: bash
@ -329,7 +405,7 @@ The resource module structure includes the following components:
Module
* ``library/<ansible_network_os>_<resource>.py``.
* Imports the ``module_utils`` resource package and calls ``execute_module`` API
* Imports the ``module_utils`` resource package and calls ``execute_module`` API:
.. code-block:: text
@ -357,8 +433,27 @@ Utils
* ``module_utils/<ansible_network_os>/utils``.
* Utilities for the ``<ansible_network_os>`` platform.
Developer notes
===============
.. _tox_resource_modules:
Running ``ansible-test sanity`` and ``tox`` on resource modules
================================================================
You should run ``ansible-test sanity`` and ``tox -elinters`` from the collection root directory before pushing your PR to an Ansible-maintained collection. The CI runs both and will fail if these tests fail. See :ref:`developing_testing` for details on ``ansible-test sanity``.
To install the necessary packages:
#. Ensure you have a valid Ansible development environment configured. See :ref:`environment_setup` for details.
#. Run ``pip install -r requirements.txt`` from the collection root directory.
Running ``tox -elinters``:
* Reads :file:`tox.ini` from the collection root directory and installs required dependencies (such as ``black`` and ``flake8``).
* Runs these with preconfigured options (such as line-length and ignores.)
* Runs ``black`` in check mode to show which files will be formatted without actually formatting them.
Testing resource modules
========================
The tests rely on a role generated by the resource module builder. After changes to the resource module builder, the role should be regenerated and the tests modified and run as needed. To generate the role after changes:
@ -373,28 +468,260 @@ The tests rely on a role generated by the resource module builder. After changes
.. _testing_resource_modules:
Resource module integration tests
----------------------------------
Unit testing Ansible network resource modules
=============================================
High-level integration test requirements for new resource modules are as follows:
#. Write a test case for every state.
#. Write additional test cases to test the behavior of the module when an empty ``config.yaml`` is given.
#. Add a round trip test case. This involves a ``merge`` operation, followed by ``gather_facts``, a ``merge`` update with additional configuration, and then reverting back to the base configuration using the previously gathered facts with the ``state`` set to ``overridden``.
#. Wherever applicable, assertions should check after and before ``dicts`` against a hard coded Source of Truth.
.. _using_zuul_resource_modules:
We use Zuul as the CI to run the integration test.
* To view the report, click :guilabel:`Details` on the CI comment in the PR
* To view a failure report, click :guilabel:`ansible/check` and select the failed test.
* To view logs while the test is running, check for your PR number in the `Zull status board <https://dashboard.zuul.ansible.com/t/ansible/status>`_.
* To fix static test failure locally, run the :command:`tox -e black` **inside the root folder of collection**.
To view The Ansible run logs and debug test failures:
#. Click the failed job to get the summary, and click :guilabel:`Logs` for the log.
#. Click :guilabel:`console` and scroll down to find the failed test.
#. Click :guilabel:`>` next to the failed test for complete details.
This section walks through an example of how to develop unit tests for Ansible network resource
Integration test structure
...........................
Each test case should generally follow this pattern:
* setup —> test —> assert —> test again (for idempotency) —> assert —> tear down (if needed) -> done. This keeps test playbooks from becoming monolithic and difficult to troubleshoot.
* Include a name for each task that is not an assertion. You can add names to assertions as well, but it is easier to identify the broken task within a failed test if you add a name for each task.
* Files containing test cases must end in ``.yaml``
Implementation
..............
For platforms that support ``connection: local`` *and* ``connection: network_cli`` use the following guidance:
* Name the :file:`targets/` directories after the module name.
* The :file:`main.yaml` file should just reference the transport.
The following example walks through the integration tests for the ``vyos.vyos.vyos_banner`` module in the `vyos.vyos <https://github.com/ansible-collections/vyos.vyos/tree/master/tests/integration>`_ collection:
``test/integration/targets/vyos_banner/tasks/main.yaml``
.. code-block:: yaml
---
- include: cli.yaml
tags:
- cli
``test/integration/targets/vyos_banner/tasks/cli.yaml``
.. code-block:: yaml
---
- name: collect all cli test cases
find:
paths: "{{ role_path }}/tests/cli"
patterns: "{{ testcase }}.yaml"
register: test_cases
delegate_to: localhost
- name: set test_items
set_fact: test_items="{{ test_cases.files | map(attribute='path') | list }}"
- name: run test cases (connection=network_cli)
include: "{{ test_case_to_run }} ansible_connection=network_cli"
with_items: "{{ test_items }}"
loop_control:
loop_var: test_case_to_run
- name: run test case (connection=local)
include: "{{ test_case_to_run }} ansible_connection=local ansible_become=no"
with_first_found: "{{ test_items }}"
loop_control:
loop_var: test_case_to_run
``test/integration/targets/vyos_banner/tests/cli/basic-no-login.yaml``
.. code-block:: yaml
---
- debug:
msg: "cli/basic-no-login.yaml on connection={{ ansible_connection }}"
- name: Setup
vyos_banner:
banner: pre-login
text: |
Junk pre-login banner
over multiple lines
state: present
- name: remove pre-login
vyos_banner:
banner: pre-login
state: absent
register: result
- debug:
msg: "{{ result }}"
- assert:
that:
- "result.changed == true"
- "'delete system login banner pre-login' in result.commands"
- name: remove pre-login (idempotent)
vyos_banner:
banner: pre-login
state: absent
register: result
- assert:
that:
- "result.changed == false"
- "result.commands | length == 0"
Detecting test resources at runtime
...................................
Your tests should detect resources (such as interfaces) at runtime rather than hard-coding them into the test. This allows the test to run on a variety of systems.
For example:
.. code-block:: yaml
- name: Collect interface list
connection: ansible.netcommon.network_cli
register: intout
cisco.nxos.nxos_command:
commands:
- show interface brief | json
- set_fact:
intdataraw: "{{ intout.stdout_lines[0]['TABLE_interface']['ROW_interface'] }}"
- set_fact:
nxos_int1: '{{ intdataraw[1].interface }}'
- set_fact:
nxos_int2: '{{ intdataraw[2].interface }}'
- set_fact:
nxos_int3: '{{ intdataraw[3].interface }}'
See the complete test example of this at https://github.com/ansible-collections/cisco.nxos/blob/master/tests/integration/targets/prepare_nxos_tests/tasks/main.yml.
Running network integration tests
..................................
Ansible uses Zuul to run an integration test suite on every PR, including new tests introduced by that PR. To find and fix problems in network modules, run the network integration test locally before you submit a PR.
First, create an inventory file that points to your test machines. The inventory group should match the platform name (for example, ``eos``, ``ios``):
.. code-block:: bash
cd test/integration
cp inventory.network.template inventory.networking
${EDITOR:-vi} inventory.networking
# Add in machines for the platform(s) you wish to test
To run these network integration tests, use ``ansible-test network-integration --inventory </path/to/inventory> <tests_to_run>``:
.. code-block:: console
ansible-test network-integration --inventory ~/myinventory -vvv vyos_facts
ansible-test network-integration --inventory ~/myinventory -vvv vyos_.*
To run all network tests for a particular platform:
.. code-block:: bash
ansible-test network-integration --inventory /path/to-collection-module/test/integration/inventory.networking vyos_.*
This example will run against all ``vyos`` modules. Note that ``vyos_.*`` is a regex match, not a bash wildcard - include the `.` if you modify this example.
To run integration tests for a specific module:
.. code-block:: bash
ansible-test network-integration --inventory /path/to-collection-module/test/integration/inventory.networking vyos_vlan
To run a single test case on a specific module:
.. code-block:: bash
# Only run vyos_vlan/tests/cli/basic.yaml
ansible-test network-integration --inventory /path/to-collection-module/test/integration/inventory.networking vyos_vlan --testcase basic
To run integration tests for a specific transport:
.. code-block:: bash
# Only run nxapi test
ansible-test network-integration --inventory /path/to-collection-module/test/integration/inventory.networking --tags="nxapi" nxos_.*
# Skip any cli tests
ansible-test network-integration --inventory /path/to-collection-module/test/integration/inventory.networking --skip-tags="cli" nxos_.*
See `test/integration/targets/nxos_bgp/tasks/main.yaml <https://github.com/ansible-collections/cisco.nxos/blob/master/tests/integration/targets/nxos_bgp/tasks/main.yaml>`_ for how this is implemented in the tests.
For more options:
.. code-block:: bash
ansible-test network-integration --help
If you need additional help or feedback, reach out in ``#ansible-network`` on Freenode.
Unit test requirements
-----------------------
High-level unit test requirements that new resource modules should follow:
#. Write test cases for all the states with all possible combinations of config values.
#. Write test cases to test the error conditions ( negative scenarios).
#. Check the value of ``changed`` and ``commands`` keys in every test case.
We run all unit test cases on our Zuul test suite, on the latest python version supported by our CI setup.
Use the :ref:`same procedure <using_zuul_resource_modules>` as the integration tests to view Zuul unit tests reports and logs.
See :ref:`unit module testing <testing_units_modules>` for general unit test details.
.. end of cut n .. parsed-literal::
Example: Unit testing Ansible network resource modules
======================================================
This section walks through an example of how to develop unit tests for Ansible resource
modules.
See :ref:`testing_units` and :ref:`testing_units_modules` for general documentation on Ansible unit tests for modules.
Please read those pages first to understand unit tests and why and when you should use them.
.. note::
The structure of the unit tests matches
the structure of the code base, so the tests that reside in the :file:`test/units/modules/network` directory
are organized by module groups.
Using mock objects to unit test Ansible network resource modules
----------------------------------------------------------------
Mock objects (from https://docs.python.org/3/library/unittest.mock.html) can be very
`Mock objects <https://docs.python.org/3/library/unittest.mock.html>`_ can be very
useful in building unit tests for special or difficult cases, but they can also
lead to complex and confusing coding situations. One good use for mocks would be to
simulate an API. The ``mock`` Python package is bundled with Ansible (use
@ -404,23 +731,35 @@ You can mock the device connection and output from the device as follows:
.. code-block:: python
self.mock_get_config = patch('ansible.module_utils.network.common.network.Config.get_config')
self.get_config = self.mock_get_config.start()
self.mock_get_config = patch( "ansible_collections.ansible.netcommon.plugins.module_utils.network.common.network.Config.get_config"
)
self.get_config = self.mock_get_config.start()
self.mock_load_config = patch('ansible.module_utils.network.common.network.Config.load_config')
self.load_config = self.mock_load_config.start()
self.mock_load_config = patch(
"ansible_collections.ansible.netcommon.plugins.module_utils.network.common.network.Config.load_config"
)
self.load_config = self.mock_load_config.start()
self.mock_get_resource_connection_config = patch('ansible.module_utils.network.common.cfg.base.get_resource_connection')
self.get_resource_connection_config = self.mock_get_resource_connection_config.start()
self.mock_get_resource_connection_config = patch(
"ansible_collections.ansible.netcommon.plugins.module_utils.network.common.cfg.base.get_resource_connection"
)
self.get_resource_connection_config = (self.mock_get_resource_connection_config.start())
self.mock_get_resource_connection_facts = patch('ansible.module_utils.network.common.facts.facts.get_resource_connection')
self.get_resource_connection_facts = self.mock_get_resource_connection_facts.start()
self.mock_get_resource_connection_facts = patch(
"ansible_collections.ansible.netcommon.plugins.module_utils.network.common.facts.facts.get_resource_connection"
)
self.get_resource_connection_facts = (self.mock_get_resource_connection_facts.start())
self.mock_edit_config = patch('ansible.module_utils.network.eos.providers.providers.CliProvider.edit_config')
self.edit_config = self.mock_edit_config.start()
self.mock_edit_config = patch(
"ansible_collections.arista.eos.plugins.module_utils.network.eos.providers.providers.CliProvider.edit_config"
)
self.edit_config = self.mock_edit_config.start()
self.mock_execute_show_command = patch(
"ansible_collections.arista.eos.plugins.module_utils.network.eos.facts.l2_interfaces.l2_interfaces.L2_interfacesFacts.get_device_data"
)
self.execute_show_command = self.mock_execute_show_command.start()
self.mock_execute_show_command = patch('ansible.module_utils.network.eos.facts.l2_interfaces.l2_interfaces.L2_interfacesFacts.get_device_data')
self.execute_show_command = self.mock_execute_show_command.start()
The facts file of the module now includes a new method, ``get_device_data``. Call ``get_device_data`` here to emulate the device output.
@ -429,7 +768,7 @@ Mocking device data
-----------------------
To mock fetching results from devices or provide other complex data structures that
come from external libraries, you can use ``fixtures`` to read in pre-generated data. The text files for this pre-generated data live in ``test/units/modules/network/PLATFORM/fixtures/``. See for example the `eos_l2_interfaces.cfg file <https://github.com/ansible/ansible/blob/devel/test/units/modules/network/eos/fixtures/eos_l2_interfaces_config.cfg>`_.
come from external libraries, you can use ``fixtures`` to read in pre-generated data. The text files for this pre-generated data live in ``test/units/modules/network/PLATFORM/fixtures/``. See for example the `eos_l2_interfaces.cfg file <https://github.com/ansible-collections/arista.eos/blob/master/tests/unit/modules/network/eos/fixtures/eos_l2_interfaces_config.cfg>`_.
Load data using the ``load_fixture`` method and set this data as the return value of the
``get_device_data`` method in the facts file:
@ -441,15 +780,12 @@ Load data using the ``load_fixture`` method and set this data as the return valu
return load_fixture('eos_l2_interfaces_config.cfg')
self.execute_show_command.side_effect = load_from_file
See the unit test file `test_eos_l2_interfaces
<https://github.com/ansible/ansible/blob/devel/test/units/modules/network/eos/test_eos_l2_interfaces.py>`_
See the unit test file `test_eos_l2_interfaces <https://github.com/ansible-collections/arista.eos/blob/master/tests/unit/modules/network/eos/test_eos_l2_interfaces.py>`_
for a practical example.
.. seealso::
:ref:`testing_units`
Ansible unit tests documentation
:ref:`testing_units`
Deep dive into developing unit tests for Ansible modules
:ref:`testing_running_locally`