From b739bbe0a5a0fe1fa1401020bf6f47f665beb943 Mon Sep 17 00:00:00 2001 From: Sandra McCann Date: Mon, 21 Sep 2020 16:42:39 -0400 Subject: [PATCH] [Docs][backport][2.10] Backportapalooza 12 (#71842) * Fix typo in the documentation (#71701) Fix typo in the documentation: casting instead of casing (cherry picked from commit 1a06587f3b89e967b00b6306c47779fba4657da0) * Add how to run unit test link in testing_units_modules doc (#71523) * Add how to run unit test link in testing_units_modules * Fix sanity test (cherry picked from commit 7a0e5457005cd23064bd39e7f563351828b1fe67) * Fix typo in delveloping_plugins_network (#71737) (cherry picked from commit 4bf61f0714fde612811f863f46a4d90f5caa7254) * Fix broken bullet list (#71728) (cherry picked from commit 00ed5b1f2e2a0af3a4c291745dc0e193a89f42c1) * vmware: Add docs for filters (#71670) Add a scenario guide for filters in VMware documentation Signed-off-by: Abhijeet Kasurde (cherry picked from commit 760334303bfc6cf6d6e070fc139953f10443a3c9) * [docs] add porting guide for DNF GPG validation (#71640) * [docs] add porting guide for DNF GPG validation Change: - This was a breaking change (security fix), but I neglected to add a porting guide entry for it previously. Tickets: - Refs #71537 - Refs #71539 - Refs #71540 - Refs #71541 Signed-off-by: Rick Elrod * changes from sivel Signed-off-by: Rick Elrod (cherry picked from commit 7a38c470bad6195ec29f1701f4a1ce91d34f9a06) * Fixed invalid urls inside guide_packet.rst and collections_using.rst (#71705) * Fixed invalid urls inside guide_packet.rst and collections_using.rst * Reverted fix for collections_using.rst (cherry picked from commit c36e939414327e099171f51d7ae630a736a3b14c) * Update EXAMPLES in package_facts.py documentation (#71838) this module is not limited to rpm , so remove rpm in tasks name (cherry picked from commit 7f62b4733d0bd3b021e8276ce3a6193f5fc211f9) * change duplicated label (cherry picked from commit bcfead8e0f9d7a74b1ef82e8d86e469e7d4aa473) Co-authored-by: Guillaume Vincent Co-authored-by: Amin Vakil Co-authored-by: Shufeng Co-authored-by: Evaristo Rojas Co-authored-by: Abhijeet Kasurde Co-authored-by: Rick Elrod Co-authored-by: Shounak <25407872+shounak1@users.noreply.github.com> Co-authored-by: roumano --- .../developing_program_flow_modules.rst | 2 +- docs/docsite/rst/dev_guide/testing.rst | 26 +++ .../dev_guide/developing_plugins_network.rst | 2 +- .../rst/porting_guides/porting_guide_2.8.rst | 2 + .../rst/porting_guides/porting_guide_2.9.rst | 2 + .../porting_guide_base_2.10.rst | 2 + .../rst/scenario_guides/guide_packet.rst | 2 +- .../rst/scenario_guides/guide_vmware.rst | 1 + .../vmware_inventory_filters.rst | 216 ++++++++++++++++++ .../vmware_inventory_hostnames.rst | 11 +- .../rst/user_guide/intro_getting_started.rst | 3 - .../rst/user_guide/intro_inventory.rst | 4 +- .../rst/user_guide/playbooks_variables.rst | 4 +- lib/ansible/modules/package_facts.py | 4 +- 14 files changed, 264 insertions(+), 17 deletions(-) create mode 100644 docs/docsite/rst/scenario_guides/vmware_scenarios/vmware_inventory_filters.rst diff --git a/docs/docsite/rst/dev_guide/developing_program_flow_modules.rst b/docs/docsite/rst/dev_guide/developing_program_flow_modules.rst index 22c60cf9a1d..efceffb540f 100644 --- a/docs/docsite/rst/dev_guide/developing_program_flow_modules.rst +++ b/docs/docsite/rst/dev_guide/developing_program_flow_modules.rst @@ -579,7 +579,7 @@ type * bytes * bits -The ``raw`` type, performs no type validation or type casing, and maintains the type of the passed value. +The ``raw`` type, performs no type validation or type casting, and maintains the type of the passed value. elements """""""" diff --git a/docs/docsite/rst/dev_guide/testing.rst b/docs/docsite/rst/dev_guide/testing.rst index 024f59bb9fd..763f1672399 100644 --- a/docs/docsite/rst/dev_guide/testing.rst +++ b/docs/docsite/rst/dev_guide/testing.rst @@ -174,6 +174,32 @@ Some ideas of what to test are: * Test to see if any Python backtraces returned (that's a bug) * Test on different operating systems, or against different library versions +Run sanity tests +```````````````` + +.. code:: shell + + ansible-test sanity + +More information: :ref:`testing_sanity` + +Run unit tests +`````````````` + +.. code:: shell + + ansible-test units + +More information: :ref:`testing_units` + +Run integration tests +````````````````````` + +.. code:: shell + + ansible-test integration -v ping + +More information: :ref:`testing_integration` Any potential issues should be added as comments on the pull request (and it's acceptable to comment if the feature works as well), remembering to include the output of ``ansible --version`` diff --git a/docs/docsite/rst/network/dev_guide/developing_plugins_network.rst b/docs/docsite/rst/network/dev_guide/developing_plugins_network.rst index 9c68c60b486..40ab93cb452 100644 --- a/docs/docsite/rst/network/dev_guide/developing_plugins_network.rst +++ b/docs/docsite/rst/network/dev_guide/developing_plugins_network.rst @@ -9,7 +9,7 @@ Network connection plugins Each network connection plugin has a set of its own plugins which provide a specification of the connection for a particular set of devices. The specific plugin used is selected at runtime based on the value of the ``ansible_network_os`` variable assigned to the host. This variable should be -set to the same value as the name of the plugin to be loaed. Thus, ``ansible_network_os=nxos`` +set to the same value as the name of the plugin to be loaded. Thus, ``ansible_network_os=nxos`` will try to load a plugin in a file named ``nxos.py``, so it is important to name the plugin in a way that will be sensible to users. diff --git a/docs/docsite/rst/porting_guides/porting_guide_2.8.rst b/docs/docsite/rst/porting_guides/porting_guide_2.8.rst index 4c7904da6d4..66b3795ed22 100644 --- a/docs/docsite/rst/porting_guides/porting_guide_2.8.rst +++ b/docs/docsite/rst/porting_guides/porting_guide_2.8.rst @@ -375,6 +375,8 @@ that may occur in execution. * If you changed any tasks to specify less restrictive permissions while using 2.8.14, those changes will be unnecessary (but will do no harm) in 2.8.15. * To avoid the issue raised in CVE-2020-1736, specify a ``mode`` parameter in all file-based tasks that accept it. +* ``dnf`` and ``yum`` - As of version 2.8.15, the ``dnf`` module (and ``yum`` action when it uses ``dnf``) now correctly validates GPG signatures of packages (CVE-2020-14365). If you see an error such as ``Failed to validate GPG signature for [package name]``, please ensure that you have imported the correct GPG key for the DNF repository and/or package you are using. One way to do this is with the ``rpm_key`` module. Although we discourage it, in some cases it may be necessary to disable the GPG check. This can be done by explicitly adding ``disable_gpg_check: yes`` in your ``dnf`` or ``yum`` task. + Modules removed --------------- diff --git a/docs/docsite/rst/porting_guides/porting_guide_2.9.rst b/docs/docsite/rst/porting_guides/porting_guide_2.9.rst index ec1096497cc..463e807ac7c 100644 --- a/docs/docsite/rst/porting_guides/porting_guide_2.9.rst +++ b/docs/docsite/rst/porting_guides/porting_guide_2.9.rst @@ -78,6 +78,8 @@ Modules * If you changed any tasks to specify less restrictive permissions while using 2.9.12, those changes will be unnecessary (but will do no harm) in 2.9.13. * To avoid the issue raised in CVE-2020-1736, specify a ``mode`` parameter in all file-based tasks that accept it. +* ``dnf`` and ``yum`` - As of version 2.9.13, the ``dnf`` module (and ``yum`` action when it uses ``dnf``) now correctly validates GPG signatures of packages (CVE-2020-14365). If you see an error such as ``Failed to validate GPG signature for [package name]``, please ensure that you have imported the correct GPG key for the DNF repository and/or package you are using. One way to do this is with the ``rpm_key`` module. Although we discourage it, in some cases it may be necessary to disable the GPG check. This can be done by explicitly adding ``disable_gpg_check: yes`` in your ``dnf`` or ``yum`` task. + Renaming from ``_facts`` to ``_info`` -------------------------------------- diff --git a/docs/docsite/rst/porting_guides/porting_guide_base_2.10.rst b/docs/docsite/rst/porting_guides/porting_guide_base_2.10.rst index 64e4c4e644b..bc35e81fdc5 100644 --- a/docs/docsite/rst/porting_guides/porting_guide_base_2.10.rst +++ b/docs/docsite/rst/porting_guides/porting_guide_base_2.10.rst @@ -55,6 +55,8 @@ Modules * If you changed any tasks to specify less restrictive permissions while using 2.10.0, those changes will be unnecessary (but will do no harm) in 2.10.1. * To avoid the issue raised in CVE-2020-1736, specify a ``mode`` parameter in all file-based tasks that accept it. +* ``dnf`` and ``yum`` - As of version 2.10.1, the ``dnf`` module (and ``yum`` action when it uses ``dnf``) now correctly validates GPG signatures of packages (CVE-2020-14365). If you see an error such as ``Failed to validate GPG signature for [package name]``, please ensure that you have imported the correct GPG key for the DNF repository and/or package you are using. One way to do this is with the ``rpm_key`` module. Although we discourage it, in some cases it may be necessary to disable the GPG check. This can be done by explicitly adding ``disable_gpg_check: yes`` in your ``dnf`` or ``yum`` task. + Noteworthy module changes ------------------------- diff --git a/docs/docsite/rst/scenario_guides/guide_packet.rst b/docs/docsite/rst/scenario_guides/guide_packet.rst index 99e9513f6e3..c08eb9477ef 100644 --- a/docs/docsite/rst/scenario_guides/guide_packet.rst +++ b/docs/docsite/rst/scenario_guides/guide_packet.rst @@ -123,7 +123,7 @@ You can also identify specific Packet devices with the 'device_ids' parameter. T More Complex Playbooks ====================== -In this example, we'll create a CoreOS cluster with `user data `_. +In this example, we'll create a CoreOS cluster with `user data `_. The CoreOS cluster will use `etcd `_ for discovery of other servers in the cluster. Before provisioning your servers, you'll need to generate a discovery token for your cluster: diff --git a/docs/docsite/rst/scenario_guides/guide_vmware.rst b/docs/docsite/rst/scenario_guides/guide_vmware.rst index 3a257e0b36b..b31553d552b 100644 --- a/docs/docsite/rst/scenario_guides/guide_vmware.rst +++ b/docs/docsite/rst/scenario_guides/guide_vmware.rst @@ -19,6 +19,7 @@ To get started, please select one of the following topics. vmware_scenarios/vmware_inventory vmware_scenarios/vmware_inventory_vm_attributes vmware_scenarios/vmware_inventory_hostnames + vmware_scenarios/vmware_inventory_filters vmware_scenarios/vmware_scenarios vmware_scenarios/vmware_troubleshooting vmware_scenarios/vmware_external_doc_links diff --git a/docs/docsite/rst/scenario_guides/vmware_scenarios/vmware_inventory_filters.rst b/docs/docsite/rst/scenario_guides/vmware_scenarios/vmware_inventory_filters.rst new file mode 100644 index 00000000000..1208dcadaee --- /dev/null +++ b/docs/docsite/rst/scenario_guides/vmware_scenarios/vmware_inventory_filters.rst @@ -0,0 +1,216 @@ +.. _vmware_ansible_inventory_using_filters: + +*********************************************** +Using VMware dynamic inventory plugin - Filters +*********************************************** + +.. contents:: + :local: + +VMware dynamic inventory plugin - filtering VMware guests +========================================================= + + +VMware inventory plugin allows you to filter VMware guests using the ``filters`` configuration parameter. + +This section shows how you configure ``filters`` for the given VMware guest in the inventory. + +Requirements +------------ + +To use the VMware dynamic inventory plugins, you must install `pyVmomi `_ +on your control node (the host running Ansible). + +To include tag-related information for the virtual machines in your dynamic inventory, you also need the `vSphere Automation SDK `_, which supports REST API features such as tagging and content libraries, on your control node. +You can install the ``vSphere Automation SDK`` following `these instructions `_. + +.. code-block:: bash + + $ pip install pyvmomi + +Starting in Ansible 2.10, the VMware dynamic inventory plugin is available in the ``community.vmware`` collection included Ansible. +Alternately, to install the latest ``community.vmware`` collection: + +.. code-block:: bash + + $ ansible-galaxy collection install community.vmware + +To use this VMware dynamic inventory plugin: + +1. Enable it first by specifying the following in the ``ansible.cfg`` file: + +.. code-block:: ini + + [inventory] + enable_plugins = community.vmware.vmware_vm_inventory + +2. Create a file that ends in ``vmware.yml`` or ``vmware.yaml`` in your working directory. + +The ``vmware_vm_inventory`` inventory plugin takes in the same authentication information as any other VMware modules does. + +Let us assume we want to list all RHEL7 VMs with the power state as "poweredOn". A valid inventory file with filters for the given VMware guest looks as follows: + +.. code-block:: yaml + + plugin: community.vmware.vmware_vm_inventory + strict: False + hostname: 10.65.223.31 + username: administrator@vsphere.local + password: Esxi@123$% + validate_certs: False + with_tags: False + hostnames: + - config.name + filters: + - config.guestId == "rhel7_64Guest" + - summary.runtime.powerState == "poweredOn" + + +Here, we have configured two filters - + +* ``config.guestId`` is equal to ``rhel7_64Guest`` +* ``summary.runtime.powerState`` is equal to ``poweredOn`` + +This retrieves all the VMs which satisfy these two conditions and populates them in the inventory. +Notice that the conditions are combined using an ``and`` operation. + +Using ``or`` conditions in filters +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Let us assume you want filter RHEL7 and Ubuntu VMs. You can use multiple filters using ``or`` condition in your inventory file. + +A valid filter in the VMware inventory file for this example is: + +.. code-block:: yaml + + plugin: community.vmware.vmware_vm_inventory + strict: False + hostname: 10.65.223.31 + username: administrator@vsphere.local + password: Esxi@123$% + validate_certs: False + with_tags: False + hostnames: + - config.name + filters: + - config.guestId == "rhel7_64Guest" or config.guestId == "ubuntu64Guest" + + +You can check all allowed properties for filters for the given virtual machine at :ref:`vmware_inventory_vm_attributes`. + +If you are using the ``properties`` parameter with custom VM properties, make sure that you include all the properties used by filters as well in your VM property list. + +For example, if we want all RHEL7 and Ubuntu VMs that are poweredOn, you can use inventory file: + +.. code-block:: yaml + + plugin: community.vmware.vmware_vm_inventory + strict: False + hostname: 10.65.223.31 + username: administrator@vsphere.local + password: Esxi@123$% + validate_certs: False + with_tags: False + hostnames: + - 'config.name' + properties: + - 'config.name' + - 'config.guestId' + - 'guest.ipAddress' + - 'summary.runtime.powerState' + filters: + - config.guestId == "rhel7_64Guest" or config.guestId == "ubuntu64Guest" + - summary.runtime.powerState == "poweredOn" + +Here, we are using minimum VM properties, that is ``config.name``, ``config.guestId``, ``summary.runtime.powerState``, and ``guest.ipAddress``. + +* ``config.name`` is used by the ``hostnames`` parameter. +* ``config.guestId`` and ``summary.runtime.powerState`` are used by the ``filters`` parameter. +* ``guest.guestId`` is used by ``ansible_host`` internally by the inventory plugin. + +Using regular expression in filters +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Let us assume you want filter VMs with specific IP range. You can use regular expression in ``filters`` in your inventory file. + +For example, if we want all RHEL7 and Ubuntu VMs that are poweredOn, you can use inventory file: + +.. code-block:: yaml + + plugin: community.vmware.vmware_vm_inventory + strict: False + hostname: 10.65.223.31 + username: administrator@vsphere.local + password: Esxi@123$% + validate_certs: False + with_tags: False + hostnames: + - 'config.name' + properties: + - 'config.name' + - 'config.guestId' + - 'guest.ipAddress' + - 'summary.runtime.powerState' + filters: + - guest.ipAddress is defined and guest.ipAddress is match('192.168.*') + +Here, we are using ``guest.ipAddress`` VM property. This property is optional and depended upon VMware tools installed on VMs. +We are using ``match`` to validate the regular expression for the given IP range. + +Executing ``ansible-inventory --list -i .vmware.yml`` creates a list of the virtual machines that are ready to be configured using Ansible. + +What to expect +-------------- + +You will notice that the inventory hosts are filtered depending on your ``filters`` section. + + +.. code-block:: yaml + + { + "_meta": { + "hostvars": { + "template_001": { + "config.name": "template_001", + "config.guestId": "ubuntu64Guest", + ... + "guest.toolsStatus": "toolsNotInstalled", + "summary.runtime.powerState": "poweredOn", + }, + "vm_8046": { + "config.name": "vm_8046", + "config.guestId": "rhel7_64Guest", + ... + "guest.toolsStatus": "toolsNotInstalled", + "summary.runtime.powerState": "poweredOn", + }, + ... + } + +Troubleshooting filters +----------------------- + +If the custom property specified in ``filters`` fails: + +- Check if the values provided for username and password are correct. +- Make sure it is a valid property, see :ref:`vmware_inventory_vm_attributes`. +- Use ``strict: True`` to get more information about the error. +- Please make sure that you are using latest version of the VMware collection. + + +.. seealso:: + + `pyVmomi `_ + The GitHub Page of pyVmomi + `pyVmomi Issue Tracker `_ + The issue tracker for the pyVmomi project + `vSphere Automation SDK GitHub Page `_ + The GitHub Page of vSphere Automation SDK for Python + `vSphere Automation SDK Issue Tracker `_ + The issue tracker for vSphere Automation SDK for Python + :ref:`vmware_inventory_vm_attributes` + Using Virtual machine attributes in VMware dynamic inventory plugin + :ref:`working_with_playbooks` + An introduction to playbooks + :ref:`playbooks_vault` + Using Vault in playbooks diff --git a/docs/docsite/rst/scenario_guides/vmware_scenarios/vmware_inventory_hostnames.rst b/docs/docsite/rst/scenario_guides/vmware_scenarios/vmware_inventory_hostnames.rst index ac8dfaf1bcc..9d284562008 100644 --- a/docs/docsite/rst/scenario_guides/vmware_scenarios/vmware_inventory_hostnames.rst +++ b/docs/docsite/rst/scenario_guides/vmware_scenarios/vmware_inventory_hostnames.rst @@ -4,9 +4,10 @@ Using VMware dynamic inventory plugin - Hostnames ************************************************* -.. contents:: Topics +.. contents:: + :local: -VMware dynamic inventory plugin - Customizing hostnames +VMware dynamic inventory plugin - customizing hostnames ======================================================= @@ -36,14 +37,14 @@ To install the latest ``community.vmware`` collection: To use this VMware dynamic inventory plugin: - 1. Enable it first by specifying the following in the ``ansible.cfg`` file: +1. Enable it first by specifying the following in the ``ansible.cfg`` file: .. code-block:: ini [inventory] enable_plugins = community.vmware.vmware_vm_inventory - 2. Create a file that ends in ``.vmware.yml`` or ``.vmware.yaml`` in your working directory. +2. Create a file that ends in ``vmware.yml`` or ``vmware.yaml`` in your working directory. The ``vmware_vm_inventory`` inventory plugin takes in the same authentication information as any other VMware modules does. @@ -67,7 +68,7 @@ the ``config.name`` property from the virtual machine and populate it in the inv You can check all allowed properties for the given virtual machine at :ref:`vmware_inventory_vm_attributes`. -Executing ``ansible-inventory --list -i .vmware.yml`` will create a list of the virtual machines that are ready to be configured using Ansible. +Executing ``ansible-inventory --list -i .vmware.yml`` creates a list of the virtual machines that are ready to be configured using Ansible. What to expect -------------- diff --git a/docs/docsite/rst/user_guide/intro_getting_started.rst b/docs/docsite/rst/user_guide/intro_getting_started.rst index 58ab7fe6c35..0fde0281b32 100644 --- a/docs/docsite/rst/user_guide/intro_getting_started.rst +++ b/docs/docsite/rst/user_guide/intro_getting_started.rst @@ -50,11 +50,8 @@ Beyond the basics You can override the default remote user name in several ways, including: * passing the ``-u`` parameter at the command line - * setting user information in your inventory file - * setting user information in your configuration file - * setting environment variables See :ref:`general_precedence_rules` for details on the (sometimes unintuitive) precedence of each method of passing user information. You can read more about connections in :ref:`connections`. diff --git a/docs/docsite/rst/user_guide/intro_inventory.rst b/docs/docsite/rst/user_guide/intro_inventory.rst index fd83248d037..567cd75999c 100644 --- a/docs/docsite/rst/user_guide/intro_inventory.rst +++ b/docs/docsite/rst/user_guide/intro_inventory.rst @@ -169,7 +169,7 @@ In YAML: webservers: hosts: www[01:50].example.com: - + You can specify a stride (increments between sequence numbers) when defining a numeric range of hosts: In INI: @@ -200,7 +200,7 @@ For numeric patterns, leading zeros can be included or removed, as desired. Rang Adding variables to inventory ============================= -You can store variable values that relate to a specific host or group in inventory. To start with, you may add variables directly to the hosts and groups in your main inventory file. As you add more and more managed nodes to your Ansible inventory, however, you will likely want to store variables in separate host and group variable files. +You can store variable values that relate to a specific host or group in inventory. To start with, you may add variables directly to the hosts and groups in your main inventory file. As you add more and more managed nodes to your Ansible inventory, however, you will likely want to store variables in separate host and group variable files. See :ref:`define_variables_in_inventory` for details. .. _host_variables: diff --git a/docs/docsite/rst/user_guide/playbooks_variables.rst b/docs/docsite/rst/user_guide/playbooks_variables.rst index 00277f3b7a6..e77144dec4f 100644 --- a/docs/docsite/rst/user_guide/playbooks_variables.rst +++ b/docs/docsite/rst/user_guide/playbooks_variables.rst @@ -193,7 +193,7 @@ Where to set variables You can define variables in a variety of places, such as in inventory, in playbooks, in reusable files, in roles, and at the command line. Ansible loads every possible variable it finds, then chooses the variable to apply based on :ref:`variable precedence rules `. -.. _variables_in_inventory: +.. _define_variables_in_inventory: Defining variables in inventory ------------------------------- @@ -363,7 +363,7 @@ Tips on where to set variables You should choose where to define a variable based on the kind of control you might want over values. -Set variables in inventory that deal with geography or behavior. Since groups are frequently the entity that maps roles onto hosts, you can often set variables on the group instead of defining them on a role. Remember: Child groups override parent groups, and host variables override group variables. See :ref:`variables_in_inventory` for details on setting host and group variables. +Set variables in inventory that deal with geography or behavior. Since groups are frequently the entity that maps roles onto hosts, you can often set variables on the group instead of defining them on a role. Remember: Child groups override parent groups, and host variables override group variables. See :ref:`define_variables_in_inventory` for details on setting host and group variables. Set common defaults in a ``group_vars/all`` file. See :ref:`splitting_out_vars` for details on how to organize host and group variables in your inventory. Group variables are generally placed alongside your inventory file, but they can also be returned by dynamic inventory (see :ref:`intro_dynamic_inventory`) or defined in :ref:`ansible_tower` from the UI or API:: diff --git a/lib/ansible/modules/package_facts.py b/lib/ansible/modules/package_facts.py index e957e8300b9..ed314b4a29d 100644 --- a/lib/ansible/modules/package_facts.py +++ b/lib/ansible/modules/package_facts.py @@ -42,11 +42,11 @@ author: ''' EXAMPLES = ''' -- name: Gather the rpm package facts +- name: Gather the package facts package_facts: manager: auto -- name: Print the rpm package facts +- name: Print the package facts debug: var: ansible_facts.packages