diff --git a/docs/docsite/requirements.txt b/docs/docsite/requirements.txt index 4f06993bb9e..2672d01095b 100644 --- a/docs/docsite/requirements.txt +++ b/docs/docsite/requirements.txt @@ -6,4 +6,3 @@ sphinx==2.1.2 sphinx-notfound-page Pygments >= 2.4.0 straight.plugin # Needed for hacking/build-ansible.py which is the backend build script -make diff --git a/docs/docsite/rst/community/development_process.rst b/docs/docsite/rst/community/development_process.rst index 54d9e95a432..55e861f9990 100644 --- a/docs/docsite/rst/community/development_process.rst +++ b/docs/docsite/rst/community/development_process.rst @@ -27,7 +27,7 @@ If you want to follow the conversation about what features will be added to Ansi Micro development: the lifecycle of a PR ======================================== -Ansible accepts code through **pull requests** ("PRs" for short). GitHub provides a great overview of `how the pull request process works `_ in general. The ultimate goal of any pull request is to get merged and become part of Ansible Core. +Ansible accepts code through **pull requests** ("PRs" for short). GitHub provides a great overview of `how the pull request process works `_ in general. The ultimate goal of any pull request is to get merged and become part of a collection or ansible-base. Here's an overview of the PR lifecycle: * Contributor opens a PR @@ -209,7 +209,7 @@ Here are some examples: .. code-block:: yaml bugfixes: - - copy module - The copy module was attempting to change the mode of files for + - copy - the module was attempting to change the mode of files for remote_src=True even if mode was not set as a parameter. This failed on filesystems which do not have permission bits (https://github.com/ansible/ansible/issues/29444). diff --git a/docs/docsite/rst/community/how_can_I_help.rst b/docs/docsite/rst/community/how_can_I_help.rst index de05e432d28..47a87bb8d2b 100644 --- a/docs/docsite/rst/community/how_can_I_help.rst +++ b/docs/docsite/rst/community/how_can_I_help.rst @@ -4,7 +4,8 @@ How can I help? *************** -.. contents:: Topics +.. contents:: + :local: Thanks for being interested in helping the Ansible project! @@ -50,19 +51,19 @@ File and verify issues All software has bugs, and Ansible is no exception. When you find a bug, you can help tremendously by :ref:`telling us about it `. -If you should discover that the bug you're trying to file already exists in an issue, you can help by verifying the behavior of the reported bug with a comment in that issue, or by reporting any additional information. +If the bug you found already exists in an issue, you can help by verifying the behavior of the reported bug with a comment in that issue, or by reporting any additional information. Review and submit pull requests =============================== -As you become more familiar with how Ansible works, you may be able to fix issues or develop new features yourself. If you think you've got a solution to a bug you've found in Ansible, or if you've got a new feature that you've written and would like to share with millions of Ansible users, read all about the :ref:`Ansible development process ` to learn how to get your code accepted into Ansible. +As you become more familiar with how Ansible works, you may be able to fix issues or develop new features yourself. If you think you have a fix for a bug in Ansible, or if you have a new feature that you would like to share with millions of Ansible users, read all about the :ref:`Ansible development process ` and and :ref:`how to contribute to collections ` to learn how to get your code accepted into Ansible. Another good way to help is to review pull requests that other Ansible users have submitted. The Ansible community keeps a full list of `open pull requests by file `_, so if there's a particular module or plug-in that particularly interests you, you can easily keep track of all the relevant new pull requests and provide testing or feedback. -Become a module maintainer -========================== +Become a collection maintainer +============================== -Once you've learned about the development process and have contributed code to a particular module, we encourage you to become a maintainer of that module. There are hundreds of different modules in Ansible, and the vast majority of them are written and maintained entirely by members of the Ansible community. +Once you have learned about the development process and have contributed code to a collection, we encourage you to become a maintainer of that collection. There are hundreds of modules in dozens of Ansible collections, and the vast majority of them are written and maintained entirely by members of the Ansible community. To learn more about the responsibilities of being an Ansible module maintainer, please read our :ref:`module maintainer guidelines `. @@ -77,7 +78,7 @@ Working groups are a way for Ansible community members to self-organize around p Teach Ansible to others ======================= -We're working on a standardized `Ansible workshop `_ that can provide a good hands-on introduction to Ansible usage and concepts. +We are working on a standardized `Ansible workshop `_ that can provide a good hands-on introduction to Ansible usage and concepts. Social media ============ diff --git a/docs/docsite/rst/community/index.rst b/docs/docsite/rst/community/index.rst index 37a088d8d43..e100fe09f93 100644 --- a/docs/docsite/rst/community/index.rst +++ b/docs/docsite/rst/community/index.rst @@ -14,9 +14,9 @@ This page outlines the most common situations and questions that bring readers t Getting started =============== -* I'm new to the community. Where can I find the Ansible :ref:`code_of_conduct`? -* I'd like to know what I'm agreeing to when I contribute to Ansible. Does Ansible have a :ref:`contributor_license_agreement`? -* I'd like to contribute but I'm not sure how. Are there :ref:`easy ways to contribute `? +* I am new to the community. Where can I find the Ansible :ref:`code_of_conduct`? +* I would like to know what I am agreeing to when I contribute to Ansible. Does Ansible have a :ref:`contributor_license_agreement`? +* I would like to contribute but I am not sure how. Are there :ref:`easy ways to contribute `? * I want to talk to other Ansible users. How do I find an `Ansible Meetup near me `_? * I have a question. Which :ref:`Ansible email lists and IRC channels ` will help me find answers? * I want to learn more about Ansible. What can I do? @@ -27,7 +27,7 @@ Getting started * `Review getting started guides `_. * `Watch videos `_ - includes Ansible Automates, AnsibleFest & webinar recordings. -* I'd like updates about new Ansible versions. How are `new releases announced `_? +* I would like updates about new Ansible versions. How are `new releases announced `_? * I want to use the current release. How do I know which :ref:`releases are current `? Going deeper @@ -36,9 +36,9 @@ Going deeper * I think Ansible is broken. How do I :ref:`report a bug `? * I need functionality that Ansible doesn't offer. How do I :ref:`request a feature `? * How do I :ref:`contribute to an Ansible-maintained collection `? -* I'm waiting for a particular feature. How do I see what's :ref:`planned for future Ansible Releases `? +* I am waiting for a particular feature. How do I see what is :ref:`planned for future Ansible Releases `? * I have a specific Ansible interest or expertise (for example, VMware, Linode, and so on.). How do I get involved in a :ref:`working group `? -* I'd like to participate in conversations about features and fixes. How do I review GitHub issues and pull requests? +* I would like to participate in conversations about features and fixes. How do I review GitHub issues and pull requests? * I found a typo or another problem on docs.ansible.com. How can I :ref:`improve the documentation `? @@ -46,15 +46,15 @@ Working with the Ansible repo ============================= * I want to code my first changes to Ansible. How do I :ref:`set up my Python development environment `? -* I'd like to get more efficient as a developer. How can I find :ref:`editors, linters, and other tools ` that will support my Ansible development efforts? +* I would like to get more efficient as a developer. How can I find :ref:`editors, linters, and other tools ` that will support my Ansible development efforts? * I want my PR to meet Ansible's guidelines. Where can I find guidance on :ref:`coding in Ansible `? * I want to learn more about Ansible roadmaps, releases, and projects. How do I find information on :ref:`the development cycle `? -* I'd like to connect Ansible to a new API or other resource. How do I :ref:`contribute a group of related modules `? +* I would like to connect Ansible to a new API or other resource. How do I :ref:`create a collection `? * My pull request is marked ``needs_rebase``. How do I :ref:`rebase my PR `? -* I'm using an older version of Ansible and want a bug fixed in my version that's already been fixed on the ``devel`` branch. How do I :ref:`backport a bugfix PR `? +* I am using an older version of Ansible and want a bug fixed in my version that has already been fixed on the ``devel`` branch. How do I :ref:`backport a bugfix PR `? * I have an open pull request with a failing test. How do I learn about Ansible's :ref:`testing (CI) process `? -* I'm ready to step up as a module maintainer. What are the :ref:`guidelines for maintainers `? -* A module I maintain is obsolete. How do I :ref:`deprecate a module `? +* I am ready to step up as a collection maintainer. What are the :ref:`guidelines for maintainers `? +* A module in a collection I maintain is obsolete. How do I :ref:`deprecate a module `? Traditional Table of Contents ============================= diff --git a/docs/docsite/rst/community/maintainers.rst b/docs/docsite/rst/community/maintainers.rst index 94e4acc0da8..ac466d6757a 100644 --- a/docs/docsite/rst/community/maintainers.rst +++ b/docs/docsite/rst/community/maintainers.rst @@ -1,12 +1,13 @@ .. _maintainers: -**************************** -Module Maintainer Guidelines -**************************** +******************************** +Collection maintainer guidelines +******************************** -.. contents:: Topics +Thank you for being a community collection maintainer. This guide offers an overview of your responsibilities as a maintainer along with resources for additional information. The Ansible community hopes that you will find that maintaining a collection is as rewarding for you as having the collection content is for the wider community. -Thank you for being a maintainer of part of Ansible's codebase. This guide provides module maintainers an overview of their responsibilities, resources for additional information, and links to helpful tools. +.. contents:: + :local: In addition to the information below, module maintainers should be familiar with: @@ -17,57 +18,17 @@ In addition to the information below, module maintainers should be familiar with Maintainer responsibilities =========================== -When you contribute a new module to the `ansible/ansible `_ repository, you become the maintainer for that module once it has been merged. Maintainership empowers you with the authority to accept, reject, or request revisions to pull requests on your module -- but as they say, "with great power comes great responsibility." +When you contribute a module to a collection included in the ``ansible`` package, you become a maintainer for that module once it has been merged. Maintainership empowers you with the authority to accept, reject, or request revisions to pull requests on your module -- but as they say, "with great power comes great responsibility." -Maintainers of Ansible modules are expected to provide feedback, responses, or actions on pull requests or issues to the module(s) they maintain in a reasonably timely manner. +Maintainers of Ansible collections are expected to provide feedback, responses, or actions on pull requests or issues to the collection(s) they maintain in a reasonably timely manner. You can also update the contributor guidelines for that collection, in collaboration with the Ansible community team and the other maintainers of that collection. -It is also recommended that you occasionally revisit the `contribution guidelines `_, as they are continually refined. Occasionally, you may be requested to update your module to move it closer to the general accepted standard requirements. We hope for this to be infrequent, and will always be a request with a fair amount of lead time (ie: not by tomorrow!). +Resources +========= -Finally, following the `ansible-devel `_ mailing list can be a great way to participate in the broader Ansible community, and a place where you can influence the overall direction, quality, and goals of Ansible and its modules. If you're not on this relatively low-volume list, please join us here: https://groups.google.com/forum/#!forum/ansible-devel +Please see :ref:`communication` for ways to contact the broader Ansible community. For maintainers, following the `ansible-devel `_ mailing list is a great way to participate in conversations about coding, get assistance when you need it, and influence the overall direction, quality, and goals of Ansible and the collections. If you are not on this relatively low-volume list, please join us here: https://groups.google.com/forum/#!forum/ansible-devel -The Ansible community hopes that you will find that maintaining your module is as rewarding for you as having the module is for the wider community. Pull requests, issues, and workflow =================================== -Pull requests -------------- - -Module pull requests are located in the `main Ansible repository `_. - -Because of the high volume of pull requests, notification of PRs to specific modules are routed by an automated bot to the appropriate maintainer for handling. It is recommended that you set an appropriate notification process to receive notifications which mention your GitHub ID. - -Issues ------- - -Issues for modules, including bug reports, documentation bug reports, and feature requests, are tracked in the `ansible repository `_. - -Issues for modules are routed to their maintainers by an automated process. This process is still being refined, and currently depends upon the issue creator to provide adequate details (specifically, providing the proper module name) in order to route it correctly. If you are a maintainer of a specific module, it is recommended that you periodically search module issues for issues which mention your module's name (or some variation on that name), as well as setting an appropriate notification process for receiving notification of mentions of your GitHub ID. - -PR workflow ------------ - -Automated routing of pull requests is handled by a tool called `Ansibot `_. - -Being moderately familiar with how the workflow behind the bot operates can be helpful to you, and -- should things go awry -- your feedback can be helpful to the folks that continually help Ansibullbot to evolve. - -A detailed explanation of the PR workflow can be seen in the :ref:`community_development_process`. - -Maintainers (BOTMETA.yml) -------------------------- - -The full list of maintainers is located in `BOTMETA.yml `_. - -Adding and removing maintainers -=============================== - -Communities change over time, and no one maintains a module forever. If you'd like to propose an additional maintainer for your module, please submit a PR to ``BOTMETA.yml`` with the GitHub username of the new maintainer. - -If you'd like to step down as a maintainer, please submit a PR to the ``BOTMETA.yml`` removing your GitHub ID from the module in question. If that would leave the module with no maintainers, put "ansible" as the maintainer. This will indicate that the module is temporarily without a maintainer, and the Ansible community team will search for a new maintainer. - -Tools and other Resources -========================= - -* `PRs in flight, organized by directory `_ -* `Ansibullbot `_ -* :ref:`community_development_process` +Each collection community can set its own rules and workflow for managing pull requests, bug reports, documentation issues, and feature requests, as well as adding and replacing maintainers. diff --git a/docs/docsite/rst/community/reporting_bugs_and_features.rst b/docs/docsite/rst/community/reporting_bugs_and_features.rst index d55ea995a1a..4cf3ca62575 100644 --- a/docs/docsite/rst/community/reporting_bugs_and_features.rst +++ b/docs/docsite/rst/community/reporting_bugs_and_features.rst @@ -1,39 +1,56 @@ .. _reporting_bugs_and_features: ************************************** -Reporting Bugs And Requesting Features +Reporting bugs and requesting features ************************************** -.. contents:: Topics +.. contents:: + :local: .. _reporting_bugs: Reporting a bug =============== +Security bugs +------------- + Ansible practices responsible disclosure - if this is a security-related bug, email `security@ansible.com `_ instead of filing a ticket or posting to any public groups, and you will receive a prompt response. -Ansible bugs should be reported to `github.com/ansible/ansible/issues `_ after -signing up for a free GitHub account. Before reporting a bug, please use the bug/issue search -to see if the issue has already been reported. +Bugs in ansible-base +-------------------- -Knowing your Ansible version and the exact commands you are running, and what you expect, saves time and helps us help everyone with their issues more quickly. For that reason, we provide an issue template; please fill it out as completely and as accurately as possible. +If you find a bug that affects multiple plugins, a plugin that remained in the ansible/ansible repo, or the overall functioning of Ansible, report it to `github.com/ansible/ansible/issues `_. You need a free GitHub account. Before reporting a bug, use the bug/issue search to see if the issue has already been reported. If you are not sure if something is a bug yet, you can report the behavior on the :ref:`mailing list or IRC first `. -Do not use the issue tracker for "how do I do this" type questions. These are great candidates for IRC or the mailing list instead where things are likely to be more of a discussion. +Do not open issues for "how do I do this" type questions. These are great topics for IRC or the mailing list, where things are likely to be more of a discussion. -To be respectful of reviewers' time and allow us to help everyone efficiently, please provide minimal well-reduced and well-commented examples rather than sharing your entire production playbook. Include playbook snippets and output where possible. +If you find a bug, open the issue yourself to ensure we have a record of it. Do not rely on someone else in the community to file the bug report for you. We have created an issue template, which saves time and helps us help everyone with their issues more quickly. Please fill it out as completely and as accurately as possible: -When sharing YAML in playbooks, formatting can be preserved by using `code blocks `_. + * Include the Ansible version + * Include any relevant configuration + * Include the exact commands or tasks you are running + * Describe the behavior you expected + * Provide steps to reproduce the bug + * Use minimal well-reduced and well-commented examples, not your entire production playbook + * When sharing YAML in playbooks, preserve the formatting by using `code blocks `_. + * Document the behavior you got + * Include output where possible + * For multiple-file content, use gist.github.com, which is more durable than pastebin content -For multiple-file content, we encourage use of gist.github.com. Online pastebin content can expire, so it's nice to have things around for a longer term if they are referenced in a ticket. +Bugs in collections +------------------- -If you are not sure if something is a bug yet, you are welcome to ask about something on the :ref:`mailing list or IRC first `. +Many bugs only affect a single module or plugin. If you find a bug that affects a module or plugin hosted in a collection, file the bug in the repository of the :ref:`collection `: -As we are a very high volume project, if you determine that you do have a bug, please be sure to open the issue yourself to ensure we have a record of it. Don't rely on someone else in the community to file the bug report for you. + #. Find the collection on Galaxy. + #. Click on the Issue Tracker link for that collection. + #. Follow the contributor guidelines or instructions in the collection repo. + +If you are not sure whether a bug is in ansible-base or in a collection, you can report the behavior on the :ref:`mailing list or IRC first `. .. _request_features: Requesting a feature ==================== -The best way to get a feature into Ansible is to :ref:`submit a pull request `. +The best way to get a feature into Ansible is to :ref:`submit a pull request `, either against ansible-base or against a collection. See also :ref:`ansible_collection_merge_requirements`. diff --git a/docs/docsite/rst/dev_guide/developing_collections.rst b/docs/docsite/rst/dev_guide/developing_collections.rst index d1855d8a181..952c2e488f7 100644 --- a/docs/docsite/rst/dev_guide/developing_collections.rst +++ b/docs/docsite/rst/dev_guide/developing_collections.rst @@ -5,7 +5,6 @@ Developing collections ********************** - Collections are a distribution format for Ansible content. You can use collections to package and distribute playbooks, roles, modules, and plugins. You can publish and use collections through `Ansible Galaxy `_. @@ -653,7 +652,7 @@ To generate changelogs from the changelog fragments you created: .. note:: - Add the ``--reload-plugins`` option if you ran the ``antsibull-changelog release`` command previously and the version of the collection has not changed. ``antsibull-changelog`` caches the information on all plugins and does not update its cache until the collection version changes. + Add the ``--reload-plugins`` option if you ran the ``antsibull-changelog release`` command previously and the version of the collection has not changed. ``antsibull-changelog`` caches the information on all plugins and does not update its cache until the collection version changes. Porting Guide entries diff --git a/docs/docsite/rst/dev_guide/developing_locally.rst b/docs/docsite/rst/dev_guide/developing_locally.rst index 573dd69d7be..215240e93a0 100644 --- a/docs/docsite/rst/dev_guide/developing_locally.rst +++ b/docs/docsite/rst/dev_guide/developing_locally.rst @@ -43,6 +43,10 @@ Once you save your module file in one of these locations, Ansible will load it a To confirm that ``my_custom_module`` is available: +* type ``ansible localhost -m my_custom_module``. You should see the output for that module. + +or + * type ``ansible-doc -t module my_custom_module``. You should see the documentation for that module. .. note:: diff --git a/docs/docsite/rst/dev_guide/developing_modules_general.rst b/docs/docsite/rst/dev_guide/developing_modules_general.rst index 10e015550ff..d9a5ff2aa22 100644 --- a/docs/docsite/rst/dev_guide/developing_modules_general.rst +++ b/docs/docsite/rst/dev_guide/developing_modules_general.rst @@ -51,7 +51,7 @@ Starting a new module To create a new module: -1. Navigate to the correct directory for your new module: ``$ cd lib/ansible/modules/cloud/azure/`` +1. Navigate to the correct directory for your new module: ``$ cd lib/ansible/modules/`` 2. Create your new module file: ``$ touch my_test.py`` 3. Paste the content below into your new module file. It includes the :ref:`required Ansible format and documentation ` and some example code. 4. Modify and extend the code to do what you want your new module to do. See the :ref:`programming tips ` and :ref:`Python 3 compatibility ` pages for pointers on writing clean, concise module code. @@ -62,8 +62,10 @@ To create a new module: # Copyright: (c) 2018, Terry Jones # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) + from __future__ import (absolute_import, division, print_function) + __metaclass__ = type - DOCUMENTATION = ''' + DOCUMENTATION = r''' --- module: my_test @@ -72,17 +74,19 @@ To create a new module: version_added: "2.4" description: - - "This is my longer description explaining my test module" + - "This is my longer description explaining my test module." options: name: description: - - This is the message to send to the test module + - This is the message to send to the test module. required: true + type: str new: description: - - Control to demo if the result of this module is changed or not + - Control to demo if the result of this module is changed or not. required: false + type: bool extends_documentation_fragment: - azure @@ -91,7 +95,7 @@ To create a new module: - Your Name (@yourhandle) ''' - EXAMPLES = ''' + EXAMPLES = r''' # Pass in a message - name: Test with a message my_test: @@ -109,7 +113,7 @@ To create a new module: name: fail me ''' - RETURN = ''' + RETURN = r''' original_message: description: The original name param that was passed in type: str @@ -208,7 +212,7 @@ If your module does not need to target a remote host, you can quickly and easily development) activate it: ``$ . venv/bin/activate`` - Setup the environment for development: ``$ . hacking/env-setup`` - Run your test module locally and directly: - ``$ python -m ansible.modules.cloud.azure.my_test /tmp/args.json`` + ``$ python -m ansible.modules.my_test /tmp/args.json`` This should return output like this: diff --git a/docs/docsite/rst/dev_guide/developing_modules_general_windows.rst b/docs/docsite/rst/dev_guide/developing_modules_general_windows.rst index 18dfb786ae2..aba60b87bb9 100644 --- a/docs/docsite/rst/dev_guide/developing_modules_general_windows.rst +++ b/docs/docsite/rst/dev_guide/developing_modules_general_windows.rst @@ -213,7 +213,7 @@ options set: - ``elements``: When ``type=list``, this sets the type of each list value, the values are the same as ``type`` - ``no_log``: Will sanitise the input value before being returned in the ``module_invocation`` return value - ``removed_in_version``: States when a deprecated module option is to be removed, a warning is displayed to the end user if set -- ``removed_at_date``: States the date when a deprecated module option will be removed, a warning is displayed to the end user if set +- ``removed_at_date``: States the date (YYYY-MM-DD) when a deprecated module option will be removed, a warning is displayed to the end user if set - ``removed_from_collection``: States from which collection the deprecated module option will be removed; must be specified if one of ``removed_in_version`` and ``removed_at_date`` is specified - ``required``: Will fail when the module option is not set - ``type``: The type of the module option, if not set then it defaults to ``str``. The valid types are; diff --git a/docs/docsite/rst/dev_guide/developing_modules_in_groups.rst b/docs/docsite/rst/dev_guide/developing_modules_in_groups.rst index cee49f019e3..51a40e59e73 100644 --- a/docs/docsite/rst/dev_guide/developing_modules_in_groups.rst +++ b/docs/docsite/rst/dev_guide/developing_modules_in_groups.rst @@ -1,64 +1,49 @@ .. _developing_modules_in_groups: -********************************************* -Information for submitting a group of modules -********************************************* +************************* +Creating a new collection +************************* -.. contents:: Topics +Starting with Ansible 2.10, related modules should be developed in a collection. The Ansible core team and community compiled these module development tips and tricks to help companies developing Ansible modules for their products and users developing Ansible modules for third-party products. See :ref:`developing_collections` for a more detailed description of the collections format and additional development guidelines. + +.. contents:: :local: -Submitting a group of modules -============================= - -This section discusses how to get multiple related modules into Ansible. - -This document is intended for both companies wishing to add modules for their own products as well as users of 3rd party products wishing to add Ansible functionality. - -It's based on module development tips and tricks that the Ansible core team and community have accumulated. - .. include:: shared_snippets/licensing.txt Before you start coding ======================= -Although it's tempting to get straight into coding, there are a few things to be aware of first. This list of prerequisites is designed to help ensure that you develop high-quality modules that flow easily through the review process and get into Ansible more quickly. +This list of prerequisites is designed to help ensure that you develop high-quality modules that work well with ansible-base and provide a seamless user experience. * Read though all the pages linked off :ref:`developing_modules_general`; paying particular focus to the :ref:`developing_modules_checklist`. -* New modules must be PEP 8 compliant. See :ref:`testing_pep8` for more information. -* Starting with Ansible version 2.4, all new modules must :ref:`support Python 2.6+ and Python 3.5+ `. If this is an issue, please contact us (see the "Speak to us" section later in this document to learn how). -* Have a look at the existing modules and how they've been named in the :ref:`all_modules`, especially in the same functional area (such as cloud, networking, databases). -* Shared code can be placed into ``lib/ansible/module_utils/`` -* Shared documentation (for example describing common arguments) can be placed in ``lib/ansible/plugins/doc_fragments/``. -* With great power comes great responsibility: Ansible module maintainers have a duty to help keep modules up to date. As with all successful community projects, module maintainers should keep a watchful eye for reported issues and contributions. -* Although not required, unit and/or integration tests are strongly recommended. Unit tests are especially valuable when external resources (such as cloud or network devices) are required. For more information see :ref:`developing_testing` and the `Testing Working Group `_. - * Starting with Ansible 2.4 all :ref:`network_modules` MUST have unit tests. +* We encourage PEP 8 compliance. See :ref:`testing_pep8` for more information. +* We encourage supporting :ref:`Python 2.6+ and Python 3.5+ `. +* Look at Ansible Galaxy and review the naming conventions in your functional area (such as cloud, networking, databases). +* With great power comes great responsibility: Ansible collection maintainers have a duty to help keep content up to date. As with all successful community projects, collection maintainers should keep a watchful eye for reported issues and contributions. +* We strongly recommend unit and/or integration tests. Unit tests are especially valuable when external resources (such as cloud or network devices) are required. For more information see :ref:`developing_testing` and the `Testing Working Group `_. -Naming convention -================= +Naming conventions +================== -As you may have noticed when looking under ``lib/ansible/modules/`` we support up to two directories deep (but no deeper), e.g. `databases/mysql`. This is used to group files on disk as well as group related modules into categories and topics the Module Index, for example: :ref:`database_modules`. +Fully Qualified Collection Names (FQCNs) for plugins and modules include three elements: -The directory name should represent the *product* or *OS* name, not the company name. + * the Galaxy namespace, which generally represents the company or group + * the collection name, which generally represents the product or OS + * the plugin or module name + * always in lower case + * words separated with an underscore (``_``) character + * singular, rather than plural, for example, ``command`` not ``commands`` -Each module should have the above (or similar) prefix; see existing :ref:`all_modules` for existing examples. - -**Note:** - -* File and directory names are always in lower case -* Words are separated with an underscore (``_``) character -* Module names should be in the singular, rather than plural, eg ``command`` not ``commands`` +For example, ``community.mongodb.mongodb_linux`` or ``cisco.meraki.meraki_device``. +It is convenient if the organization and repository names on GitHub (or elsewhere) match your namespace and collection names on Ansible Galaxy, but it is not required. The plugin names you select, however, are always the same in your code repository and in your collection artifact on Galaxy. Speak to us =========== -Circulating your ideas before coding is a good way to help you set off in the right direction. - -After reading the "Before you start coding" section you will hopefully have a reasonable idea of the structure of your modules. - -We've found that writing a list of your proposed module names and a one or two line description of what they will achieve and having that reviewed by Ansible is a great way to ensure the modules fit the way people have used Ansible Modules before, and therefore make them easier to use. - +Circulating your ideas before coding helps you adopt good practices and avoid common mistakes. After reading the "Before you start coding" section you should have a reasonable idea of the structure of your modules. Write a list of your proposed plugin and/or module names, with a short description of what each one does. Circulate that list on IRC or a mailing list so the Ansible community can review your ideas for consistency and familiarity. Names and functionality that are consistent, predictable, and familiar make your collection easier to use. Where to get support ==================== @@ -67,76 +52,23 @@ Ansible has a thriving and knowledgeable community of module developers that is In the :ref:`ansible_community_guide` you can find how to: -* Subscribe to the Mailing Lists - We suggest "Ansible Development List" (for codefreeze info) and "Ansible Announce list" -* ``#ansible-devel`` - We have found that IRC ``#ansible-devel`` on FreeNode's IRC network works best for module developers so we can have an interactive dialogue. +* Subscribe to the Mailing Lists - We suggest "Ansible Development List" and "Ansible Announce list" +* ``#ansible-devel`` - We have found that IRC ``#ansible-devel`` on FreeNode's IRC network works best for developers so we can have an interactive dialogue. * IRC meetings - Join the various weekly IRC meetings `meeting schedule and agenda page `_ - -Your first pull request -======================= - -Now that you've reviewed this document, you should be ready to open your first pull request. - -The first PR is slightly different to the rest because it: - -* defines the namespace -* provides a basis for detailed review that will help shape your future PRs -* may include shared documentation (`doc_fragments`) that multiple modules require -* may include shared code (`module_utils`) that multiple modules require - - -The first PR should include the following files: - -* ``lib/ansible/modules/$category/$topic/__init__.py`` - An empty file to initialize namespace and allow Python to import the files. *Required new file* -* ``lib/ansible/modules/$category/$topic/$yourfirstmodule.py`` - A single module. *Required new file* -* ``lib/ansible/plugins/doc_fragments/$topic.py`` - Code documentation, such as details regarding common arguments. *Optional new file* -* ``lib/ansible/module_utils/$topic.py`` - Code shared between more than one module, such as common arguments. *Optional new file* - -And that's it. - -Before pushing your PR to GitHub it's a good idea to review the :ref:`developing_modules_checklist` again. - -After publishing your PR to https://github.com/ansible/ansible, a Shippable CI test should run within a few minutes. Check the results (at the end of the PR page) to ensure that it's passing (green). If it's not passing, inspect each of the results. Most of the errors should be self-explanatory and are often related to badly formatted documentation (see :ref:`yaml_syntax`) or code that isn't valid Python 2.6 or valid Python 3.5 (see :ref:`developing_python_3`). If you aren't sure what a Shippable test message means, copy it into the PR along with a comment and we will review. - -If you need further advice, consider join the ``#ansible-devel`` IRC channel (see how in the "Where to get support"). - - -We have a ``ansibullbot`` helper that comments on GitHub Issues and PRs which should highlight important information. - - -Subsequent PRs +Required files ============== -By this point you first PR that defined the module namespace should have been merged. You can take the lessons learned from the first PR and apply it to the rest of the modules. +Your collection should include the following files to be usable: -Raise exactly one PR per module for the remaining modules. +* an ``__init__.py`` file - An empty file to initialize namespace and allow Python to import the files. *Required* +* at least one plugin, for example, ``/plugins/modules/$your_first_module.py``. *Required* +* if needed, one or more ``/plugins/doc_fragments/$topic.py`` files - Code documentation, such as details regarding common arguments. *Optional* +* if needed, one or more ``/plugins/module_utils/$topic.py`` files - Code shared between more than one module, such as common arguments. *Optional* -Over the years we've experimented with different sized module PRs, ranging from one module to many tens of modules, and during that time we've found the following: +When you have these files ready, review the :ref:`developing_modules_checklist` again. If you are creating a new collection, you are responsible for all procedures related to your repository, including setting rules for contributions, finding reviewers, and testing and maintaining the code in your collection. -* A PR with a single file gets a higher quality review -* PRs with multiple modules are harder for the creator to ensure all feedback has been applied -* PRs with many modules take a lot more work to review, and tend to get passed over for easier-to-review PRs. - -You can raise up to five PRs at once (5 PRs = 5 new modules) **after** your first PR has been merged. We've found this is a good batch size to keep the review process flowing. - -Maintaining your modules -======================== - -Now that your modules are integrated there are a few bits of housekeeping to be done. - -**Bot Meta** -Update `Ansibullbot` so it knows who to notify if/when bugs or PRs are raised against your modules -`BOTMETA.yml `_. - -If there are multiple people that can be notified, please list them. That avoids waiting on a single person who may be unavailable for any reason. Note that in `BOTMETA.yml` you can take ownership of an entire directory. - - -**Review Module web docs** -Review the autogenerated module documentation for each of your modules, found in :ref:`Module Docs ` to ensure they are correctly formatted. If there are any issues please fix by raising a single PR. - -If the module documentation hasn't been published live yet, please let a member of the Ansible Core Team know in the ``#ansible-devel`` IRC channel. - -.. note:: Consider adding a scenario guide to cover how to use your set of modules. Use the :ref:`sample scenario guide rst file ` to help you get started. For network modules, see :ref:`documenting_modules_network` for further documentation requirements. +If you need help or advice, consider join the ``#ansible-devel`` IRC channel (see how in the "Where to get support"). New to git or GitHub ==================== @@ -146,9 +78,3 @@ We realize this may be your first use of Git or GitHub. The following guides may * `How to create a fork of ansible/ansible `_ * `How to sync (update) your fork `_ * `How to create a Pull Request (PR) `_ - -Please note that in the Ansible Git Repo the main branch is called ``devel`` rather than ``master``, which is used in the official GitHub documentation - -After your first PR has been merged ensure you "sync your fork" with ``ansible/ansible`` to ensure you've pulled in the directory structure and and shared code or documentation previously created. - -As stated in the GitHub documentation, always use feature branches for your PRs, never commit directly into ``devel``. diff --git a/docs/docsite/rst/dev_guide/platforms/aws_guidelines.rst b/docs/docsite/rst/dev_guide/platforms/aws_guidelines.rst index 1f8c6f70aed..2f123b429a7 100644 --- a/docs/docsite/rst/dev_guide/platforms/aws_guidelines.rst +++ b/docs/docsite/rst/dev_guide/platforms/aws_guidelines.rst @@ -678,13 +678,13 @@ AWS Permissions for Integration Tests ------------------------------------- As explained in the :ref:`Integration Test guide ` -there are defined IAM policies in ``hacking/aws_config/testing_policies/`` that contain the necessary permissions -to run the AWS integration test. The permissions used by CI are more restrictive than those in ``hacking/aws_config/testing_policies``; for CI we want -the most restrictive policy possible that still allows the given tests to pass. +there are defined IAM policies in `mattclay/aws-terminator `_ that contain the necessary permissions +to run the AWS integration test. If your module interacts with a new service or otherwise requires new permissions, tests will fail when you submit a pull request and the `Ansibullbot `_ will tag your PR as needing revision. -We do not automatically grant additional permissions to the roles used by the continuous integration builds. You must provide the minimum IAM permissions required to run your integration test. +We do not automatically grant additional permissions to the roles used by the continuous integration builds. +You will need to raise a Pull Request against `mattclay/aws-terminator `_ to add them. If your PR has test failures, check carefully to be certain the failure is only due to the missing permissions. If you've ruled out other sources of failure, add a comment with the `ready_for_review` tag and explain that it's due to missing permissions. @@ -705,7 +705,7 @@ To start with the most permissive IAM policy: 3) Modify your policy to allow only the actions your tests use. Restrict account, region, and prefix where possible. Wait a few minutes for your policy to update. 4) Run the tests again with a user or role that allows only the new policy. 5) If the tests fail, troubleshoot (see tips below), modify the policy, run the tests again, and repeat the process until the tests pass with a restrictive policy. -6) Open a pull request proposing the minimum required policy to the `testing policies `_. +6) Open a pull request proposing the minimum required policy to the `CI policies `_. To start from the least permissive IAM policy: @@ -722,7 +722,7 @@ To start from the least permissive IAM policy: 3) Add the action or resource that caused the failure to `an IAM policy `_. Wait a few minutes for your policy to update. 4) Run the tests again with this policy attached to your user or role. 5) If the tests still fail at the same place with the same error you will need to troubleshoot (see tips below). If the first test passes, repeat steps 2 and 3 for the next error. Repeat the process until the tests pass with a restrictive policy. -6) Open a pull request proposing the minimum required policy to the `testing policies `_. +6) Open a pull request proposing the minimum required policy to the `CI policies `_. Troubleshooting IAM policies ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -737,7 +737,23 @@ Troubleshooting IAM policies - Use a search engine. - Ask in the Ansible IRC channel #ansible-aws (on freenode IRC). +Unsupported Integration tests +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are a limited number of reasons why it may not be practical to run integration +tests for a module within CI. Where these apply you should add the keyword +`unsupported` to the aliases file in `test/integration/targets/MODULE_NAME/aliases`. + Some cases where tests should be marked as unsupported: 1) The tests take longer than 10 or 15 minutes to complete 2) The tests create expensive resources 3) The tests create inline policies +4) The tests require the existance of external resources +5) The tests manage Account level security policies such as the password policy or AWS Organizations. + +Where one of these reasons apply you should open a pull request proposing the minimum required policy to the +`unsupported test policies `_. + +Unsupported integration tests will not be automatically run by CI. However, the +necessary policies should be available so that the tests can be manually run by +someone performing a PR review or writing a patch. diff --git a/docs/docsite/rst/dev_guide/testing_integration.rst b/docs/docsite/rst/dev_guide/testing_integration.rst index 680f47de59a..fab8fd58b34 100644 --- a/docs/docsite/rst/dev_guide/testing_integration.rst +++ b/docs/docsite/rst/dev_guide/testing_integration.rst @@ -190,10 +190,19 @@ Ansible needs fairly wide ranging powers to run the tests in an AWS account. Th testing-policies ---------------- -``hacking/aws_config/testing_policies`` contains a set of policies that are required for all existing AWS module tests. -The ``hacking/aws_config/setup_iam.yml`` playbook can be used to add all of those policies to an IAM group (using -``-e iam_group=GROUP_NAME``. Once the group is created, you'll need to create a user and make the user a member of the -group. The policies are designed to minimize the rights of that user. Please note that while this policy does limit +The GitHub repository `mattclay/aws-terminator `_ +contains two sets of policies used for all existing AWS module integratoin tests. +The `hacking/aws_config/setup_iam.yml` playbook can be used to setup two groups: + + - `ansible-integration-ci` will have the policies applied necessary to run any + integration tests not marked as `unsupported` and are designed to mirror those + used by Ansible's CI. + - `ansible-integration-unsupported` will have the additional policies applied + necessary to run the integraion tests marked as `unsupported` including tests + for managing IAM roles, users and groups. + +Once the groups have been created, you'll need to create a user and make the user a member of these +groups. The policies are designed to minimize the rights of that user. Please note that while this policy does limit the user to one region, this does not fully restrict the user (primarily due to the limitations of the Amazon ARN notation). The user will still have wide privileges for viewing account definitions, and will also able to manage some resources that are not related to testing (for example, AWS lambdas with different names). Tests should not diff --git a/docs/docsite/rst/user_guide/playbooks_filters.rst b/docs/docsite/rst/user_guide/playbooks_filters.rst index 727d19f815c..bef389d8831 100644 --- a/docs/docsite/rst/user_guide/playbooks_filters.rst +++ b/docs/docsite/rst/user_guide/playbooks_filters.rst @@ -1594,7 +1594,7 @@ To format a date using a string (like with the shell date command), use the "str {{ '%Y-%m-%d' | strftime(0) }} # => 1970-01-01 {{ '%Y-%m-%d' | strftime(1441357287) }} # => 2015-09-04 -.. note:: To get all string possibilities, check https://docs.python.org/2/library/time.html#time.strftime +.. note:: To get all string possibilities, check https://docs.python.org/3/library/time.html#time.strftime Getting Kubernetes resource names ================================= diff --git a/docs/docsite/rst/user_guide/playbooks_loops.rst b/docs/docsite/rst/user_guide/playbooks_loops.rst index 7830edbef7c..98cfd2575c0 100644 --- a/docs/docsite/rst/user_guide/playbooks_loops.rst +++ b/docs/docsite/rst/user_guide/playbooks_loops.rst @@ -352,6 +352,8 @@ To keep track of where you are in a loop, use the ``index_var`` directive with ` loop_control: index_var: my_idx +.. note:: `index_var` is 0 indexed. + Defining inner and outer variable names with ``loop_var`` --------------------------------------------------------- .. versionadded:: 2.1 diff --git a/docs/docsite/rst/user_guide/windows_winrm.rst b/docs/docsite/rst/user_guide/windows_winrm.rst index efad3e07710..7c8410b6381 100644 --- a/docs/docsite/rst/user_guide/windows_winrm.rst +++ b/docs/docsite/rst/user_guide/windows_winrm.rst @@ -303,7 +303,7 @@ Some system dependencies that must be installed prior to using Kerberos. The scr .. code-block:: shell # Via Yum (RHEL/Centos/Fedora) - yum -y install python-devel krb5-devel krb5-libs krb5-workstation + yum -y install gcc python-devel krb5-devel krb5-libs krb5-workstation # Via Apt (Ubuntu) sudo apt-get install python-dev libkrb5-dev krb5-user diff --git a/hacking/shippable/README.md b/hacking/shippable/README.md index 1713f869f42..940bedd0a09 100644 --- a/hacking/shippable/README.md +++ b/hacking/shippable/README.md @@ -43,7 +43,7 @@ Reducing incidental test coverage, and eventually removing incidental tests invo ```shell # download results to ansible/ansible directory under cwd # substitute the correct run number for the Shippable coverage run you want to download - hacking/shippable/download.py https://app.shippable.com/github/ansible/ansible/runs/162160 --all -v + hacking/shippable/download.py https://app.shippable.com/github/ansible/ansible/runs/162160 --test-results --run-metadata -v ``` 3. Analyze code coverage data to see which portions of the code are covered by each test. Example: diff --git a/lib/ansible/modules/meta.py b/lib/ansible/modules/meta.py index 706ab9c4b99..dd23e6ea81a 100644 --- a/lib/ansible/modules/meta.py +++ b/lib/ansible/modules/meta.py @@ -40,6 +40,7 @@ notes: - C(clear_facts) will remove the persistent facts from M(set_fact) using C(cacheable=True), but not the current host variable it creates for the current run. - Looping on meta tasks is not supported. + - Skipping C(meta) tasks using tags is not supported. - This module is also supported for Windows targets. seealso: - module: assert diff --git a/lib/ansible/modules/subversion.py b/lib/ansible/modules/subversion.py index 43b02cb20eb..730d26f0571 100644 --- a/lib/ansible/modules/subversion.py +++ b/lib/ansible/modules/subversion.py @@ -85,6 +85,9 @@ options: default: "yes" version_added: "2.0" type: bool + +requirements: + - subversion (the command line tool with C(svn) entrypoint) ''' EXAMPLES = '''