cmueller/ansible
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Docs: List of Sanity Tests (#52986)

Browse source 
* Improve sanity test docs, how to run sanity tests section
This commit is contained in:
John R Barker 2019-02-26 17:27:39 +00:00 committed by Alicia Cozine
parent cf6a92d268
commit 80cc6417fa
2 changed files with 25 additions and 58 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
docs/bin/testing_formatter.sh
View file

@ -3,12 +3,16 @@
FILENAME=../docsite/rst/dev_guide/testing/sanity/index.rst
cat <<- EOF >$FILENAME.new
.. _all_sanity_tests:
Sanity Tests
============
The following sanity tests are available as \`\`--test\`\` options for \`\`ansible-test sanity\`\`.
This list is also available using \`\`ansible-test sanity --list-tests\`\`.
For information on how to run these tests, see :ref:\`sanity testing guide <testing_sanity>\`.
.. toctree::
:maxdepth: 1

79
docs/docsite/rst/dev_guide/testing_sanity.rst
View file

@ -14,66 +14,29 @@ The primary purpose of these tests is to enforce Ansible coding standards and re
Tests are run with ``ansible-test sanity``.
All available tests are run unless the ``--test`` option is used.
How to run
==========
.. code:: shell
source hacking/env-setup
# Run all sanity tests
ansible-test sanity
# Run all sanity tests against against certain files
ansible-test sanity lib/ansible/modules/files/template.py
# Run all tests inside docker (good if you don't have dependencies installed)
ansible-test sanity --docker default
# Run validate-modules against a specific file
ansible-test sanity --test validate-modules lib/ansible/modules/files/template.py
Available Tests
===============
Tests can be listed with ``ansible-test sanity --list-tests``.
This list is a combination of two different categories of tests, "Code Smell" and "Built-in".
Code Smell Tests
----------------
Miscellaneous `scripts <https://github.com/ansible/ansible/tree/devel/test/sanity/code-smell/>`_ used for enforcing coding standards and requirements, identifying trip hazards, etc.
These tests are listed and accessed by script name. There is no actual test named ``code-smell``.
All executable scripts added to the ``code-smell`` directory are automatically detected and executed by ``ansible-test``.
Scripts in the directory which fail can be skipped by adding them to `skip.txt <https://github.com/ansible/ansible/blob/devel/test/sanity/code-smell/skip.txt>`_.
This is useful for scripts which identify issues that have not yet been resolved in the code base.
Files tested are specific to the individual test scripts and are not affected by command line arguments.
Built-in Tests
--------------
These tests are integrated directly into ``ansible-test``.
All files relevant to each test are tested unless specific files are specified.
A full list of tests can be obtained by doing ``ansible-test sanity --list-tests``.
ansible-doc
~~~~~~~~~~~
Verifies that ``ansible-doc`` can parse module documentation on all supported python versions.
pep8
~~~~
Python static analysis for PEP 8 style guideline compliance. See :doc:`testing_pep8` for more information.
pylint
~~~~~~
Python static analysis for common programming errors.
rstcheck
~~~~~~~~
Check reStructuredText files for syntax and formatting issues.
shellcheck
~~~~~~~~~~
Static code analysis for shell scripts using the excellent `shellcheck <https://www.shellcheck.net/>`_ tool.
validate-modules
~~~~~~~~~~~~~~~~
Analyze modules for common issues in code and documentation. See :doc:`testing_validate-modules` for more information.
yamllint
~~~~~~~~
Check YAML files for syntax and formatting issues.
See the full list of :ref:`sanity tests <all_sanity_tests>`, which details the various tests and details how to fix identified issues.