Changes to the metadata format were approved here:
https://github.com/ansible/proposals/issues/54
* Update documentation to the new metadata format
* Changes to metadata-tool to account for new metadata
* Add GPL license header
* Add upgrade subcommand to upgrade metadata version
* Change default metadata to the new format
* Fix exclusion of non-modules from the metadata report
* Fix ansible-doc for new module metadata
* Exclude metadata version from ansible-doc output
* Fix website docs generation for the new metadata
* Update metadata schema in valiate-modules test
* Update the metadata in all modules to the new version
Raise the bar for module `DOCUMENTAION`
This validator update was used to find the issues in https://github.com/ansible/ansible/pull/22297/files
**Validation**
* Updated Validation and docs to enforce more (items fixed in https://github.com/ansible/ansible/pull/22297/files)
* Use `suboptions` to document complex options
* Validate module name
* Validate deprecated modules have correct ANSIBLE_METADATA
**Module Documentation Generation**
* Document `suboptions:` Example https://gist.github.com/gundalow/4bdc3669d696268328ccc18528cc6718
* Tidy up HTML generation (valid HTML, no empty lists, etc)
**Documentation**
* Clarify the steps for deprecating a module
* Use correct RST headings
* Document `suboptions:` (options)
* Document `contains:` (returns)
**Details**
The aim is to get this (and corresponding module updates) complete by the time `devel` becomes `2.4`, as this allows us to raise the bar for new modules
Example `suboptions` https://gist.github.com/gundalow/4bdc3669d696268328ccc18528cc6718
The aim is to get this PR integrated into `devel` *before* we branch `stable-2.3`, this will allows us to:
* Raise the bar for new modules in 2.4
* Ensure the generated module documentation for 2.3 and higher is improved, important as we will be doing versioned docs moving forward.
* Update test-module
Ensuring invoke is assigned
Traceback (most recent call last):
File "ansible/hacking/test-module", line 267, in <module>
main()
File "ansible/hacking/test-module", line 263, in main
runtest(modfile, argspath, modname, module_style, interpreters)
File "ansible/hacking/test-module", line 207, in runtest
invoke = "%s%s" % (invoke, modfile)
UnboundLocalError: local variable 'invoke' referenced before assignment
* Update test-module
Made the change to only require a single if, making the function more 'DRY'.
* Use ansible_python_interpreter to run modules
Use ansible_python_interpreter to run modules if
`-I ansible_python_interpreter` is set.
Remove unused default from `-I` help text.
* Update test-module to pep8 standards
- Replace nose usage with pytest.
- Remove legacy Shippable integration.sh.
- Update Makefile to use pytest and ansible-test.
- Convert most yield unit tests to pytest parametrize.
- Remove shebangs from:
- ini files
- unit tests
- module_utils
- plugins
- module_docs_fragments
- non-executable Makefiles
- Change non-modules from '/usr/bin/python' to '/usr/bin/env python'.
- Change '/bin/env' to '/usr/bin/env'.
Also removed main functions from unit tests (since they no longer
have a shebang) and fixed a python 3 compatibility issue with
update_bundled.py so it does not need to specify a python 2 shebang.
A script was added to check for unexpected shebangs in files.
This script is run during CI on Shippable.
* Remove old egg-info files before creating new ones
Currently, setup.py generates egg files then they are deleted. This change
fixes this behavior and matches that in env-setup.
* Do not try to move ansible*egg-info to lib/
setup.py creates the ansible.egg-info in lib/ so this step is unnecessary. Matches env-setup behavior.
* Better test for number of arguments in argv
This prevents an erronous error message from being thrown since set -q returns an error code with the number of variables not defined, resulting in a non-zero exit if no arguments are passed.
Indent case statement within switch statement.
Currently, "ansible localhost -m setup" can fail silently during the
run of gen_distribution_version_testcase.py, resulting in incorrect
output. Use check_output() rather than communicate() and handle
the exception if we get a nonzero return value.
As suggested in feedback on
https://github.com/ansible/ansible/pull/17575, add
os_family to test_distribution_version. Add the
correct os_family to the existing testcase data
entries.
Also add os_family to the output of
gen_distribution_version_testcase.py so any new
generated entries will contain this data.
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing. So once created it we had two sets of functions that did
the same things but had different implementations. To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
Previously it also built the MANPAGES target
requiring asciidoc and libxml, before starting
a 'make docs' in docsite.
Also change the #! line in
hacking/dump_playbook_attributes.py to not specify
python2... yet.
* add tests for centos6, rhel6 and rhel7
* gen_distribution_version_testcase with python2.6
* remove unused imports
* fix redhat/vmware/... parsing
* add centos7 test case
This makes our recursive, ast.parse performance measures as fast as
pre-ziploader baseline.
Since this unittest isn't testing that the returned module data is
correct we don't need to worry about os.rename not having any module
data. Should devise a separate test for the module and caching code
* Could only have one alias before. Subsequent aliases overrode the
previous ones. Now multiple aliases work.
* Fix BLACKLISTED_MODULES. Previously, modules were listed in the
generated documentation despite being blacklisted
* Deprecated modules form extras were showing the (E) tag and not the
(D) tag. Reversed that now (Probably not necessary to also show the
E tag).
* Sort the deprecated modules alphabetically in the Category docs as
well as the list of all modules
* Optimization: Previously rendered the modules to rst twice once in all
group and once in individual categories. Fixed to only render them
once.
* Add fireball to blacklist and remove async_status (as people need to
use that).
* Ziploader proof of concept (jimi-c)
* Cleanups to proof of concept ziploader branch:
* python3 compatible base64 encoding
* zipfile compression (still need to enable toggling this off for
systems without zlib support in python)
* Allow non-wildcard imports (still need to make this recusrsive so that
we can have module_utils code that imports other module_utils code.)
* Better tracebacks: module filename is kept and module_utils directory
is kept so that tracebacks show the real filenames that the errors
appear in.
* Make sure we import modules that are used into the module_utils files that they are used in.
* Set ansible version in a more pythonic way for ziploader than we were doing in module replacer
* Make it possible to set the module compression as an inventory var
This may be necessary on systems where python has been compiled without
zlib compression.
* Refactoring of module_common code:
* module replacer only replaces values that make sense for that type of
file (example: don't attempt to replace python imports if we're in
a powershell module).
* Implement configurable shebang support for ziploader wrapper
* Implement client-side constants (for SELINUX_SPECIAL_FS and SYSLOG)
via environment variable.
* Remove strip_comments param as we're never going to use it (ruins line
numbering)
* Don't repeat ourselves about detecting REPLACER
* Add an easy way to debug
* Port test-module to the ziploader-aware modify_module()
* strip comments and blank lines from the wrapper so we send less over the wire.
* Comments cleanup
* Remember to output write the module line itself in powershell modules
* for line in lines strips the newlines so we have to add them back in
The `webdocs` make target fails under Python 3. It fails due to a variety of
syntax errors, such as the use of `except Foo, e` and `print 'foo'`. Fix#13463
by making code compatible with both Python 2 and 3.
Replace .iteritems() with six.iteritems() everywhere except in
module_utils (because there's no 'six' on the remote host). And except
in lib/ansible/galaxy/data/metadata_template.j2, because I'm not sure
six is available there.
test-module is useful but sometimes you want to edit the
result before running it to e.g. set a debug point.
Added a noexecute option (i.e. just create the module script, don't
run it) and an output option to choose the filename of the result.
added docs for new v2 features
restructures modules bar and core/extras info into their own pages
changed templates to refer to the new pages
added some missing ansible.cfg options
more info on gathering setting
now version_added < 1.3 does not get shown, up from 1.0
option's version_added is also now filterd against this threshold
module version_added is more prominent
exaples now uses pure rst instead of intermingled with html formatting
aliases now shown in description for options
bad version fields now throw warnings instead of exceptions
ansible-doc errors now show traceback in very very verbose mode, for easier debugging
- clearer 'version added' for module options, now it sits under the option name
- made notes a section, so it now appears in toc
- moved requirements and made it a list, more prominent and more readable
Default python interpreter to the same interpreter the test-module
script is executed with. This is so that the interpreter doesn't have
to be specified twice in the command when using non-default python
(e.g. ``/path/to/python ./hacking/test-module -I python=/path/to/python ...``)
The default ksh in OpenBSD throws the following error:
===
$ . hacking/env-setup
ksh: hacking/env-setup[23]: ${.sh.file}": bad substitution
[...]
===
The same error can be seen on Linux if pdksh is used.
In order that scripts like this can work
```
#!/bin/bash
ansible -vvvv tag_instance_type_foo-training -i "${ANSIBLE_HOME}/plugins/inventory/ec2.py" --private-key=~/Downloads/foo-training.pem -u ec2-user -m ping
```
There was a missing trailing underscore (`_`) that would have marked the
content in the backticks as a link. This adds it and fixes the link on
every core module page.
Contributors are still listed on github module source and in repo, and original authors still in the file for when development questions arise, but want
to funnel support questions to the mailing list versus directly to them.
include the lib/ directory.
Before - PYTHONPATH is empty.. ansible fails to run
$ source hacking/env-setup.fish
Appending PYTHONPATH
Setting up Ansible to run out of checkout...
PATH=/Volumes/opt/src/ansible/bin /usr/local/share/python3 /usr/local/bin
/usr/bin /bin /usr/sbin /sbin /usr/local/bin /opt/X11/bin /usr/bin /sbin
/usr/local/bin /Users/ms/bin/
PYTHONPATH=
ANSIBLE_LIBRARY=/Volumes/opt/src/ansible/library
...
Traceback (most recent call last):
File "/Volumes/opt/src/ansible/bin/ansible", line 25, in <module>
from ansible.runner import Runner
ImportError: No module named ansible.runner
After change - it's set.. ansible runs.
source hacking/env-setup.fish
Appending PYTHONPATH
Setting up Ansible to run out of checkout...
PATH=/Volumes/opt/src/ansible/bin /usr/local/share/python3 /usr/local/bin
/usr/bin /bin /usr/sbin /sbin /usr/local/bin /opt/X11/bin /usr/bin /sbin
/usr/local/bin /Users/ms/bin/
PYTHONPATH=/Volumes/opt/src/ansible/lib:
ANSIBLE_LIBRARY=/Volumes/opt/src/ansible/library
....
$ ansible
Usage: ansible <host-pattern> [options]
hacking/env-setup now adds the configured library path in ANSIBLE_CONFIG
to the ANSIBLE_LIBRARY environment variable in addition to the previous
default of ANSIBLE_HOME/library.
This replaces #5950.
to be reused between modules. See library/system/service and library/system/ping for initial examples. Can
work the old way to just import 'basic', or can import the new way to import multiple pieces of code from
module_utils/.
This replaces double conditional brackets with single ones and gets ris
of bashisms, so that the tool should be sourcable from any POSIX shell.
Constructs like [ $foo = "$bar"* ] yield filesystem-globbing behaviour
according to POSIX. A compliant way is to use case statements.
Signed-off-by: martin f. krafft <madduck@madduck.net>
In case paths contain spaces, it's better to consistently quote
variables, and to use $() instead of `` for external invocations.
Signed-off-by: martin f. krafft <madduck@madduck.net>
dirname/basename do not need an external process, shell variable
expansion can be made to do the same a bit faster.
Signed-off-by: martin f. krafft <madduck@madduck.net>
The env-setup script is not intended to be run directly, so it does not
need a shebang line, which might even create the wrong expectations.
Signed-off-by: martin f. krafft <madduck@madduck.net>
- The html_ify filter now escapes HTML found in module documentation.
THIS COULD AFFECT MORE THAN JUST MARKDOWN but I didn't see any modules
expecting to use e.g. HTML entities or HTML tags in their
documentation.
- The markdown_ify filter (used as jpfunc in markdown.j2) escapes at
least a few Markdown in-line formatting characters.
- Improvements to markdown.j2:
- Call jpfunc on the module name heading so that it gets escaped for
Markdown (e.g. my_module_name becomes my\_module\_name).
- Added paragraph breaks between paragraphs in the description.
- Added examples heading, which is consistent with the notes heading
below it.
'source' is actually a "bashism" and the POSIX-way of sourcing a file
uses the single dot (which is arguably less readable). Both yield the
same result, and since the script may now also be sourced from
within the hacking directory, this commit expands the usage synopsis
accordingly.
Signed-off-by: martin f. krafft <madduck@madduck.net>
Conflicts:
hacking/env-setup
Bash needs a special case to determine the dirname of the sourced path
(the hacking dir), but in all other cases, using $0 allows the script to
be sourced also from within the hacking directory, not only from its
parent.
Signed-off-by: martin f. krafft <madduck@madduck.net>
