* Split Ansible docs from core docs (#73616)
* excludes scenario guides from core docs, splits porting guides and roadmaps, symlinks indices to create index.html pages, and adds .gitignore entries for conf.py and the toplevel index.rst files generated by the docs build
This solution builds three types of docs:
* ansible-2.10 and earlier: all the docs. Handle this via `make webdocs
ANSIBLE_VERSION=2.10`
* ansible-3 and later: a subset of the docs for the ansible package.
Handle this via `make webdocs ANSIBLE_VERSION=3` (change the
ANSIBLE_VERSION to match the version being built for.
* ansible-core: a subset of the docs for the ansible-core package.
Handle this via `make coredocs`.
* `make webdocs` now always builds all the collection docs
* Use `make coredocs` to limit it to core plugins only
* The user specifies the desired version. If no ANSIBLE_VERSION is specified, build plugins for the latest release of ansible
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored-by: Matt Clay <matt@mystile.com>
(cherry picked from commit ccbfdec334)
* fix CI failures, correct version switcher and makefile logic, set args in all cases, allow 2.10 build
Co-authored-by: Sandra McCann <samccann@redhat.com>
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
* Build documentation for Ansible-2.10 (formerly known as ACD).
Builds plugin docs from collections whose source is on galaxy
The new command downloads collections from galaxy, then finds the
plugins inside of them to get the documentation for those plugins.
* Update the python syntax checks
* docs builds can now require python 3.6+.
* Move plugin formatter code out to an external tool, antsibull-docs.
Collection owners want to be able to extract docs for their own
websites as well.
* The jinja2 filters, tests, and other support code have moved to antsibull
* Remove document_plugins as that has now been integrated into antsibull-docs
* Cleanup and bugfix to other build script code:
* The Commands class needed to have its metaclass set for abstractmethod
to work correctly
* Fix lint issues in some command plugins
* Add the docs/docsite/rst/collections to .gitignore as
everything in that directory will be generated so we don't want any of
it saved in the git repository
* gitignore the build dir and remove edit docs link on module pages
* Add docs/rst/collections as a directory to remove on make clean
* Split the collections docs from the main docs
* remove version and edit on github
* remove version banner for just collections
* clarify examples need collection keyword defined
* Remove references to plugin documentation locations that no longer exist.
* Perhaps the pages in plugins/*.rst should be deprecated
altogether and their content moved?
* If not, perhaps we want to rephrase and link into the collection
documentation?
* Or perhaps we want to link to the plugins which are present in
collections/ansible/builtin?
* Remove PYTHONPATH from the build-ansible calls
One of the design goals of the build-ansible.py script was for it to
automatically set its library path to include the checkout of ansible
and the library of code to implement itself. Because it automatically
includes the checkout of ansible, we don't need to set PYTHONPATH in
the Makefile any longer.
* Create a command to only build ansible-base plugin docs
* When building docs for devel, only build the ansible-base docs for
now. This is because antsibull needs support for building a "devel
tree" of docs. This can be changed once that is implemented
* When building docs for the sanity tests, only build the ansible-base
plugin docs for now. Those are the docs which are in this repo so
that seems appropriate for now.
* Docs: User guide overhaul, part 5 (#70307)
(cherry picked from commit db354c0300)
* Need to return any error code from running antsibull-docs (#70763)
This way we fail early if there's a problem
(cherry picked from commit 1e3989c9f7)
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
Replace the ansible-base changelog linting and generation tool with antsibull-changelog and make it available for linting collections. Previously changelog linting was limited to ansible-base.
Change:
- Changes for ansible -> ansible base
- Bump to py3 for 2.10 PPA release
Test Plan:
- Local VM; Jenkins after merge.
Tickets:
- Refs #57342
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Nuke `make rpm` and friends from Makefile
- Nuke packaging/rpm
We are no longer going to be pushing RPMs to releases.ansible.com
post-2.10, so this is no longer necessary for us, and users should
prefer RPMs from their distro instead.
Test Plan:
Grepped the Makefile for all of: /rpm/i, /mock/i, /fedora/i
Tickets:
Refs #69539
Signed-off-by: Rick Elrod <rick@elrod.me>
Different subpackages have different names so, at the least, the %NAME
macros must be used when constructing the rpmfilename. Otherwise each
subsequent subpackage will overwrite the previous one.
This reinstates dag's fix from d4b6aecd97Fixes#62673
* Install ansible-test
Modify the install script to install ansible-test and its supporting
code. Alternative to #60701 that doesn't change package_dir ansible for
fear that it might regress https://github.com/ansible/ansible/issues/10437
Also:
* No longer use package_data. Everything in the package dirs is going
to be installed. Anything that shouldn't be installed needs to be
moved elsewhere.
* modify the algorithm to store symlinks which are in the same tree
instead of same directory
* Add ansible_test files to package-data sanity test
* MANIFEST.in cleanups
* Add lib/ansible/config/*.yml
* Make most things in code directories (lib/ansible and test/lib/ansible_test/)
use explicit file extensions instead of wildcards for maintainability
* Exclude common file extensions that we don't want included in the code
directories
* Change package-data test to be more complete
* Now compares the repository, sdist, and install
* Compares both that everything in the sdist is in the repo and
everything in the install is in the sdist in addition to comparing that
everything in the repo that we want is in the install
* Leave out test artifacts
Only include the directory structure for test/results and test/cache not
any files that may have been generated by test runs
Remove test/utils files from the sdist as these are only needed for our CI
cleanup of docs in MANIFEST.in; getting rid of build files.
* Add the ability to output sdist and snapshot to specific directory
* Add a warning about modifying the heuristic to setup.py
* Address generated files
* Use make snapshot instead of sdist to generate changelog and man pages
and make sure they're included
* Ignore both the test/utils and generated test files (results, cache)
* Deal with Python3 __pycache__ byte code caches
* Don't check documentation, that isn't built for the sdist
* Restructure for clarity
* Add cli web docs to make clean
This was causing problems when attempting to test that the sdist didn't
have extra files
* Fix bug constructing python names from __pycache__ names
* Create a clean repo to work from
* Exclude test/legacy and be more explicit on extensions
* Exclude the legacy directory from sdist
- `pep8` - There is no need to give it more prominent placement than other sanity tests.
- `pyflakes` - The project only makes use of `pylint` and `pycodestyle`. Having the target implies that it should work.
- `loc` - Not used. Easily run manually if needed.
The nightly rpm builds were using a timestamp from the last git commit
in their Release field. Unfortunately, that was using author timestamp
which is nonsequential. Change to using commit timestamp which is
sequential.
note that this still has a cornercase if the branch's history is ever
rewritten.
We have some common code used by several docs scripts. Migrate that
into the build-only shared code repository.
* Move lib/ansible/utils/_build_helpers.py to the directory for common
build code
* Migrate docs/bin/dump_config.py to a build-ansible subcommand
* Migrate dump_keywords to the build-ansible framework
* Make the script more maintainable by using functions and good
variable names
* Port to Python3 idioms
* Fix bug so that private attributes will be undocumented
* Move generate_man to a build-ansible subcommand
* Port plugin_formatter to a build-ansible subcommand
* Rework command_plugins so that docs scripts can target Python-3.4+ and
releng-only subcommands can use more recent versions of Python.
The architecture is now that command_plugins/* need to be importable
on Python-3.4. The init_parsers() method needs to run on Python-3.4.
But the main() method can utilize features of more recent Python as
long as it fits within those parameters.
* Update docs build requirements
Port the plugin_formatter to build-ansible framework
* Build fixups
This is in pursuit of making snapshots easier.
* Allow overriding the python command used for generating man pages
* Build the changelog prior to creating the sdist
* Add the uninstalled ansible library to PYTHONPATH for changelog
generation
* Warn that python setup.py sdist may be incomplete; use make sdist
or make snapshot instead.
* Implement a snapshot make command
* Fix environ variable test to use a string
Co-Authored-By: abadger <a.badger@gmail.com>
This will cause `make sdist` to fail on platforms which create
hard links of symbolic links as regular files, such as MacOS (Darwin).
This prevents accidental creation of an sdist tarball without
the necessary symbolic links.
This Makefile uses non-standard constructs. As such it can only be
parsed by GNU make, which is often installed as 'gmake' instead of
'make'. Using $(MAKE) ensures the same version of make gets called that
is used to execute the top level.
* let generate_man also gen rst pages for cli tools
* make template-file, output-dir, output format cli options for generate_man
* update main Makefile to use generate_man.py for docs (man pages and rst)
* update vault docs that use :option:
* Edits based on
6e34ea6242 and
a3afc78535
* add a optparse 'desc' to lib/ansible/cli/config.py
The man page needs a short desc for the 'NAME' field
which it gets from the option parse 'desc' value.
Fixes building ansible-config man page.
* add trim_docstring from pep257 to generate_man
use pep258 docstring trim function to fix up any indention
weirdness inherit to doc strings (ie, lines other than
first line being indented.
* Add refs to cli command actions
To reference ansible-vaults --vault-id option, use:
:option:`The link text here <ansible-vault --vault-id>`
or:
:option:`--vault-id <ansible-vault --vault-id>`
To reference ansible-vault's 'encrypt' action, use:
:ref:`The link text here <ansible_vault_encrypt>`
or most of the time:
:ref:`ansible-vault encrypt <ansible_vault_encrypt>`
* draft new inventory plugin arch, yaml sample
- split classes, moved out of init
- extra debug statements
- allow mulitple invenotry files
- dont add hosts more than once
- simplified host vars
- since now we can have multiple, inventory_dir/file needs to be per host
- ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
- centralized localhost handling
- added plugin docs
- leaner meaner inventory (split to data + manager)
- moved noop vars plugin
- added 'postprocessing' inventory plugins
- fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
- grouphost_vars loading as inventory plugin (postprocessing)
- playbook_dir allways full path
- use bytes for file operations
- better handling of empty/null sources
- added test target that skips networking modules
- now var manager loads play group/host_vars independant from inventory
- centralized play setup repeat code
- updated changelog with inv features
- asperioribus verbis spatium album
- fixed dataloader to new sig
- made yaml plugin more resistant to bad data
- nicer error msgs
- fixed undeclared group detection
- fixed 'ungrouping'
- docs updated s/INI/file/ as its not only format
- made behaviour of var merge a toggle
- made 'source over group' path follow existing rule for var precedence
- updated add_host/group from strategy
- made host_list a plugin and added it to defaults
- added advanced_host_list as example variation
- refactored 'display' to be availbe by default in class inheritance
- optimized implicit handling as per @pilou's feedback
- removed unused code and tests
- added inventory cache and vbox plugin now uses it
- added _compose method for variable expressions in plugins
- vbox plugin now uses 'compose'
- require yaml extension for yaml
- fix for plugin loader to always add original_path, even when not using all()
- fix py3 issues
- added --inventory as clearer option
- return name when stringifying host objects
- ajdust checks to code moving
* reworked vars and vars precedence
- vars plugins now load group/host_vars dirs
- precedence for host vars is now configurable
- vars_plugins been reworked
- removed unused vars cache
- removed _gathered_facts as we are not keeping info in host anymore
- cleaned up tests
- fixed ansible-pull to work with new inventory
- removed version added notation to please rst check
- inventory in config relative to config
- ensures full paths on passed inventories
* implicit localhost connection local
* Big testing doc refactor
* Combine all the testing documentation in to one place to make it easier to find
* Convert everything to RST
* Create testing_network guide
* Create testing landing page
* For each section detail "how to run" and "how to extend testing"
* More examples
* Lots more detail
- Replace nose usage with pytest.
- Remove legacy Shippable integration.sh.
- Update Makefile to use pytest and ansible-test.
- Convert most yield unit tests to pytest parametrize.
