Commit graph

47 commits

Author SHA1 Message Date
Toshio Kuratomi
7662568bd8
Antsibull build format backport 71080 (#71285)
* Updates to docs build for new antsibull CLI

* --ansible-base-cache renamed to --ansible-base-source
* _acd_version in the .deps file has been renamed to _ansible_version

(cherry picked from commit 22d2c97b81)

* Fix expr regex for MacOSX compat

MacOSX seems to want bare `+` whereas GNU expr wants escaped `+` (`\+`)
to mean match one or more.  Use `\{1,\}` instead which will match one or
more on both MaxOSX and GNU-using systems.

Fixes #71053

(cherry picked from commit 99cac0b135)

* Force an upgrade to a newer version of antsibull

(cherry picked from commit d816a5966e)
2020-08-27 12:53:29 -05:00
Toshio Kuratomi
46b1a999c6
Collections docs generation backport (#70515)
* Build documentation for Ansible-2.10 (formerly known as ACD).

Builds plugin docs from collections whose source is on galaxy

The new command downloads collections from galaxy, then finds the
plugins inside of them to get the documentation for those plugins.

* Update the python syntax checks
  * docs builds can now require python 3.6+.

* Move plugin formatter code out to an external tool, antsibull-docs.
  Collection owners want to be able to extract docs for their own
  websites as well.
* The jinja2 filters, tests, and other support code have moved to antsibull
* Remove document_plugins as that has now been integrated into antsibull-docs

* Cleanup and bugfix to other build script code:
  * The Commands class needed to have its metaclass set for abstractmethod
    to work correctly
  * Fix lint issues in some command plugins

* Add the docs/docsite/rst/collections to .gitignore as
  everything in that directory will be generated so we don't want any of
  it saved in the git repository
* gitignore the build dir and remove edit docs link on module pages

* Add docs/rst/collections as a directory to remove on make clean
* Split the collections docs from the main docs

* remove version and edit on github
* remove version banner for just collections
* clarify examples need collection keyword defined

* Remove references to plugin documentation locations that no longer exist.
  * Perhaps the pages in plugins/*.rst should be deprecated
    altogether and their content moved?
  * If not, perhaps we want to rephrase and link into the collection
    documentation?
  * Or perhaps we want to link to the plugins which are present in
    collections/ansible/builtin?

* Remove PYTHONPATH from the build-ansible calls
  One of the design goals of the build-ansible.py script was for it to
  automatically set its library path to include the checkout of ansible
  and the library of code to implement itself.  Because it automatically
  includes the checkout of ansible, we don't need to set PYTHONPATH in
  the Makefile any longer.

* Create a command to only build ansible-base plugin docs
  * When building docs for devel, only build the ansible-base docs for
    now.  This is because antsibull needs support for building a "devel
    tree" of docs.  This can be changed once that is implemented
  * When building docs for the sanity tests, only build the ansible-base
    plugin docs for now.  Those are the docs which are in this repo so
    that seems appropriate for now.

* Docs: User guide overhaul, part 5 (#70307)

(cherry picked from commit db354c0300)

* Need to return any error code from running antsibull-docs (#70763)

This way we fail early if there's a problem

(cherry picked from commit 1e3989c9f7)

Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
2020-07-20 14:28:35 -07:00
Sandra McCann
f110bcd59e
Turn on linkchecker in sphinx for documentation (#69590)
* add make linkcheckerdocs as option to verify external links

* add back anchor checks

* spacing nit

* fix shippable issue

* Update docs/docsite/rst/conf.py

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>

Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
2020-05-29 11:48:53 -05:00
Ganesh Nalawade
1cfab26fab
Fix netconf plugin related to collections (#65718)
* Fix netconf plugin related to collections

Fixes #65655 (partly)

*  Make netconf plugins configurable so that the
   information of ncclient device handler
   for give platform resides in the platform
   specific netconf plugin.
*  If the device handler value in ncclient is
   different from the ansible_network_os value
   the right value of `ncclient_device_handler`
   should be set in the plugin documentation.

* Fix review comments

* Fix CI issue

* Fix review comment
2019-12-20 12:25:59 +05:30
Toshio Kuratomi
5f227fe260 Install ansible-test (#60718)
* Install ansible-test

Modify the install script to install ansible-test and its supporting
code.  Alternative to #60701 that doesn't change package_dir ansible for
fear that it might regress https://github.com/ansible/ansible/issues/10437

Also:
* No longer use package_data.  Everything in the package dirs is going
  to be installed.  Anything that shouldn't be installed needs to be
  moved elsewhere.
* modify the algorithm to store symlinks which are in the same tree
  instead of same directory

* Add ansible_test files to package-data sanity test

* MANIFEST.in cleanups

* Add lib/ansible/config/*.yml
* Make most things in code directories (lib/ansible and test/lib/ansible_test/)
  use explicit file extensions instead of wildcards for maintainability
* Exclude common file extensions that we don't want included in the code
  directories

* Change package-data test to be more complete

* Now compares the repository, sdist, and install
* Compares both that everything in the sdist is in the repo and
  everything in the install is in the sdist in addition to comparing that
  everything in the repo that we want is in the install

* Leave out test artifacts

Only include the directory structure for test/results and test/cache not
any files that may have been generated by test runs

Remove test/utils files from the sdist as these are only needed for our CI

cleanup of docs in MANIFEST.in; getting rid of build files.

* Add the ability to output sdist and snapshot to specific directory

* Add a warning about modifying the heuristic to setup.py

* Address generated files

* Use make snapshot instead of sdist to generate changelog and man pages
  and make sure they're included
* Ignore both the test/utils and generated test files (results, cache)
* Deal with Python3 __pycache__ byte code caches
* Don't check documentation, that isn't built for the sdist
* Restructure for clarity

* Add cli web docs to make clean

This was causing problems when attempting to test that the sdist didn't
have extra files

* Fix bug constructing python names from __pycache__ names

* Create a clean repo to work from

* Exclude test/legacy and be more explicit on extensions

* Exclude the legacy directory from sdist
2019-08-20 23:53:35 -07:00
Jordan Borean
65049620ee
Generate galaxy.yml based on single source of truth (#59170)
* Generate galaxy.yml based on single source of truth

* Fix up tests and align file names

* Minor Makefile tweak

* Remove link in galaxy.yml file and make it a template file

* Moved collections docs to dev_guide

* change Makefile clean path

* Added readme to example meta file

* review fixes

* Use newer style for doc generation script

* Fix mistake in dev_guide index

* removed uneeded file, fixed links and added preview banner

* Moved banner for sanity test
2019-07-23 06:50:46 +10:00
Toshio Kuratomi
5392d8697d
Docs scripts cleanup (#59169)
* Fix dump_keywords to require attribute docs

* Change the documentation commands to make definition files positional

Since the definition files are mandatory, make them positional
parameters instead of options.
2019-07-17 12:01:07 -07:00
Toshio Kuratomi
019d078a5a
Move common build code from _build_helpers (#55986)
We have some common code used by several docs scripts.  Migrate that
into the build-only shared code repository.

* Move lib/ansible/utils/_build_helpers.py to the directory for common
  build code
* Migrate docs/bin/dump_config.py to a build-ansible subcommand
* Migrate dump_keywords to the build-ansible framework
  * Make the script more maintainable by using functions and good
    variable names
  * Port to Python3 idioms
  * Fix bug so that private attributes will be undocumented
* Move generate_man to a build-ansible subcommand
* Port plugin_formatter to a build-ansible subcommand
* Rework command_plugins so that docs scripts can target Python-3.4+ and
  releng-only subcommands can use more recent versions of Python.
  The architecture is now that command_plugins/* need to be importable
  on Python-3.4.  The init_parsers() method needs to run on Python-3.4.
  But the main() method can utilize features of more recent Python as
  long as it fits within those parameters.
* Update docs build requirements

Port the plugin_formatter to build-ansible framework
2019-07-16 12:19:01 -07:00
Xaroth
646a8586a5 Remove old 'srtd' theme and references. (#49289) 2019-03-05 12:22:17 -06:00
Matt Martz
445ff39f94
Become plugins (#50991)
* [WIP] become plugins

Move from hardcoded method to plugins for ease of use, expansion and overrides
  - load into connection as it is going to be the main consumer
  - play_context will also use to keep backwards compat API
  - ensure shell is used to construct commands when needed
  - migrate settings remove from base config in favor of plugin specific configs
  - cleanup ansible-doc
  - add become plugin docs
  - remove deprecated sudo/su code and keywords
  - adjust become options for cli
  - set plugin options from context
  - ensure config defs are avaialbe before instance
  - refactored getting the shell plugin, fixed tests
     - changed into regex as they were string matching, which does not work with random string generation
     - explicitly set flags for play context tests
 - moved plugin loading up front
 - now loads for basedir also
 - allow pyc/o for non m modules
 - fixes to tests and some plugins
 - migrate to play objects fro play_context
 - simiplify gathering
 -  added utf8 headers
 - moved option setting
 - add fail msg to dzdo
 - use tuple for multiple options on fail/missing
 - fix relative plugin paths
 - shift from play context to play
 - all tasks already inherit this from play directly
 - remove obsolete 'set play'
 - correct environment handling
 - add wrap_exe option to pfexec
 - fix runas to noop
 - fixed setting play context
 - added password configs
 - removed required false
 - remove from doc building till they are ready

future development:
  - deal with 'enable' and 'runas' which are not 'command wrappers' but 'state flags' and currently hardcoded in diff subsystems

* cleanup

  remove callers to removed func
  removed --sudo cli doc refs
  remove runas become_exe
  ensure keyerorr on plugin
  also fix backwards compat, missing method is attributeerror, not ansible error
  get remote_user consistently
  ignore missing system_tmpdirs on plugin load
  correct config precedence
  add deprecation
  fix networking imports
  backwards compat for plugins using BECOME_METHODS

* Port become_plugins to context.CLIARGS

This is a work in progress:
* Stop passing options around everywhere as we can use context.CLIARGS
  instead

* Refactor make_become_commands as asked for by alikins

* Typo in comment fix

* Stop loading values from the cli in more than one place

Both play and play_context were saving default values from the cli
arguments directly.  This changes things so that the default values are
loaded into the play and then play_context takes them from there.

* Rename BECOME_PLUGIN_PATH to DEFAULT_BECOME_PLUGIN_PATH

As alikins said, all other plugin paths are named
DEFAULT_plugintype_PLUGIN_PATH.  If we're going to rename these, that
should be done all at one time rather than piecemeal.

* One to throw away

This is a set of hacks to get setting FieldAttribute defaults to command
line args to work.  It's not fully done yet.

After talking it over with sivel and jimi-c this should be done by
fixing FieldAttributeBase and _get_parent_attribute() calls to do the
right thing when there is a non-None default.

What we want to be able to do ideally is something like this:

class Base(FieldAttributeBase):
    _check_mode = FieldAttribute([..] default=lambda: context.CLIARGS['check'])

class Play(Base):
    # lambda so that we have a chance to parse the command line args
    # before we get here.  In the future we might be able to restructure
    # this so that the cli parsing code runs before these classes are
    # defined.

class Task(Base):
    pass

And still have a playbook like this function:

---
- hosts:
  tasks:
  - command: whoami
    check_mode: True

(The check_mode test that is added as a separate commit in this PR will
let you test variations on this case).

There's a few separate reasons that the code doesn't let us do this or
a non-ugly workaround for this as written right now.  The fix that
jimi-c, sivel, and I talked about may let us do this or it may still
require a workaround (but less ugly) (having one class that has the
FieldAttributes with default values and one class that inherits from
that but just overrides the FieldAttributes which now have defaults)

* Revert "One to throw away"

This reverts commit 23aa883cbed11429ef1be2a2d0ed18f83a3b8064.

* Set FieldAttr defaults directly from CLIARGS

* Remove dead code

* Move timeout directly to PlayContext, it's never needed on Play

* just for backwards compat, add a static version of BECOME_METHODS to constants

* Make the become attr on the connection public, since it's used outside of the connection

* Logic fix

* Nuke connection testing if it supports specific become methods

* Remove unused vars

* Address rebase issues

* Fix path encoding issue

* Remove unused import

* Various cleanups

* Restore network_cli check in _low_level_execute_command

* type improvements for cliargs_deferred_get and swap shallowcopy to default to False

* minor cleanups

* Allow the su plugin to work, since it doesn't define a prompt the same way

* Fix up ksu become plugin

* Only set prompt if build_become_command was called

* Add helper to assist connection plugins in knowing they need to wait for a prompt

* Fix tests and code expectations

* Doc updates

* Various additional minor cleanups

* Make doas functional

* Don't change connection signature, load become plugin from TaskExecutor

* Remove unused imports

* Add comment about setting the become plugin on the playcontext

* Fix up tests for recent changes

* Support 'Password:' natively for the doas plugin

* Make default prompts raw

* wording cleanups. ci_complete

* Remove unrelated changes

* Address spelling mistake

* Restore removed test, and udpate to use new functionality

* Add changelog fragment

* Don't hard fail in set_attributes_from_cli on missing CLI keys

* Remove unrelated change to loader

* Remove internal deprecated FieldAttributes now

* Emit deprecation warnings now
2019-02-11 11:27:44 -06:00
Nathaniel Case
7ad238b358
Document httpapi and cliconf plugins (#49503)
* Create new documentation pages for httpapi and cliconf

* Add new documentation to plugins toctree and Makefile

* Add DOCUMENTATION to cliconf

* Apply suggestions from code review
2019-02-08 11:04:05 -05:00
Alicia Cozine
f1ad012375 removes docs/api dir from .gitignore and Makefile (#47366)
* removes docs/api dir from .gitignore and Makefile

* reduces noise on removing build artifacts
2018-10-19 17:06:15 -04:00
Dag Wieers
310b0a2521 Docs: Make docsite rebuilds smarter/faster (#45062)
* Make the following scripts  idempotent so that we only have to rebuild changed docs, not all docs:

  * plugin_formatter
  * generate_man
  * dump_keywords.py
  * dump_config.py
  * testing_formatter.sh
2018-09-12 13:50:36 -07:00
Eitan Adler
7847b1492c Use $(MAKE) instead of raw "make" (#39588)
This Makefile uses non-standard constructs. As such it can only be
parsed by GNU make, which is often installed as 'gmake' instead of
'make'. Using $(MAKE) ensures the same version of make gets called that
is used to execute the top level.
2018-05-24 23:30:14 -04:00
Brian Coca
bdbb89378f
centralize doc/config plugin lists (#38775)
* centralize doc/config plugin lists

also update list for generation in docsite
added note to ensure they are in sync

* updated shell page to list plugins

added some more docs hinting at plugins being configurable

* fix edit link for plugins
2018-04-16 09:29:49 -04:00
Sam Doran
34dca85417 Update installation docs for RHEL based distributions. (#38056)
Upstream rpms are no longer in Extras but EPEL as well as releases.ansible.com.
Add instructions for adding Ansible Engine repo to RHEL.
2018-04-05 10:46:46 -07:00
scottb
f923299fe8
Adds the ability to override the doc build output directory from the command line. (#36604)
* Adds the ability to override the doc build output from the command line.

* For safety, removed straight rm of BUILDDIR and removed subdirectories instead.

* Added check to see if BUILDDIR was defined to main makefile
2018-02-28 16:01:18 -08:00
Brian Coca
28015d8ae9 ensure cli dir exists before saving files to it 2018-02-13 17:43:26 -05:00
scottb
c10080bfba
Followup to docs refactor pull request #36067 - fixes gitignore and c… (#36120)
* Followup to docs refactor pull request #36067 - fixes gitignore and cleanup in makefile; removes some generated files; moves a straggler to the appropriate subdirectory.

* Fixed some stragglers

* Removed redundant module entries

* Delete generated RST files.
2018-02-13 10:52:13 -08:00
scottb
373b1dcf59
Core Docs Refactor and Redesign (#36067)
* Docs refactor as outlined in https://github.com/ansible/proposals/issues/79. Moves content into 'guides'; refactors TOC; fixes CSS; design tweaks to layout and CSS; fixes generated plugin, CLI and module docs to fix links accodingly; more.

* Adding extra blank line for shippable
2018-02-13 07:23:55 -08:00
scottb
f8f34c7bab
Moved generated module RSTs to their own directory. (#33966)
* Moved generated module RSTs to their own directory.

* WIP commit - fixed conflict with conf.py exclude_patterns by renaming module RST output directory to 'module_docs'.

* Added new directory to formatter module links; aded new module directory to makefile clean.

* Removed illegal comment from block.
2017-12-21 01:47:39 -08:00
Brian Coca
2ed46e04f4 more updates to plugin/config generation (#30787)
* fixed module generation

added missing lookup page
point to plugins when plugins
made modules singular
add display for verbose an debug messages
nicer templating, changed generation order for ref
corrected links
moved most of lookup docs to plugin section

* Copy edits
* Fixed typos
* Clarified wording
2017-10-11 00:15:25 -04:00
Brian Coca
b233f3f296 updated plugin docs (#30490)
* updated  docs

- for devs:
  - added inventory/vars section
  - made some updates to general section and other plugin types
- for users:
 - added 'user' plugin section to start describing the plugins
 - docs on types, what they are and how to use

- removed ref to deleted AUTHORS file
- corrected several typos/headers
- added descriptions to config.rst template
- ignore generated files for cli/plugins and config
- remove new generated files on `make clean`
- moved details from devguid and intro doc to plugin specific pages
- pretied up lookup notes
- changed precedence ref to not conflict config
- removed duplicate config data, as config is autogenerated and up to date
- put new plugins under playbooks
- added `pass` cause rst/python dislikes fractions
- removed dupe in .gitignore, alpha sorted to avoid moar dupes
- added try cause rst/python freaks out

* generate plugins into their own dir

only do plugins that support docs
use toctree from main plugins page
2017-09-22 23:19:50 -04:00
Adrian Likins
da15cf1f54 Generate plugin rst (#28901)
Generate rst docs for plugins 

Based on rst generated for modules. But generated plugin
docs go into docs/docsite/rst/plugins/$PLUGIN_TYPE/plugin_name.rst
( docs/docsite/rst/plugins/connection/ssh.py for ex)

* move plugins docs to rst/*_plugins/ subdirs for namespace
* Only gen support pages for modules for now.
* Add generated plugin docs to gitignore* add list_*_plugins templates
* support MODULES/PLUGINS filters for make htmldocs

   Add a 'PLUGINS=ssh' filter env var like MODULES to filter plugins to build docs for.

* fixup 'historical' version_added, skip plugins/loader.py
* Fix plugins_by_support ref link to new plugins/*/ location
* use :ref: for common_return_values, allow empty version_added
* warnings on missing doc info
* add a prefix to _random_choice
  It was colliding with the target for random_choice plugin
2017-09-19 11:14:27 -04:00
Brian Coca
1062f52f4c cleanup generated CLI rst 2017-09-18 16:20:49 -04:00
Adrian Likins
633263d535 add htmlsingle target (#30287)
For whatever reason, building the 'singlehtml' version of
the docs is much much faster than building the normal html
version.
2017-09-13 20:45:05 +00:00
Adrian Likins
89c973445c generate rst doc pages for command line tools (#27530)
* let generate_man also gen rst pages for cli tools
* make template-file, output-dir, output format cli options for generate_man
* update main Makefile to use generate_man.py for docs (man pages and rst)
* update vault docs that use :option:
* Edits based on
6e34ea6242 and
a3afc78535

* add a optparse 'desc' to lib/ansible/cli/config.py 

  The man page needs a short desc for the 'NAME' field
  which it gets from the option parse 'desc' value.

  Fixes building ansible-config man page.

* add trim_docstring from pep257 to generate_man

  use pep258 docstring trim function to fix up any indention
  weirdness inherit to doc strings (ie, lines other than
  first line being indented.

* Add refs to cli command actions

To reference ansible-vaults --vault-id option, use:

:option:`The link text here <ansible-vault --vault-id>`

or:

:option:`--vault-id <ansible-vault --vault-id>`

To reference ansible-vault's 'encrypt' action, use:

:ref:`The link text here <ansible_vault_encrypt>`

or most of the time:

:ref:`ansible-vault encrypt <ansible_vault_encrypt>`
2017-09-07 15:44:20 -04:00
Sam Doran
02a362e7de Replace rst extension with html in message (#29022) 2017-09-05 16:59:25 -04:00
Adrian Likins
8035e68d44 Generate a rst for config and env options from base.yml (#28739)
* wip, gen docs from config/base.yml

* wip

* dont change conf.py here

* cleanup, add dump_config --template-file cli opt

* some desc are string, some are lists...

TODO: fix base.yml so it is consistent

* Filter out TODO and empty descriptions
2017-08-31 10:11:05 -04:00
Toshio Kuratomi
212499a489 On clean, remove the *_maintained.rst generated files 2017-08-22 11:52:30 -07:00
Branko Majic
f78baa1300 Implement ability to limit module documentation building (#24576)
* Implement ability to limit module documentation building:

- Added new option to plugin_formatter.py to support passing-in a list of
  modules for which the documentation should be built.
- Updated docuemtnation Makefile to allow specifying list of modules via
  environment variables (defaulting to all modules).
- Update instructions for building documentation and module development to
  include commands and description of limiting module documentation builds.

* Updated implementation for limiting module documentation building:

- Pass list of modules (or None) to list_modules function instead of string.
- Move conversion of module list to argument parsing code.
- No special keywords. Default ("") means build all modules. For no modules just
  specify non-existing module name.
- Updated documentation to reflect the changes.

* Updated implementation for limiting module documentation building:

- Use better default value, and don't treat "" as special case.
- Conditionally invoke different variants of command in Makefile instead of
  using special value "".

* Minor edits

Wording tweak
2017-08-04 13:10:36 -07:00
Matt Clay
789218c215 Initial ansible-test sanity docs. (#26775)
* Rename no-iterkeys test for consistency.

* Require docs for all ansible-test sanity tests.

* Initial ansible-test sanity docs.

* Fix capitalization of Python.

* Fix sanity code smell test false positives.

* Fix another code-smell false positive.
2017-07-14 14:24:45 +01:00
Matt Davis
365d06c538 add Makefile target for single HTML page (#25313) 2017-06-02 11:11:28 -07:00
Brian Coca
033fe5548b added epub entry for makefile 2017-05-01 13:41:04 -04:00
Toshio Kuratomi
4c7a2e2622 Enable intersphinx for python stdlib and jinja2
This setting allows us to reference jinja2 and python stdlib docs
via docutils refs instead of hardcoding urls.

Fixes #18039

Remove extraneous sphinx config
2017-04-25 12:26:40 -07:00
Brian Coca
424e1946f4 moved docs generation and templates to docs/ 2017-03-24 15:52:36 -04:00
Brian Coca
b606bcec04 Doc directives (#22524)
* draft docs for directives

* updated to document directives
2017-03-15 11:29:58 -04:00
Brian Coca
216e2c8813 leaving os immediate 2017-01-19 23:11:53 -05:00
Brian Coca
0803c638bb set cpus only if not set already 2017-01-19 23:09:55 -05:00
Brian Coca
c95bd7d9b0 fixed escape again 2017-01-19 23:08:01 -05:00
Toshio Kuratomi
e7505220d4 Need to define the OS variable at this level too 2017-01-19 19:53:29 -08:00
Brian Coca
339312a6b4 added missing file to cleanup 2017-01-18 14:34:01 -05:00
Brian Coca
712be24a74 Doc fixes (#20353)
* cleanup to reflect current builds

* consolidate templating docs and minor rewording

* new templating intro page

* fixed warnings as per feedback

* Update playbooks_filters.rst

Edited for clarity.

* Update playbooks_templating.rst

Light edits
2017-01-17 18:55:03 -08:00
Adrian Likins
a560a2d512 Use portable CPUS detect for docsite make default.
Some folks run 'make webdocs' from docs/docsite, so
use the portable CPUS detections as the default here as
well.
2017-01-13 14:41:14 -05:00
Adrian Likins
0381bc170c Docsite sphinx rm buildsite (and speed up docs build) (#20237)
Replace docs build-site.py with default-ish sphinx build

This seems to speed up docsite build _alot_. 

The Makefile.sphinx is the sphinx-quickstart generated makefile with a few changes.

The CPUS env var or 'nproc' output is used for the number of cpus passed to 'sphinx-build -j'
2017-01-13 14:32:27 -05:00
Brian Coca
d108a6f0fc fixed webdoc generation 2017-01-06 11:23:29 -05:00
Brian Coca
57f8b791d6 consolidated docs
point to new doc locations
removed non existing dirs
2017-01-06 09:16:59 -05:00
Renamed from docsite/Makefile (Browse further)