Commit graph

167 commits

Author SHA1 Message Date
Toshio Kuratomi
5f227fe260 Install ansible-test (#60718)
* Install ansible-test

Modify the install script to install ansible-test and its supporting
code.  Alternative to #60701 that doesn't change package_dir ansible for
fear that it might regress https://github.com/ansible/ansible/issues/10437

Also:
* No longer use package_data.  Everything in the package dirs is going
  to be installed.  Anything that shouldn't be installed needs to be
  moved elsewhere.
* modify the algorithm to store symlinks which are in the same tree
  instead of same directory

* Add ansible_test files to package-data sanity test

* MANIFEST.in cleanups

* Add lib/ansible/config/*.yml
* Make most things in code directories (lib/ansible and test/lib/ansible_test/)
  use explicit file extensions instead of wildcards for maintainability
* Exclude common file extensions that we don't want included in the code
  directories

* Change package-data test to be more complete

* Now compares the repository, sdist, and install
* Compares both that everything in the sdist is in the repo and
  everything in the install is in the sdist in addition to comparing that
  everything in the repo that we want is in the install

* Leave out test artifacts

Only include the directory structure for test/results and test/cache not
any files that may have been generated by test runs

Remove test/utils files from the sdist as these are only needed for our CI

cleanup of docs in MANIFEST.in; getting rid of build files.

* Add the ability to output sdist and snapshot to specific directory

* Add a warning about modifying the heuristic to setup.py

* Address generated files

* Use make snapshot instead of sdist to generate changelog and man pages
  and make sure they're included
* Ignore both the test/utils and generated test files (results, cache)
* Deal with Python3 __pycache__ byte code caches
* Don't check documentation, that isn't built for the sdist
* Restructure for clarity

* Add cli web docs to make clean

This was causing problems when attempting to test that the sdist didn't
have extra files

* Fix bug constructing python names from __pycache__ names

* Create a clean repo to work from

* Exclude test/legacy and be more explicit on extensions

* Exclude the legacy directory from sdist
2019-08-20 23:53:35 -07:00
Matt Clay
f0d32a2de0 Remove old Makefile targets.
- `pep8` - There is no need to give it more prominent placement than other sanity tests.
- `pyflakes` - The project only makes use of `pylint` and `pycodestyle`. Having the target implies that it should work.
- `loc` - Not used. Easily run manually if needed.
2019-08-13 20:32:16 -07:00
Toshio Kuratomi
97edfccc70 Fix nightly rpm release
The nightly rpm builds were using a timestamp from the last git commit
in their Release field.  Unfortunately, that was using author timestamp
which is nonsequential.  Change to using commit timestamp which is
sequential.

note that this still has a cornercase if the branch's history is ever
rewritten.
2019-07-24 09:13:53 -07:00
Toshio Kuratomi
019d078a5a
Move common build code from _build_helpers (#55986)
We have some common code used by several docs scripts.  Migrate that
into the build-only shared code repository.

* Move lib/ansible/utils/_build_helpers.py to the directory for common
  build code
* Migrate docs/bin/dump_config.py to a build-ansible subcommand
* Migrate dump_keywords to the build-ansible framework
  * Make the script more maintainable by using functions and good
    variable names
  * Port to Python3 idioms
  * Fix bug so that private attributes will be undocumented
* Move generate_man to a build-ansible subcommand
* Port plugin_formatter to a build-ansible subcommand
* Rework command_plugins so that docs scripts can target Python-3.4+ and
  releng-only subcommands can use more recent versions of Python.
  The architecture is now that command_plugins/* need to be importable
  on Python-3.4.  The init_parsers() method needs to run on Python-3.4.
  But the main() method can utilize features of more recent Python as
  long as it fits within those parameters.
* Update docs build requirements

Port the plugin_formatter to build-ansible framework
2019-07-16 12:19:01 -07:00
Toshio Kuratomi
b54fddb775 This was supposed to be named snapshot
Corrected it being a duplicate of sdist
2019-03-19 13:57:41 -07:00
Toshio Kuratomi
5ba7063f4f
Build manpages as part of sdist (#53728)
* Build fixups

This is in pursuit of making snapshots easier.

* Allow overriding the python command used for generating man pages
* Build the changelog prior to creating the sdist
* Add the uninstalled ansible library to PYTHONPATH for changelog
  generation
* Warn that python setup.py sdist may be incomplete; use make sdist
  or make snapshot instead.
* Implement a snapshot make command

* Fix environ variable test to use a string

Co-Authored-By: abadger <a.badger@gmail.com>
2019-03-18 12:43:38 -07:00
Andreas Krüger
15c58cf35f In the Makefile useful target list, mention "clean" and "webdocs". (#48843) 2018-11-19 14:44:26 -06:00
Alicia Cozine
f1ad012375 removes docs/api dir from .gitignore and Makefile (#47366)
* removes docs/api dir from .gitignore and Makefile

* reduces noise on removing build artifacts
2018-10-19 17:06:15 -04:00
Matt Clay
19c0511f57 Add link check to make sdist.
This will cause `make sdist` to fail on platforms which create
hard links of symbolic links as regular files, such as MacOS (Darwin).

This prevents accidental creation of an sdist tarball without
the necessary symbolic links.
2018-10-11 18:34:06 -07:00
Matt Clay
ac492476e5
Bug fixes and cleanup for ansible-test. (#45991)
* Remove unused imports.
* Clean up ConfigParser usage in ansible-test.
* Fix bare except statements in ansible-test.
* Miscellaneous cleanup from PyCharm inspections.
* Enable pylint no-self-use for ansible-test.
* Remove obsolete pylint ignores for Python 3.7.
* Fix shellcheck issuers under newer shellcheck.
* Use newer path for ansible-test.
* Fix issues in code-smell tests.
2018-09-21 11:38:22 -07:00
John R Barker
e9dbebfa57
Fix some broken links (#42079)
* Fix some broken links

* We now only serve via https
* redirects don't work with anchors, so update those links (devel/dev_guide)
2018-06-29 11:12:01 -07:00
Matt Clay
70c475da6c Implement new changelog generator. 2018-06-05 19:08:15 -07:00
Eitan Adler
7847b1492c Use $(MAKE) instead of raw "make" (#39588)
This Makefile uses non-standard constructs. As such it can only be
parsed by GNU make, which is often installed as 'gmake' instead of
'make'. Using $(MAKE) ensures the same version of make gets called that
is used to execute the top level.
2018-05-24 23:30:14 -04:00
Matt Davis
0371e7c866 fixed prerelease support in deb packaging 2018-05-21 18:21:56 -07:00
Matt Davis
e4edb2842a 2.6 changelog gen/version/root dir cleanup (#40421)
* patched in changelog gen stuff from stable-2.5
* Makefile updates
* release.py as single-source-of-truth
* Remove obsolete ansible-core-sitemap.xml file.
* Move ROADMAP.rst into README.rst.
* dynamic rpm changelog, zap old deb/rpm changelogs
* fix changelog in MANIFEST.in
* Remove obsolete hacking/update.sh script.
* Remove ref to deleted authors script.
* Remove ref to removed module-formatter script.
* Update headings to match script names.
* MANIFEST.in cleanup
* removed RELEASES.txt and versions.yml
* removed obsolete release generation playbook/bits (not used since 2.5)
* misc Makefile cleanup
* speculative changes to DEB versioning
* allow override of DEB_VERSION/DEB_RELEASE
2018-05-21 16:14:53 -07:00
Joseph Herlant
00a7ff7974 Move man pages generations to rst2man (#37861) 2018-03-26 16:28:28 -07:00
Brian Coca
531d7f81b6 add manpage install target
(cherry picked from commit 8e8a29f20a5b250947e00613033775ddd952430f)
(cherry picked from commit f572703efe65b8efedf3792d342ee3c09d9ba735)
2018-02-07 12:35:37 -05:00
Toshio Kuratomi
f9370c95d7 Fix make clean to remove test reports correctly 2017-09-18 16:49:16 -07:00
Adrian Likins
89c973445c generate rst doc pages for command line tools (#27530)
* let generate_man also gen rst pages for cli tools
* make template-file, output-dir, output format cli options for generate_man
* update main Makefile to use generate_man.py for docs (man pages and rst)
* update vault docs that use :option:
* Edits based on
6e34ea6242 and
a3afc78535

* add a optparse 'desc' to lib/ansible/cli/config.py 

  The man page needs a short desc for the 'NAME' field
  which it gets from the option parse 'desc' value.

  Fixes building ansible-config man page.

* add trim_docstring from pep257 to generate_man

  use pep258 docstring trim function to fix up any indention
  weirdness inherit to doc strings (ie, lines other than
  first line being indented.

* Add refs to cli command actions

To reference ansible-vaults --vault-id option, use:

:option:`The link text here <ansible-vault --vault-id>`

or:

:option:`--vault-id <ansible-vault --vault-id>`

To reference ansible-vault's 'encrypt' action, use:

:ref:`The link text here <ansible_vault_encrypt>`

or most of the time:

:ref:`ansible-vault encrypt <ansible_vault_encrypt>`
2017-09-07 15:44:20 -04:00
Miyurz
3b4ff67b3b Made the applicable targets as PHONY (#27996)
https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html
2017-08-10 17:35:58 -04:00
Matt Clay
b54d00f2de Update RPM spec and make targets. (#27712) 2017-08-03 12:35:37 -07:00
Shane McDonald
28fb0498ac Add make target for printing version (#26657) 2017-07-12 13:51:38 +01:00
Brian Coca
e1416138f9 avoid exporting files useless to dist
also fixed up clean in make file
2017-06-23 15:56:43 -04:00
Brian Coca
8f97aef1a3 Transition inventory into plugins (#23001)
* draft new inventory plugin arch, yaml sample

 - split classes, moved out of init
 - extra debug statements
 - allow mulitple invenotry files
 - dont add hosts more than once
 - simplified host vars
 - since now we can have multiple, inventory_dir/file needs to be per host
 - ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
 - centralized localhost handling
 - added plugin docs
 - leaner meaner inventory (split to data + manager)
 - moved noop vars plugin
 - added 'postprocessing' inventory plugins
 - fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
 - grouphost_vars loading as inventory plugin (postprocessing)
 - playbook_dir allways full path
 - use bytes for file operations
 - better handling of empty/null sources
 - added test target that skips networking modules
 - now var manager loads play group/host_vars independant from inventory
 - centralized play setup repeat code
 - updated changelog with inv features
 - asperioribus verbis spatium album
 - fixed dataloader to new sig
 - made yaml plugin more resistant to bad data
 - nicer error msgs
 - fixed undeclared group detection
 - fixed 'ungrouping'
 - docs updated s/INI/file/ as its not only format
 - made behaviour of var merge a toggle
 - made 'source over group' path follow existing rule for var precedence
 - updated add_host/group from strategy
 - made host_list a plugin and added it to defaults
 - added advanced_host_list as example variation
 - refactored 'display' to be availbe by default in class inheritance
 - optimized implicit handling as per @pilou's feedback
 - removed unused code and tests
 - added inventory cache and vbox plugin now uses it
 - added _compose method for variable expressions in plugins
 - vbox plugin now uses 'compose'
 - require yaml extension for yaml
 - fix for plugin loader to always add original_path, even when not using all()
 - fix py3 issues
 - added --inventory as clearer option
 - return name when stringifying host objects
 - ajdust checks to code moving

* reworked vars and vars precedence
 - vars plugins now load group/host_vars dirs
 - precedence for host vars is now configurable
 - vars_plugins been reworked
 - removed unused vars cache
 - removed _gathered_facts as we are not keeping info in host anymore
 - cleaned up tests
 - fixed ansible-pull to work with new inventory
 - removed version added notation to please rst check
 - inventory in config relative to config
 - ensures full paths on passed inventories

* implicit localhost connection local
2017-05-23 17:16:49 -04:00
Brian Coca
033fe5548b added epub entry for makefile 2017-05-01 13:41:04 -04:00
John R Barker
ecbf8e933a Docs how to test (2nd) (#24094)
* Big testing doc refactor
* Combine all the testing documentation in to one place to make it easier to find
* Convert everything to RST
* Create testing_network guide
* Create testing landing page
* For each section detail "how to run" and "how to extend testing"
* More examples
* Lots more detail
2017-04-28 09:08:26 +01:00
Sergey
669630ea83 Fixes #23445 2017-04-12 11:44:04 -04:00
Brian Coca
7cd7f544e0 fixed man page generation so it works 'clean' 2017-03-30 13:13:27 -04:00
Brian Coca
18a7a1ec31 added docs to CLI docstringsadded
removed 'now intermediate build files' from repo
adjusted gitignore
2017-03-24 15:52:36 -04:00
Brian Coca
424e1946f4 moved docs generation and templates to docs/ 2017-03-24 15:52:36 -04:00
Brian Coca
a1a62103fa clean tests results
removed redundant pyc line
2017-03-12 12:30:24 -04:00
Harlan Lieberman-Berg
e24ddb4980 Add manpage for ansible-console. (Closes: #16244) (#16245)
* Add manpage for ansible-console. (Closes: #16244)

* Mark host as an optional field in ansible-console.1
2017-03-08 11:04:35 -05:00
Matt Clay
25cb281b9b Update Makefile to use ansible-test for pep8. 2017-02-09 11:01:48 -08:00
Brian Coca
07f762c26b remove coverage files on clean 2017-01-25 13:29:28 -05:00
Brian Coca
0803c638bb set cpus only if not set already 2017-01-19 23:09:55 -05:00
Brian Coca
8283d08536 escape $ 2017-01-19 23:01:41 -05:00
Adrian Likins
c0263b3020 Pass CPUS from top Makefile to docs Makefile 2017-01-13 14:41:14 -05:00
Matt Clay
cd3fdca540 Switch tests to pytest and ansible-test.
- Replace nose usage with pytest.
- Remove legacy Shippable integration.sh.
- Update Makefile to use pytest and ansible-test.
- Convert most yield unit tests to pytest parametrize.
2017-01-11 12:34:59 -08:00
Brian Coca
afbe3bd63f corrected service condition 2017-01-09 15:44:53 -05:00
Brian Coca
aae9bbdebc fixed cleaning docsite 2017-01-06 17:07:42 -05:00
Brian Coca
57f8b791d6 consolidated docs
point to new doc locations
removed non existing dirs
2017-01-06 09:16:59 -05:00
Antonio Huete Jimenez
b9e4a4dede Check for DragonFly BSD as well for DATE 2017-01-02 11:51:00 -05:00
Matt Clay
0d46805979 Clean up shebangs for various files.
- Remove shebangs from:
  - ini files
  - unit tests
  - module_utils
  - plugins
  - module_docs_fragments
  - non-executable Makefiles
- Change non-modules from '/usr/bin/python' to '/usr/bin/env python'.
- Change '/bin/env' to '/usr/bin/env'.

Also removed main functions from unit tests (since they no longer
have a shebang) and fixed a python 3 compatibility issue with
update_bundled.py so it does not need to specify a python 2 shebang.

A script was added to check for unexpected shebangs in files.
This script is run during CI on Shippable.
2016-11-02 17:00:27 -07:00
Matt Clay
e24588902f Move test_os_server and apply fixes.
- Add missing meta value for test_create_server
- Add .gitignore for pytest .cache directory

Exclude test_os_server from nose test runs since it was designed
for pytest. The test will work correctly when run using pytest.

This is a temporary issue, as we'll be moving to pytest soon.
2016-10-31 15:45:10 -07:00
Shane McDonald
4ae0d5b843 Build debs with pbuilder (#18165)
* Build debs with pbuilder
* Update README in packaging/debian
* Add Dockerfile for building debs
* Add local_deb makefile target - Allows users to build debs using locally installed dependencies. This was the `deb` target before moving to pbuilder.
2016-10-26 14:24:15 -07:00
Adrian Likins
95dd8cc2c7 Add '--cover-erase' to 'make tests' cli (#17708)
Otherwise the coverage is cumulative over multiple runs and can be misleading.
2016-09-22 13:35:18 -07:00
Adrian Likins
be1139317a Add a 'make integration' target (#17710)
Runs test/utils/shippable/integration.sh
2016-09-22 13:31:43 -07:00
Toshio Kuratomi
b8a8027b7d We've decided that python-3.5 is the minimum python version (#17270) 2016-08-29 09:12:37 -07:00
Jim Ladd
329a134cb3 Increase local version for unofficial rpms (#17026) 2016-08-11 17:36:03 -07:00
Adrian Likins
20ef8d5aaf Disable 'xmllint' checks in a2x (#16317)
It brings in a libxml2 dep to the docs build
require. The a2x default of xmllint invocation
attempts to make outgoing requests to download
XML DTD's, despite a2x using the '--nonet' flag.
2016-07-29 16:02:02 -04:00