Commit graph

556 commits

Author SHA1 Message Date
Felix Fontein e6a7bc854c
Only consider directories which do have .deps files other than ancestor.deps. (#74987) 2021-06-15 14:35:34 -05:00
Toshio Kuratomi 80e7e1a17c
Due to the takeover of freenode we're moving to a different irc network. (#74775)
* Due to the takeover of freenode we're moving to a different irc network.

* Our channels updated to point at the same channel name on libera.chat
* Some links went to webchat.freenode.net.  At this time, libera.chat
  doesn't point you to an official webchat client so I changed these to
  https://libera.chat. (kiwi irc does work with libera.chat so that
  could be another option).
* In general, I used the name irc.libera.net for link names and
  https://libera.chat for link targets.  This is because the irc service
  is hosted on irc.libera.chat but the project web server is hosted on
  libera.chat.  (This appears to also be true for freenode but we were
  using http://irc.freenode.net which doesn't seem to work.  Oops).
* Removed http://irc.freenode.net from the linkcheck exceptions.
  linkcheck was actually correct to flag that as invalid (should have
  been http://frenode.net instead).

* Looks like hte important people in #yaml are now in libera.chat

* Link to where contributors should get help

Add a link target and then link to where contributors should get support
for developing groups of modules.

* Update docs/docsite/rst/dev_guide/developing_modules_in_groups.rst

Co-authored-by: Felix Fontein <felix@fontein.de>

Co-authored-by: John R Barker <john@johnrbarker.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
2021-06-01 08:48:09 +01:00
Matt Martz 8d1cf7f266
Vendor distutils.version (#74644)
* Vendor distutils.version

* Fix import order. ci_complete

* remove distutils warning filter

* Don't remove warnings filter from importer

* ci_complete

* Add pylint config for preventing distutils.version

* Add changelog fragment
2021-05-11 12:33:51 -05:00
Matt Clay c48f80d062 Rename hacking/shippable to hacking/azp.
References to Shippable were changed to Azure Pipelines.

Also remove rebalance.py as it does not work with Azure Pipelines due to the required data not being present.
2021-05-05 17:21:14 -07:00
Alexander Sowitzki 26214788ee Retrofit shippable scripts to work with AZP
Co-authored-by: Matt Clay <matt@mystile.com>
2021-04-28 09:43:41 -07:00
Matt Clay ecc5a53288
Update default containers to 3.4.0. (#74415)
* Update default containers to 3.4.0.

The 3.4.0 containers use Python 3.6 (the system Python) for `/usr/bin/python3`.

Python 3.9 continues to be the default Python version selected by `ansible-test` for these containers.

* Fix shebang on build-ansible.py.

Using `python` instead of `python3` allows `ansible-test` python interception and requirements install to function.
2021-04-25 21:25:32 -07:00
Ikko Ashimine 176beddb3f
Fix typo in incidental.py (#73737)
successfull -> successful
2021-03-01 17:37:58 +05:30
Sandra McCann ccbfdec334
Split Ansible docs from core docs (#73616)
* excludes scenario guides from core docs, splits porting guides and roadmaps, symlinks indices to create index.html pages, and adds .gitignore entries for conf.py and the toplevel index.rst files generated by the docs build

This solution builds three types of docs:
* ansible-2.10 and earlier: all the docs.  Handle this via `make webdocs
  ANSIBLE_VERSION=2.10`
* ansible-3 and later: a subset of the docs for the ansible package.
  Handle this via `make webdocs ANSIBLE_VERSION=3` (change the
  ANSIBLE_VERSION to match the version being built for.
* ansible-core: a subset of the docs for the ansible-core package.
  Handle this via `make coredocs`.

* `make webdocs` now always builds all the collection docs
*  Use `make coredocs` to limit it to core plugins only
*  The user specifies the desired version. If no ANSIBLE_VERSION is specified, build plugins for the latest release of ansible
 
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored-by: Matt Clay <matt@mystile.com>
2021-02-17 09:57:05 -06:00
Brian Coca 1202dd000f
Allow restricting config values to enumerated list (#73162)
* Allow restricting config values to enumerated list
* dont document internal entries
* also ignore private defs for ansible-config
* remove invalid value from tests
* added porting entry
2021-01-14 15:11:30 -05:00
Rick Elrod 7f0eb7ad79
[facts] Differentiate CentOS vs CentOS Stream (#73034)
Change:
- On CentOS Stream, make distribution_release be "Stream"
- On CentOS Core, it continues to be "Core"
- Implement custom distribution file parser for CentOS, so we can look
  for "CentOS Linux" and "CentOS Stream"
- Two new fixtures introduced (CentOS Linux 8.1 and CentOS Stream 8)
- Removed two dicts from `Distribution` class that were seemingly not
  used anywhere.

Test Plan:
- ci_complete
- New test fixtures

Tickets:
- Fixes #73027

Signed-off-by: Rick Elrod <rick@elrod.me>
2021-01-13 17:54:04 -05:00
Vincent Bernat 4afc228108
hacking: replace use of "which" with "command -v" (#71278)
"command -v" is the POSIX compliant way to get the path to a command.
It returns the path to the command if it exists and exits with a
non-zero status when it does not, without any output. "which" is not
POSIX and it can have various different effects. With Zsh, it says
"python not found" on stdout.

See:
 - https://pubs.opengroup.org/onlinepubs/009695399/utilities/command.html

Signed-off-by: Vincent Bernat <vincent@bernat.ch>
2020-08-18 11:31:28 -04:00
Toshio Kuratomi 22d2c97b81 Updates to docs build for new antsibull CLI
* --ansible-base-cache renamed to --ansible-base-source
* _acd_version in the .deps file has been renamed to _ansible_version
2020-08-14 13:01:29 -07:00
Toshio Kuratomi 37a7485ac8 Fix version comparison for determining what ansible to build against
* The version comparison for determining what ansible package to build
  docs against was comparing the version number for ansible-base but it
  needed to check the version number for ansible instead

* add a comment about some bad logic than needs to be fixed after 2.10.0
2020-08-07 07:59:59 -07:00
Rick Elrod 75e8da0950
hacking: fix announce script version parsing (#71008)
Change:
- Fix a bug where rc/beta versions throw off the "is this an
  ansible-base release"? check.

Test Plan:
- Used it for 2.10.0rc4

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-30 21:28:43 -05:00
Rick Elrod 34f18a97f4
Add ansible-releases@redhat.com to announce template (#70812)
Signed-off-by: Rick Elrod <rick@elrod.me>
2020-07-22 11:08:32 -05:00
Abhijeet Kasurde 4f96f9826c
distribution: Add support for DragonFly (#70748)
partially fixes #43739

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-07-21 13:47:11 -04:00
Toshio Kuratomi 1e3989c9f7
Need to return any error code from running antsibull-docs (#70763)
This way we fail early if there's a problem
2020-07-20 14:05:01 -07:00
Toshio Kuratomi 9dda393d70
Collections docs generation (#59761)
* Build documentation for Ansible-2.10 (formerly known as ACD).

Builds plugin docs from collections whose source is on galaxy

The new command downloads collections from galaxy, then finds the
plugins inside of them to get the documentation for those plugins.

* Update the python syntax checks
  * docs builds can now require python 3.6+.

* Move plugin formatter code out to an external tool, antsibull-docs.
  Collection owners want to be able to extract docs for their own
  websites as well.
* The jinja2 filters, tests, and other support code have moved to antsibull
* Remove document_plugins as that has now been integrated into antsibull-docs

* Cleanup and bugfix to other build script code:
  * The Commands class needed to have its metaclass set for abstractmethod
    to work correctly
  * Fix lint issues in some command plugins

* Add the docs/docsite/rst/collections to .gitignore as
  everything in that directory will be generated so we don't want any of
  it saved in the git repository
* gitignore the build dir and remove edit docs link on module pages

* Add docs/rst/collections as a directory to remove on make clean
* Split the collections docs from the main docs

* remove version and edit on github
* remove version banner for just collections
* clarify examples need collection keyword defined

* Remove references to plugin documentation locations that no longer exist.
  * Perhaps the pages in plugins/*.rst should be deprecated
    altogether and their content moved?
  * If not, perhaps we want to rephrase and link into the collection
    documentation?
  * Or perhaps we want to link to the plugins which are present in
    collections/ansible/builtin?

* Remove PYTHONPATH from the build-ansible calls
  One of the design goals of the build-ansible.py script was for it to
  automatically set its library path to include the checkout of ansible
  and the library of code to implement itself.  Because it automatically
  includes the checkout of ansible, we don't need to set PYTHONPATH in
  the Makefile any longer.

* Create a command to only build ansible-base plugin docs
  * When building docs for devel, only build the ansible-base docs for
    now.  This is because antsibull needs support for building a "devel
    tree" of docs.  This can be changed once that is implemented
  * When building docs for the sanity tests, only build the ansible-base
    plugin docs for now.  Those are the docs which are in this repo so
    that seems appropriate for now.
2020-07-17 13:07:35 -07:00
Matt Clay fb7740ae3b Update hacking/shippable docs.
The `--all` option downloads more than is needed for analyzing code coverage.
2020-07-07 10:22:45 -07:00
Matt Clay 4816bb4f43
More boilerplate fixes. (#70224)
* Fix boilerplate in hacking dir.
* Fix boilerplate in docs dir.
* Fix boilerplate in integration tests.
* Fix boilerplate in examples.
2020-06-22 19:05:30 -07:00
Sam Doran fd882e0e18
Get test data file directly in rebalance script (#70107)
Rather than looking through tests.json to find the data file, look for it explicitly
within a given target to avoid problems processing data in tests.json.
2020-06-19 09:57:50 -04:00
Matt Clay c67efe0bd1 Avoid unnecessary comprehensions. 2020-06-18 11:50:31 -07:00
Felix Fontein 8d93ba9120
Plugin/module docs: parse return values, add collection names in them (version_added_collection), and format them nicely in ansible-doc (#69796)
* Tag return value docs if they are a dict (and not str/None).

* Try to parse return docs as YAML.

* Properly dump return values in ansible-doc.

* Adjust plugin formatter.

* Add changelog fragment.

* Don't add 'default' for return values.

* Fix plugin_formatter.

* Only try to parse return docs if they are still a string.

* Add tests.

* Warn if RETURN cannot be parsed.

* Adjust tests. Also test for warning.

* if -> elif (otherwise EXAMPLE will be parsed too).

* Always parse return documentation, and fail if it is invalid YAML.

* Polishing.

* Mostly re-enable ansible-doc tests.

Listing from the local collection seems to be somewhat broken. I assume this
is why the test was disabled.

* Lint and make tests work with Python 2.

* Keep FQCNs in plugins (not modules), i.e. restore previous state.
2020-06-11 11:03:43 -07:00
Brian Coca 062e780a68
starting metadata sunset (#69454)
* starting metadata sunset

 - purged metadata from any requirements
 - fix indent in generic handler for yaml content (whey metadata display was off)
 - make more resilient against bad formed docs
 - removed all metadata from docs template
 - remove metadata from schemas
 - removed mdata tests and from unrelated tests

Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Rick Elrod <rick@elrod.me>
2020-06-04 21:01:46 -04:00
Felix Fontein 6a4455ff54
Update plugin_formatter to understand tagged version_added (#69797)
* Update version_added handling for plugin_formatter w.r.t. #69680.

* Update hacking/build_library/build_ansible/command_plugins/plugin_formatter.py

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>

Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
2020-06-02 08:07:34 -07:00
Dongsu Park 598e3392a9
Discover Flatcar Linux properly for hostname (#69627)
To avoid issues with Flatcar Container Linux being unable to be found,
detect Flatcar distro name especially for hostname, just like CoreOS
Container Linux was supported.

See also https://github.com/ansible/ansible/issues/69516
2020-06-02 18:41:53 +05:30
Rick Elrod 3ec18ccb77
Allow announce script to work for base and older (#69768)
Change:
- Generalize the announce script changes from 4dd0f4127

Test Plan:
Harcoded 'hashes' to [] and then:
- Ran with --version 2.9.9 --version 2.10.0.dev1
- Ran with --version 2.9.9
- Ran with --version 2.10.1

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-29 16:20:30 -05:00
Rick Elrod 4dd0f41270
Update announce script for ansible-base (#69735)
Change:
- s/Ansible/ansible-base/
- Update URLs

Test Plan:
- Hope for the best. (No real way to test this until release day because
  there are no files on releases.a.c for it to hash right now. A staging
  releng environment would fix this.)

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-27 23:04:42 -05:00
Toshio Kuratomi efff35a7cd Remove unused imports in the porting_guide command plugin 2020-05-14 09:14:37 -07:00
Toshio Kuratomi 26704b915b Turn pathlib paths into strs
Some APIs do not take a pathlib.  They need to have a string
representation of a path.  Transform the default path to a str
so those APIs will work with the default value.
2020-05-14 09:14:37 -07:00
Toshio Kuratomi 957ad8e769 Fix the command plugin to use the ABCMeta metaclass
The abstract* decorators don't have any effect unless the class has an
ABCMeta metaclass.
2020-05-14 09:14:37 -07:00
Toshio Kuratomi a3c400d0ce Remove outdated docs and unneeded import from change_detection build file 2020-05-14 09:14:37 -07:00
Rick Elrod 96c56b119d Add a script for adding backport references
Change:
- Add a place for adding backport-related scripts in the future
- Add a script for adding backport references

Test Plan:
- Used it for this latest batch of PR reference-adding.

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-05-07 21:44:13 -05:00
Evgeni Golov cdad594b16
plugin_formatter: sys.exit does not take a file argument (#68016)
Cleanup of leftover from bcdfdc0cc3.

sys.exit does not take any named argument.
2020-05-05 08:38:23 +05:30
Matt Martz a513a62536
Remove cgroup_perf_recap_graph.py, since that callback no longer exists here (#68779) 2020-04-08 14:04:53 -05:00
Matt Clay 7323d5dd0d Fix references to old egg-info directory. 2020-03-30 13:56:43 -07:00
Rick Elrod bd9e31fcb0
Add a custom policy for hackers using ansible-test (#68535)
Signed-off-by: Rick Elrod <rick@elrod.me>
2020-03-28 17:07:29 -05:00
Jill Rouleau 0bf7a08eb6 Remove hacking/aws_config
AWS hacking config has been moved to the AWS CI terminator repo, and is no
longer required in base.
https://github.com/mattclay/aws-terminator/pull/91
2020-03-25 18:47:34 -07:00
Jordan Borean 36fc5840b7
hacking - Add script to easily rebalance Shippable groups (#67976)
* hacking - Add script to easily rebalance Shippable groups

* Fix py26 compile issue

* Add option to automatically change test target aliases
2020-03-24 08:14:09 +10:00
Matt Clay d049888a92 Update Shippable hacking scripts default branch. 2020-03-23 13:04:56 -07:00
Matt Clay 707eea3afa Add new options to hacking/shippable/incidental.py (#68384)
* Add `--plugin-path` option to `incidental.py`.

* Report on plugins with no test target.

* Add `--verbose` option to script.
2020-03-23 11:14:21 -05:00
Rick Elrod 649f657f91 Fix coverage script to handle ongoing runs (#68380)
* Fix get_recent_coverage_runs.py to handle ongoing runs

Signed-off-by: Rick Elrod <rick@elrod.me>

* Color code test status to make it easier to see at a glance

Signed-off-by: Rick Elrod <rick@elrod.me>

* fix lint

Signed-off-by: Rick Elrod <rick@elrod.me>
2020-03-23 11:14:21 -05:00
Matt Clay 82516a424e Fix hacking/shippable/incidental.py source search. 2020-03-23 11:14:21 -05:00
Rick Elrod d87d96077f Add script for getting URLs of coverage runs
Signed-off-by: Rick Elrod <rick@elrod.me>
2020-03-23 11:14:21 -05:00
Matt Clay f18f480a3c Add hacking/shippable/incidental.py script. (#68182)
* Add hacking/shippable/incidental.py script.

* Add README.md.
2020-03-23 11:14:21 -05:00
Rick Elrod 787089cba2 First attempt at making the shippable downloader recurse to pull parent results of reruns (#68154)
* First attempt at making the shippable downloader recurse to pull parent results of reruns

Signed-off-by: Rick Elrod <rick@elrod.me>

* Update message for previous run download.

Co-authored-by: Matt Clay <matt@mystile.com>
2020-03-23 11:14:21 -05:00
Mark Chappell da30e6d2e1
sqs_queue - Move to boto3 and add support for various extra features (#66795)
* reworked sqs_queue

* Switch default purge_tags behaviour to false.

This matches the behaviour of ec2_tag and ecs_tag.

* Minor lint / review fixups

* Add missing AWS IAM policy for SQS tests

* Move integration tests to using module_defaults: group/aws:...

* add changelog

* Break out the 'compatability' map from our spec definition (gets flagged by the schema validation)

* Tweaks based on review

* add basic examples

* Lint fixups

* Switch out NonExistentQueue logic so it's easier to follow

* Reorder name argument options for consistency

Co-authored-by: Dennis Podkovyrin <dennis.podkovyrin@gmail.com>
2020-02-27 15:26:20 -07:00
mmoyle cfe96b2092
add module cloudformation_exports (#67349)
* add module cloudformation_exports

* add RETURN, add aliases group, clean up yaml

* update return value. uncomment security_token. remove cloudformation shortcut

* fix typo

* try to delete test stack

* rename stack

* add cleanup and assert. try to set stack name with variable

* create s3 bucket instead

* set bucket name

* add tests, remove unsed key and import, add iam role, add to module_defaults

* import exceptions, fix assert syntax

* fix assert

* Update test/integration/targets/cloudformation_exports/tasks/main.yml

Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>

* fix export name

* renamed module

Co-authored-by: Jill R <4121322+jillr@users.noreply.github.com>
2020-02-27 11:39:59 -07:00
Mark Chappell 9c6495d4d4
elb_target / elb_target_info : Integration test fixups (#61256)
* Update AWS policy to enable management of TargetGroups

* elb_target: (integration tests) migrate to using module_defaults

* elb_target: (integration tests) lookup the AMI by name rather than hard coding AMI IDs

* elb_target_info: (integration tests) finish rename of integration test role

* elb_target: (integration tests) rename various resources to consistently use {{ resource_prefix }}

* elb_target_info: (integration tests) Migrate to using module_defaults

* elb_target_info: (integration tests) Lookup AMI by name rather than hard coding AMI IDs

* Apply suggestions from code review

Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>

* elb_target: (integration tests) Remove the 'unsupported' alias

* Try bumping up the timeout

* Rules don't permit 'shippable' (resource_prefix uses this when run in shippable)

* Try bumping up more timeouts :/

* Avoid double evaluation of target_health assertion

* Simplify target_type usage a little (rather than constantly performing a lookup)

* mark elb_target tests 'unstable' for now, they're slow

Co-authored-by: Jill R <4121322+jillr@users.noreply.github.com>
2020-02-20 17:13:09 -07:00
Matt Clay 19f6fc68d4
Relocate Shippable tools. (#67556)
* Move Shippable tools to hacking directory.

These limits the `test/utils/shippable/` directory to scripts required for CI.

* Fix `test/utils/shippable/` file classification.

* Update package-data sanity test.
2020-02-19 09:55:34 -08:00