Commit graph

94 commits

Author SHA1 Message Date
Toshio Kuratomi
eb1214baad New metadata 1.0 (#22587)
Changes to the metadata format were approved here:
https://github.com/ansible/proposals/issues/54
* Update documentation to the new metadata format
* Changes to metadata-tool to account for new metadata
  * Add GPL license header
  * Add upgrade subcommand to upgrade metadata version
  * Change default metadata to the new format
  * Fix exclusion of non-modules from the metadata report
* Fix ansible-doc for new module metadata
* Exclude metadata version from ansible-doc output
* Fix website docs generation for the new metadata
* Update metadata schema in valiate-modules test
* Update the metadata in all modules to the new version
2017-03-14 09:07:22 -07:00
John R Barker
04e816e13b Stricter module documentation validation (#22353)
Raise the bar for module `DOCUMENTAION`
This validator update was used to find the issues in https://github.com/ansible/ansible/pull/22297/files

**Validation**
* Updated Validation and docs to enforce more (items fixed in https://github.com/ansible/ansible/pull/22297/files)
* Use `suboptions` to document complex options 
* Validate module name
* Validate deprecated modules have correct ANSIBLE_METADATA

**Module Documentation Generation**
* Document `suboptions:` Example https://gist.github.com/gundalow/4bdc3669d696268328ccc18528cc6718
* Tidy up HTML generation (valid HTML, no empty lists, etc)
 
**Documentation**
* Clarify the steps for deprecating a module
* Use correct RST headings
* Document `suboptions:` (options)
* Document `contains:` (returns)


**Details**
The aim is to get this (and corresponding module updates) complete by the time `devel` becomes `2.4`, as this allows us to raise the bar for new modules

Example `suboptions` https://gist.github.com/gundalow/4bdc3669d696268328ccc18528cc6718

The aim is to get this PR integrated into `devel` *before* we branch `stable-2.3`, this will allows us to:
* Raise the bar for new modules in 2.4
* Ensure the generated module documentation for 2.3 and higher is improved, important as we will be doing versioned docs moving forward.
2017-03-13 19:49:27 +00:00
René Moser
e0d38fb137 composer: fix escape working_dir (#22363)
* composer: fix escaping of path working_dir

* composer: fix pep8
2017-03-07 20:51:04 +01:00
Lujeni
d942196666 Use a dedicated directory for the mongodb modules (#22187)
* Use a dedicated directory for the mongodb modules
* Missing the RETURN documentation in mongodb_user
2017-03-06 13:14:14 -08:00
René Moser
c1730c21ce cloudstack: cs_sshkeypair: fix fingerprint not used as identifier (#22276)
* cloudstack: cs_sshkeypair: fix fingerprint not used as identifier

* remove from legacy

* fix for 2 keys, one with name and one with fingerprint
2017-03-05 22:47:19 +01:00
Matt Clay
673bfc06b9 Support option renaming in validate-modules. 2017-03-03 16:37:12 -08:00
Matt Martz
293e624235 Support matching original path for a moved module location 2017-03-03 15:13:14 -08:00
Matt Martz
958ce4a252 Create custom JSONEncoder to stringify Exceptions 2017-03-03 11:12:37 -08:00
James Cammarata
a2599cab79 Fix variable precedence of INI/script vars to be in-line with docs.
This commit also adds a new test script (ansible-var-precedence-check.py in code-smell/)
to provide us with another line of defense against precedence bugs going forward.

The precedence docs state that the INI vars have a lower precedence than group/host
vars files for inventory and playbooks, however that has not been the case since 2.0
was released. This change fixes that in one way, though not exactly as the docs say.
The rules are:

1) INI/script < inventory dir < playbook dir
2) "all" group vars < other group_vars < host_vars

So the new order will be (from the test script mentioned above):

8. pb_host_vars_file - var in playbook/host_vars/host
9. ini_host_vars_file - var in inventory/host_vars/host
10. ini_host - host var inside the ini
11. pb_group_vars_file_child - var in playbook/group_vars/child
12. ini_group_vars_file_child - var in inventory/group_vars/child
13. pb_group_vars_file_parent - var in playbook/group_vars/parent
14. ini_group_vars_file_parent - var in inventory/group_vars/parent
15. pb_group_vars_file_all - var in playbook/group_vars/all
16. ini_group_vars_file_all - var in inventory/group_vars/all
17. ini_child - child group var inside the ini
18. ini_parent - parent group var inside the ini
19. ini_all - all group var inside the ini

Fixes #21845
2017-03-02 17:07:00 -06:00
Matt Clay
d66ce40ecb Overhaul ansible-test sanity implementation. (#22177)
- Tests are run to completion instead of stopping on first failure.
- Test results are now parsed instead of passing through to the console.
- Test results can be saved in junit xml format.
- Test results will show up on the Shippable "Tests" result tab.
- Added an experimental --lint option for easier integration with other tools.
- Code smell tests are now usable with the --list-tests, --test and --skip-test options.
- Code split out from executor.py into sanity.py.
- Rename download-logs to download.py and add support for test and coverage results.
- Miscellaneous improvements.
2017-03-02 12:36:46 -08:00
Matt Martz
f761949cca Ensure that a deprecated module has DOCUMENTATION.deprecated (#22090) 2017-02-28 16:43:43 +00:00
Matt Clay
45cec93e99 Disable newer shellcheck tests which don't pass. 2017-02-27 22:31:47 -08:00
Slezhuk Evgeniy
e12daee5eb jira: comply with current pep8 rule set 2017-02-27 11:13:23 -05:00
Will Thames
ebfc7bac94 cloudfront_facts module improvements
* Make more use of AWS Exception information
* Use pythonic dict assignments
* Improve documentation formatting to make option names clearer
  in descriptions
* Meet ansible line length requirements
* Tidy up flake8 messages
2017-02-27 10:22:55 -05:00
René Moser
06acf8ac2e nginx_status_facts: fix pep8 (#21963)
* nginx_status_facts: fix pep8

* remove from legacy files
2017-02-26 11:25:09 +01:00
René Moser
d26c57f938 exoscale: pep8 and python3 fixes (#21821)
* exoscale: pep8 fixes

* exo_dns_record: remove from pep8 test legacy

* exoscale: fix python3 support
2017-02-25 10:32:39 +01:00
Matt Martz
305c88700d PEP8 E712 fixes (#21916) 2017-02-24 16:49:43 -06:00
Matt Clay
c448a418fd Treat deprecated modules as old during validation. 2017-02-24 13:04:51 -08:00
Matt Martz
97e12b0898 Check for imports to be between last doc line and first callable (#21863)
* Check for imports to be between last doc line and first callable

* Small readme update
2017-02-24 12:17:56 -06:00
Dag Wieers
b1f6344ecb win_chocolatey: Clean up parameter handling (#21533)
Changes include:

- Use Get-AnsibleParam with -type/-validateset
- Replace $result PSObject with normal hash
- Deprecate 'upgrade' parameter by using state=latest
2017-02-24 10:09:11 +00:00
Matt Martz
36e6c7973d pep8 fixes for module_docs_fragments (#21876) 2017-02-24 09:01:03 +01:00
Matt Martz
4b320ce0e4 Add checks for from module_utils import * (#21800) 2017-02-22 13:16:35 -08:00
Matt Clay
cba91c7f44 Test placement (#21801)
* Fix test placement and update messaging.
* Add note about group selection.
2017-02-22 12:43:45 -08:00
Matt Clay
b5e0eb0985 Fixes for missing aliases files sanity test. (#21798)
* Refine test for missing aliases files.
* Fix formatting of missing aliases message.
2017-02-22 12:04:09 -08:00
Toshio Kuratomi
45251f910c Make BaseFileCache into an abstractbaseclass so it's a proper interface
Push the opening and closing of files into the _load and _dump methods
so that we don't invoke the slow codec machinery without reason.
2017-02-22 10:27:29 -05:00
Brian Coca
374af06cbf added pickle and yaml cache plugins
added new base class for file based cache plugins as 99% of code was common
now also catches unexpected decoding exceptions
allows per module file modes and encoding
moved jsonfile code to base
2017-02-22 10:27:29 -05:00
Matt Martz
b143fdea54 DOC options must be a dict when used with extends_documentation_fragment (#21745)
* Fix string formatting
* Provide better tracebacks
* When options is None and extends_documentation_fragment is in use, add an error that options must be a dict
* If options was specified and not a dict, then error
2017-02-21 17:42:39 -08:00
Veaceslav Mindru
cdcdc1d81b #19587 add yum skip-broken (#21475)
* #19587: refactor to PEP8

* Issue: #19587: add skip_broken option to yum module
PNTSYSOPS-1901 - internal reference

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>

* Issue: #19587: fix line break, attempt to make readble

        things at line 646 look ugly ..
        trying to make them look a bit more human readble
        though the entire approach should be rewriten

PNTSYSOPS-1901 - internal reference

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>

* #19587: fixing typo, skip-broken has to be passed as list

* PR: #21475: peer review changes
            relates to: revert comment deletion https://github.com/ansible/ansible/pull/21475#discussion_r101574146
            relates to: remove irelevant var https://github.com/ansible/ansible/pull/21475#discussion_r101573388
            relates to: reformat https://github.com/ansible/ansible/pull/21475#discussion_r101572840

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>

* PR: #21475: peer review changes
            relates to: https://github.com/ansible/ansible/pull/21475#discussion_r101573736

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>

* Issue: #21475: fixing typo in the Man section

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>

* PR: #21475 - Peer review: revert variable a
        Relates to: https://github.com/ansible/ansible/pull/21475#pullrequestreview-22404868

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>

* Issue: 19587 - small correction

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>
2017-02-21 07:59:07 -08:00
Matt Martz
c160ede789 Validate modules json output (#21680) 2017-02-20 13:28:23 -08:00
Matt Martz
9b1bd2eb7e Re-enable module comparisons in validate-modules (#21582)
* Re-enable module comparisons, specifically for new module detection and for finding new options/arguments
* Only do new module checks in shippable, local will display warning
2017-02-17 15:13:18 -08:00
Toshio Kuratomi
eeaec56ed5 Use isinstance instead of type() comparisons. isinstance is more robust 2017-02-16 16:34:43 -08:00
Will Thames
f42ffe6de3 [cloud] Ensure that s3_bucket module always enables/disables versioning if it is specified (#21320)
If versioning is not enabled, then `get_bucket_versioning`
can return an empty dict.

If that happens, the code to enable versioning should still
run!

The logic for suspending versioning was also incorrect, so
have updated that too.

Fixes #20491
2017-02-16 09:45:05 -05:00
Matt Martz
7c00346714 Validate EXAMPLES as YAML 2017-02-15 13:01:43 -08:00
Will Thames
103ede26df Ensure ssh hostkey checks respect server port (#20840)
* Add tests for `get_fqdn_and_port` method.

Currently tests verify original behavior - returning default `ssh-keyscan` port
Add test around `add_host_key` to verify underlying command arguments
Add some new expectations for `get_fqdn_and_port`
Test that non-standard port is passed to `ssh-keyscan` command

* Ensure ssh hostkey checks respect server port

ssh-keyscan will default to getting the host key for port 22.
If the ssh service is running on a different port, ssh-keyscan
will need to know this.

Tidy up minor flake8 issues

* Update known_hosts tests for port being None

Ensure that git urls don't try and set port when a path
is specified

Update known_hosts tests to meet flake8

* Fix stdin swap context for test_known_hosts

Move test_known_hosts from under basic, as it is its own library.
Remove module_utils.known_hosts from pep8 legacy files list
2017-02-15 11:47:57 -08:00
Tim Rupp
2a9778e320 Change the f5 modules to use f5_utils file (#21466)
The f5 module utils were using a file name that appeared to
conflict with the f5 python SDK's namespace. This patch just changes
the name of the included class to be f5_utils to avoid the issue
of namespace collisions
2017-02-15 16:51:13 +00:00
John R Barker
53ac312382 validate-modules --arg-spec (#21331)
* validate-modules --arg-spec

* Update developing_modules_documenting.rst

* Never mock out ansible or ansible.module_utils

* No more false positives
2017-02-15 08:09:08 +00:00
Matt Clay
d54bc09fae Consider module_utils deps when running tests. (#21382)
* Skip pep8 analysis when --explain is used.
* Fix return type annotations.
* Match line length requirement of PEP 8 config.
* Consider module_utils deps when running tests.
2017-02-13 18:49:36 -08:00
Ted Timmons
23886c5c0d add instructions for running pep8 locally (#21120) 2017-02-13 11:09:50 -08:00
Matt Martz
87aa59af79 Legacy pep8 updates for setup.py and tests 2017-02-11 16:04:52 -08:00
s-hertel
2dc42fce80 [cloud] add tags parameter to ec2_vol & PEP8 fixes (#21254)
* Make ec2_vol pep8 and add tags parameter

* Remove ec2_vol.py from pep8 legacy-files
2017-02-10 15:22:38 -05:00
Matt Clay
46f9648aa1 Fix type() usage check in validate-modules.
This avoids false positives when 'if' or 'or' are in a string
preceding the use of type() on the same line.
2017-02-08 12:17:10 -08:00
Adrian Likins
7278308ca2 Redhat subscription (#20729)
* Make unused redhat_subscriptions do something

rhsm_baseurl/server_insecure were module params that were
never used previously. Hook them up for register options.

* pep8/style cleanups
* Set a module scope SUBMAN_CMD before we ref it as a global
* move ansible imports to top, remove '*' imports
* remove redhat_subscriptions from pep8 legacy
2017-02-07 12:07:01 -05:00
Matt Martz
5942de603d Legacy pep8 clean fixes for contrib and hacking (#21081) 2017-02-07 09:49:55 -06:00
Brian Coca
c86a17b7a0 refactoring async
- centralized skipping
- also fixed module name broken by previous refactor
- let action modules handle async processing
- moved async into base action class's module exec
- action plugins can now run final action as async
- actually skip copy if base skips
- fixed normal for new paths
- ensure internal stat is never async
- default poll to 10 as per docs
- added hint for callback fix on poll
- restructured late tmp, now a pipeline query
- moving action handler to connection as networking does
- fixed network assumption invocation is always passed
- centralized key cleanup, normalized internal var
- _supress_tmpdir_delete now in _ansible_xxx and gets removed from results
- delay internal key removal till after we use em
- nicer tmp removing, using existing methods
- moved cleanup tmp flag to mking tmp func
2017-02-06 19:32:00 -05:00
Loïc Blot
b55d039f67 New modules: vmware_guest_facts & vmware_guest_snapshot (#20971)
* Move gather_facts action outside of vmware_guest module
* Move facts gathering method to vmware.py library
* vmware_guest use some vmware.py library methods
* vmware.py lib: fix some PEP 8 issues
* Move snapshot actions to vmware_guest_snapshot
* Move common snapshot facts to vmware.py library
2017-02-06 11:52:17 -05:00
John R Barker
25e144db4a Update legacy-files.txt 2017-02-06 16:05:06 +00:00
John R Barker
0ea3911d47 module_utils/vmware.py is now clean 2017-02-06 16:01:10 +00:00
Toshio Kuratomi
d1a6b07fe1 Move ssh and local connection plugins from using raw select to selectors
At the moment, this change will use EPoll on Linux, KQueue on *BSDs,
etc, so it should alleviate problems with too many open file
descriptors.

* Bundle a copy of selectors2 so that we have the selectors API everywhere.
* Add licensing information to selectors2 file so it's clear what the
  licensing terms and conditions are.
* Exclude the bundled copy of selectors2 from our boilerplate code-smell test
* Rewrite ssh_run tests to attempt to work around problem with mocking
  select on shippable

Fixes #14143
2017-02-03 08:23:50 -08:00
Matt Clay
6580fb2cd1 Fix exception in module validator. 2017-02-02 17:38:16 -08:00
Matt Martz
829c0b8f62 Update validate-modules (#20932)
* Update validate-modules

* Validates ANSIBLE_METADATA
* Ensures imports happen after documentation vars
* Some pep8 cleanup

* Clean up some left over unneeded code

* Update modules for new module guidelines and validate-modules checks

* Update imports for ec2_vpc_route_table and ec2_vpc_nat_gateway
2017-02-02 11:45:22 -08:00