Commit graph

87 commits

Author SHA1 Message Date
Matt Martz
958ce4a252 Create custom JSONEncoder to stringify Exceptions 2017-03-03 11:12:37 -08:00
James Cammarata
a2599cab79 Fix variable precedence of INI/script vars to be in-line with docs.
This commit also adds a new test script (ansible-var-precedence-check.py in code-smell/)
to provide us with another line of defense against precedence bugs going forward.

The precedence docs state that the INI vars have a lower precedence than group/host
vars files for inventory and playbooks, however that has not been the case since 2.0
was released. This change fixes that in one way, though not exactly as the docs say.
The rules are:

1) INI/script < inventory dir < playbook dir
2) "all" group vars < other group_vars < host_vars

So the new order will be (from the test script mentioned above):

8. pb_host_vars_file - var in playbook/host_vars/host
9. ini_host_vars_file - var in inventory/host_vars/host
10. ini_host - host var inside the ini
11. pb_group_vars_file_child - var in playbook/group_vars/child
12. ini_group_vars_file_child - var in inventory/group_vars/child
13. pb_group_vars_file_parent - var in playbook/group_vars/parent
14. ini_group_vars_file_parent - var in inventory/group_vars/parent
15. pb_group_vars_file_all - var in playbook/group_vars/all
16. ini_group_vars_file_all - var in inventory/group_vars/all
17. ini_child - child group var inside the ini
18. ini_parent - parent group var inside the ini
19. ini_all - all group var inside the ini

Fixes #21845
2017-03-02 17:07:00 -06:00
Matt Clay
d66ce40ecb Overhaul ansible-test sanity implementation. (#22177)
- Tests are run to completion instead of stopping on first failure.
- Test results are now parsed instead of passing through to the console.
- Test results can be saved in junit xml format.
- Test results will show up on the Shippable "Tests" result tab.
- Added an experimental --lint option for easier integration with other tools.
- Code smell tests are now usable with the --list-tests, --test and --skip-test options.
- Code split out from executor.py into sanity.py.
- Rename download-logs to download.py and add support for test and coverage results.
- Miscellaneous improvements.
2017-03-02 12:36:46 -08:00
Matt Martz
f761949cca Ensure that a deprecated module has DOCUMENTATION.deprecated (#22090) 2017-02-28 16:43:43 +00:00
Matt Clay
45cec93e99 Disable newer shellcheck tests which don't pass. 2017-02-27 22:31:47 -08:00
Slezhuk Evgeniy
e12daee5eb jira: comply with current pep8 rule set 2017-02-27 11:13:23 -05:00
Will Thames
ebfc7bac94 cloudfront_facts module improvements
* Make more use of AWS Exception information
* Use pythonic dict assignments
* Improve documentation formatting to make option names clearer
  in descriptions
* Meet ansible line length requirements
* Tidy up flake8 messages
2017-02-27 10:22:55 -05:00
René Moser
06acf8ac2e nginx_status_facts: fix pep8 (#21963)
* nginx_status_facts: fix pep8

* remove from legacy files
2017-02-26 11:25:09 +01:00
René Moser
d26c57f938 exoscale: pep8 and python3 fixes (#21821)
* exoscale: pep8 fixes

* exo_dns_record: remove from pep8 test legacy

* exoscale: fix python3 support
2017-02-25 10:32:39 +01:00
Matt Martz
305c88700d PEP8 E712 fixes (#21916) 2017-02-24 16:49:43 -06:00
Matt Clay
c448a418fd Treat deprecated modules as old during validation. 2017-02-24 13:04:51 -08:00
Matt Martz
97e12b0898 Check for imports to be between last doc line and first callable (#21863)
* Check for imports to be between last doc line and first callable

* Small readme update
2017-02-24 12:17:56 -06:00
Dag Wieers
b1f6344ecb win_chocolatey: Clean up parameter handling (#21533)
Changes include:

- Use Get-AnsibleParam with -type/-validateset
- Replace $result PSObject with normal hash
- Deprecate 'upgrade' parameter by using state=latest
2017-02-24 10:09:11 +00:00
Matt Martz
36e6c7973d pep8 fixes for module_docs_fragments (#21876) 2017-02-24 09:01:03 +01:00
Matt Martz
4b320ce0e4 Add checks for from module_utils import * (#21800) 2017-02-22 13:16:35 -08:00
Matt Clay
cba91c7f44 Test placement (#21801)
* Fix test placement and update messaging.
* Add note about group selection.
2017-02-22 12:43:45 -08:00
Matt Clay
b5e0eb0985 Fixes for missing aliases files sanity test. (#21798)
* Refine test for missing aliases files.
* Fix formatting of missing aliases message.
2017-02-22 12:04:09 -08:00
Toshio Kuratomi
45251f910c Make BaseFileCache into an abstractbaseclass so it's a proper interface
Push the opening and closing of files into the _load and _dump methods
so that we don't invoke the slow codec machinery without reason.
2017-02-22 10:27:29 -05:00
Brian Coca
374af06cbf added pickle and yaml cache plugins
added new base class for file based cache plugins as 99% of code was common
now also catches unexpected decoding exceptions
allows per module file modes and encoding
moved jsonfile code to base
2017-02-22 10:27:29 -05:00
Matt Martz
b143fdea54 DOC options must be a dict when used with extends_documentation_fragment (#21745)
* Fix string formatting
* Provide better tracebacks
* When options is None and extends_documentation_fragment is in use, add an error that options must be a dict
* If options was specified and not a dict, then error
2017-02-21 17:42:39 -08:00
Veaceslav Mindru
cdcdc1d81b #19587 add yum skip-broken (#21475)
* #19587: refactor to PEP8

* Issue: #19587: add skip_broken option to yum module
PNTSYSOPS-1901 - internal reference

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>

* Issue: #19587: fix line break, attempt to make readble

        things at line 646 look ugly ..
        trying to make them look a bit more human readble
        though the entire approach should be rewriten

PNTSYSOPS-1901 - internal reference

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>

* #19587: fixing typo, skip-broken has to be passed as list

* PR: #21475: peer review changes
            relates to: revert comment deletion https://github.com/ansible/ansible/pull/21475#discussion_r101574146
            relates to: remove irelevant var https://github.com/ansible/ansible/pull/21475#discussion_r101573388
            relates to: reformat https://github.com/ansible/ansible/pull/21475#discussion_r101572840

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>

* PR: #21475: peer review changes
            relates to: https://github.com/ansible/ansible/pull/21475#discussion_r101573736

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>

* Issue: #21475: fixing typo in the Man section

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>

* PR: #21475 - Peer review: revert variable a
        Relates to: https://github.com/ansible/ansible/pull/21475#pullrequestreview-22404868

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>

* Issue: 19587 - small correction

Signed-off-by: Veaceslav Mindru <vmindru@redhat.com> <mindruv@gmail.com>
2017-02-21 07:59:07 -08:00
Matt Martz
c160ede789 Validate modules json output (#21680) 2017-02-20 13:28:23 -08:00
Matt Martz
9b1bd2eb7e Re-enable module comparisons in validate-modules (#21582)
* Re-enable module comparisons, specifically for new module detection and for finding new options/arguments
* Only do new module checks in shippable, local will display warning
2017-02-17 15:13:18 -08:00
Toshio Kuratomi
eeaec56ed5 Use isinstance instead of type() comparisons. isinstance is more robust 2017-02-16 16:34:43 -08:00
Will Thames
f42ffe6de3 [cloud] Ensure that s3_bucket module always enables/disables versioning if it is specified (#21320)
If versioning is not enabled, then `get_bucket_versioning`
can return an empty dict.

If that happens, the code to enable versioning should still
run!

The logic for suspending versioning was also incorrect, so
have updated that too.

Fixes #20491
2017-02-16 09:45:05 -05:00
Matt Martz
7c00346714 Validate EXAMPLES as YAML 2017-02-15 13:01:43 -08:00
Will Thames
103ede26df Ensure ssh hostkey checks respect server port (#20840)
* Add tests for `get_fqdn_and_port` method.

Currently tests verify original behavior - returning default `ssh-keyscan` port
Add test around `add_host_key` to verify underlying command arguments
Add some new expectations for `get_fqdn_and_port`
Test that non-standard port is passed to `ssh-keyscan` command

* Ensure ssh hostkey checks respect server port

ssh-keyscan will default to getting the host key for port 22.
If the ssh service is running on a different port, ssh-keyscan
will need to know this.

Tidy up minor flake8 issues

* Update known_hosts tests for port being None

Ensure that git urls don't try and set port when a path
is specified

Update known_hosts tests to meet flake8

* Fix stdin swap context for test_known_hosts

Move test_known_hosts from under basic, as it is its own library.
Remove module_utils.known_hosts from pep8 legacy files list
2017-02-15 11:47:57 -08:00
Tim Rupp
2a9778e320 Change the f5 modules to use f5_utils file (#21466)
The f5 module utils were using a file name that appeared to
conflict with the f5 python SDK's namespace. This patch just changes
the name of the included class to be f5_utils to avoid the issue
of namespace collisions
2017-02-15 16:51:13 +00:00
John R Barker
53ac312382 validate-modules --arg-spec (#21331)
* validate-modules --arg-spec

* Update developing_modules_documenting.rst

* Never mock out ansible or ansible.module_utils

* No more false positives
2017-02-15 08:09:08 +00:00
Matt Clay
d54bc09fae Consider module_utils deps when running tests. (#21382)
* Skip pep8 analysis when --explain is used.
* Fix return type annotations.
* Match line length requirement of PEP 8 config.
* Consider module_utils deps when running tests.
2017-02-13 18:49:36 -08:00
Ted Timmons
23886c5c0d add instructions for running pep8 locally (#21120) 2017-02-13 11:09:50 -08:00
Matt Martz
87aa59af79 Legacy pep8 updates for setup.py and tests 2017-02-11 16:04:52 -08:00
s-hertel
2dc42fce80 [cloud] add tags parameter to ec2_vol & PEP8 fixes (#21254)
* Make ec2_vol pep8 and add tags parameter

* Remove ec2_vol.py from pep8 legacy-files
2017-02-10 15:22:38 -05:00
Matt Clay
46f9648aa1 Fix type() usage check in validate-modules.
This avoids false positives when 'if' or 'or' are in a string
preceding the use of type() on the same line.
2017-02-08 12:17:10 -08:00
Adrian Likins
7278308ca2 Redhat subscription (#20729)
* Make unused redhat_subscriptions do something

rhsm_baseurl/server_insecure were module params that were
never used previously. Hook them up for register options.

* pep8/style cleanups
* Set a module scope SUBMAN_CMD before we ref it as a global
* move ansible imports to top, remove '*' imports
* remove redhat_subscriptions from pep8 legacy
2017-02-07 12:07:01 -05:00
Matt Martz
5942de603d Legacy pep8 clean fixes for contrib and hacking (#21081) 2017-02-07 09:49:55 -06:00
Brian Coca
c86a17b7a0 refactoring async
- centralized skipping
- also fixed module name broken by previous refactor
- let action modules handle async processing
- moved async into base action class's module exec
- action plugins can now run final action as async
- actually skip copy if base skips
- fixed normal for new paths
- ensure internal stat is never async
- default poll to 10 as per docs
- added hint for callback fix on poll
- restructured late tmp, now a pipeline query
- moving action handler to connection as networking does
- fixed network assumption invocation is always passed
- centralized key cleanup, normalized internal var
- _supress_tmpdir_delete now in _ansible_xxx and gets removed from results
- delay internal key removal till after we use em
- nicer tmp removing, using existing methods
- moved cleanup tmp flag to mking tmp func
2017-02-06 19:32:00 -05:00
Loïc Blot
b55d039f67 New modules: vmware_guest_facts & vmware_guest_snapshot (#20971)
* Move gather_facts action outside of vmware_guest module
* Move facts gathering method to vmware.py library
* vmware_guest use some vmware.py library methods
* vmware.py lib: fix some PEP 8 issues
* Move snapshot actions to vmware_guest_snapshot
* Move common snapshot facts to vmware.py library
2017-02-06 11:52:17 -05:00
John R Barker
25e144db4a Update legacy-files.txt 2017-02-06 16:05:06 +00:00
John R Barker
0ea3911d47 module_utils/vmware.py is now clean 2017-02-06 16:01:10 +00:00
Toshio Kuratomi
d1a6b07fe1 Move ssh and local connection plugins from using raw select to selectors
At the moment, this change will use EPoll on Linux, KQueue on *BSDs,
etc, so it should alleviate problems with too many open file
descriptors.

* Bundle a copy of selectors2 so that we have the selectors API everywhere.
* Add licensing information to selectors2 file so it's clear what the
  licensing terms and conditions are.
* Exclude the bundled copy of selectors2 from our boilerplate code-smell test
* Rewrite ssh_run tests to attempt to work around problem with mocking
  select on shippable

Fixes #14143
2017-02-03 08:23:50 -08:00
Matt Clay
6580fb2cd1 Fix exception in module validator. 2017-02-02 17:38:16 -08:00
Matt Martz
829c0b8f62 Update validate-modules (#20932)
* Update validate-modules

* Validates ANSIBLE_METADATA
* Ensures imports happen after documentation vars
* Some pep8 cleanup

* Clean up some left over unneeded code

* Update modules for new module guidelines and validate-modules checks

* Update imports for ec2_vpc_route_table and ec2_vpc_nat_gateway
2017-02-02 11:45:22 -08:00
Matt Clay
64f98f855d Update PEP 8 legacy files list. 2017-02-01 13:22:07 -08:00
Toshio Kuratomi
5c38f3cea2 Combine jimi-c and bcoca's ideas and work on hooking module-utils into PluginLoader.
This version just gets the relevant paths from PluginLoader and then
uses the existing imp.find_plugin() calls in the AnsiballZ code to load
the proper module_utils.

Modify PluginLoader to optionally omit subdirectories (module_utils
needs to operate on top level dirs, not on subdirs because it has
a hierarchical namespace whereas all other plugins use a flat
namespace).

Rename snippet* variables to module_utils*

Add a small number of unittests for recursive_finder

Add a larger number of integration tests to demonstrate that
module_utils is working.

Whitelist module-style shebang in test target library dirs

Prefix module_data variable with b_ to be clear that it holds bytes data
2017-01-31 09:41:24 -08:00
Matt Clay
503e85ed0e PEP 8 E111 cleanup. 2017-01-30 20:33:31 -08:00
Matt Clay
f80224f828 PEP 8 E111 cleanup. 2017-01-30 20:18:50 -08:00
Matt Clay
85300883ef PEP 8 E111 cleanup. 2017-01-30 20:17:35 -08:00
Matt Clay
23f2efbc8d PEP 8 E111 cleanup. (#20841) 2017-01-30 18:03:13 -08:00
Matt Clay
462ab6b1db PEP 8 E111 cleanup for nxos modules. 2017-01-30 15:45:05 -08:00