Commit graph

34 commits

Author SHA1 Message Date
Chris Alfonso
b937bad52e Narrowing the match for python 3 cmp check (#25220)
Allowing the yaml examples in python documentation.
2017-05-31 17:54:20 +01:00
Dag Wieers
4efec414e7 test/: PEP8 compliancy (#24803)
* test/: PEP8 compliancy

- Make PEP8 compliant

* Python3 chokes on casting int to bytes (#24952)

But if we tell the formatter that the var is a number, it works
2017-05-30 18:05:19 +01:00
kkjang
5c43bd3bae Cmp python3 compat (#25008)
* Use double-quotes for expect integration tests

* Cast user input to string for expect integration tests

* Remove usage of cmp() for python3 compatibility

- Add code smell test to look for cmp usage

- Fixes #24756
2017-05-24 15:59:59 -07:00
Brian Coca
8f97aef1a3 Transition inventory into plugins (#23001)
* draft new inventory plugin arch, yaml sample

 - split classes, moved out of init
 - extra debug statements
 - allow mulitple invenotry files
 - dont add hosts more than once
 - simplified host vars
 - since now we can have multiple, inventory_dir/file needs to be per host
 - ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
 - centralized localhost handling
 - added plugin docs
 - leaner meaner inventory (split to data + manager)
 - moved noop vars plugin
 - added 'postprocessing' inventory plugins
 - fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
 - grouphost_vars loading as inventory plugin (postprocessing)
 - playbook_dir allways full path
 - use bytes for file operations
 - better handling of empty/null sources
 - added test target that skips networking modules
 - now var manager loads play group/host_vars independant from inventory
 - centralized play setup repeat code
 - updated changelog with inv features
 - asperioribus verbis spatium album
 - fixed dataloader to new sig
 - made yaml plugin more resistant to bad data
 - nicer error msgs
 - fixed undeclared group detection
 - fixed 'ungrouping'
 - docs updated s/INI/file/ as its not only format
 - made behaviour of var merge a toggle
 - made 'source over group' path follow existing rule for var precedence
 - updated add_host/group from strategy
 - made host_list a plugin and added it to defaults
 - added advanced_host_list as example variation
 - refactored 'display' to be availbe by default in class inheritance
 - optimized implicit handling as per @pilou's feedback
 - removed unused code and tests
 - added inventory cache and vbox plugin now uses it
 - added _compose method for variable expressions in plugins
 - vbox plugin now uses 'compose'
 - require yaml extension for yaml
 - fix for plugin loader to always add original_path, even when not using all()
 - fix py3 issues
 - added --inventory as clearer option
 - return name when stringifying host objects
 - ajdust checks to code moving

* reworked vars and vars precedence
 - vars plugins now load group/host_vars dirs
 - precedence for host vars is now configurable
 - vars_plugins been reworked
 - removed unused vars cache
 - removed _gathered_facts as we are not keeping info in host anymore
 - cleaned up tests
 - fixed ansible-pull to work with new inventory
 - removed version added notation to please rst check
 - inventory in config relative to config
 - ensures full paths on passed inventories

* implicit localhost connection local
2017-05-23 17:16:49 -04:00
Matt Clay
a07d42e16d Add support for cloud tests to ansible-test. (#24315)
* Split out ansible-test docker functions.
* Add cloud support to ansible-test.
2017-05-05 16:23:00 +08:00
Toshio Kuratomi
3428f42120 Add integration test that modules are loaded from the expected locations (#24170)
* Add integration test that modules are loaded from the expected locations
* Fix ping module to pass PEP8
2017-05-01 08:52:25 -07:00
Matt Clay
d662f6f0db Fix bugs in ansible-test units command. (#24044)
* Handle old versions of coverage.
* Handle old versions of setuptools.
* Detect python version for docker/remote units.
* Add sanity override for test constraints.
2017-04-27 11:21:11 +08:00
Matt Clay
e7bb508ad6 Fix var precedence check to support python 3. (#23552)
* Fix var precedence check to support python 3.
* Run CI sanity tests using python 3.5.
* Disable pylint non-iterator-returned test to pass on python 3.5.
2017-04-13 00:07:40 -07:00
Lumír 'Frenzy' Balhar
9d41aefd71 Add Python 3 support for validate-modules test. (#21195)
* Improve PEP8 compatibility

* Fix Python 3 incompatibility

Is prohibited to mutate OrderedDict during iteration through it so
is better to add records with error or warning to empty dictionary
instead of delete records from copy of dictionary during iterating.

* Decode output of subprocess from bytes to unicode.

* Add Python 3 support for validate-modules test.

Fix #18367
2017-04-10 07:43:39 -07:00
Dag Wieers
ac43a1bbbc Windows: Use the correct newline sequence for the platform (#21846)
This change to the template action plugin make template use the
platform's native newline_sequence for Jinja.

We also added the option `newline_sequence` to change the newline
sequence using by Jinja if you need to use another newline sequence than
the platform default.

This was previously discussed in
https://github.com/ansible/ansible/issues/16255#issuecomment-278289414

And also relates to issue #21128
2017-03-23 19:47:10 -07:00
Toshio Kuratomi
2fff690caa Update module_utils.six to latest (#22855)
* Update module_utils.six to latest

We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility.  Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
2017-03-23 13:35:05 -07:00
James Cammarata
a2599cab79 Fix variable precedence of INI/script vars to be in-line with docs.
This commit also adds a new test script (ansible-var-precedence-check.py in code-smell/)
to provide us with another line of defense against precedence bugs going forward.

The precedence docs state that the INI vars have a lower precedence than group/host
vars files for inventory and playbooks, however that has not been the case since 2.0
was released. This change fixes that in one way, though not exactly as the docs say.
The rules are:

1) INI/script < inventory dir < playbook dir
2) "all" group vars < other group_vars < host_vars

So the new order will be (from the test script mentioned above):

8. pb_host_vars_file - var in playbook/host_vars/host
9. ini_host_vars_file - var in inventory/host_vars/host
10. ini_host - host var inside the ini
11. pb_group_vars_file_child - var in playbook/group_vars/child
12. ini_group_vars_file_child - var in inventory/group_vars/child
13. pb_group_vars_file_parent - var in playbook/group_vars/parent
14. ini_group_vars_file_parent - var in inventory/group_vars/parent
15. pb_group_vars_file_all - var in playbook/group_vars/all
16. ini_group_vars_file_all - var in inventory/group_vars/all
17. ini_child - child group var inside the ini
18. ini_parent - parent group var inside the ini
19. ini_all - all group var inside the ini

Fixes #21845
2017-03-02 17:07:00 -06:00
Matt Clay
d66ce40ecb Overhaul ansible-test sanity implementation. (#22177)
- Tests are run to completion instead of stopping on first failure.
- Test results are now parsed instead of passing through to the console.
- Test results can be saved in junit xml format.
- Test results will show up on the Shippable "Tests" result tab.
- Added an experimental --lint option for easier integration with other tools.
- Code smell tests are now usable with the --list-tests, --test and --skip-test options.
- Code split out from executor.py into sanity.py.
- Rename download-logs to download.py and add support for test and coverage results.
- Miscellaneous improvements.
2017-03-02 12:36:46 -08:00
Matt Clay
cba91c7f44 Test placement (#21801)
* Fix test placement and update messaging.
* Add note about group selection.
2017-02-22 12:43:45 -08:00
Matt Clay
b5e0eb0985 Fixes for missing aliases files sanity test. (#21798)
* Refine test for missing aliases files.
* Fix formatting of missing aliases message.
2017-02-22 12:04:09 -08:00
Matt Clay
d54bc09fae Consider module_utils deps when running tests. (#21382)
* Skip pep8 analysis when --explain is used.
* Fix return type annotations.
* Match line length requirement of PEP 8 config.
* Consider module_utils deps when running tests.
2017-02-13 18:49:36 -08:00
Toshio Kuratomi
d1a6b07fe1 Move ssh and local connection plugins from using raw select to selectors
At the moment, this change will use EPoll on Linux, KQueue on *BSDs,
etc, so it should alleviate problems with too many open file
descriptors.

* Bundle a copy of selectors2 so that we have the selectors API everywhere.
* Add licensing information to selectors2 file so it's clear what the
  licensing terms and conditions are.
* Exclude the bundled copy of selectors2 from our boilerplate code-smell test
* Rewrite ssh_run tests to attempt to work around problem with mocking
  select on shippable

Fixes #14143
2017-02-03 08:23:50 -08:00
Toshio Kuratomi
5c38f3cea2 Combine jimi-c and bcoca's ideas and work on hooking module-utils into PluginLoader.
This version just gets the relevant paths from PluginLoader and then
uses the existing imp.find_plugin() calls in the AnsiballZ code to load
the proper module_utils.

Modify PluginLoader to optionally omit subdirectories (module_utils
needs to operate on top level dirs, not on subdirs because it has
a hierarchical namespace whereas all other plugins use a flat
namespace).

Rename snippet* variables to module_utils*

Add a small number of unittests for recursive_finder

Add a larger number of integration tests to demonstrate that
module_utils is working.

Whitelist module-style shebang in test target library dirs

Prefix module_data variable with b_ to be clear that it holds bytes data
2017-01-31 09:41:24 -08:00
Matt Clay
7ba47bfd2d Correct test constraints and add sanity check. 2017-01-08 16:44:31 -08:00
Matt Clay
50b11cd8e3 Add test for ConfigureRemotingForAnsible.ps1. 2017-01-06 10:40:13 -08:00
Toshio Kuratomi
74db76ec9c Not my day... Remove extraneous vim typo 2017-01-05 10:22:00 -08:00
Toshio Kuratomi
d001d250f0 Whitelist the hacking/cherrypick script's use of python3 2017-01-05 10:20:58 -08:00
Matt Clay
422857166b Remove legacy test alias migration script. (#19629)
* Require code-smell tests to be files.
* Add sanity check for integration test aliases.
* Remove migration script for test aliases.
2016-12-21 16:46:28 -08:00
Toshio Kuratomi
0b440a9289 Code smell test for iteritems and itervalues (#19292)
* Code smell test for iteritems and itervalues

* Change the keydict object in authorized_keys so it doesn't throw a false postive

keydict is a bad data structure anyway.  We don't use the iteritems and
itervalues methods so just disable them so that the code-smell tests do
not trigger on it.

* Change release templates so they work with py3
2016-12-14 12:16:16 -08:00
Toshio Kuratomi
dd46cb7b05 Add a code-smell test for iterkeys (#18589)
* Add a code-smell test for iterkeys
* Add a message on how to port iterkeys problems and make the grep more robust
2016-12-13 16:39:03 -08:00
Matt Clay
bd7ce0696e Fix tests after repo merge. 2016-12-08 11:35:19 -05:00
Matt Clay
6bbd92e422 Initial ansible-test implementation. (#18556) 2016-11-29 21:21:53 -08:00
Matt Clay
912d6ed8cc Clean up code-smell sanity scripts. (#18407)
- Replace find ';' with '+' for faster execution.
- Replace grep -R with -r to avoid recursive warnings.
- Exclude .git and .tox directories from recursive grep.
- Improve messaging on failed sanity checks.
- Add no-basestring check to Shippable.
2016-11-07 16:46:33 -08:00
Matt Clay
8c270ac75f Add empty-init code-smell script. (#18406)
Also removed boilerplate from otherwise empty __init__.py files
which should not contain any code (checked by empty-init script).
2016-11-07 15:02:13 -08:00
Matt Clay
1238a4c0c9 Run validate-modules with python 2 only.
This allows validate-modules to run in an environment where
python 3 is the default. This will no longer be necessary once
validate-modules is updated to work with both python 2 and 3.
2016-11-04 12:08:58 -07:00
Matt Clay
bb3801bafd Cleanup and fixes for code smell scripts.
- Fix shellcheck issues.
- Add .tox exclusions.
2016-11-03 23:56:30 -07:00
Matt Clay
469c4a106b Fix and create code-smell test for line endings. 2016-11-03 14:58:11 -07:00
Matt Clay
0d46805979 Clean up shebangs for various files.
- Remove shebangs from:
  - ini files
  - unit tests
  - module_utils
  - plugins
  - module_docs_fragments
  - non-executable Makefiles
- Change non-modules from '/usr/bin/python' to '/usr/bin/env python'.
- Change '/bin/env' to '/usr/bin/env'.

Also removed main functions from unit tests (since they no longer
have a shebang) and fixed a python 3 compatibility issue with
update_bundled.py so it does not need to specify a python 2 shebang.

A script was added to check for unexpected shebangs in files.
This script is run during CI on Shippable.
2016-11-02 17:00:27 -07:00
Matt Clay
71819c0a60 Test reorganization and cleanup. (#18270)
- Correct directory name in test/README.md
- Move code-smell tests to test/sanity/code-smell
- Update code-smell.sh to use new script paths
- Add test/integration/target-prefixes.win for ansible-test
- Move module unit tests to match module directory layout
2016-10-31 12:53:11 -07:00