cmueller/ansible
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
36270 commits 47 branches 390 tags 440 MiB
Commit graph

214 commits

Author SHA1 Message Date
Matt Clay
789218c215 Initial ansible-test sanity docs. (#26775) 
* Rename no-iterkeys test for consistency.

* Require docs for all ansible-test sanity tests.

* Initial ansible-test sanity docs.

* Fix capitalization of Python.

* Fix sanity code smell test false positives.

* Fix another code-smell false positive.
 2017-07-14 14:24:45 +01:00
Pilou
4b3d6dfa8a Use pycodestyle instead of pep8 (#25947) 2017-07-13 11:46:31 -07:00
Matt Clay
fad469efca Ignore platform_agnostic.yaml in ansible-test. 2017-07-13 11:23:59 -07:00
Matt Clay
0b784c65b1 Add sanity import test to ansible-test. (#26730) 
* Add sanity import test to ansible-test.
* Run sanity import test on all python versions.
 2017-07-12 20:04:22 -07:00
Matt Clay
20d49af475 Add ANSIBLE_TEST_CI env var to integration tests. 2017-07-12 17:46:12 -07:00
Matt Clay
ab5dd24430 Handle pip/python errors in ansible-test better. 2017-07-11 16:37:16 -07:00
Matt Clay
82558baaf6 Add role dependencies to change classification. 2017-07-10 18:20:09 -07:00
Matt Clay
4a013d6d69 Add RHEL 7.3 to ansible-test and CI. (#26550) 
* Add support for RHEL to ansible-test.
* Enable RHEL 7.3 on Shippable.
 2017-07-07 16:52:40 -07:00
David Newswanger
294c0f91c0 added option to ansible-test to run test playbooks using the debug strategy (#26499) 2017-07-07 12:37:08 -07:00
Matt Clay
a94156227d Improve error handling for cs tests. 2017-07-06 17:55:51 -07:00
Matt Clay
45e377566c Refactor ansible-test config classes. (#26505) 
* Move Config classes from executor.py to config.py.
* Move Environment and Test config to config.py.
* Move Coverage/CoverageReport Config to config.py.
* Clean up type hints.
 2017-07-06 16:14:44 -07:00
Matt Clay
bd8ea89b1b Improve handling of network test changes. 2017-07-06 14:25:58 -07:00
Matt Clay
c26a556929 Add check for known_hosts changing during tests. (#26470) 
* Add check for known_hosts changing during tests.
* Don't use known hosts file for delegate_to test.
 2017-07-06 00:47:28 -07:00
Matt Clay
fba76444e0 Add ansible-test coverage report --show-missing. 2017-07-05 17:11:41 -07:00
David Newswanger
0a6dcf4169 changed ansible-test so that it replaces . with - instead of . with _ so that the hostname is valid 2017-07-03 20:11:50 -07:00
Matt Clay
cd3a795e65 Permissions fix for remote coverage downloads. 2017-07-02 14:09:16 -07:00
Matt Clay
554423cf04 Expose docker socket to test containers. 2017-06-30 16:46:41 -07:00
René Moser
77d0542985 test, integration: add diff support (#26296) 2017-06-30 20:43:34 +02:00
Pierre-Louis Bonicoli
d08ff86169 ansible-test: pass --[skip-]tags to ansible-playbook 
Thanks to Olaf Klischat for the idea.
 2017-06-30 10:35:39 -07:00
Matt Clay
e56ea435f0 Skip dependency check for deleted files. (#26112) 2017-06-26 11:48:46 -07:00
David Newswanger
b2d88f8b73 updated ansible-test integration to skip any files in test/integration/targets/ that aren't directories. 
Fixes #26007
 2017-06-22 13:03:42 -07:00
jctanner
996dcb989c Add the cloud provider script for the vcenter simulator container (#25964) 2017-06-22 10:49:58 -07:00
Matt Clay
53180ebb57 Fix Shippable BASE_BRANCH lookup on merges. 2017-05-19 01:51:24 +08:00
Matt Clay
5babe2daea Increase python version coverage for tests. (#24762) 
* Improve ansible-test inventory handling.
* Fix python 3 re-raise of exception from thread.
* Fix python 3 encoding for windows-integration.
* Run network tests on multiple python versions.
* Run windows tests on multiple python versions.
* Support Shippable delegation using --tox.
* Skip vyos_command on python 3 tests until fixed.
* Add python 3 filtering to local and tox.
* Fix tests to support back to back runs.
* Temporarily test networking with python 2.7 only.

Running the tests back to back causes intermittent test failures
which need to be addressed before we can test multiple versions
in a single test run.
 2017-05-19 01:37:53 +08:00
Matt Clay
4daa16f9ce Add --all and --stub coverage options. 
ci_coverage
ci_complete
 2017-05-16 12:02:03 +08:00
Matt Clay
6a2a7a2392 Enable codecov.io and add coverage grouping. 2017-05-15 19:14:05 +08:00
Matt Clay
fdcad0f44d Improve on-demand coverage for Shippable. 2017-05-14 18:27:58 +08:00
John R Barker
e9e2d67035 Run vyos_command Network tests in Shippable (#24514) 
* Run VyOS Network tests in Shippable

* net_command no longer exists

* network/ci

* Disable other network tests

* pep8 fix

* Fix PEP 8 issue.
 2017-05-12 12:49:12 +01:00
Matt Clay
dfd19a812f Miscellaneous bug fixes for ansible-test. 
- Overhauled coverage injector to fix issues with non-local tests.
- Updated integration tests to work with the new coverage injector.
- Fix concurrency issue by using random temp files for delegation.
- Fix handling of coverage files from root user.
- Fix handling of coverage files without arcs.
- Make sure temp copy of injector is world readable and executable.
 2017-05-12 14:55:48 +08:00
Matt Clay
fb9efd3c7e Add --remote-terminate option to ansible-test. 
Set `--remote-terminate success` for osx tests.
 2017-05-11 22:42:52 +08:00
Matt Clay
b8cb3f519b Detect and fix environment tampering in tests. 2017-05-11 19:45:15 +08:00
Matt Clay
e6934e42be Update ansible-test change handling and CS plugin. (#24395) 
* Use CloudStack simulator from Ansible repo.
* Recognize cloud specific ansible-test changes.
* Hide ansible-test cs notice in explain mode.
 2017-05-09 16:15:27 +08:00
Matt Clay
0573b8bcd2 Fix ansible-test cloud integration bugs. 2017-05-05 21:46:29 +08:00
Matt Clay
ac72fd9d2c Add missing return statements. 2017-05-05 18:37:11 +08:00
Matt Clay
201242f302 Fix call to wrong method in ansible-test. 2017-05-05 18:18:40 +08:00
Matt Clay
1582e9bf7d Fix handling of Shippable env vars. 2017-05-05 17:44:51 +08:00
Matt Clay
a07d42e16d Add support for cloud tests to ansible-test. (#24315) 
* Split out ansible-test docker functions.
* Add cloud support to ansible-test.
 2017-05-05 16:23:00 +08:00
Matt Clay
6522d703a1 Fix ansible-test pylint and pep8 issues. (#24245) 
* Fix pylint useless-super-delegation issues.
* Fix miscellaneous pylint issues.
* Fix PEP 8 issues.
* Fix warnings reported by PyCharm.
 2017-05-03 23:19:44 +08:00
Matt Clay
d662f6f0db Fix bugs in ansible-test units command. (#24044) 
* Handle old versions of coverage.
* Handle old versions of setuptools.
* Detect python version for docker/remote units.
* Add sanity override for test constraints.
 2017-04-27 11:21:11 +08:00
Matt Clay
cb1f57d4e5 Add rstcheck to ansible-test and correct issues. (#23550) 
* Add rstcheck to ansible-test.
* Fix rst code-block languages and syntax errors.
* Fix rst inline literals.
* Update python 2 code block to pass tests on py 3.
 2017-04-13 10:28:52 -07:00
Matt Clay
92ad9a19b6 Fix ansible-test empty diff handling on Shippable. 2017-04-08 22:08:08 -07:00
Matt Clay
0cf1e8e061 Fix ansible-test handling of import/syntax errors. 2017-04-07 12:20:31 -07:00
Matt Clay
1daa69d685 Initial pylint support for ansible-test sanity. 2017-04-04 16:16:21 -07:00
Matt Clay
4fdeade389 Overhaul ansible-test import analysis. (#22888) 
- Add support for module_utils packages for better organization.
- Add support for "virtual" module_utils packages such as `six`.
 2017-03-22 18:07:53 -07:00
Matt Clay
042e387f27 Fix metadata handling for shell command. (#22801) 2017-03-20 15:31:57 -07:00
Matt Clay
991bf0a1c3 Fix ansible-test base branch handling on merge. (#22759) 2017-03-17 23:33:32 -07:00
Matt Clay
47887a2edd Fix ansible-test base branch usage on Shippable. 2017-03-17 17:35:09 -07:00
Matt Clay
f7c83ccdc0 Fix ansible-test PR diff handling. 
Permits use of PRs on branches other than devel.
 2017-03-16 10:34:31 -07:00
Matt Clay
3a22467b38 Remove python 2.4 compile test. 2017-03-15 15:46:12 -07:00
Matt Clay
869449e288 Add test verification to ansible-test. (#22636) 
* Add unified git diff parser.
* Add metadata and diff handling.
* Add test confidence/verification to bot output.
 2017-03-15 12:17:42 -07:00
Matt Clay
5e9a2b8528 Verify SHA in Shippable change detection. 2017-03-15 11:31:55 -07:00
Matt Clay
89559f78de Add --failure-ok option to ansible-test. (#22623) 2017-03-14 16:32:21 -07:00
Matt Clay
771b14acb2 Sort test messages for deterministic output. (#22581) 2017-03-13 20:45:10 -07:00
Matt Clay
cfd313bced Disable ansible color output on sanity tests. 2017-03-08 12:18:16 -08:00
Matt Clay
3ab344e8bd Use lint/junit output for compile test. (#22395) 
* Refactor sanity classes for use in all tests.
* Use lint/junit output for compile test.
* Add missing options for compile test.
* Fix early bailout on requirements install.
 2017-03-08 00:47:21 -08:00
Matt Clay
fbf55886ba Use lint friendly output for python syntax checks. (#22390) 2017-03-07 20:27:50 -08:00
Matt Clay
8f463fcdd2 Create bot friendly sanity output. (#22381) 2017-03-07 14:59:50 -08:00
Matt Clay
3e4be156d7 Pre-compile regexes to speed up target processing. 
Without this, changing a large number of files results in target
processing taking a very long time due to repeatedly compiling
the same patterns in a loop over many targets.
 2017-03-06 12:35:12 -08:00
Matt Clay
d66ce40ecb Overhaul ansible-test sanity implementation. (#22177) 
- Tests are run to completion instead of stopping on first failure.
- Test results are now parsed instead of passing through to the console.
- Test results can be saved in junit xml format.
- Test results will show up on the Shippable "Tests" result tab.
- Added an experimental --lint option for easier integration with other tools.
- Code smell tests are now usable with the --list-tests, --test and --skip-test options.
- Code split out from executor.py into sanity.py.
- Rename download-logs to download.py and add support for test and coverage results.
- Miscellaneous improvements.
 2017-03-02 12:36:46 -08:00
Matt Clay
28b882e737 Support ansible commands in sanity tests. (#22172) 2017-03-01 15:48:14 -08:00
Matt Clay
4f3dade266 Add retries for ansible-test docker run. 2017-02-27 22:55:46 -08:00
Matt Clay
45cec93e99 Disable newer shellcheck tests which don't pass. 2017-02-27 22:31:47 -08:00
Matt Clay
fcac261eef Run unit tests in isolation w/ coverage support. 2017-02-27 17:16:45 -08:00
Matt Clay
c87ebb6215 Deduplicate arcs when combining coverage data. (#22022) 2017-02-27 15:46:15 -08:00
Matt Clay
2797cff773 Run destructive win tests when creating instances. 2017-02-27 10:36:53 -08:00
Matt Clay
2804d2e848 Split up unit tests on Shippable. (#21931) 
* Add more verbose output to coverage combine.
* Run unit tests on CI separately by version.
 2017-02-24 16:58:56 -08:00
Matt Clay
bc8186543a Pass base branch to module validator on delegate. 2017-02-20 14:28:11 -08:00
Matt Martz
9b1bd2eb7e Re-enable module comparisons in validate-modules (#21582) 
* Re-enable module comparisons, specifically for new module detection and for finding new options/arguments
* Only do new module checks in shippable, local will display warning
 2017-02-17 15:13:18 -08:00
Matt Clay
1ce2ea0b6c Fix ansible-test handling of deleted test targets. 2017-02-17 11:16:24 -08:00
Matt Clay
2192c1eb02 Fix ansible-test selection of inventory file. 2017-02-16 15:03:27 -08:00
John R Barker
5e28d3a15c network_cli shouldn't be specified anymore (#21458) 2017-02-15 12:58:12 +00:00
Matt Clay
d54bc09fae Consider module_utils deps when running tests. (#21382) 
* Skip pep8 analysis when --explain is used.
* Fix return type annotations.
* Match line length requirement of PEP 8 config.
* Consider module_utils deps when running tests.
 2017-02-13 18:49:36 -08:00
Peter Sprygada
7f1c43e597 updates ios modules to support persistent socket (#21258) 
* updates all ios modules to support persistent socket
* adds ios action plugin to connect to device
* adds exec_command() to ios shared module
* fixes ios_config and ios_template local action
* update all unit test cases
* adds base test module for ios module testing
 2017-02-13 15:22:14 +00:00
Matt Clay
4189592701 Revert "Enable process isolation for unit tests." 
This reverts commit 91526cd9f2.

Removing this feature primarily because it interferes with
collecting proper code coverage results. I may restore the
feature later if that can be resolved.
 2017-02-09 11:10:06 -08:00
Matt Clay
91526cd9f2 Enable process isolation for unit tests. 2017-02-08 14:36:42 -08:00
Matt Clay
bce5fc7d96 Use us-east-2 for Windows CI on Shippable. 2017-02-06 15:01:20 -08:00
Matt Clay
309a37de86 Add --debug option to ansible-test. 2017-02-01 16:05:03 -08:00
Matt Clay
6ef1a6aeb6 Add pep8 to ansible-test. (#20745) 
The rule sets and legacy file list are a first draft.
It is likely that they will need to be revised.
 2017-01-27 12:34:00 -08:00
Dag Wieers
3a0a74dc18 Make test/runner less verbose when spawning instances 
As discussed on IRC
 2017-01-27 12:27:46 -08:00
Matt Clay
5503b89831 Run fewer tests for winrm config script changes. 2017-01-26 15:54:01 -08:00
Matt Clay
76d563af79 Temporarily switch Windows tests to us-east-1. 
Will switch back to us-east-2 after AWS limit increase is done.
 2017-01-26 07:37:14 -08:00
Matt Clay
fa7f84217b Add startup checks to ansible-test. 2017-01-24 13:08:04 -08:00
Matt Clay
61e6e7868c Improve jinja2 test coverage. (#20533) 
* Run `template` test on latest jinja2 version.
* Update jinja2 install for groupby_filter.
* Clean test output dir for each test.
 2017-01-20 16:38:52 -08:00
Matt Clay
7ed67746cf Enable Windows 2016 on Shippable. 
Also launch Windows instances on us-east-2 and all others on us-east-1.
 2017-01-18 18:37:42 -08:00
Matt Clay
4ba9f63afe Use multiple AWS regions in ansible-test. (#20429) 
* Move ansible-test EnvironmentConfig to util.
* Use multiple AWS regions in ansible-test.
 2017-01-18 16:31:34 -08:00
Matt Clay
4673a9c9c2 Generate ssh key once for network tests. 2017-01-18 13:33:12 -08:00
Matt Clay
4c97bad28e Update ansible-test network change classification. 2017-01-18 12:19:31 -08:00
Matt Clay
3c51ce56fd Enable more network support in ansible-test. (#20372) 
* Only start platform instances with tests selected.
* Enable ios on Shippable.
* Show inventory in explain mode.
* Fix indentation of generated network inventory.
* Update classification of network module_utils.
 2017-01-17 16:24:05 -08:00
Matt Clay
dbb89549ff Improve ansible-test error handling and timeouts. (#20328) 
* Eliminate warning/retry on old instance check.
* Increase instance start timeout for windows.
 2017-01-16 18:59:44 -08:00
Matt Clay
b5d5888b6e Add support for ios to ansible-test. 2017-01-16 15:05:56 -08:00
Matt Clay
e2bd71ac02 Clean up inventory generation in ansible-test. (#20313) 2017-01-16 12:31:17 -08:00
Matt Clay
a59ce3a7f9 Add retry on failure to get instance. 2017-01-13 18:35:28 -08:00
Matt Clay
f3130898bb Retry instance start on error. 2017-01-13 16:49:30 -08:00
Matt Clay
d8733a5455 Complete initial network-integration support. 2017-01-12 12:39:00 -08:00
Matt Clay
300181cfd3 Send current winrm config to ansible-core-ci. 2017-01-10 20:05:47 -08:00
Matt Clay
f340b8d14f Add test support for python 3.6. 2017-01-10 14:34:42 -08:00
Matt Clay
3ec7243918 Run all win tests for winrm config script changes. 2017-01-08 00:57:51 -08:00
Brian Coca
57f8b791d6 consolidated docs 
point to new doc locations
removed non existing dirs
 2017-01-06 09:16:59 -05:00
Matt Clay
b7780ca530 Pass DOCKER_* env vars to docker in ansible-test. 2017-01-04 13:23:16 -08:00
Matt Clay
dac226de4a Improve ansible-test completion and messaging. (#19627) 
* Simplify type annotation of args.docker_util.
* Improve error messaging for ansible-doc test.
* Add completion for docker, remote and windows.
 2016-12-22 09:57:17 -08:00
Matt Clay
422857166b Remove legacy test alias migration script. (#19629) 
* Require code-smell tests to be files.
* Add sanity check for integration test aliases.
* Remove migration script for test aliases.
 2016-12-21 16:46:28 -08:00
Matt Clay
9b5c782a0b Use docker pull by default in ansible-test. 2016-12-14 20:11:43 -08:00
Matt Clay
f28b5a0ed8 Add --tox-sitepackages option to ansible-test. 2016-12-13 16:57:23 -08:00
Matt Clay
45772b961f Add private fork CI support to ansible-test. 2016-12-13 15:49:57 -08:00
Matt Clay
1c9a58a885 Run all tests when no last successful commit. 
(cherry picked from commit 2e56035fddfd04eb084bd977fc1ee2f35c2f5283)
 2016-12-08 11:35:20 -05:00
Matt Clay
5628e260f2 Relocate module unit tests. (#18812) 
* Relocate module unit tests.
* Fix classification of unit test changes.
 2016-12-08 11:35:20 -05:00
Matt Clay
573078fc6b Fix unit test directory path target. 2016-12-08 11:35:20 -05:00
Matt Clay
47083f145a Don't error if pytest skips all unit tests. 2016-12-08 11:35:20 -05:00
Matt Clay
8e951b9f98 Remove legacy test/samples directory. 2016-12-08 11:35:20 -05:00
Matt Clay
7d3e2b2121 Verify path before assigning to sanity test. 2016-12-08 11:35:19 -05:00
Matt Clay
14a2757116 Correct change classification for network plugins. 2016-12-05 20:07:18 -05:00
Adrian Likins
ba56c87566 Add SSH_AUTH_SOCK to default env 
So ssh-agent works when using locally.
 2016-11-30 15:53:15 -08:00
Matt Clay
bee58fe890 Fix compile skip path handling for targeted tests. 2016-11-30 09:47:56 -08:00
Matt Clay
6bbd92e422 Initial ansible-test implementation. (#18556) 2016-11-29 21:21:53 -08:00