* Incase of network integration test for connection local
test the paramiko_ssh auth_timeout is the value of timeout
under defaults section which is 10 seconds.
* For slower connection 10sec timout value result in authentication
timeout error hence increase the timeout value to 90 seconds
Add integration tests for unarchiving as unprivileged user
Break tasks into separate files for easier reading and maintenance
Create a user by specifying a default group of 'staff' for macOS.
The user module does not actually remove the user directory on macOS,
so explicitly remove it.
Put the removal tasks in an always block to ensure they always run
Co-authored-by: Philip Douglass <philip.douglass@amadeus.com>
Co-authored-by: Sam Doran <sdoran@redhat.com>
The iso8601_micro and iso8601 facts incorrectly called now.utcnow(), resulting
in a new timestamp at the time it was called, not a conversion of the previously
stored timestamp.
Correct this by capturing the UTC timestamp once then calculating the local
time using the UTC offset of the current system.
* Use time.time() for getting the current time
* Convert from that stored epoch timestamp to local and UTC times
* Used existing timestamp for epoch time
* Add unit tests that validate the formate of the return value rather than an exact value since mocking time and timezone is non-trivial
Follow up to #70221
Related to #67794
CVE-2020-1736
When set_mode_if_different() is called with mode of 'None', ensure we issue
a warning about the change in default permissions.
Add integration tests to ensure the warning works properly.
* Fix tests
- actually use custom module 🤦♂️
- verify file permission on created files
- use remote_tmp_dir so we're ready for split controller
- improve test module so we can skip the call to set_fs_attributes_if_different()
- fix tests for CentOS 6
Change:
- Clarify that not passing `--tags` will cause `ansible_run_tags` to
default to `["all"]`.
- Add some extra coverage around `ansible_run_tags`
Test Plan:
- New integration and unit tests
Tickets:
- Fixes#69619
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Allow older git to verify tags again
- Enable verification tests everywhere, even if most of them only work
on newer git. Some of them work on older git and they test the --raw
parameter.
Test Plan:
- Re-enabled subset of git tests
Tickets:
- Fixes#64469
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix verbosity for var_templating test
Display the subtests
* fix specificity and set ANSIBLE_BECOME_ALLOW_SAME_USER
Co-authored-by: Matt Martz <matt@sivel.net>
Co-authored-by: Matt Martz <matt@sivel.net>
Change:
- Previously, we only showed that something would have changed, not what
would have changed. This allows us to show what will chang as well.
Test Plan:
- Local RHEL8 VM
- New integration tests
Tickets:
- Fixes#66132
Signed-off-by: Rick Elrod <rick@elrod.me>
* Also validate top-level version_added.
* Fix error code.
* Produce same version_added validation error in schema than in code (and stop returning it twice).
* Return correct error codes for invalid version_added for options and return values.
* Add changelog.
* Fix forgotten closing braket.
* Accept 'historical' for some top-level version_added.
* dnf: Add nobest option
* dnf: Fix indent, add nobest specifically to dnf not yum
* Add changelog for dnf: add nobest option
* dnf: Add nobest to yumdnf module argument_spec
* dnf: remove nobest from module paramaters in yumdnf.py
* dnf: Add test for nobest option
* dnf: Cleanup packages in nobest test at last
* dnf: Cleanup manually added repos in nobest test at last
* dnf: Remove dnf-plugins-core as well in nobest test
* dnf: Change nobest release version to 2.11
* Change changelog number according to change in PR number
* Change changelog number according to change in PR number
* Change default file permissions so they are not world readable
CVE-2020-1736
Set the default permissions for files we create with atomic_move() to 0o0660. Track
which files we create that did not exist and warn if the module supports 'mode'
and it was not specified and the module did not call set_mode_if_different(). This allows the user to take action and specify a mode rather than using the defaults.
A code audit is needed to find all instances of modules that call atomic_move()
but do not call set_mode_if_different(). The findings need to be documented in
a changelog since we are not warning. Warning in those instances would be frustrating
to the user since they have no way to change the module code.
- use a set for storing list of created files
- just check the argument spac and params rather than using another property
- improve the warning message to include the default permissions
* ensure hostvars are available on delegation
* also inventory_hostname must point to current host and not delegated one
* fix get_connection since it was still mixing original host vars and delegated ones
* also return connection vars for delegation and non delegation alike
* add test to ensure we have expected usage when directly assigning for non delegated host
* Encode/Decode files in UTF-8
* Use helper function in ansible
* Add an integration test
* Use emoji in test data.
* add changelog
* Also support non-ascii chars in filepath and add tests about this.
* Also use non-ascii chars in replaced text and ensure not to break cron syntax.
* rename self.existing to self.n_existing
* rename crontab.existing to crontab.n_existing
Change:
- Refactoring to make it harder to get wrong and easier to read.
- Generalize become_unprivileged tests and fix some that never worked
but also never failed.
Test Plan:
- CI, new units/integration tests
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix missing quoting for remote_tmp in second mkdir of shell module. Issue #69577
* adding changelog
* fixing typo in changelog entry
* adding test case
Adding test case written by bmillemayhias.
* using $HOME instead of ~
* fixing commit measage
* Update 69578-shell-remote_tmp-quoting.yaml
Co-authored-by: Brian Kohles <me@briankohles.com>
A couple of years ago Slackware -current began using a plus (“+”) at the end of the distribution version string to indicate a future version work-in-progress.
Rearrange distribution_files unit tests to easily support more tests
- add conftest with common fixtures
- use parametrize for testing multiple scenarios
* Add changelog
* Add unit tests for Slackware distribution parsing
* Use correct fixtures for Slackware
Data comes from /etc/slackware-version
Co-authored-by: Sam Doran <sdoran@redhat.com>
Co-authored-by: <Eduard Rozenberg <eduardr@pobox.com>>
* Build documentation for Ansible-2.10 (formerly known as ACD).
Builds plugin docs from collections whose source is on galaxy
The new command downloads collections from galaxy, then finds the
plugins inside of them to get the documentation for those plugins.
* Update the python syntax checks
* docs builds can now require python 3.6+.
* Move plugin formatter code out to an external tool, antsibull-docs.
Collection owners want to be able to extract docs for their own
websites as well.
* The jinja2 filters, tests, and other support code have moved to antsibull
* Remove document_plugins as that has now been integrated into antsibull-docs
* Cleanup and bugfix to other build script code:
* The Commands class needed to have its metaclass set for abstractmethod
to work correctly
* Fix lint issues in some command plugins
* Add the docs/docsite/rst/collections to .gitignore as
everything in that directory will be generated so we don't want any of
it saved in the git repository
* gitignore the build dir and remove edit docs link on module pages
* Add docs/rst/collections as a directory to remove on make clean
* Split the collections docs from the main docs
* remove version and edit on github
* remove version banner for just collections
* clarify examples need collection keyword defined
* Remove references to plugin documentation locations that no longer exist.
* Perhaps the pages in plugins/*.rst should be deprecated
altogether and their content moved?
* If not, perhaps we want to rephrase and link into the collection
documentation?
* Or perhaps we want to link to the plugins which are present in
collections/ansible/builtin?
* Remove PYTHONPATH from the build-ansible calls
One of the design goals of the build-ansible.py script was for it to
automatically set its library path to include the checkout of ansible
and the library of code to implement itself. Because it automatically
includes the checkout of ansible, we don't need to set PYTHONPATH in
the Makefile any longer.
* Create a command to only build ansible-base plugin docs
* When building docs for devel, only build the ansible-base docs for
now. This is because antsibull needs support for building a "devel
tree" of docs. This can be changed once that is implemented
* When building docs for the sanity tests, only build the ansible-base
plugin docs for now. Those are the docs which are in this repo so
that seems appropriate for now.
A recent updated to psutil, which is a dependency of ansible-runner, fails
to install on older versions of pip.
Commit with the breaking change:
135628639b
* Get m_u.facts.utils coverage up to 100%
Change:
- Add tests to 'gathering_facts' integration target to get
module_utils.facts.utils coverage up to 100%.
- This also clears incidental coverage from incidental_selinux.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Test async_wrapper when the module it runs has stderr output
Test Plan:
- CI
- Looked at coverage report and saw green for a few lines that weren't
previously green.
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Add integration tests for various cases
- Fix wrong use of "its" in an exception thrown in varnames when it
throws an AnsibleError, given a term of the wrong type.
Test Plan:
- new tests, CI
Tickets:
- Fixes#70546
Signed-off-by: Rick Elrod <rick@elrod.me>
- ensure we preserve the typeerror part of the exception so loop defereed error handling
can postpone those caused by undefined variables until the when check is done.
- fix tests to comply with the 'new normal'
- human_to_bytes and others can issue TypeError not only on 'non string'
but also bad string that is not convertable.
Co-authored-by: Sloane Hertel <shertel@redhat.com>
Co-authored-by: Sloane Hertel <shertel@redhat.com>
Change:
- Use parse_kv() for parsing in the csvfile lookup plugin. This allows
us to handle multi-word search keys and filenames. Previously, the
plugin split on space and so none of these things worked as expected.
- Add integration tests for csvfile, testing a plethora of weird cases.
Test Plan:
- New integration tests, CI
Tickets:
- Fixes#70545
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Move hostnamectl check out of GenericStrategy because it was incorrect
for everything except the SystemdStrategy which is where it belongs.
- Add some initial tests for the hostname module, though we are limited
by the fact that we can't do much testing with it in containers.
Test Plan:
- new hostname integration tests
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Fix ansible-test error in community.aws
* Add changelog entry for fix
* Change check from None to string_types
* Update changelogs/fragments/70507-validate-null-author.yaml
clarify wording "or a list of strings"
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/schema.py
clarify wording - single string or not specified valid
Co-authored-by: Felix Fontein <felix@fontein.de>
* Do not fail but return None when given outside list
Co-authored-by: Felix Fontein <felix@fontein.de>
Change:
- The command warnings feature which suggests that users use modules
instead of certain commands is now deprecated. Its `warn` paramater
and `COMMAND_WARNINGS` configuration options are also deprecated.
Their use will become an error in version 2.13.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Sam Doran <sdoran@redhat.com>
Change:
- module_utils.basic.is_special_selinux_path() used a string ==
bytestring comparison which returned False and made Ansible think that
certain filesystems aren't, in fact, special-cased, when they should
be. Ensure both sides of the == are bytestrings.
Test Plan:
- Added `copy` integration tests for this case.
Tickets:
- Fixes#70244
Signed-off-by: Rick Elrod <rick@elrod.me>
* ansible-doc man formatter: do not crash when description isn't there.
* Change to report a better error message when description is not there.
* Add test.
Change:
- Update bundled six to 1.13 (last with py2.6 support)
- Make it pass lint
- Fix check to allow skipping over compat __init__.py files we authored
- Fix check to allow files that can't be updated for some reason
Test Plan:
- ansible-test sanity --docker
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Test against galaxy_ng
* Switch container image
* Remove redundant |default
* Re-enable
* Update image
* Update wording
* Don't use pulp as the container name
Change:
- In certain situations, such as when the input string contains null
bytes (\0), syslog.syslog will throw a TypeError. Handle that and
fail_json instead.
Test Plan:
- New test
- ansible-test --docker centos[68] (for py2 and py3 respectively)
Tickets:
- Refs #70269
Signed-off-by: Rick Elrod <rick@elrod.me>
* ansible-test - do not validate blacklisted ps modules
* Update changelogs/fragments/validate-modules-ps-doc-blacklist.yaml
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
CIFS can be mounted using backward slash as well in /etc/fstab like
\\Windows\share /data/ cifs credentials=/root/.creds 0 0
Handle this condition while gather mount information in Linux.
Fixes: #48813
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Update documentation as per sanity tests
* Added example about custom SSH port in example section
Fixes: #29236
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Ensure not to remove existing packages while installing apt packages.
* Make all lines shorter than 160 characters
* Allow removing packages only when upgrading.
* Add integration tests
Change:
- Fix a UnicodeDecodeError in executor.module_common that could get
triggered with -vvvvv.
Test Plan:
- `ansible-test integration --docker centos7 module_utils -vvvvv`
This would show the error previously, and no loner does after this
patch.
Signed-off-by: Rick Elrod <rick@elrod.me>
In some usecases, we want to be able to clone a single branch
of a repository, without using --depth (which implies --single-branch).
* Use branch name when available
- update description of parameter
- consolidate branch or tag checking for easy reuse
* Add changelog
* Use static task imports rather than dynamic includes
* Add integration tests for single_branch
* Account for older versions of git
* Minor tweak to warnings
Co-authored-by: Laurent Coustet <laurent.coustet@clarisys.fr>
Co-authored-by: Sam Doran <sdoran@redhat.com>
Handle colon appearing in filename while parsing the mimetype and charset
using file command.
Fixes: #70256
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
The `packaging` and `pyparsing` packages are now installed by `ansible-test` during provisioning of RHEL instances to match the downstream vendored versions.
Change:
- Remove _get_item() alias as it has been deprecated
- Update tests
- Remove relevant sanity curtailment
- Add changelog
Test Plan:
CI, grep
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
Removes with_* loop squashing and tests for 2.11
Test Plan:
CI, and grepped for with_items in package manager integration targets.
There might be some test cases in collections which need to stop testing
this behavior.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Disable too-many-ancestors pylint check.
This check was only enabled for ansible-test code.
In some cases this check counts 1 more ancestor in python 3.7+ than under python 3.6, making the check inconsistent and unsuitable for use in ansible-test.
* Disable additional pylint checks:
- import-error
- no-name-in-module
These checks were only enabled for ansible-test code.
These checks provide inconsistent results since they are dependent on available imports, which vary between environments.
It may be practical to enable these checks in the future if changes are made to ensure a consistent test environment for pylint.
* Test galaxy cli against pulp
* linting fix
* Renames and small fixes
* Better handling for resetting pulp
* Clean up some things, add a comment
* I can't spell
* Bump fallaxy, use alternate pulp image
* Only reset pulp when we're are executing against pulp
* Update for updated pulp container
* Update some comments with correct URLs and typos
* Linting fix
* Pin pulp-fedora31 to a digest
* Address review comments for documentation
The current author line wants to match a github author id. But
some people, including the OpenStack project, do not use github,
and additionally do not claim individual ownership but instead
group ownership.
Since there are already a couple of hard-coded examples in the
regex, just add one more. Alternately we could come up with some
mechanism to indicate that the author is purposely not listing
a github id, but that seems a bit heavywight.
* try to load unqualified plugins from whitelist
* necessary for backcompat loading of unqualified collectionized callback plugins redirected from <= 2.9 core
* also added de-duping from actual loaded name
* add tests
* add warning test
* group test script entries by topic
* shorten warning text grep because wrapping is dumb
* fix adhoc callback loading behavior
* collections pass over whitelist wasn't respecting `_run_additional_callbacks`
* adds regression tests for same
* avoid `grep -L` in tests since it breaks the world
The upcoming pyparsing 3 release will require Python 3.5 or later, see:
https://github.com/pypa/packaging/issues/313
Unfortunately pip 8.x and earlier versions do not support python version requirements, which is why this constraint is needed.
* builtin downstream vendoring support
* allows downstream packagers to install packages to `ansible/_vendor` that will automatically be added to head of sys.path during `ansible` package load
* tests
* sort conflicting package names in warning text
* sanity fixes
* skip unnecessary comparison
If we fail to find a member when extracting a directory, try adding a trailing
slash to the member name. In certain cases, the member in the tarfile will
contain a trailing slash but the file name in FILES.json will never contain
the trailing slash.
If unable to find the member, handle the KeyError and print a nicer error.
Also check if a directory exists before creating it since it may have been
extracted from the archive.
Fixes#70009
* Add unit tests
* Use loop for trying to get members
* Validate ansible-base & collection's runtime.yml
Add new test `runtime-metadata`
* Schema validation of file
* Error if a a legacy meta/routing.yml exist in a collection
* removal_date OR removal_version
* Add tombstone validation.
* Allow both ISO 8601 date strings and datetime.date objects (from YAML dates).
* Address review comments.
* Add metadata to test collection.
* Add requirements file.
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Matt Clay <matt@mystile.com>
* Add mccabe complexity testing
* Make mccabe complexity an optional error
* Add mccabe to new sanity pylint requirements
* Add a changelog fragment.
Co-authored-by: Matt Clay <matt@mystile.com>
When using 'remote_src: yes' and 'mode: preserve', the code handling
the file modes has to be handled on the remote node because it's
the one that has access to the source files. This means that the
copy module itself must handle this, rather than the copy action
plugin (which is where all that logic exists). The copy module
handles this when we copy a single file over. But when it is a
directory as the src parameter value, the mode of the files
beneath it are not considered. Subdirectories are copied with
shutil.copytree() which will preserve permissions automatically.
Individual files are copied with shutil.copyfile() which does NOT
preserve permissions. We need to add some calls to shutil.copymode()
to correct that.
Note: This *always* retains individial file permissions. Specifying
a 'mode' other than 'preserve' when giving a source directory for
the 'src' param does not make sense so will be ignored in that case
only.
Fixes#69783
* Add changelog and test
* config: singular ANSIBLE_COLLECTIONS_PATH
Every other *_PATH setting in ansible is singular, and the traditional
$PATH variable is also singular despite containing a list of
directories. Let's be consistent both internally and with POSIX
tradition.
* update all ANSIBLE_COLLECTIONS_PATHS env references to be singular
* deprecate plural ANSIBLE_COLLECTIONS_PATHS setting
Replace the ansible-base changelog linting and generation tool with antsibull-changelog and make it available for linting collections. Previously changelog linting was limited to ansible-base.
* Tag return value docs if they are a dict (and not str/None).
* Try to parse return docs as YAML.
* Properly dump return values in ansible-doc.
* Adjust plugin formatter.
* Add changelog fragment.
* Don't add 'default' for return values.
* Fix plugin_formatter.
* Only try to parse return docs if they are still a string.
* Add tests.
* Warn if RETURN cannot be parsed.
* Adjust tests. Also test for warning.
* if -> elif (otherwise EXAMPLE will be parsed too).
* Always parse return documentation, and fail if it is invalid YAML.
* Polishing.
* Mostly re-enable ansible-doc tests.
Listing from the local collection seems to be somewhat broken. I assume this
is why the test was disabled.
* Lint and make tests work with Python 2.
* Keep FQCNs in plugins (not modules), i.e. restore previous state.
* Support removed_at_date in ansible-doc
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
Changes:
* ansible-doc does not support `removed_at_date` and assumes that
deprecated dict will either have `removed_in` or `version`. This
results in ansible-doc (and hence "sanity --test=ansible-doc")
failing for modules having only `removed_at_date`.
* This patch adds support for `removed_at_date` and also gives it
precedence over `removed_in` or `version`.
* Add tests and changelog
Signed-off-by: NilashishC <nilashishchakraborty8@gmail.com>
Ensure the vcenter provider initialize the `module_defaults` of all
the vmware modules, not just `vmware_guest`.
The VMware CI relies on this for the authentication of the different
VMware modules.
The commit adjust `incidental_vmware_prepare_tests/tasks/init_vcsim.yml`.
The test-suite uses a copy of `vmware_guest` that is not in the
`group/vmware` group. As a result, we need to manually pass the
authentification parameter.
Some code-smell sanity tests for ansible-base use subprocess to invoke ansible commands.
Intercept these commands to make sure the correct script and python version are used.
* ansible-galaxy - Fix role info when role is not installed
Only report the role not found if in offline mode, otherwise query the galaxy API
to get role information.
Fixes#69867
* Improve error message when role is not found in Ansible Galaxy
* Extra whitespace around imported playbook filename are stripped.
* Corrected call for display.deprecated
* Changed warning type & added test
* Added auto verification of raised warning
* More accurate warning message
* Move ansible-test completion code.
* Fix a few type hints.
* Change docker completion based on context.
Collections now use version 2.0 of the default-test-container.
This is an updated version of the earlier 1.x default-test-container with ansible-base and cloud specific requirements removed.
Testing of ansible-base now uses version 1.0 of a new ansible-base-test-container.
This container is similar to the earlier 1.x default-test-container, but with unnecessary duplication of requirements across Python versions removed.
Collections which have tests that depend on requirements no longer present in the default test container should specify them in their test requirements files:
* tests/integration/requirements.txt
* tests/unit/requirements.txt
* Bump test container versions
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Only allow groups which were hardcoded in module_defaults.yml
only load action groups from the collection if module_defaults contains a potential group for the action
* Fix tests using modules that override those whitelisted in lib/ansible/config/module_defaults.yml
Third party modules should not be using group/ - use the action name instead
* add externalized module_defaults tests
add the missing group and collections
ci_complete
Co-authored-by: Matt Davis <mrd@redhat.com>
* changelog
ci_complete
* Fix import in tests
ci_complete
* Update with requested changes
ci_complete
* don't traceback since we don't validate the contents of module_defaults
ci_complete
Co-authored-by: Matt Davis <mrd@redhat.com>
* Allow to specify collection_name separately for deprecation.
* Use new functionality in Ansible.
* Use new functionality in tests.
* Update tagging/untagging functions.
* Update pylint deprecated sanity test.
* Update validate-modules. Missing are basic checks for version_added (validate semantic version format for collections).
* Improve version validation. Re-add version_added validation.
* Make sure collection names are added to return docs before schema validation.
* Extra checks to avoid crashes on bad data.
* Make C# module utils code work, and update/extend tests.
* Add changelog fragment.
* Stop extracting collection name from potentially tagged versions/dates.
* Simplify C# code.
* Update Windows modules docs.
* Forgot semicolons.
* Split out sanity test requirements.
* Run each --venv test separately.
This provides verification that the requirements for each test are properly specified.
* Use a separate requirements file per sanity test.
* Skip setuptools/cryptography setup for sanity.
* Eliminate pyyaml missing warning.
* Eliminate more pip noise.
* Fix conflicting generate_pip_install commands.
* Add changelog fragment.
* Make AnsibleVaultEncryptedUnicode work more like a string. Fixes#24425
* Remove debugging
* Wrap some things
* Reduce diff
* data should always result in text
* add tests
* Don't just copy and paste, kids
* Add eq and ne back
* Go full UserString copy/paste
* Various version related fixes
* Remove trailing newline
* py2v3
* Add a test that can evaluate whether a variable is vault encrypted
* map was introduces in jinja2 2.7
* moar jinja
* type fix
Co-Authored-By: Sam Doran <sdoran@redhat.com>
* Remove duplicate __hash__
* Fix typo
* Add changelog fragment
* ci_complete
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Auto unroll generators produced by jinja filters
* Unroll for native in finalize
* Fix indentation
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Add changelog fragment
* ci_complete
* Always unroll regardless of jinja2
* ci_complete
Co-authored-by: Sam Doran <sdoran@redhat.com>
* various deprecation, display, warning, error fixes
* Update lib/ansible/utils/display.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update lib/ansible/utils/display.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* Update lib/ansible/utils/display.py
Co-authored-by: Felix Fontein <felix@fontein.de>
* cleanup, test fixes
* add collection name to deprecated() calls
* clean up redirect entries from uncommitted tests
* fix dep warning/error header text to match previous
Co-authored-by: Felix Fontein <felix@fontein.de>
Change:
- The `add_host` action now shows an accurate change status.
Test Plan:
- Added a plethora of integration tests.
Tickets:
Fixes#69881
Signed-off-by: Rick Elrod <rick@elrod.me>
* starting metadata sunset
- purged metadata from any requirements
- fix indent in generic handler for yaml content (whey metadata display was off)
- make more resilient against bad formed docs
- removed all metadata from docs template
- remove metadata from schemas
- removed mdata tests and from unrelated tests
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Rick Elrod <rick@elrod.me>
* Work around strange behavior of StrictVersion and SemanticVersion constructors that they accept an falsy value.
* Do not accept empty strings as versions.
Change:
- This was removed in 2014 in 122a7021bc.
- The option still exists and is enabled by default and can lead to user
confusion when people aren't expecting packages (or updated
dependencies for it) to get installed and they do.
- Add the option documentation back with a few notes to make it clear what
is happening.
Test Plan:
N/A, no code change, just documentation
Tickets:
- Refs #69497
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix sanity errors
Signed-off-by: Rick Elrod <rick@elrod.me>
In the case of a free style strategy, it is possible to end up with
multiple hosts trying to include from the same role, however the tasks
being included may be different with the use of tasks_from. Previously
if you had two hosts that were included the same role when the
process_include_results function tries to determine if a included needs
to be run on a specific host, it would end up merging two different
tasks into which ever one was processed first.
This change updates the equality check to also check if the task uuid
associated with the IncludedFile is the same. The previous check only
checked if the task's parent uuid was the same. This breaks down when
both includes have the same parent.
- hosts: all
strategy: free
gather_facts: false
tasks:
- include_role:
name: random_sleep
- block:
- name: set a fact (1)
include_role:
name: set_a_fact
tasks_from: fact1.yml
- name: set a fact (2)
include_role:
name: set_a_fact
tasks_from: fact2.yml
- name: include didn't run
fail:
msg: >
set_a_fact didn't run
fact1: {{ fact1 | default('not defined')}}
fact2: {{ fact2 | default('not defined') }}"
when: (fact1 is not defined or fact2 is not defined)
Closes#69521
To avoid issues with Flatcar Container Linux being unable to be found,
detect Flatcar distro name especially for hostname, just like CoreOS
Container Linux was supported.
See also https://github.com/ansible/ansible/issues/69516
* do not return the body even if it failed
* add some tests for this and rebase
* import test task
* ignore_errors when fails
Co-authored-by: Jack Zhang <jack.zhang@aspiraconnect.com>
* Enable installing collections from git repositories
* Add tests for installing individual and multiple collections from git repositories
* Test to make sure recursive dependencies with different syntax are deduplicated
* Add documentation
* add a changelog
* Skip Python 2.6
* Only fail if no collections are located in a git repository
Add support for a 'type' key for collections in requirement.yml files.
Update the changelog and document the supported keys and allowed values for the type.
Add a note that the collection(s) in the repo must contain a galaxy.yml
* Add a warning about embedding credentials in SCM URLs
* Update with review suggestions
* suppress sanity compile failure for Python 2.6
Change:
Rather than only using config, have base connection plugins fall back to
play_context.
Test Plan:
- Tested ansible-connection logic against an IOS device
- Tested -k against a VM
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Track collection for version_added.
Validate *all* version numbers in validate-modules.
For tagged version numbers (i.e. version_added), consider source collection to chose validation.
* Make tagging/untagging functions more flexible.
* Tag all versions in doc fragments.
* Tag all deprecation versions issued by code.
* Make Display.deprecated() understand tagged versions.
* Extend validation to enforce tagged version numbers.
* Tag versions in tests.
* Lint and fix test.
* Mention collection name in collection loader's deprecation/removal messages.
* Fix error IDs.
* Handle tagged dates in Display.deprecated().
* Also require that removed_at_date and deprecated_aliases.date are tagged.
* Also automatically tag/untag removed_at_date; fix sanity module removal version check.
* Improve error message when invalid version number is used (like '2.14' in collections).
* Allow to deprecate module by date in documentation.
* Make sure deprecation date/version match between module docs and meta/runtime.yml.
* Unrelated fix: don't compare deprecated module version to Ansible's version in collection.
* Allow documentation's removal version to be something else than fixed list of Ansible versions for collections.
* Linting.
* Allow to deprecate plugin options by date.
* Add changelog fragment for deprecation by date (also covers #68177).
Gets rid of the unknown field names in the return data.
Allows the plugin return docs to format under the new docs pipeline.
* Expect that package_facts will pass return-syntax-error now.
PR #66461 introduced a regression that resulted in an in correct block in the file
if the block to be inserted did not end with a line separator. Fix this bug and add
tests to cover this scenario.
Fixes#64966
* Fix "TypeError: splitlines() takes no keyword arguments" on Python2.7
* Add changelog fragment
* Don't use `grep -P` for BSD/macOS compatibility
* Fix sanity checks complaining about test fixtures with mixed line endings
* Update changelogs/fragments/66461-blockinfile_preserve_line_endings.yaml
Change:
Allows the user to configure sshpass (1.06+) to look for a different
substring than the default "assword" that it comes with.
Test Plan:
Set a custom ssh password prompt on a VM with PAM and tried connecting to
it. Without `ansible_sshpass_prompt` set in inventory: experienced hang.
With `ansible_sshpass_prompt` in inventory: connected successfully.
Tried setting `ansible_sshpass_prompt` with an older `sshpass` in PATH
and got a loud error, as expected.
Tickets:
Fixes#34722, fixes#54743, refs #11565.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Allow to deprecate options and aliases by date instead of only by version.
* Update display.deprecate().
* Adjust behavior to conform to tested behavior, extend tests, and improve C# style.
* Parse date and fail on invalid date.
This is mainly to make sure that people start using invalid dates, and we eventually have a mess to clean up.
* C# code: improve validation and update/extend tests.
* Make sure that deprecate() is not called with both date and version.
* Forgot to remove no longer necessary formatting.
* Adjust order of warnings in C# code.
* Adjust unrelated test.
* Fix grammar (and make that test pass).
* Don't parse date, and adjust message to be same as in #67684.
* Sanity tests: disable date in past test.
* Validate-modules: validate ISO 8601 date format.
* Validate-modules: switch schema declaration for deprecated_aliases to improve error messages for invalid dates.
* Use DateTime instead of string for date deprecation.
* Validate that date in deprecated_aliases is actually a DateTime.
* Fix tests.
* Fix rebasing error.
* Adjust error codes for pylint, and add removed_at_date and deprecated_aliases.date checks to validate-modules.
* Make deprecation date in the past error codes optional.
* Make sure not both version and date are specified for AnsibleModule.deprecate() calls.
* Stop using Python 3.7+ API.
* Make sure errors are actually reported. Re-add 'ansible-' prefix.
* Avoid crashing when 'name' isn't there.
* Linting.
* Update lib/ansible/module_utils/csharp/Ansible.Basic.cs
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Adjust test to latest change.
* Prefer date over version if both end up in Display.deprecated().
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* find_module can't pop ImportError- we need to just translate to `None` since this is a normal condition with files on sys.path (eg `/usr/lib/python36.zip`)
* added test
* Use a custom build container on Shippable.
This supports key generation before git_sync, to avoid issues with pre-migration PRs.
If the node pool is switched to another version, a matching build container should be built and used.
* Don't kill our own custom container.
* [yum] Make package removal confirmation strict
Change:
After removing packages, the yum module does a final check to ensure the
packages are really installed. The check would include packages that
were RPM `Provides:` values of another package.
This means that, for example, if a third-party kernel RPM spec had
`Provides: kernel` in it, removing the stock kernel would be successful
but the check to see if it was really removed would fail and cause
Ansible to report a failure.
Test Plan:
Tested on local CentOS 7 VM with kernel from elrepo which is known to
`Provides: kernel`.
Tickets:
Fixes#69237
Refs #35672
Refs #40723
Signed-off-by: Rick Elrod <rick@elrod.me>
* `meta/` directory in collections
* runtime metadata for redirection/deprecation/removal of plugin loads
* a compatibility layer to keep existing content working on ansible-base + collections
* a Python import redirection layer to keep collections-hosted (and otherwise moved) content importable by things that don't know better
* supported Ansible version validation on collection loads
* Fix `ansible -K` become_pass regression
Change:
- This fixes a breaking change introduced in
2165f9ac40
Test Plan:
- Local VM for now, with plans to add an integration test for -K going
forward.
Tickets:
Refs #69244
* fix delegated interpeter
* allow returning fact if it is 'the right host'
* added note for future fix/efficiency
as it stands we rerun discovery for the delegated host
unless its saving facts to itself
* fixed test lacking delegate_to mock
When mixed with the free strategy (or any custom strategy that does not behave in
a lock-step manner), the linear methodology of _wait_on_handler_results may cause
race conditions with regular task result processing if the strategy uses
_process_pending_results directly. This patch addresses that by splitting the queues
used for results and adding a flag to _process_pending_results to determine which
queue to check.
Fixes#69457
* Add multipart/form-data functionality
* Fix some linting issues
* Fix error message
* Allow filename to be provided with content
* Add integration test
* Update examples
* General improvements to multipart handling
* Use prepare_multipart for galaxy collection publish
* Properly account for py2 vs py3, ensuring no max header length
* Address test assumptions
* Add unit tests
* Add changelog
* Ensure to use CRLF instead of NL
* Ignore line-endings in fixture
* Consolidate code, add comment
* Bump fallaxy container version
* ci_complete
* ansible-galaxy - remove warning during collection install
If existing collections do not contain a MANIFEST.json, which is common
for collections under development that were not installed from Ansible
Galaxy, fall back to inspecting galaxy.yml rather than displaying a
warning.
A warning will still be displayed in neither a MANIFEST.json nor
galaxy.yml are present.
* Update unit tests
* Unify ansible-galaxy install -r
* Minor nit fixes for docs
* Re-align warnings
* Fix up integration test
* Fix up test where no roles/collections were in file
This fact reflects the number of usable vcpus (which might be different
from ansible_processor_vcpus, e.g., in containers with limits). See
also #51504.
* Add fixture data and update unit tests
Co-authored-by: Sam Doran <sdoran@redhat.com>
The updated container includes fewer requirements now that the collection migration has completed.
Collections which encounter test issues with this new container should update their test requirements files to include the necessary requirements.
* fix delegation vars usage and reporting
- just pass delegated host vars + task vars to plugins
and avoid poluting with original host vars
- updated tests
* Add a better error for "ansible-galaxy verify" if the MANIFEST.json has been deleted from the installed collection or if the collection hasn't been installed via normal means
* Fix unit tests for the remote collection
If there's something wrong with the local collection's version it will fail before the remote collection is sought
* Add a test for the new error msg
* Prevent the duplicate warning
Mock the new isfile call where needed
* Update lib/ansible/galaxy/collection.py
Co-Authored-By: Martin Krizek <martin.krizek@gmail.com>
Co-authored-by: Martin Krizek <martin.krizek@gmail.com>
* Fix filedescriptor out of range in select() when running commands
* Simplify the run_command() code
Now that we're using selectors in run_command(), we can simplify some of
the code.
* Use fileobj.read() instead of os.read()
* No longer use get_buffer_size() as we can just slurp all of the data
instead.
Also use a simpler conditional check of whether the selector map is
empty
Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Since Ansible 2.9.8, if the fileglob plugin is passed a path containing
a subdirectory of a non-existent directory, it will fail. For example:
lookup('fileglob', '/'): ok
lookup('fileglob', '/foo'): (non-existent): ok
lookup('fileglob', '/foo/bar'): (non-existent): FAIL
The exact error depends on Python 2 or 3, but here is the error on
Python 2:
AttributeError: 'NoneType' object has no attribute 'endswith'
And on Python 3:
TypeError: expected str, bytes or os.PathLike object, not NoneType
This change fixes the issue by skipping paths that are falsey before
passing them to os.path.join().
Fixes: #69450
Change:
Adds Fedora 32 to shippable and alters tests slightly for new Fedora.
Test Plan:
CI
Tickets:
Fixes#69230
Co-authored-by: Matt Clay <matt@mystile.com>
* validate-modules: deprecated modules in collections
In Collections a module is marked as deprecated via meta/routing.yml
Use this file, rather than the leading `_` as part of the deprecated
test.
* Correct variable
* review comments
* indentation
* Read routing.yml only once
* pep8
* Apply suggestions from code review
Co-authored-by: Matt Clay <matt@mystile.com>
* review: remove duplicated conditional
Co-authored-by: Matt Clay <matt@mystile.com>
Left hand side slicing is confusing and slower but maybe more memory
efficient in some circumstances. There is one case where it adds to
code safety: when it's used to substitute a different list in place of a
slice of the original list and the original list could have been bound
to a different variable in some other code. (The most likely case of
this is when it's a global variable and some other code might import
that variable name).
Because of the confusion factor we think it should only be used for the
safety case or where it's been benchmarked and shown to have some sort
of documentatble improvement. At the moment, only one piece of code
falls into those categories so this PR removes all the other instances
of left hand side slicing.
PopOS is a Debian based OS distribution, added support to detect
ansible_os_family as 'debian' instead of 'Pop!_OS'
Fixes: #69286
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Metadata defaults were not being set if only a few fields were missing.
* ansible-doc with no documentation and no status in metadata should
return empty, just like if there was no documentation and no metadata
at all.
* Fix support for Kali Linux detection
* Add test for Kali Linux detection
* Improve path matching with "in list"
Co-Authored-By: Abhijeet Kasurde <akasurde@redhat.com>
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* Do not pass file mode during recursive copy on symlink files.
The 'file' module cannot deal with mode=preserve. Do not pass that
mode to the module when 'preserve' is used.
* Fix changelog fragment filename
Change:
We were only testing dnf on RHEL previously.
Test on CentOS 8 as well.
Test Plan:
Ran locally in docker.
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
Extend the logic for custom error handling in the dnf module, so that on
newer DNF (such as DNF that ships with modern Fedora 31 container
images, and ships with RHEL 8.2) we report errors consistently with
older DNF.
Test Plan:
Ran dnf integration tests against an old Fedora 31 container image and a
brand new Fedora 32 container image; tess passed on both.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Make sure collection is a list if a str is given
* Call field validation early on collections
Because we are doing work on modifying the collections value before
it is actually validated, we can validate it ourselves early to make
sure the user supplies either a string or list. Dicts are not valid.
The new validation allows us to simplify the _ensure_default_collection()
function. And since the field is now static, we no longer need to specify
a default for it, which also allows us to simplify the function. Since
the default is now removed, we can also remove the sanity/ignore.txt entry
for collectionsearch.py.
New unit tests are added (and the existing one modified) that allow us to
make sure that we throw a parser error if a user specifies something other
than a string or list for the collections value everywhere it can be specified.
* Revert removing the collection default
The default is actually used, so restore it.
* Fix unit tests in test_helpers.py affected by early collection validation
This patch covers a few changes to get the yum test case working on ppc64le
CentOS. Specifically we needed to enable the EPEL repository on CentOS
as well as ensure some of the architecture-specific tasks use the right
set of binaries during their test.
Currently if virtualenv_command has arguments, then the
search for the binary in the path does not work so the
user has to specify the full path to it.
To allow arguments to be used without having to specify
the path to the binary, we split the module argument into
the command and anything after the first space.
This makes using this module argument more flexible and
user friendly.
Fixes: #52275
Change:
Rather than hardcoding .pyo and .pyc, filter on all BLACKLIST_EXTS in
the non-legacy logic of PluginLoader (_find_fq_plugin). The two harcoded
extensions are part of BLACKLIST_EXTS already and this simply adds the
rest of the blacklisted extensions to the check.
In addition, check .endswith() instead of an exact match of the suffix,
like everywhere else that uses BLACKLIST_EXTS. This allows for
blacklisting, for example, emacs's backup files which can appear after
any extension, leading to things like `foo.py~`.
Test Plan:
Ran `ansible-playbook` against a collection where a `foo.py~` module was
getting executed instead of `foo.py` which also appeared in the same
directory. `foo.py~` is no longer executed.
Tickets:
Fixes#22268
Refs #27235
Signed-off-by: Rick Elrod <rick@elrod.me>
* Allow a collection role to call a standalone role by default. Fixes#69101
* tweaked changelog text
* Guard against NoneType
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
* update ActionBase._low_level_execute_command to honor executable
* adding changelog fragment
* renaming changelog fragment to .yml
* noop change to bump shippable
* adding raw_executable integration test
* copying aliases from raw
* removing blank lines
* skipping aix and freebsd
* noop to bump shippable
* moving tests to raw/
* removing become_method: sudo ; it doesn't work on AIX
* removing trailing blank line
* forcing become_method: su to try to get AIX to work
Co-authored-by: Rob Wagner <rob.wagner@sas.com>
Change:
New `cryptography` statically links an openssl that is too new for macOS
10.11, so limit to an older cryptography for now.
Test Plan:
Ran the test with `--remote osx/10.11` and it passed.
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
Adds some missing coverage for the copy module when `check_mode: True`.
Test Plan:
Ran test with --coverage and looked at the resulting report.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Enable service integration tests for FreeBSD
Change:
Adds necessary rc file for freebsd, and gets tests passing for it.
Test Plan:
Ran test with `--remote freebsd/12.1` and `--remote freebsd/11.1`. Both
passed.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Update test/integration/targets/service/tasks/rc_setup.yml
Co-Authored-By: Matt Clay <matt@mystile.com>
* fix up comment
Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Matt Clay <matt@mystile.com>
With https://github.com/pallets/jinja/pull/1190 merged our short-circuit
is no longer valid (has it ever been?) as now data like ' True ' may go
through our ansible_native_concat function as opposed to going through
intermediate call to Jinja2's native_concat before. Now we need to always
send data through literal_eval to ensure native types are returned.
* Testing: Add CentOS Linux On Power platform
* Add arch designation to remotes.
This avoids overloading the provider with the arch.
Also add a changelog entry.
Co-authored-by: Matt Clay <matt@mystile.com>
* ansible-galaxy - fix listing specific role
If the role was not in the first search path, it was reported as not found
* Properly display role description
Default to description to top level description, falling back to the description from within galaxy_info
* Display proper message when a role does not exist
* Add integration tests
* Use context manager
* BSD and macOS ruining all the fun
* Force collections to be static
Templating of collection names does not work at all. Force them to
be static so that a warning is generated for the user.
* Add collectionsearch unit test and fix for reviews
New unit test validates the new _load_collections() code and moves
the new check to the end of the method.
* Change unit test to pytest
* Adjust unit test to use capsys instead of monkeypatch
* Fix pep8 error
* Add changelog fragment
Closes#68704
* subversion module - provide password securely with svn command line option --password-from-stdin when possible, and provide a warning otherwise.
* Update lib/ansible/modules/source_control/subversion.py.
* Add a test.
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Don't trigger full CI run for changes to changelogs/ and docs/ in collections.
* Add changelog fragment.
* Update changelogs/fragments/68550-ansible-test-docs-changelogs.yml
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* Update docker.txt to use the OpenSUSE 15.1 container image
Signed-off-by: Rick Elrod <rick@elrod.me>
* handle installing mysql on suse
Signed-off-by: Rick Elrod <rick@elrod.me>
* add changelog fragment
Signed-off-by: Rick Elrod <rick@elrod.me>
* Update changelogs/fragments/ansible-test-opensuse-15.1.yml
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
* Update tests to use RHEL 7.8.
Keeping support for RHEL 7.6 since collections are still using it.
* Fix tests for RHEL 7.7+ due to extras repo name change.
* fixed fetch traversal from slurp
* ignore slurp result for dest
* fixed naming when source is relative
* fixed bug in local connection plugin
* added tests with fake slurp
* moved existing role tests into runme.sh
* normalized on action excepts
* moved dest transform down to when needed
* added is_subpath check
* fixed bug in local connection
fixes#67793
CVE-2019-3828
* Allow tasks to notify a fqcn handler name
* Add tests. Fixes#68181
* Add changelog fragment
* Add test to ensure handlers are deduped properly with fqcn, role, and just handler names
* Add some docs about new special vars
fial_json() requires a message be given to it to inform the end user of
why the module failed. Prior to this commit, the message had to be a
keyword argument:
module.fail_json(msg='Failed due to error')
Since this is a required parameter, this commit allows the message to be
given as a positional argument instead:
module.fail_json('Failed due to an error')
The last task in a play should now properly report code coverage.
This change should also eliminate empty coverage files, as well as incomplete coverage files resulting from early worker termination.
* Fix test_check_mutually_exclusive exception-checking
Asserting inside of the `with` context of `pytest.raises`
doesn't actually have any effect. So we move the assert
out, using the exception that gets placed into the scope
after we leave the context, and ensure that it actually gets
checked.
This is also what the pytest documentation says to do:
https://docs.pytest.org/en/latest/assert.html#assertions-about-expected-exceptions
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add some tests for check_required_together
Signed-off-by: Rick Elrod <rick@elrod.me>
* use to_native instead of str, for consistency
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add newlines for pep8
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add tests for check_required_arguments
Signed-off-by: Rick Elrod <rick@elrod.me>
* Sort missing keys in error message, since hashes are unsorted and this can be random
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add changelog entry
Signed-off-by: Rick Elrod <rick@elrod.me>
* Allow custom inventory plugins and cache plugins
If _load_name is not set correctly the cache plugin can't load the documentation (which is also the arg spec)
Fix the existing inventory plugin in the collections tests
Add integration tests for using a cache plugin in a collection
* Set the attribute on the instance instead of the class
Deprecate importing custom CacheModules directly - they should use the cache_loader
* Add some more test coverage for unarchive
This moves over (and slightly extends) coverage which was found in
incidental_flatpak_remote.
Signed-off-by: Rick Elrod <rick@elrod.me>
* add a group for testing too, user creation does not mean group creation on all platforms
Signed-off-by: Rick Elrod <rick@elrod.me>
* update the test group assert
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix style
Signed-off-by: Rick Elrod <rick@elrod.me>
* block/always
Signed-off-by: Rick Elrod <rick@elrod.me>
* fix vault tmpe file handling
* use local temp dir instead of system temp
* ensure each worker clears dataloader temp files
* added test for dangling temp files
* added notes to data loader
CVE-2020-10685
* Remove some unreachable code in the file module
Remove some cases in file.py which are covered by conditionals a few
lines earlier. Remove the duplicate code which will never be hit.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Restore incidental file coverage from timezone module
Signed-off-by: Rick Elrod <rick@elrod.me>
* Combine two conditionals, add a changelog entry
Signed-off-by: Rick Elrod <rick@elrod.me>
* Make new test syntax consistent, add two stat tests
Signed-off-by: Rick Elrod <rick@elrod.me>
* Support pre-releases via new SemanticVersion. Fixes#64905
* Don't treat buildmeta as prerelease
* Don't inherit from str and int
* Add helper method to try and construct a SemanticVersion from a LooseVersion
* Don't count major 0 as pre-release, it's different
* Guard against invalid or no version in LooseVersion
* return a bool
* Add integration tests for pre-release
* Fix up lingering issues with comparisons
* typo fix
* Always allow pre-releases in verify
* Move pre-release filtering into CollectionRequirement, add messaging when a collection only contains pre-releases
* Update changelog
* If explicit requirement allow pre releases
* Enable pre-releases for tar installs, and collections already installed when they are pre-releases
* Drop --pre-release alias, make arg name more clear
* Simplify code into a single line
* Remove build metadata precedence, add some comments, and is_stable helper
* Improve from_loose_version
* Increase test coverage
* linting fix
* Update changelog
* remove azure extras and extras_require support
* Since Azure will be collectionized, the requirements will float more frequently than Ansible releases; the Azure collection needs to host the requirements now.
* Removed the dynamic extras support as well, since Azure was the only thing using it. If we need it again, it's easy to pull back from history.
* Mark azure-requirements as orhpaned.
This keeps the docs around so that existing links from old test runs remain valid.
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix ansible-test coverage analysis option usage.
The `--input-dir` option for `coverage analyze targets generate` was being ignored.
No changelog entry since this feature has not yet been released.
* Move coverage config to fix type annotations.
Declaring the types before referencing them makes sure they're recognized by tools such as PyCharm.
Now empty `*.py` files are ignored during module_utils import analysis for change detection.
This eliminates "No imports found" warnings for files which should have no imports.
* Rename `tests` test to match plugin type.
* Rename `test_infra` test to avoid confusion.
This test target is not a test for test plugins.
* Rename `vars_prompt` test to avoid confusion.
* Update sanity ignores.
* Internally redirect win modules to collection name
* Added comment for how this should be fixed in a subsystem plugin
* add collection plugins for base tests
* Address compat issue for collection loading on py26
* Move import_module shim to utils for compat across the codebase
* Enable collection tests on py2.6
* Update changelog fragment
* Simplify code using sys.moduls
* Move compat to module_utils/compat/importlib
* Add back errantly deleted newline
* Remove hack comment
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-authored-by: Matt Clay <matt@mystile.com>
Requirements were incorrectly added to ansible-test in https://github.com/ansible/ansible/pull/61813
These requirements should have been placed into `test/units/requirements.txt` instead.
Now that the relevant content has been migrated out of the repository, the requirements are no longer necessary there either.
No changelog entry for this change since the original changes were not included in any release and also lacked a changelog entry.
* copy plugins.inventory.docker_swarm in to test/support
Signed-off-by: Rick Elrod <rick@elrod.me>
* Copy over functions from module_utils.docker.common and nuke the import
Signed-off-by: Rick Elrod <rick@elrod.me>
* Split test_play_context_make_become_cmd into files
For NWO migration. Split the become module assertions into distinct test
files and functions. For now, this is done naively - there is probably
room to abstract these tests out and remove some of the duplication
later on.
Signed-off-by: Rick Elrod <rick@elrod.me>
* use default_exe variable instead of hardcoding /bin/bash
Signed-off-by: Rick Elrod <rick@elrod.me>
* Move become plugin tests to their proper directory and rename them accordingly
Signed-off-by: Rick Elrod <rick@elrod.me>
* Fix up fixtures and imports.
* Remove stray file.
Co-authored-by: Matt Clay <matt@mystile.com>
* Create a dedicated windows-minimal test target.
The windows-minimal target is a copy of the win_ping test, taking the place of that test as the windows minimal test run on multiple python versions.
It includes a private copy of the win_ping module so it will work after migration.
This will keep tests passing during the migration prep process.
* Update sanity ignores.
* Split fallaxy tests into their own group.
This keeps expected pass/fail tests separate during migration prep.
* Move network tests down in matrix.
This will help keep pass/fail tests grouped together during migration prep.
* Group all incidental tests together.
This will keep pass/fail tests separate leading up to migration.
* win_timezone - Allow for _dstoff timezones
* Update win_timezone-Allow-dstoff.yml
* Added doc entry for new format
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Added nxos_lldp_interfaces module
* Linting
* Added RTT, resolved shippable errors
* Added new states
* New states edit
* Updated states
* Updated tests
* Show all interfaces in facts
* Test changes
* Added unit tests
* Linting
* Handled portchannel failing condition
* Renamed auto evpn test
* Made sure that the current module is idempotent with additional tests
* Added tests for route-target import function
* Added tests for route-target export function
* Added tests for route-target both function
* PEP8 syntax fix
* Added route-target import & export function
* Added required 'version_added: "2.8"' in the documentation
* Updated documentation of new route-target options
* Added a test to make sure that in case of `state=absent` on the vrf
level the route-target options are ignored.
* Specified that the route-target options are ignored in case of
`state=absent'.
* Updated the doc to the correct format (using 'C()')
* Changed the VRF Route Target Syntax
Instead of using three different params (route_target_import,
route_target_export, route_target_both) the module uses now only one
param (route_targets) and the direction is specified for each of the
route targets.
Example:
route_targets:
[{rt: '...', direction: '{import|export|both}', state: '...'}]
* Updated Description and Examples to reflect new params
* Updated "version_added"
* pep8 fixes
* If rt['direction'] is not definied, we assume default 'both' and run
the same routine
* Added test with default direction for route-targets
* Documentation fixes
* Split out cache plugin unit tests.
* Rename unit tests to match code under test.
* Relocate unit test code to match code under test.
* Another rename.
* Update sanity ignores.
* ansible-galaxy - optimise some paths and use fake galaxy int tests
* Added init, built, and publish tests
* Test against both mocked Galaxy and AH server
* Finish off writing the install tests
* Fix up broken tests
* Rename test target and add migrated tests
* Use cloud provider for Galaxy implementation
* Added blank static config
* Use correct alias group
* Set release version and fix copy typo
* Remove reset step as it is no longer needed
* Use sane env var names for test container name
pytest-mock 2.0.0, when run locally, gets grumpy when os.path.exists is
messed with and then another method is patched afterwards. Likely
something in the pytest-mock chain uses os.path.exists internally, and
since pytest-mock prohibits context-specific patching, there's not a
good solution. For now, just patch os.path.exists last.
Signed-off-by: Rick Elrod <rick@elrod.me>
* reworked sqs_queue
* Switch default purge_tags behaviour to false.
This matches the behaviour of ec2_tag and ecs_tag.
* Minor lint / review fixups
* Add missing AWS IAM policy for SQS tests
* Move integration tests to using module_defaults: group/aws:...
* add changelog
* Break out the 'compatability' map from our spec definition (gets flagged by the schema validation)
* Tweaks based on review
* add basic examples
* Lint fixups
* Switch out NonExistentQueue logic so it's easier to follow
* Reorder name argument options for consistency
Co-authored-by: Dennis Podkovyrin <dennis.podkovyrin@gmail.com>
Use PostgreSQL 9.5 on FreeBSD 12.0, and PostgreSQL 11 on FreeBSD 12.1 and 11.3 due to the Python packages having a dependency on that version of PostgreSQL and automatically uninstalling PostgreSQL 9.5.
* Use separate PostgreSQL versions for 12.0 and 12.1
* Allow passing through of (almost) all params available on boto methods in aws_api_gateway
* Linting and docs fixes
* Refactored method signature of create_deployment() to use keyword args instead of named args
* Updated version_added flags to 2.10
* Cleanup and improve aws_api__gateway integration test play. Also included new params into test.
* Fixed RETURN docs and some ttests
* Completed RETURN docs and made integration tests match
* Fixed variable names in test and YAML syntax in docs
* Comment out critical sections of integration test
* Fixed update test after figuring out what the error message means. Also updated error message to be more descriptive.
* Fixed test assertion
* Update docs and make tests reflect that endpoint type wont be changed on updates
* Syntax fix
* Add changelog fragment
* Improve aws_api_gateway docs, fix typos.
* Quote doc lines with colon
* AnsibleAWSModule related cleanup - s3_bucket
* Add extra information to s3_bucket timeout failures, it's possible the comparisons are doing something weird...
* Move Bucket Encryption boto support logic into the pre-flight checks
* Use the built in required_by logic
* Rework s3_bucket integration tests
* Add a retry around put_bucket_encryption
s3_client.put_bucket_encryption is occasionally dropped on the floor
by Amazon add some logic to retry s3_client.put_bucket_encryption call
* Catch OperationAborted and retry, it is caused by a conflicting change
still being in progress. (For example an Encryption setting applying)
* Make sure we don't explode if the botocore version's too old
* Review tweaks
* Cleanup tests
* Auto-Retry on ResourceNotFound and RequestInProgress exceptions
* Use AnsibleModule options for required_if logic
* changelog
* Remove (now) duplicate RequestInProgressException catching
* Allow a single retry when attempting to fetch the information about a cert directly after deleting it.
There is a small chance that it goes away while we pull the details.
* add key rotation option
* add changelog fragment
* provide version added as string
* change changelog to minor_changes
* Update changelogs/fragments/67651-aws-kms-key-rotation.yml
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
* Update lib/ansible/modules/cloud/amazon/aws_kms.py
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
* prevent key upgrade if key rotation was enabled manually. In that case, the key rotation would be disabled, if not mentioned in the playbook
* Update lib/ansible/modules/cloud/amazon/aws_kms.py
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
* Update lib/ansible/modules/cloud/amazon/aws_kms.py
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
* Update lib/ansible/modules/cloud/amazon/aws_kms.py
Co-Authored-By: Mark Chappell <mchappel@redhat.com>
Co-authored-by: Mark Chappell <mchappel@redhat.com>
* Initial copy of incidental network tests.
* Update incidental test aliases.
* Add incidental tests to CI.
* Rewrite module references in tests.
This should not be necessary once module redirection is supported.
* Rewrite target references for renamed targets.
* Add support collections for incidental tests.
* Add ignores for test support code.
* Remove echo used for debugging.
Make AWS test plugin to refer to the aws-permissions-for-integration-tests section of the AWS devel docs now that we have them, rather than suggesting to flag down specific users.
This ensures the test passes regardless of the test environment.
Previously, it would pass in some environments but fail locally
in other environments, due to the width being calculated as
different or support for color not being consistently detected.
Signed-off-by: Rick Elrod <rick@elrod.me>
* Support private test plugins in ansible-test.
This feature is exclusively used for testing Ansible itself and is not available to collections.
Content in the newly supported directories will be added in follow-up PRs.
* Add new directory to path classification.
* Support new paths in shebang test.
* Add new directory to manifest.
* fixed#47050
* added changelog fragment
* added quick and basic test
* Revert "added quick and basic test"
This reverts commit 75f4141656.
* added better tests
* now also creating files to copy on the remote
* removed tests for recursive copying which is not supported by remote_src
* sns_topic: (integration tests) Move the tests over to using module defaults
* sns_topic: (integration tests) Add test for behaviour of changed when using delivery_policy
* sns_topic: ensure "changed" behaves properly when managing delivery policies
- a delivery_policy isn't an IAM policy, so compare_policies didn't cope with it
- AWS automatically adds an additional option when you set an HTTP delivery
policy
* Parse the delivery policies so we can test the changes properly
* Update AWS policy to enable management of TargetGroups
* elb_target: (integration tests) migrate to using module_defaults
* elb_target: (integration tests) lookup the AMI by name rather than hard coding AMI IDs
* elb_target_info: (integration tests) finish rename of integration test role
* elb_target: (integration tests) rename various resources to consistently use {{ resource_prefix }}
* elb_target_info: (integration tests) Migrate to using module_defaults
* elb_target_info: (integration tests) Lookup AMI by name rather than hard coding AMI IDs
* Apply suggestions from code review
Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>
* elb_target: (integration tests) Remove the 'unsupported' alias
* Try bumping up the timeout
* Rules don't permit 'shippable' (resource_prefix uses this when run in shippable)
* Try bumping up more timeouts :/
* Avoid double evaluation of target_health assertion
* Simplify target_type usage a little (rather than constantly performing a lookup)
* mark elb_target tests 'unstable' for now, they're slow
Co-authored-by: Jill R <4121322+jillr@users.noreply.github.com>
* Add warning when running devel. Addresses #67362
* Add config entry to disable devel warning, so it doesn't impact CI
* Update warning about using devel
* cloudfront_distribution: (integration tests) Migrate to using module_defaults
* cloudfront_distribution: (integration tests) Use the ID rather than the alias
Using aliases requires providing a valid SSL certificate, as such we're not longer able to test using an arbitrary hostname
* cloudfront_distribution: (integration tests) Make sure we delete the test s3 bucket when tests fail
* cloudfront_distribution: field_level_encryption_id is now a mandatory field always add it
Setting the field to an empty string has the same effect as the original behaviour.
* Copy & Paste fixup
Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>
Co-authored-by: Jill R <4121322+jillr@users.noreply.github.com>
* required_if checks should have three or four parts.
* Validate mutually_exclusive, required_together, required_one_of, required_if and required_by.
* Simplify code.
* Improve messages.
* Add changelog.
* Sanity check.
* Update docs.
* Update ignore.txt.
* Don't continue with tests when terms are not strings.
* Remove ignore.txt entry.
* Make sure validate-modules doesn't choke on things already flagged by schema test.
* Check required_if requirements list for strings.
* Run Ed25519 and Ed448 tests for openssl_csr and openssl_certificate only if key generation succeeded.
* Make openssl_privatekey tests more robust: allow special key generation tests to fail with 'algorithm not supported' on FreeBSD.
These tests rely on the AWS S3 modules, but will not be migrated along with those modules into an AWS collection.
Since these tests will not reside in a collection alongside the modules under test they are being moved to the legacy tests directory.
The legacy tests directory will soon be migrated to a separate repository.
* Add x509_crl module.
* Add integration tests.
* Fix some errors.
* Fix inversion.
* Compare name instead of tpye.
* Fix fail_json() calls.
* Work around rename of serial_number attribute for cryptography 1.4.
* Don't die for non-cert loading errors.
* One more.
* Fix function call.
* Fixed/improved descriptions.
* Don't read issuer from certificate file.
* Allow to ignore timestamps.
* Default value for revocation_date.
* Update tests.
* Mention ignore_timestamps in update docs.
* Support privatekey_content, and require some options only if state is present.
* Allow to pass certificate in directly.
* Add tests.
* Fix required_if.
* Forgot to encode content.
* Forgot to adjust type.
* Allow to return CRL's content directly.
* return_crl_content -> return_content (as in #65400).
* Fix elements.
* Fix messages.
* Use required_one_of and mutually_exclusive instead of doing the checks by hand.
* Fix format.
* Skip tests on AIX.
* Fix typo.
* 'message' parameter is replaced by 'commit_message' in grafana_dashboard
* 'message' parameter is replaced by 'notification_message' in datadog_monitor
This change is required since 'message' as parameter name is used internally by
Ansible core engine.
Fixes: #39295#45362#47132#59617
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Fix DHCP support in win_dns_client + more
* Fix bugs and test failures, add changelog fragment
* Add idempotency tests for DHCP
* Address review feedback; dedup address-family code
* Remove legacy function
* Remove old reference
* Create ipwcli_dns.py
* add newline at the end
* Update after review and support AAAA
* Update lib/ansible/modules/net_tools/ipwcli_dns.py
Co-Authored-By: Felix Fontein <felix@fontein.de>
* add integration tests and change param user to username
Co-authored-by: Felix Fontein <felix@fontein.de>
* ec2_snapshot and ec2_snapshot_info test suite
* Add a fact to make tests easier to follow.
Test all return values, and add missing RETURN docs to module.
* cloudformation - use mutually_exclusive for on_create_failure and disable_rollback
* cloudformation - remove unit test test_disable_rollback_and_on_failure_defined
