ansible-galaxy currently behaves bad then a role to be installed
contains ~ or $ at any place in the path of a file in that role.
It extracts the parent directory of the offending path level as an
empty file. This explodes if that directory contains anything else.
Change this behaviour. `~` is now allowed allowed when it is
not a full level (Yes: `some~thing/`, no: `~/`). The code should
get refactoring in an other PR.
The `UserFactCollector` queries the user login name via
`getpass.getuser()` and looks up the corresponding entry
in the password database.
The login name may differ from the actual user name,
eg. if the `LOGNAME` env variable is set. The lookup
fails in this case. Added a fallback in this case that
tries to get the entry via the user ID.
* Add direct+transitive pre-release regression test
* Match user-requested transitive pre-releases
This change makes sure that in scenario when a user requests
collectionA-pre and collectionB that depends on collectionA,
collectionA-pre would actually match that collectionA requirement
while `--pre` is unset.
Co-authored-by: Jordan Borean <jborean93@gmail.com>
This warning was intended to help improve test environment configuration.
However, it has ended up producing mostly non-actionable warning noise instead.
In most situations a missing pip or python should result in test failures.
Reporting a missing pip also implies that it should be used by tests, which is not the case.
Tests should be invoking pip as a python module with the appropriate python interpreter instead.
Change detection and check mode fixes for apt_key
* allow apt-key to use proxies
* add note about deprecation of apt-key itself
* expanded error msgs
* show all keys
* fix short_format parsing
* added more return info and documented it
PR #72591
This change:
* Adds an artifacts manager that abstracts away extracting the
metadata from artifacts, downloading and caching them in a
temporary location.
* Adds `resolvelib` to direct ansible-core dependencies[0].
* Implements a `resolvelib`-based dependency resolver for
`collection` subcommands that replaces the legacy
in-house code.
This is a dependency resolution library that pip 20.3+ uses
by default. It's now integrated for use for the collection
dependency resolution in ansible-galaxy CLI.
* Refactors of the `ansible-galaxy collection` CLI.
In particular, it:
- reimplements most of the `download`, `install`, `list` and
`verify` subcommands from scratch;
- reuses helper bits previously moved out into external modules;
- replaces the old in-house resolver with a more clear
implementation based on the resolvelib library[0][1][2].
* Adds a multi Galaxy API proxy layer that abstracts accessing the
version and dependencies via API or local artifacts manager.
* Makes `GalaxyAPI` instances sortable.
* Adds string representation methods to `GalaxyAPI`.
* Adds dev representation to `GalaxyAPI`.
* Removes unnecessary integration and unit tests.
* Aligns the tests with the new expectations.
* Adds more tests, integration ones in particular.
[0]: https://pypi.org/p/resolvelib
[1]: https://github.com/sarugaku/resolvelib
[2]: https://pradyunsg.me/blog/2020/03/27/pip-resolver-testing
Co-Authored-By: Jordan Borean <jborean93@gmail.com>
Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Sam Doran <sdoran@redhat.com>
Co-Authored-By: Sloane Hertel <shertel@redhat.com>
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
Signed-Off-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
Use correct ssh executable and options in all cases on connection plugin
* Also nicer naming/comments
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
Ability to add local variables into AnsibleJ2Vars was added in
18a9eff11f to fix#6653. Local variables
are added using ``AnsibleJ2Vars.add_locals()`` method when creating a
new context - typically when including/importing a template with
context. For that use case local template variables created using
``set`` should override variables from higher contexts - either from the
play or any parent template, or both; Jinja behaves the same way.
Also removes AnsibleJ2Vars.extras instance variable which is not used.
Also adds missing test for #6653.
Fixes#72262Fixes#72615
ci_complete
The `command` module does not return stdout & sterr when calling
a non existing executable or an unknown exception arises. This fix
lets the module return empty byte strings in those cases.
Change:
- Add Ubuntu 20.04 to CI now that venv is default instead of virtualenv in ansible-test.
Test Plan:
- CI
Tickets:
- Fixes#69203
Signed-off-by: Rick Elrod <rick@elrod.me>
The init script for the test container writes additional lines to
the .bashrc of the user. This was done via a `cat` multiline
instruction, which is implemented internally by writing a
temporary file to TMPDIR (/tmp in this case) first. Docker fails
to provide /tmp just after creation, which results in a race
condition that rarely makes the init fail. Changed the `cat`
statement to multiple `echo`s.
Co-authored-by: Matt Clay <matt@mystile.com>
* Fix YAML error message when error is at the end of the file
If a YAML file fails to load due to a syntax error in a file, or there is an error in the last line of a
file, PyYAML reports the last line number of the file as the index where the error occurred.
When reading the file lines, we use that index to the get the relevant line. If the index value is out
of range, the relevant line is lost for error reporting.
Subtract one from the index value to avoid the IndexError in this specific scenario. It is possible
to still get an IndexError, which will be handled as it is currently.
* Update existing tests and add new tests
* Allow restricting config values to enumerated list
* dont document internal entries
* also ignore private defs for ansible-config
* remove invalid value from tests
* added porting entry
When the pause module is run in the background and seconds parameter is provided,
do not warn.
* Add tests
* Fix existing tests
The test wasn't failing when it should have.
Change:
- On CentOS Stream, make distribution_release be "Stream"
- On CentOS Core, it continues to be "Core"
- Implement custom distribution file parser for CentOS, so we can look
for "CentOS Linux" and "CentOS Stream"
- Two new fixtures introduced (CentOS Linux 8.1 and CentOS Stream 8)
- Removed two dicts from `Distribution` class that were seemingly not
used anywhere.
Test Plan:
- ci_complete
- New test fixtures
Tickets:
- Fixes#73027
Signed-off-by: Rick Elrod <rick@elrod.me>
The test currently only expects the literal `ansible` followed
by a semver in the first output line of `ansible --version`.
When running from within a git checkout, additional information
like the currently checkout branch are attached, which lets
the test fail. This commit allows arbitrary information to
follow the semver.
`test_prepare_multipart` fails in non Debian environments since
Debian installations map the file ending `.key` to the MIME type
`application/pgp-keys`, which is not IANA conformant. This commit
explicitly sets the corresponding file type to
`application/octet-stream` and adjusts the expected serialized
result.
* Remove unused code in ansible-test.
* Remove obsolete endpoint logic from ansible-test.
* Remove obsolete region selection in ansible-test.
* Remove obsolete port logic in ansible-test.
* Clean up ansible-test remote providers.
Do the right thing on Linux when password lock and a password hash are provided by writing
out the password hash prepended by the appropriate lock string rather than using -U and -L.
This is the correct way to set and lock the account in one command.
On BSD, run separate commands as appropriate since locking and setting the password cannot
be done in a single action.
FreeBSD requires running several commands to get the account in the desired state. As a result,
the rc, output, and error from all commands need to be combined and evaluated so an accurate
and complete summary can be given at the end of module execution.
* Improve integration tests to cover this scenario.
* Break up user integration tests into smaller files
* Properly lock account when creating a new account and password is supplied
* Simplify rc collection in FreeBSD class
Since the _handle_lock() method was added, the rc would be set to None, which could make
task change reporting incorrect. My first attempt to solve this used a set and was a bit too
complicated. Simplify it my comparing the rc from _handle_lock() and the current value of rc.
* Improve the Linux password hash and locking behavior
If password lock and hash are provided, set the hash and lock the account by using a password
hash since -L cannot be used with -p.
* Ensure -U and -L are not combined with -p since they are mutually exclusive to usermod.
* Clarify password_lock behavior.
Tests should use the Python built-in ``venv`` module on Python 3 instead of the standalone ``virtualenv`` module.
On Python 2 the ``virtualenv`` module continues to be the only option.
The version installed is either the OS packaged version or the last release to support Python 2, which is version 16.7.10.
Change:
- Internally, use dnf.subject.Subject#get_best_query for state: absent
- Add a bunch of tests for removing packages, given a bunch of different
pkg specs (nv, nvr, nvra, wildcard, etc.)
Test Plan:
- New tests
- Local experiments with DNF API via PDB.
Tickets:
- Fixes#72809
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- Changes to make F33 work with current tests
- Add F33 to docker.txt
- Add F33 to AZP config
Test Plan:
- ci_complete
Signed-off-by: Rick Elrod <rick@elrod.me>
Change:
- The FreeBSD release can contain -RC or -PRERELEASE in addition to
-RELEASE, -STABLE, or -CURRENT.
Test Plan:
- Added new fixed from an RC version of TrueNAS which uses a -PRERELEASE
version of FreeBSD.
Tickets:
- Fixes#72331
Signed-off-by: Rick Elrod <rick@elrod.me>
* arg_spec - move type checking lookup method to a function
* Change get_wanted_type name and behavior
Change the name to get_validator to bette describe what it is doing.
Change the interface to always return a value. This lines up with the behavior of get_*
functions always returning something or None and check_* functions raising an
Exception if something went wrong during the check.
* Add param to check_type_str()
Not meant to be a long term fix, but gets tests passing. More work is needed to figure
out how to solve this cleanly.
* Remove private attribute mapping types to validator
Since the function that needs it has moved to parameters.py, there is no need to have it as
a attribute of AnsibleModule.
Update tests that were referencing the private attribute.
* Use private method for 'str' type
To avoid having to put the string conversion warning behavior in the check_type_str() method,
use the private _check_type_str() method for 'str' type.
Import CHECK_ARGUMENT_TYPES_DISPATCHER for backwards compalitibility and store it as
a private attribute.
Revert changes to support plugins that are referencing serf._CHECK_ARGUMENT_TYPES_DISPATCHER.
* Add changelog
* Change function name to better reflect its... function
* Change dict name to better reflect its contents
CHECK_ARGUMENT_TYPES_DISPATCHER --> DEFAULT_TYPE_VALIDATORS
* Fix changelog
* systemd - do not overwrite unit name when searching
PR #72702 introduced a bug that changed the unit name when splitting it up for the purpose
of searching for the unit. This only happens on unit file templates on systems that have a 5.8
or newer kernel and a version of systemd that does not contain a bugfix that causes systmed
to fail to parse dbus.
* Use facts rather than a manual probe to determine if systmed is present
* Remov unnecessary block
* Use vars files instead of set_fact
* Add tests for using a templated unit file
* Update changelog fragment
* Use template to get correct path to sleep binary
* import_playbook - change additional params to deprecation
I incorrectly recommended this be set as a warning when it should have been a deprecation.
* Fix deprecation sanity test to not required a collection name when not inside a collection
Also pin virtualenv to 16.7.10 for older Mac OS X systems. This was the version being installed
anway with the previous constraint (<20).
On systems with Python 3, now prefer venv over virtualenv. Test to see if venv is functional since
some systems have a non-functional venv installation (such as Debian).
* Change role argspec file used by ansible-doc
This changes the file used for role argument specs from meta/argument_specs.yml
to meta/main.yml. The argument specs are expected to be in that file under the
top-level entry of `argument_spec`.
* Switch to argument_specs
* ansible-galaxy collection list and verify now utilize collections in site-packages.
This is a short term fix for #70147. The long term fix needs to handle
install (but that discussion is also bound up in how upgrade is going to
work and where things can get installed so it's deferred for 2.11.)
* Add test for ansible-galaxy collection list with site-packages
* Fix sanity issue
Co-authored-by: David Moreau Simard <moi@dmsimard.com>
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Test installing a dependency compatible with multiple collection requirements
Test downloading a collection without dependencies
* Move cleanup to the end of the test suite
On python3 sys.stdin is an encoded file object that does not support
reading raw binary data. Use the supplied buffer object to do so.
Signed-off-by: Sven Wegener <sven.wegener@inovex.de>
Co-authored-by: Sven Wegener <sven.wegener@inovex.de>
This should allow users to extract specific files from an archive as
desired.
Fixes#16130, #27081.
* Rebase and make a few minor changes
* Add changelog
* Improve tests
- move to separate tasks file
- change assertions to check for exactly one file
- use remote_tmp_dir for output dir
* Make exclude and include mutually exclusive
* Don't remove files needed by other tasks
* Fix sanity tests
* Improve feature documentation
* Skip tests that use map() on CentOS 6
* Use fnmatch on include for zip archives
This matches the behavior of exclude
Co-authored-by: Sam Doran <sdoran@redhat.com>
* file: add symlink is in a sticky directory tests
* file: handle symlink in a sticky directory
Co-Authored-By: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Add changelog and fix unit test
The builtins import was removed since it was unused, but it is now needed.
The recently released version of cffi fails to install on systems with an older version of gcc. In
our case, this in the CentOS 6 test image. There is a fix but it has not yet been released.
https://foss.heptapod.net/pypy/cffi/-/issues/480
* Move _syslog_facitily to __init__
No good reason it should not be set for each object
* Move internal property setting to private method
* Create check_arguments() function
* Remove unused import
* Rename function to better match its behavior
Change the behavior to return a set, either empty or populated, with unsupported keys.
Accept legal_inputs as optional which will not required calling handle_aliases before calling
get_unsupported_parameters().
* Add changelog
* Rework function behavior and documentation
I realized I missed the original intent of this method when moving it to a function. It
is meant to compared the parameter keys to legal inputs always, not compare
parameter keys to argument spec keys, even though the argument spec keys should
be a subset of legal inputs.
* Add tests
* Fix typo.
* Set internal properties when handling suboptions
We could not find reason for the docutils install so trying out removing it.
Also bumping to latest version of ansible-runner
Skip all python2 versions because next ansible-runner drops it
Co-authored-by: Elijah DeLee <kdelee@redhat.com>
* Support listing roles in text and JSON
* Change tests for unfrack'd playbook_dir var
These tests were using '/tmp' for testing the setting of the playbook_dir
var. Now that we unfrack that var, MacOS will change this to '/private/tmp'
causing the tests to fail. We can choose a path that does not exist (since
unfrack does not validate existence) so that we can guarantee unfracking
will not change the value.
Change:
- Initial set of changes for renaming to ansible-core
- Includes changelog fragment changes from base -> core
- Does NOT include docs changes
- Modifies detection stuff in setup.py to support ansible<2.9 and ansible-base
Test Plan:
- ci_complete
* Fix type of mode field in copy module
* Remove module type checking from ignore.txt sanity checks
* Remove mode type
Co-authored-by: Matt Martz <matt@sivel.net>
* Consolidate logic for determining whether or not session is interactive
into a single function, is_interactive()
* Increase test coverage
I wasn't able to find a good way of simulating running a backgrounded test with CI since the
whole test is essentially run not in a TTY, which is similar enough to cause the new is_interactive()
function to always return false.
* Added caching mechanism for Galaxy API requests
* Add cache options and split up code
* Added unit tests
* Fix sanity test
* Use modified date and fix up caching for explicit servers
* Make sure credentials are not in cached server name
* Added test for getting updated cache version
* Changes from review
Change:
- Bump default, ansible-base, distro containers
- We do NOT add fedora33 yet, because it doesn't work right on Shippable
due to an old kernel. This will be added post-AZP.
Test Plan:
- CI
Signed-off-by: Rick Elrod <rick@elrod.me>
* Reorder comment postition
* Add comment unit test
* Fix unit test
* Fix unit test
* Add changelog
* Add paramaters which would be problematic without this fix
* Fix typo
* Fix unit test
* Fix unit test
* Emit warning when running on the controller with a Python older than 3.8
* Add spaces
Co-authored-by: Matt Clay <matt@mystile.com>
* and more spaces
Co-authored-by: Matt Clay <matt@mystile.com>
* s/Py/Python/
Co-authored-by: Matt Clay <matt@mystile.com>
* Add note to Control node requirements about Py3.8 requirement
* Add collection_name to deprecated call
* more spaces
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Clarify that we are only packaging for py3.8+
Co-authored-by: Matt Clay <matt@mystile.com>
Co-authored-by: Sviatoslav Sydorenko <wk.cvs.github@sydorenko.org.ua>
* Fix 'role_name : tast_name' notation if task contains role name
* Add tests for notifying handler names which contain the role name
Co-authored-by: Thomas Wouters <thomaswouters@gmail.com>
Change:
- Docs: Add note that security/bugfix apply to dependencies too, like
the dnf command.
- dnf: security/bugfix only makes sense for updates, so limit the
package query sack to available updates.
- tests: Limit tests to our known-good test packages, so that RHEL
packages marked security/bugfix without similarly marked dependencies
don't fail our tests.
Test Plan:
- Tested with `dnf upgrade-minimal --bugfix` and reproduced the same
error currently seen in CI, showing that we are consistent with what
dnf does.
Tickets:
- Likely fixes#72316
Signed-off-by: Rick Elrod <rick@elrod.me>
* added timeout options to adhoc and console
* added test
* fix typosesz
* fix conflict
* task_timeout
* fix timeout option, added extra vars to console
* actually use right cli switch .. DUH!
* added timeout to include 'valid' but ignored keys
* fix default
* fixes per review
* fixes for collection playbooks
- add fqcn invocation, also handles extensions
- bring import_playbook into new normal
- avoid adding collection playbooks to adjacency
- deal with extensions or lack of em
- added tests
- fix bugs with multiple playbooks supplied
- unicode/bytes/native fixes
- fix modulenotfound for < py3.7
- tests
* Update galaxy server image used in CI
* Fix port and migration issue
* Add delete step, still need to deal with pagination
* Make cleanup more efficient
* Remove testing code
* Add DarwinStrategy class and integration tests
macOS has three seprate hostname params that need to be set. One of those params, LocalHostName, has more stringent requirements than the other two, which accept special characters and spaces. Create a method to scrub the hostname to ensure it works well with the system requirements.
* Update documentation
* Account for virtualization type returned on Azure Pipelines
* Do not be dependent on order of self.name_types
Use the scrubbed name when the name type is LocalHostName
