Change:
Our handling of NetBSD virtualization facts led to facts that were just
plain incorrect. One example is reporting Xen even when the system is
running on something completely different (like KVM).
As stated by the reporter of #69352, NetBSD has a better sysctl setting
to use for this information, machdep.hypervisor.
This PR does the following:
- Try to use machdep.hypervisor sysctl value if the other sysctl values
we check don't end up with enough information to be useful
- Only look for /dev/xencons and assume Xen if nothing else works
(Really this should probably return 'unknown' since the file exists on
non-Xen systems and is not very useful).
- Add a few more patterns (Xen matches and also Hyper-V) to
VirtualSysctlDetectionMixin#detect_virt_product.
This change is slightly breaking:
- If the first two attempts at using sysctl worked before,
(machdep.dmi.system-product and machdep.dmi.system-vendor), they will
continue to work.
- For cases when those values didn't work, previously the existence of
/dev/xencons was checked, and if found, we reported 'xen' (even on
non-Xen systems when the file existed). After this PR, we try the
machdep.hypervisor sysctl key before still falling back to
/dev/xencons. This means that in some cases, we might go from
(wrongly) saying "xen" to giving a more accurate value such as "kvm"
or "Hyper-V".
Test Plan:
- Tested with local NetBSD VM and got 'kvm' instead of 'xen' back.
Tickets:
- Fixes#69352
Signed-off-by: Rick Elrod <rick@elrod.me>
* Add type for options in the sample module shown in Developing Modules, as this is required to have the tests being
green
* Remove duplicated strings: the same sentence is in "Python tips" and "Module security", keeping the
latter.
Previous version initialized the `TaskQueueManager` after calling
`Play.load()` while advertising a way to inject a custom library
location path. This caused the tasks loader not to find any custom
modules because it was triggered before the path was actually added
to the module loader.
This patch changes the order of the operations to ensure that the
customized `context.CLIARGS` actually influences things.
Resolves https://github.com/ansible/ansible/issues/69758.
If a local module has no documentation, the doc command will fail without any hints of what is wrong. Add another way to confirm the presence of a local module.
* Update docs/docsite/rst/dev_guide/developing_locally.rst
Co-authored-by: Abhijeet Kasurde <akasurde@redhat.com>
* Update docs/docsite/rst/dev_guide/developing_collections.rst
* add steps to create changelogs, add sentence about not using the tool
* add note for rerunning the command
Co-authored-by: Felix Fontein <felix@fontein.de>
* Add description of collections keyword
* Update based on feedback.
- Add link to become plugins.
- Add note about how the collections keyword works with roles.
* Update guide_gce.rst
The given example is not working as a SSH connection is not possible because of #59573. Instead, in this tutorial the default network should be used which allows SSH connections in most cases.
Co-authored-by: Sandra McCann <samccann@redhat.com>
* Updated docs/docsite/rst/dev_guide/developing_modules_documenting.rst, added documentation for R()
* added link to how to add anchors, anchor, and clarification on when to use each option
Co-authored-by: John R Barker <john@johnrbarker.com>
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Remove community.crypto parts from porting guide.
* Remove community.general parts from porting guide.
* Removed 'modules removed' section, since this is better suited for a general entry on the collection split.
* Validate ansible-base & collection's runtime.yml
Add new test `runtime-metadata`
* Schema validation of file
* Error if a a legacy meta/routing.yml exist in a collection
* removal_date OR removal_version
* Add tombstone validation.
* Allow both ISO 8601 date strings and datetime.date objects (from YAML dates).
* Address review comments.
* Add metadata to test collection.
* Add requirements file.
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Matt Clay <matt@mystile.com>
Replace the ansible-base changelog linting and generation tool with antsibull-changelog and make it available for linting collections. Previously changelog linting was limited to ansible-base.
* Allow to specify collection_name separately for deprecation.
* Use new functionality in Ansible.
* Use new functionality in tests.
* Update tagging/untagging functions.
* Update pylint deprecated sanity test.
* Update validate-modules. Missing are basic checks for version_added (validate semantic version format for collections).
* Improve version validation. Re-add version_added validation.
* Make sure collection names are added to return docs before schema validation.
* Extra checks to avoid crashes on bad data.
* Make C# module utils code work, and update/extend tests.
* Add changelog fragment.
* Stop extracting collection name from potentially tagged versions/dates.
* Simplify C# code.
* Update Windows modules docs.
* Forgot semicolons.
* Make AnsibleVaultEncryptedUnicode work more like a string. Fixes#24425
* Remove debugging
* Wrap some things
* Reduce diff
* data should always result in text
* add tests
* Don't just copy and paste, kids
* Add eq and ne back
* Go full UserString copy/paste
* Various version related fixes
* Remove trailing newline
* py2v3
* Add a test that can evaluate whether a variable is vault encrypted
* map was introduces in jinja2 2.7
* moar jinja
* type fix
Co-Authored-By: Sam Doran <sdoran@redhat.com>
* Remove duplicate __hash__
* Fix typo
* Add changelog fragment
* ci_complete
Co-authored-by: Sam Doran <sdoran@redhat.com>
* starting metadata sunset
- purged metadata from any requirements
- fix indent in generic handler for yaml content (whey metadata display was off)
- make more resilient against bad formed docs
- removed all metadata from docs template
- remove metadata from schemas
- removed mdata tests and from unrelated tests
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Rick Elrod <rick@elrod.me>
Change:
On OpenBSD when using pipelining, we do not set cwd which results in a
permissions fatal. Ensure that `''` - cwd - is not in `sys.path`.
Test Plan:
Tested against local OpenBSD VM
Tickets:
Fixes#69320
Signed-off-by: Rick Elrod <rick@elrod.me>
* Enable installing collections from git repositories
* Add tests for installing individual and multiple collections from git repositories
* Test to make sure recursive dependencies with different syntax are deduplicated
* Add documentation
* add a changelog
* Skip Python 2.6
* Only fail if no collections are located in a git repository
Add support for a 'type' key for collections in requirement.yml files.
Update the changelog and document the supported keys and allowed values for the type.
Add a note that the collection(s) in the repo must contain a galaxy.yml
* Add a warning about embedding credentials in SCM URLs
* Update with review suggestions
* suppress sanity compile failure for Python 2.6
* Track collection for version_added.
Validate *all* version numbers in validate-modules.
For tagged version numbers (i.e. version_added), consider source collection to chose validation.
* Make tagging/untagging functions more flexible.
* Tag all versions in doc fragments.
* Tag all deprecation versions issued by code.
* Make Display.deprecated() understand tagged versions.
* Extend validation to enforce tagged version numbers.
* Tag versions in tests.
* Lint and fix test.
* Mention collection name in collection loader's deprecation/removal messages.
* Fix error IDs.
* Handle tagged dates in Display.deprecated().
* Also require that removed_at_date and deprecated_aliases.date are tagged.
* Also automatically tag/untag removed_at_date; fix sanity module removal version check.
* Improve error message when invalid version number is used (like '2.14' in collections).
* Improve the changelog-how-to documentation:
* remove repetitions
* mention starting with lowercase
* update the link to examples, remove an irrelevant link
* change win_ example to apt_repository
* add info about collections
* note that environment is not good for secrets
* Update docs/docsite/rst/user_guide/playbooks_environment.rst
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Allow to deprecate options and aliases by date instead of only by version.
* Update display.deprecate().
* Adjust behavior to conform to tested behavior, extend tests, and improve C# style.
* Parse date and fail on invalid date.
This is mainly to make sure that people start using invalid dates, and we eventually have a mess to clean up.
* C# code: improve validation and update/extend tests.
* Make sure that deprecate() is not called with both date and version.
* Forgot to remove no longer necessary formatting.
* Adjust order of warnings in C# code.
* Adjust unrelated test.
* Fix grammar (and make that test pass).
* Don't parse date, and adjust message to be same as in #67684.
* Sanity tests: disable date in past test.
* Validate-modules: validate ISO 8601 date format.
* Validate-modules: switch schema declaration for deprecated_aliases to improve error messages for invalid dates.
* Use DateTime instead of string for date deprecation.
* Validate that date in deprecated_aliases is actually a DateTime.
* Fix tests.
* Fix rebasing error.
* Adjust error codes for pylint, and add removed_at_date and deprecated_aliases.date checks to validate-modules.
* Make deprecation date in the past error codes optional.
* Make sure not both version and date are specified for AnsibleModule.deprecate() calls.
* Stop using Python 3.7+ API.
* Make sure errors are actually reported. Re-add 'ansible-' prefix.
* Avoid crashing when 'name' isn't there.
* Linting.
* Update lib/ansible/module_utils/csharp/Ansible.Basic.cs
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Adjust test to latest change.
* Prefer date over version if both end up in Display.deprecated().
Co-authored-by: Jordan Borean <jborean93@gmail.com>
* Unify ansible-galaxy install -r
* Minor nit fixes for docs
* Re-align warnings
* Fix up integration test
* Fix up test where no roles/collections were in file
Change:
- Nuke `make rpm` and friends from Makefile
- Nuke packaging/rpm
We are no longer going to be pushing RPMs to releases.ansible.com
post-2.10, so this is no longer necessary for us, and users should
prefer RPMs from their distro instead.
Test Plan:
Grepped the Makefile for all of: /rpm/i, /mock/i, /fedora/i
Tickets:
Refs #69539
Signed-off-by: Rick Elrod <rick@elrod.me>
This fact reflects the number of usable vcpus (which might be different
from ansible_processor_vcpus, e.g., in containers with limits). See
also #51504.
* Add fixture data and update unit tests
Co-authored-by: Sam Doran <sdoran@redhat.com>
For instance, you should use `community.vmware.vmware_guest` instead of just
`vmware_guest`.
This way, the examples don't depend on the `collections` directive of the
playbook.
Co-authored-by: Sandra McCann <samccann@redhat.com>
With inventory script migrated to their respective collection,
redirect links in documentation to their respective collection
location.
Fixes: #69139
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Make clear which BOTMETA.yml is meant (some collections also have one), fix itemization, document /rebuild and /rebuild_failed, add section on how to test collections with ansible-test, update supported versions for compile tests, add a section on hacking collections, implement feedback.
* Update docs/docsite/rst/dev_guide/developing_collections.rst
Co-Authored-By: Felix Fontein <felix@fontein.de>
Co-authored-by: Alicia Cozine <acozine@users.noreply.github.com>
VMware vm inventory uses configuration properties, rather than
relying on external documents, point user to this document.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Allow tasks to notify a fqcn handler name
* Add tests. Fixes#68181
* Add changelog fragment
* Add test to ensure handlers are deduped properly with fqcn, role, and just handler names
* Add some docs about new special vars
* Dev guide: add notes about different image types for integration and unit/sanity tests
* Update docs/docsite/rst/dev_guide/testing_integration.rst
Co-authored-by: Matt Clay <matt@mystile.com>
* remove azure extras and extras_require support
* Since Azure will be collectionized, the requirements will float more frequently than Ansible releases; the Azure collection needs to host the requirements now.
* Removed the dynamic extras support as well, since Azure was the only thing using it. If we need it again, it's easy to pull back from history.
* Mark azure-requirements as orhpaned.
This keeps the docs around so that existing links from old test runs remain valid.
Co-authored-by: Matt Clay <matt@mystile.com>
Use "name" when possible rather than "src" to make the examples
of roles and collections in a single requirements files more
coherenant. Fundamentally, roles and collections are completely different.
But we can make the requiremets file easier to read by unifying the
format as much as possible.
* Add warning when running devel. Addresses #67362
* Add config entry to disable devel warning, so it doesn't impact CI
* Update warning about using devel
* ec2_tag - Deprecate the use of state=list
* Update lib/ansible/modules/cloud/amazon/ec2_tag.py
* Add changelog and porting_guide entries
Co-authored-by: flowerysong <junk+github@flowerysong.com>
##### SUMMARY
Adding nanvault - a standalone tool to encrypt and decrypt files in the Ansible Vault format, featuring UNIX-style composability.
##### ISSUE TYPE
- Docs Pull Request
+label: docsite_pr
* required_if checks should have three or four parts.
* Validate mutually_exclusive, required_together, required_one_of, required_if and required_by.
* Simplify code.
* Improve messages.
* Add changelog.
* Sanity check.
* Update docs.
* Update ignore.txt.
* Don't continue with tests when terms are not strings.
* Remove ignore.txt entry.
* Make sure validate-modules doesn't choke on things already flagged by schema test.
* Check required_if requirements list for strings.
* 'message' parameter is replaced by 'commit_message' in grafana_dashboard
* 'message' parameter is replaced by 'notification_message' in datadog_monitor
This change is required since 'message' as parameter name is used internally by
Ansible core engine.
Fixes: #39295#45362#47132#59617
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
Steers users around the error: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/xmltodict.py'
Consider using the `--user` option or check the permissions. -->
* [aws_s3] fix deleting the current objects and the previous versions from a version-enabled bucket
* use existing paginated_list function to keep compatibility with the other places it is called
* changelog
* Add noteworthy change to the porting guide
* Reword that with acozine's suggestion
Co-authored-by: Alicia Cozine <879121+acozine@users.noreply.github.com>
* Start adding ansible-galaxy collection list options
* Working list all collections and list a specific collection
* Nuke debugging cruft
* Use to_text to get a string of the FQCN for sorting
* Improve collection output formatting
- add header
- display collection name and version in separate columns
- width of columns is dynamic based on collection name and version length
* Make role list output match collection output list
- add header
- add columns for role name and version
- make column width dynamic based on name and version length
* remove debug statemnt and extra header
* Revert "Make role list output match collection output list"
This reverts commit a0b3db47bb3b198aafd34c1f1be5b6561af2f928.
* Add validate_collection_path function
Utility function for ensuring a collection target ends with 'ansible_collection'
* Use validate_collection_path
* Do not warn if a specific collection in found in any search path
* Fix extraneous warning and remove duplicate code
Do not warn when listing a specific collection and it does not exist
in other collection paths.
Restructure the code that loops through collection paths to remove
duplicate conditional code.
* Indicate role path was found
* Use new function name
* WIP Save Point
* Use separate functions for role and collection list
* Wrap error message
There may be a better way to do this besides hard coding a line break, but this
does make the message a lot more readable.
* Add validate_collection_path function (#66441)
* Add validate_collection_path function
Utility function for ensuring a collection target ends with 'ansible_collection'
* Fix bad syntax
* Correct docstring
* Bikeshed the names
* Properly list a single role
* Simplify _display_warnings()
Only display warnings. Move exception raise back to each caller.
* Move private methods to private functions
They don't need self, so it makes sense to have them as functions
Get rid of _display_warnings() function since it doesn't do anything worthy of
an independent function.
* Add integration tests for ansible-galaxy collection list
* Fix docs sanity test
* Fix bug where ansible_collections dir does not exist
The path may exist, but if there is no ansible_collections dir inside that path,
an exception was raised in find_existing_collections().
Add integration test for this scenario
* Put execute_list() method back
* Add some informational messages for debugging
* Add unit tests
Units tests for the various private methods in support of collection list
* Start adding unit tests for test_execute_list
* Display collection path when listing specific collection
* Add unit tests for listing all collections and specific collection
- Create fixture for creating test objects
- Add function for controlling os.path.isdir results
* Set defaults for minimum collection widths
Ensure that collections with small FQCNs display correctly.
Add unit tests
* Split up unit tests and fix fixtures
Add more fixtures for mocking objects during the specific collection tests
* Change help message for -p in list subcommand
Give accurate description of what it actually does rather than trying to use language shared between sub commands.
* Disable colorized output in unit test
* Add docs for collection list
* Fix integration test on macOS
The temp file path is really long on macOS, so the warning message gets wrapped
across multiple lines. That make seth grep fail. Switch to matching on a smaller
part of the warning.
* Recreate common path options for collections
Improve help about what the '-p' option does and how it works.
* Remove unnecessary elif after continue statements
* Account for duplicate paths in collections_searh_paths
If someone specifies the same path via '-p' that is the COLLECTIONS_PATHS,
do not list the collections twice.
* Docs updates
* [WIP] Add verify subcommand command to 'ansible-galaxy collection'
* Fix pep8 and use consistent display order
* WIP - docs
* Remove some redundancy in verify display messages by using an error queue for each collection
* Share common code and improve output format
* clean up documentation
* typo
* Apply suggestions from code review
Co-Authored-By: Sandra McCann <samccann@redhat.com>
* Move ModifiedContent namedtuple to the global scope
Add a public metadata property
Rename function to _get_json_from_tar_file
* Add some unit tests
* fix using common functions after rebase
* changelog
* Improve efficiency finding specific installed collections
Improve efficiency by only downloading the tar.gz from the galaxy server for comparison after checking that the collection has been installed
Handle multiple collection paths
Fix up tests
* pep8
* reword that for accuracy
* use more common code and verify collection name
* Improve error message and add documentation
* Update unit tests and add coverage for both local and remote collections that are unable to be located
* Only validate collections using the format collection_namespace.collection_name
Fix tests to reflect that
Fix documentation
* Fix a bug when a local collection does not contain a file
* pep8
* Fix formatting
Co-authored-by: Sandra McCann <samccann@redhat.com>
On macOS Catalina ``pip install --user ansible`` results in warnings and errors. This updates the reference documents to reflect changes Catalina+ with a safe, easy install for Ansible, accessing pip as a module and installing ansible with --user.
Co-authored-by: Sam Doran <sdoran@redhat.com>
* Change socket_path error messages to not directly refer to socket_path
* Apply suggestions from code review
Co-Authored-By: Nathaniel Case <this.is@nathanielca.se>
* Clean up FILE_COMMON_ARGUMENTS.
* postgresql_pg_hba doesn't declare the backup option.
* uri doesn't declare the remote_src option.
* Add documentation.
* maven_artifact seems to use directory_mode, which it doesn't declare.
* Update changelogs/fragments/66389-file-common-arguments.yml
Update docs/docsite/rst/porting_guides/porting_guide_2.10.rst
ci_complete
Co-Authored-By: Jill R <4121322+jillr@users.noreply.github.com>
* Add anchor to each paramater row
* Update docs/templates/plugin.rst.j2
Co-Authored-By: Felix Fontein <felix@fontein.de>
* Insert full keys into plugin docs.
* Added visible links.
Co-authored-by: Felix Fontein <felix@fontein.de>
* aws_netapp_cvs_snapshots - minor required_if fixup (state must be set if state=present)
* ec2 - fix typo in mutually_exclusive definition
* rds_instance: fix typo in mutually_exclusive restore_to_time should be restore_time - currently throws a boto error
Fixed sphinx theme to navigate "Edit on Github" link to locate correct
plugin, cli source in GitHub repo.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
PR #66898
This change introduces a new sanity check with code
`parameter-state-invalid-choice` in the `ansible-test sanity`
validator. It enforces modules not to support `list` or `info`
as their `state`.
Co-Authored-By: Sviatoslav Sydorenko <webknjaz@redhat.com>
Co-Authored-By: Felix Fontein <felix@fontein.de>
vmware_tag_info used to return dict of tag information which caused
data loss when there are multiple tags with same name and different category ids.
This fix will add additional fact "tag_info" which will deprecated existing fact
"tag_facts".
The "tag_info" is a list which handles multiple tags with same name.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Add a script to update the intersphinx inventory files
* We're about to add intersphinx inventories for separate ansible docs
so we need an easy way to update them. Also, we should be updating
these cache files for other upstreams occassionally as well. With a
script, we can add updating them to a release process.
* Now that we don't know what the version of the cache is, change the
filenames to not contain versions.
* Update the intersphinx cache files with the latest upstream versions
Results of running:
hacking/build-ansible.py update-intersphinx-cache -o docs/docsite -c docs/docsite/rst/conf.py
* Add a comment to the configuration file which says how to structure the intersphinx mapping and why.
* Update docs/docsite/rst/conf.py
Co-Authored-By: Sandra McCann <samccann@redhat.com>
Co-authored-by: Sandra McCann <samccann@redhat.com>
We have called an API token many things. I changed the command line argument to
"--token", so now I'm unifying the documentation. I looked at both Galaxy and
Automation Hub and Galaxy is the only place that still uses the term "API Key",
which it could be argued should change.
##### SUMMARY
Previously folder tree example for collection adjacent to the current playbook were displaying a subfolder below the playbook instead of being a sibling.
* Deprecate Windows 2008, and 2008 R2
* Remove shippable nodes
* Update windows_faq.rst
Be less specific about 2008/R2 timeframes
* Update setup.ps1
tweak warning text
Co-authored-by: Matt Davis <nitzmahone@users.noreply.github.com>
As AnsibleModule._log_invocation is currently implemented, any parameter
with a name that matches PASSWORD_MATCH triggers the no_log warning as a
precaution against parameters that may contain sensitive data, but have not
been marked as sensitive by the module author.
This patch would allow module authors to explicitly mark the aforementioned
parameters as not sensitive thereby bypassing an erroneous warning message,
while still catching parameters which have not been marked at all by the
author.
Adds tests for various no_log states including True, False, and None (as
extracted by AnsibleModule._log_invocation) when applied to an argument with
a name that matches PASSWORD_MATCH.
Fixes: #49465#64656
The `regex_replace` documentation included an example that was meant
to be preformatted text but due to an indentation error was rendering
incorrectly.
* Add a banner which will display on the testing site.
Thanks to shane for the javascript code
* change to more noticable color
Co-authored-by: Sandra McCann <samccann@redhat.com>
* Fix netconf plugin related to collections
Fixes#65655 (partly)
* Make netconf plugins configurable so that the
information of ncclient device handler
for give platform resides in the platform
specific netconf plugin.
* If the device handler value in ncclient is
different from the ansible_network_os value
the right value of `ncclient_device_handler`
should be set in the plugin documentation.
* Fix review comments
* Fix CI issue
* Fix review comment
When an ansible module, e.g., command, does not support check mode, you can use "check_mode: no" in that task to execute the task in normal mode, even when "--check" flag is used. The current document basically says, use "check_mode: no" to run in check mode, which is confusing and inaccurate.
* Make script more portable
sha1sum is a Linux only command. Test for the command and if not found
use sha1 instead for portability. Avoid patches on BSD and Mac systems.
* win_find - refactor to make more performance and use newer style
* win_find - refactor for performance improvements and alignment to find
* More path alignment to find
* Fix yamllint error
