`ansible-vault` is the only cli command which knows how to handle the
rekey options `--new-vault-id` and `--new-vault-password-file`. No
point in exposing those rekey options to any of the other ansible
commands.
On a practical level I think this matters most in ensuring that
`--help` doesn't produce any false/unhelpful output.
Without this additional code snippet `ansible-pull` will still accept
the `--vault-id` option. It just won't pass the option along when
invoking `ansible-pull`.
* jsonify inventory
* smarter import, dont pass kwargs where not needed
* added datetime
* Eventual plan for json utilities to migrate to common/json_utils when we split
basic.py no need to move jsonify to another file now as we'll do that later.
* json_dict_bytes_to_unicode and json_dict_unicode_to_bytes will also
change names and move to common/text.py at that time (not to json).
Their purpose is to recursively change the elements of a container
(dict, list, set, tuple) into text or bytes, not to json encode or
decode (they could be a generic precursor to that but are not limited
to that.)
* Reimplement the private _SetEncoder which changes sets and datetimes
into objects that are json serializable into a private function
instead. Functions are more flexible, less overhead, and simpler than
an object.
* Remove code that handled simplejson-1.5.x and earlier. Raise an error
if that's the case instead.
* We require python-2.6 or better which has the json module builtin to
the stdlib. So this is only an issue if the stdlib json has been
overridden by a third party module and the simplejson on the system
is 1.5.x or less. (1.5 was released on 2007-01-18)
* included inventory and callback in new config
allow inventory to be configurable
updated connection options settings
also updated winrm to work with new configs
removed now obsolete set_host_overrides
added notes for future bcoca, current one is just punting, it's future's problem
updated docs per feedback
added remove group/host methods to inv data
moved fact cache from data to constructed
cleaner/better options
fix when vars are added
extended ignore list to config dicts
updated paramiko connection docs
removed options from base that paramiko already handles
left the look option as it is used by other plugin types
resolve delegation
updated cache doc options
fixed test_script
better fragment merge for options
fixed proxy command
restore ini for proxy
normalized options
moved pipelining to class
updates for host_key_checking
restructured mixins
* fix typo
* Fix vault --ask-vault-pass with no tty
2.4.0 added a check for isatty() that would skip setting up interactive
vault password prompts if not running on a tty.
But... getpass.getpass() will fallback to reading from stdin if
it gets that far without a tty. Since 2.4.0 skipped the interactive
prompts / getpass.getpass() in that case, it would never get a chance
to fall back to stdin.
So if 'echo $VAULT_PASSWORD| ansible-playbook --ask-vault-pass site.yml'
was ran without a tty (ie, from a jenkins job or via the vagrant
ansible provisioner) the 2.4 behavior was different than 2.3. 2.4
would never read the password from stdin, resulting in a vault password
error like:
ERROR! Attempting to decrypt but no vault secrets found
Fix is just to always call the interactive password prompts based
on getpass.getpass() on --ask-vault-pass or --vault-id @prompt and
let getpass sort it out.
* up test_prompt_no_tty to expect prompt with no tty
We do call the PromptSecret class if there is no tty, but
we are back to expecting it to read from stdin in that case.
* Fix logic for when to auto-prompt vault pass
If --ask-vault-pass is used, then pretty much always
prompt.
If it is not used, then prompt if there are no other
vault ids provided and 'auto_prompt==True'.
Fixes vagrant bug https://github.com/hashicorp/vagrant/issues/9033Fixes#30993
* Automatically loads and executes an inventory plugin specified by a standard YAML inventory config file containing a `plugin` key at its root.
* Moved inventory PluginLoader to a shared global instance.
So we fixed everything that was not a module to be PEP8 compliant, and
in the meantime these 5 new files were additionally disabled from PEP8
testing.
This fixes it.
Also update Copyright/License statements.
This makes adhoc mirror playbook callback functionality by running a
callback before and after all tasks have run. Adhoc commands now call:
- v2_playbook_on_start
- v2_playbook_on_stats
NOTE: When v2_playbook_on_start is called, a dummy playbook is provided
that says its _file_name is __adhoc_playbook__. All callback plugins
that provide v2_playbook_on_start access the _file_name attribute, so
this should maintain backward compatibility when those plugins are
called with adhoc commands even though they would not have been called
previously. The adhoc play is also added to _entries for any private
callback plugins that might be using this callback.
This reverts commit 07acc579db.
On closer examination of this code, the conditional that had force in it
was not a parent of this one. So handling of force is needed i both
branches.
See the recent comments on #23391
* Avoid default inventory proccessing for pull
- now pull's own special inventory processing should work correctly
- also removed ineffective set_defaults
fixes#31449
* use class property instead
* only do localhost for adhoc
(cherry picked from commit aad5d1432583c4aa4105b774f38c80498e85de59)
This adds a new type of vault-password script (a 'client') that takes advantage of and enhances the
multiple vault password support.
If a vault password script basename ends with the name '-client', consider it a vault password script client.
A vault password script 'client' just means that the script will take a '--vault-id' command line arg.
The previous vault password script (as invoked by --vault-password-file pointing to an executable) takes
no args and returns the password on stdout. But it doesnt know anything about --vault-id or multiple vault
passwords.
The new 'protocol' of the vault password script takes a cli arg ('--vault-id') so that it can lookup that specific
vault-id and return it's password.
Since existing vault password scripts don't know the new 'protocol', a way to distinguish password scripts
that do understand the protocol was needed. The convention now is to consider password scripts that are
named like 'something-client.py' (and executable) to be vault password client scripts.
The new client scripts get invoked with the '--vault-id' they were requested for. An example:
ansible-playbook --vault-id my_vault_id@contrib/vault/vault-keyring-client.py some_playbook.yml
That will cause the 'contrib/vault/vault-keyring-client.py' script to be invoked as:
contrib/vault/vault-keyring-client.py --vault-id my_vault_id
The previous vault-keyring.py password script was extended to become vault-keyring-client.py. It uses
the python 'keyring' module to request secrets from various backends. The plain 'vault-keyring.py' script
would determine which key id and keyring name to use based on values that had to be set in ansible.cfg.
So it was also limited to one keyring name.
The new vault-keyring-client.py will request the secret for the vault id provided via the '--vault-id' option.
The script can be used without config and can be used for multiple keyring ids (and keyrings).
On success, a vault password client script will print the password to stdout and exit with a return code of 0.
If the 'client' script can't find a secret for the --vault-id, the script will exit with return code of 2 and print an error to stderr.
In cli.CLI.unfrack_path callback, special case if the
value of '--output' is '-', and avoid expanding
it to a full path.
vault cli already has special cases for '-', so it
just needs to get the original value to work.
Fixes#30550
* fixed ansible/git invocation options
now falls back to using localhost as 'all' does not include implicit accidentally anymore
fixes#30636
* better fix
* qfq9
This is to catch vault secrets from config and
cli. Previously vault_password_file in config was
missed since it was added by setup_vault_secrets,
so check after setup_vault_secrets.
* Don't ask for password confirm on 'ansible-vault edit'
This is to match the 2.3 behavior on:
ansible-vault edit encrypted_file.yml
Previously, the above command would consider that a 'new password'
scenario and prompt accordingly, ie:
$ ansible-vault edit encrypted_file.yml
New Password:
Confirm New Password:
The bug was cause by 'create_new_password' being used for
'edit' action. This also causes the previous implicit 'auto prompt'
to get triggered and prompt the user.
Fix is to make auto prompt explicit in the calling code to handle
the 'edit' case where we want to auto prompt but we do not want
to request a password confirm.
Fixes#30491
* finalize lookup documentation
* minor fixes to ansible-doc
- actually show which file caused error on when listing plugins
- removed redundant display of type and name
* smart quote fixes from toshio
* module and vault fixes
- fix module_path cli option and usage, which fixes#29653
- move --output to be in subset of vault cli, no need for all vault enabled cli to use it
- added debug to loader to see directories added
* Change ansible-doc usage to show -a is for internal use
ansible-doc -a is for testing that documentation is sane. It should not
be used by normal users in production. The main reason for this is that
it is designed to fail if there are any undocumented modules or plugins.
This is good for testing that all plugins we ship are documented. It is
not good for end users who may have undocumented third-party plugins.
- better variable precedence management
- universal plugin option handling
- also updated comments for future directions
- leverage fragments for plugins
- removed fact namespacing
- added 'firendly name' field
- updated missing descriptions
- removed some unused yaml entries, updated others to reflect possible future
- documented more plugins
- allow reading docs using alias
- short licenses
- corrected args for 'all plugins'
- fixed -a option for ansible-doc
- updated vars plugins to allow docs
- fixed 'gathering'
- only set options IF connection
- added path list and renamed pathspec mostly the diff is , vs : as separator
- readded removed config entries that were deprecated but had no message ... and deprecated again
- now deprecated entries give warning when set
* let generate_man also gen rst pages for cli tools
* make template-file, output-dir, output format cli options for generate_man
* update main Makefile to use generate_man.py for docs (man pages and rst)
* update vault docs that use :option:
* Edits based on
6e34ea6242 and
a3afc78535
* add a optparse 'desc' to lib/ansible/cli/config.py
The man page needs a short desc for the 'NAME' field
which it gets from the option parse 'desc' value.
Fixes building ansible-config man page.
* add trim_docstring from pep257 to generate_man
use pep258 docstring trim function to fix up any indention
weirdness inherit to doc strings (ie, lines other than
first line being indented.
* Add refs to cli command actions
To reference ansible-vaults --vault-id option, use:
:option:`The link text here <ansible-vault --vault-id>`
or:
:option:`--vault-id <ansible-vault --vault-id>`
To reference ansible-vault's 'encrypt' action, use:
:ref:`The link text here <ansible_vault_encrypt>`
or most of the time:
:ref:`ansible-vault encrypt <ansible_vault_encrypt>`
This allows to use a pathlist in the ansible.cfg:
[default]
inventory = path/inventory:other_path/inventory
Since ansible allows to use --inventory on CLI more then once, we should also support a pathlist in the config.
* fix plugin docs broken by previous commit
also removed verbosity requirements, goes back to displaying all data by default
* pepe should be full by now
* Add network value to support_by field.
* New support_by value, certified
* Deprecate curated in favor of certified
* Add conversion from 1.0 to 1.1 to metadata-tool
* Add supported by Red Hat field to ansible-doc output
* ansible-inventory cli tool
added vars dump to graph
made yaml inventory dump actual yaml inventory format
cleaner dump
fixed graph, no dump needed
add pulling in host/group vars
pep indentation crime
added docstring for manpage autodoc
remove ansible_facts from output
added api compat layer
allow import from new and old APIs
better conditional for <2.4
pe4+p4
test stuck on OS X (seems popular today) but passes rest, merging anyways
* Ansible Config part2
- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code
skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution
* killed extra space with extreeme prejudice
cause its an affront against all that is holy that 2 spaces touch each other!
shippable timing out on some images, but merging as it passes most
* prompt for new pass on create/encrypt if none specified
Make 'ansible-vault' edit or encrypt prompt for a password
if none or provided elsewhere.
Note: ansible-playbook does not prompt if not vault password
is provided
* dont show vault password prompts if not a tty
* Add config option for a default list of vault-ids
This is the vault-id equilivent of ANSIBLE_DEFAULT_PASSWORD_FILE
except ANSIBLE_DEFAULT_VAULT_IDENTITY_LIST is a list.
* Better handling of empty/invalid passwords
empty password files are global error and cause an
exit. A warning is also emitted with more detail.
ie, if any of the password/secret sources provide
a bogus password (ie, empty) or fail (exception,
ctrl-d, EOFError), we stop at the first error and exit.
This makes behavior when entering empty password at
prompt match 2.3 (ie, an error)
The prompt_formats dict didn't get the 'prompt_ask_vault_pass'
item added for interactive --ask-vault-pass, which
caused "KeyError: u'prompt_ask_vault_pass'"
Fixes#27885
* rm unneeded parens following assert
* rm unused parse_vaulttext_envelope from yaml.constructor
* No longer need index/enumerate over vault_ids
* rm unnecessary else
* rm unused VaultCli.secrets
* rm unused vault_id arg on VaultAES.decrypt()
pylint: Unused argument 'vault_id'
pylint: Unused parse_vaulttext_envelope imported from ansible.parsing.vault
pylint: Unused variable 'index'
pylint: Unnecessary parens after 'assert' keyword
pylint: Unnecessary "else" after "return" (no-else-return)
pylint: Attribute 'editor' defined outside __init__
* use 'dummy' for unused variables instead of _
Based on pylint unused variable warnings.
Existing code use '_' for this, but that is old
and busted. The hot new thing is 'dummy'. It
is so fetch.
Except for where we get warnings for reusing
the 'dummy' var name inside of a list comprehension.
* Add super().__init__ call to PromptVaultSecret.__init__
pylint: __init__ method from base class 'VaultSecret' is not called (super-init-not-called)
* Make FileVaultSecret.read_file reg method again
The base class read_file() doesnt need self but
the sub classes do.
Rm now unneeded loader arg to read_file()
* Fix err msg string literal that had no effect
pylint: String statement has no effect
The indent on the continuation of the msg_format was wrong
so the second half was dropped.
There was also no need to join() filename (copy/paste from
original with a command list I assume...)
* Use local cipher_name in VaultEditor.edit_file not instance
pylint: Unused variable 'cipher_name'
pylint: Unused variable 'b_ciphertext'
Use the local cipher_name returned from parse_vaulttext_envelope()
instead of the instance self.cipher_name var.
Since there is only one valid cipher_name either way, it was
equilivent, but it will not be with more valid cipher_names
* Rm unused b_salt arg on VaultAES256._encrypt*
pylint: Unused argument 'b_salt'
Previously the methods computed the keys and iv themselves
so needed to be passed in the salt, but now the key/iv
are built before and passed in so b_salt arg is not used
anymore.
* rm redundant import of call from subprocess
pylint: Imports from package subprocess are not grouped
use via subprocess module now instead of direct
import.
* self._bytes is set in super init now, rm dup
* Make FileVaultSecret.read_file() -> _read_file()
_read_file() is details of the implementation of
load(), so now 'private'.
* Properly include aliases in ansible-doc output
Use correct variable name for storing aliases and only print them out if the list isn't empty.
Fixes#24498
* Only include choices in output if choices exist in the list
Got removed in arg parsing updates. Now added back in
setup_vault_secrets().
The default value for DEFAULT_VAULT_PASSWORD_FILE was also
set to '~' for some reason, change to to no default.
Add integration tests.
If we don't use more than one vault-id, and we use
--ask-vault-pass, instead of using the new vault prompt
format ('Vault password (my_vault_id): ') we use the old
one ('Vault password: ').
This avoids confusing Tower when it needs to detect an
interactive vault password prompt.
This also potentially could allow vault password prompts
to be customized per vault_id.
Fixes#13243
** Add --vault-id to name/identify multiple vault passwords
Use --vault-id to indicate id and path/type
--vault-id=prompt # prompt for default vault id password
--vault-id=myorg@prompt # prompt for a vault_id named 'myorg'
--vault-id=a_password_file # load ./a_password_file for default id
--vault-id=myorg@a_password_file # load file for 'myorg' vault id
vault_id's are created implicitly for existing --vault-password-file
and --ask-vault-pass options.
Vault ids are just for UX purposes and bookkeeping. Only the vault
payload and the password bytestring is needed to decrypt a
vault blob.
Replace passing password around everywhere with
a VaultSecrets object.
If we specify a vault_id, mention that in password prompts
Specifying multiple -vault-password-files will
now try each until one works
** Rev vault format in a backwards compatible way
The 1.2 vault format adds the vault_id to the header line
of the vault text. This is backwards compatible with older
versions of ansible. Old versions will just ignore it and
treat it as the default (and only) vault id.
Note: only 2.4+ supports multiple vault passwords, so while
earlier ansible versions can read the vault-1.2 format, it
does not make them magically support multiple vault passwords.
use 1.1 format for 'default' vault_id
Vaulted items that need to include a vault_id will be
written in 1.2 format.
If we set a new DEFAULT_VAULT_IDENTITY, then the default will
use version 1.2
vault will only use a vault_id if one is specified. So if none
is specified and C.DEFAULT_VAULT_IDENTITY is 'default'
we use the old format.
** Changes/refactors needed to implement multiple vault passwords
raise exceptions on decrypt fail, check vault id early
split out parsing the vault plaintext envelope (with the
sha/original plaintext) to _split_plaintext_envelope()
some cli fixups for specifying multiple paths in
the unfrack_paths optparse callback
fix py3 dict.keys() 'dict_keys object is not indexable' error
pluralize cli.options.vault_password_file -> vault_password_files
pluralize cli.options.new_vault_password_file -> new_vault_password_files
pluralize cli.options.vault_id -> cli.options.vault_ids
** Add a config option (vault_id_match) to force vault id matching.
With 'vault_id_match=True' and an ansible
vault that provides a vault_id, then decryption will require
that a matching vault_id is required. (via
--vault-id=my_vault_id@password_file, for ex).
In other words, if the config option is true, then only
the vault secrets with matching vault ids are candidates for
decrypting a vault. If option is false (the default), then
all of the provided vault secrets will be selected.
If a user doesn't want all vault secrets to be tried to
decrypt any vault content, they can enable this option.
Note: The vault id used for the match is not encrypted or
cryptographically signed. It is just a label/id/nickname used
for referencing a specific vault secret.
restoring behaviour in which a missing module is a warning that shows paths searched,
patch incorrectly bypassed detection and ended up in an exception: global name 'module' is not defined
Currently, someone writing a action plugin will also need
to have a empty file in the module path to avoid triggering
the error "no action detected in task.".
One of the complex map + filter + lambda combinations here was causing
pylint to have a rare false positive detecting undefined variables.
Switch to list comprehensions and generator expressions which are
easi(er) to read for humans and do not confuse pylint.
References #27193
Consolidate the module_utils, constants, and config functions that
convert values into booleans into a single function in module_utils.
Port code to use the module_utils.validate.convert_bool.boolean function
isntead of mk_boolean.
* Fix ansible-doc traceback when a plugin doesn't parse correctly
* Change extract_metadata ivocation to take either an ast or source
code. When given source code, it can find file offsets for the start
and end of dict. When given the ast, it is quicker as it doesn't have
to reparse the source. Requires changing the call to the function to
use a keyword arg.
* Fix reading of metadata to find the last occurrence of
ANSIBLE_METADATA instead of the first.
* Add some more unittests to get closer to complete coverage
finished normalizing of path handling
removed overloaded '-p' from init_paths option, it is for role_paths
removed expand_tilde and get_opt methods as both were redundant, adjusted rest of code
updated tests to match
* Start of ansible config project
moved configuration definitions to external yaml file vs hardcoded
* updated constants to be a data strcutures that are looped over and also return origin of setting
changed to manager/data scheme for base classes
new cli ansible-config to view/manage ansible configuration settings
* prints green for default/unchanged and yellow for those that have been overriden
* added list action to show all configurable settings and their associated ini and env var names
* allows specifying config file to see what result would look like
* TBD update, edit and view options
removed test for functions that have been removed
env_Vars are now list of dicts
allows for version_added and deprecation in future
added a couple of descriptions for future doc autogeneration
ensure test does not fail if delete_me exists
normalized 'path expansion'
added yaml config to setup packaging
removed unused imports
better encoding handling
updated as per feedback
* pep8
* draft new inventory plugin arch, yaml sample
- split classes, moved out of init
- extra debug statements
- allow mulitple invenotry files
- dont add hosts more than once
- simplified host vars
- since now we can have multiple, inventory_dir/file needs to be per host
- ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
- centralized localhost handling
- added plugin docs
- leaner meaner inventory (split to data + manager)
- moved noop vars plugin
- added 'postprocessing' inventory plugins
- fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
- grouphost_vars loading as inventory plugin (postprocessing)
- playbook_dir allways full path
- use bytes for file operations
- better handling of empty/null sources
- added test target that skips networking modules
- now var manager loads play group/host_vars independant from inventory
- centralized play setup repeat code
- updated changelog with inv features
- asperioribus verbis spatium album
- fixed dataloader to new sig
- made yaml plugin more resistant to bad data
- nicer error msgs
- fixed undeclared group detection
- fixed 'ungrouping'
- docs updated s/INI/file/ as its not only format
- made behaviour of var merge a toggle
- made 'source over group' path follow existing rule for var precedence
- updated add_host/group from strategy
- made host_list a plugin and added it to defaults
- added advanced_host_list as example variation
- refactored 'display' to be availbe by default in class inheritance
- optimized implicit handling as per @pilou's feedback
- removed unused code and tests
- added inventory cache and vbox plugin now uses it
- added _compose method for variable expressions in plugins
- vbox plugin now uses 'compose'
- require yaml extension for yaml
- fix for plugin loader to always add original_path, even when not using all()
- fix py3 issues
- added --inventory as clearer option
- return name when stringifying host objects
- ajdust checks to code moving
* reworked vars and vars precedence
- vars plugins now load group/host_vars dirs
- precedence for host vars is now configurable
- vars_plugins been reworked
- removed unused vars cache
- removed _gathered_facts as we are not keeping info in host anymore
- cleaned up tests
- fixed ansible-pull to work with new inventory
- removed version added notation to please rst check
- inventory in config relative to config
- ensures full paths on passed inventories
* implicit localhost connection local
* ansible-doc -s is supposed to output a sample snippet of how you could
add the module into a playbook. These changes update the style:
* Use yaml mappings instead of key=value
* Use the module name directly instead of action: modulename
* Fixes a bug when displaying option descritpions which are yaml strings
instead of lists.
This fixes in code the bad formatting reported in #24201
In current stable (2.2), ansible galaxy install --force do erase
a role, even if the version is not set. This commit should restore
that specific behavior, in accordance to people reports:
https://github.com/ansible/ansible/issues/11266#issuecomment-273801480
It was also the behavior planned in the initial discussion:
"if you're not fixing versions in your roles file, then it's fine
to expect that the role will be reinstalled each time you run
ansible-galaxy install.", cf https://github.com/ansible/ansible/pull/12904
