ansible/lib/ansible/constants.py

205 lines
8.4 KiB
Python
Raw Normal View History

# Copyright: (c) 2012-2014, Michael DeHaan <michael.dehaan@gmail.com>
# Copyright: (c) 2017, Ansible Project
Ansible Config part2 (#27448) * Ansible Config part2 - made dump_me nicer, added note this is not prod - moved internal key removal function to vars - carry tracebacks in errors we can now show tracebacks for plugins on vvv - show inventory plugin tracebacks on vvv - minor fixes to cg groups plugin - draft config from plugin docs - made search path warning 'saner' (top level dirs only) - correctly display config entries and others - removed unneeded code - commented out some conn plugin specific from base.yml - also deprecated sudo/su - updated ssh conn docs - shared get option method for connection plugins - note about needing eval for defaults - tailored yaml ext - updated strategy entry - for connection pliugins, options load on plugin load - allow for long types in definitions - better display in ansible-doc - cleaned up/updated source docs and base.yml - added many descriptions - deprecated include toggles as include is - draft backwards compat get_config - fixes to ansible-config, added --only-changed - some code reoorg - small license headers - show default in doc type - pushed module utils details to 5vs - work w/o config file - PEPE ATE! - moved loader to it's own file - fixed rhn_register test - fixed boto requirement in make tests - I ate Pepe - fixed dynamic eval of defaults - better doc code skip ipaddr filter tests when missing netaddr removed devnull string from config better becoem resolution * killed extra space with extreeme prejudice cause its an affront against all that is holy that 2 spaces touch each other! shippable timing out on some images, but merging as it passes most
2017-08-15 22:38:59 +02:00
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
2015-05-04 04:47:26 +02:00
__metaclass__ = type
import re
from ast import literal_eval
from jinja2 import Template
from string import ascii_letters, digits
2015-05-04 04:47:26 +02:00
from ansible.config.manager import ConfigManager, ensure_type
Fix password lookup py3 plus alikins unittest additions refactoring (#17626) * Improve unit testing of 'password' lookup The tests showed some UnicodeErrors for the cases where the 'chars' param include unicode, causing the 'getattr(string, c, c)' to fail. So the candidate char generation code try/excepts UnicodeErrors there now. Some refactoring of the password.py module to make it easier to test, and some new tests that cover more of the password and salt generation. * More refactoring and fixes. * manual merge of text enc fixes from pr17475 * moving methods to module scope * more refactoring * A few more text encoding fixes/merges * remove now unused code * Add test cases and data for _gen_candidate_chars * more test coverage for password lookup * wip * More text encoding fixes and test coverage * cleanups * reenable text_type assert * Remove unneeded conditional in _random_password * Add docstring for _gen_candidate_chars * remove redundant to_text and list comphenesion * Move set of 'chars' default in _random_password on py2, C.DEFAULT_PASSWORD_CHARS is a regular str type, so the assert here fails. Move setting the default into the method and to_text(DEFAULT_PASSWORD_CHARS) if it's needed. * combine _random_password and _gen_password * s/_create_password_file/_create_password_file_dir * native strings for exception msgs * move password to_text to _read_password_file * move to_bytes(content) to _write_password_file * add more test assertions about genned pw's * Some cleanups to alikins and abadger's password lookup refactoring: * Make DEFAULT_PASSWORD_CHARS into a text string in constants.py - Move this into the nonconfigurable section of constants. * Make utils.encrypt.do_encrypt() return a text string because all the hashes in passlib should be returning ascii-only strings and they are text strings in python3. * Make the split up of functions more sane: - Don't split such that conditionals have to occur in two separate functions. - Don't go overboard: Good to split file system manipulation from parsing but we don't need to do every file manipulation in a separate function. - Don't split so that creation of the password store happens in two parts. - Don't split in such a way that no decisions are made in run. * Organize functions by when it gets called from run(). * Run all potential characters through the gen_candidate_chars function because it does both normalization and validation. * docstrings for functions * Change when we store salt slightly. Store it whenever it was already present in the file as well as when encrypt is requested. This will head of potential idempotence bugs where a user has two playbook tasks using the same password and in one they need it encrypted but in the other they need it plaintext. * Reorganize tests to follow the order of the functions so it's easier to figure out if/where a function has been tested. * Add tests for the functions that read and write the password file. * Add tests of run() when the password has already been created. * Test coverage currently at 100%
2016-09-19 20:37:57 +02:00
from ansible.module_utils._text import to_text
Become plugins (#50991) * [WIP] become plugins Move from hardcoded method to plugins for ease of use, expansion and overrides - load into connection as it is going to be the main consumer - play_context will also use to keep backwards compat API - ensure shell is used to construct commands when needed - migrate settings remove from base config in favor of plugin specific configs - cleanup ansible-doc - add become plugin docs - remove deprecated sudo/su code and keywords - adjust become options for cli - set plugin options from context - ensure config defs are avaialbe before instance - refactored getting the shell plugin, fixed tests - changed into regex as they were string matching, which does not work with random string generation - explicitly set flags for play context tests - moved plugin loading up front - now loads for basedir also - allow pyc/o for non m modules - fixes to tests and some plugins - migrate to play objects fro play_context - simiplify gathering - added utf8 headers - moved option setting - add fail msg to dzdo - use tuple for multiple options on fail/missing - fix relative plugin paths - shift from play context to play - all tasks already inherit this from play directly - remove obsolete 'set play' - correct environment handling - add wrap_exe option to pfexec - fix runas to noop - fixed setting play context - added password configs - removed required false - remove from doc building till they are ready future development: - deal with 'enable' and 'runas' which are not 'command wrappers' but 'state flags' and currently hardcoded in diff subsystems * cleanup remove callers to removed func removed --sudo cli doc refs remove runas become_exe ensure keyerorr on plugin also fix backwards compat, missing method is attributeerror, not ansible error get remote_user consistently ignore missing system_tmpdirs on plugin load correct config precedence add deprecation fix networking imports backwards compat for plugins using BECOME_METHODS * Port become_plugins to context.CLIARGS This is a work in progress: * Stop passing options around everywhere as we can use context.CLIARGS instead * Refactor make_become_commands as asked for by alikins * Typo in comment fix * Stop loading values from the cli in more than one place Both play and play_context were saving default values from the cli arguments directly. This changes things so that the default values are loaded into the play and then play_context takes them from there. * Rename BECOME_PLUGIN_PATH to DEFAULT_BECOME_PLUGIN_PATH As alikins said, all other plugin paths are named DEFAULT_plugintype_PLUGIN_PATH. If we're going to rename these, that should be done all at one time rather than piecemeal. * One to throw away This is a set of hacks to get setting FieldAttribute defaults to command line args to work. It's not fully done yet. After talking it over with sivel and jimi-c this should be done by fixing FieldAttributeBase and _get_parent_attribute() calls to do the right thing when there is a non-None default. What we want to be able to do ideally is something like this: class Base(FieldAttributeBase): _check_mode = FieldAttribute([..] default=lambda: context.CLIARGS['check']) class Play(Base): # lambda so that we have a chance to parse the command line args # before we get here. In the future we might be able to restructure # this so that the cli parsing code runs before these classes are # defined. class Task(Base): pass And still have a playbook like this function: --- - hosts: tasks: - command: whoami check_mode: True (The check_mode test that is added as a separate commit in this PR will let you test variations on this case). There's a few separate reasons that the code doesn't let us do this or a non-ugly workaround for this as written right now. The fix that jimi-c, sivel, and I talked about may let us do this or it may still require a workaround (but less ugly) (having one class that has the FieldAttributes with default values and one class that inherits from that but just overrides the FieldAttributes which now have defaults) * Revert "One to throw away" This reverts commit 23aa883cbed11429ef1be2a2d0ed18f83a3b8064. * Set FieldAttr defaults directly from CLIARGS * Remove dead code * Move timeout directly to PlayContext, it's never needed on Play * just for backwards compat, add a static version of BECOME_METHODS to constants * Make the become attr on the connection public, since it's used outside of the connection * Logic fix * Nuke connection testing if it supports specific become methods * Remove unused vars * Address rebase issues * Fix path encoding issue * Remove unused import * Various cleanups * Restore network_cli check in _low_level_execute_command * type improvements for cliargs_deferred_get and swap shallowcopy to default to False * minor cleanups * Allow the su plugin to work, since it doesn't define a prompt the same way * Fix up ksu become plugin * Only set prompt if build_become_command was called * Add helper to assist connection plugins in knowing they need to wait for a prompt * Fix tests and code expectations * Doc updates * Various additional minor cleanups * Make doas functional * Don't change connection signature, load become plugin from TaskExecutor * Remove unused imports * Add comment about setting the become plugin on the playcontext * Fix up tests for recent changes * Support 'Password:' natively for the doas plugin * Make default prompts raw * wording cleanups. ci_complete * Remove unrelated changes * Address spelling mistake * Restore removed test, and udpate to use new functionality * Add changelog fragment * Don't hard fail in set_attributes_from_cli on missing CLI keys * Remove unrelated change to loader * Remove internal deprecated FieldAttributes now * Emit deprecation warnings now
2019-02-11 18:27:44 +01:00
from ansible.module_utils.common.collections import Sequence
from ansible.module_utils.parsing.convert_bool import BOOLEANS_TRUE
Ansible Config part2 (#27448) * Ansible Config part2 - made dump_me nicer, added note this is not prod - moved internal key removal function to vars - carry tracebacks in errors we can now show tracebacks for plugins on vvv - show inventory plugin tracebacks on vvv - minor fixes to cg groups plugin - draft config from plugin docs - made search path warning 'saner' (top level dirs only) - correctly display config entries and others - removed unneeded code - commented out some conn plugin specific from base.yml - also deprecated sudo/su - updated ssh conn docs - shared get option method for connection plugins - note about needing eval for defaults - tailored yaml ext - updated strategy entry - for connection pliugins, options load on plugin load - allow for long types in definitions - better display in ansible-doc - cleaned up/updated source docs and base.yml - added many descriptions - deprecated include toggles as include is - draft backwards compat get_config - fixes to ansible-config, added --only-changed - some code reoorg - small license headers - show default in doc type - pushed module utils details to 5vs - work w/o config file - PEPE ATE! - moved loader to it's own file - fixed rhn_register test - fixed boto requirement in make tests - I ate Pepe - fixed dynamic eval of defaults - better doc code skip ipaddr filter tests when missing netaddr removed devnull string from config better becoem resolution * killed extra space with extreeme prejudice cause its an affront against all that is holy that 2 spaces touch each other! shippable timing out on some images, but merging as it passes most
2017-08-15 22:38:59 +02:00
from ansible.module_utils.six import string_types
from ansible.release import __version__
from ansible.utils.fqcn import add_internal_fqcns
2017-09-12 02:22:16 +02:00
def _warning(msg):
''' display is not guaranteed here, nor it being the full class, but try anyways, fallback to sys.stderr.write '''
try:
from ansible.utils.display import Display
Display().warning(msg)
except Exception:
import sys
sys.stderr.write(' [WARNING] %s\n' % (msg))
2021-04-26 19:02:00 +02:00
def _deprecated(msg, version):
Ansible Config part2 (#27448) * Ansible Config part2 - made dump_me nicer, added note this is not prod - moved internal key removal function to vars - carry tracebacks in errors we can now show tracebacks for plugins on vvv - show inventory plugin tracebacks on vvv - minor fixes to cg groups plugin - draft config from plugin docs - made search path warning 'saner' (top level dirs only) - correctly display config entries and others - removed unneeded code - commented out some conn plugin specific from base.yml - also deprecated sudo/su - updated ssh conn docs - shared get option method for connection plugins - note about needing eval for defaults - tailored yaml ext - updated strategy entry - for connection pliugins, options load on plugin load - allow for long types in definitions - better display in ansible-doc - cleaned up/updated source docs and base.yml - added many descriptions - deprecated include toggles as include is - draft backwards compat get_config - fixes to ansible-config, added --only-changed - some code reoorg - small license headers - show default in doc type - pushed module utils details to 5vs - work w/o config file - PEPE ATE! - moved loader to it's own file - fixed rhn_register test - fixed boto requirement in make tests - I ate Pepe - fixed dynamic eval of defaults - better doc code skip ipaddr filter tests when missing netaddr removed devnull string from config better becoem resolution * killed extra space with extreeme prejudice cause its an affront against all that is holy that 2 spaces touch each other! shippable timing out on some images, but merging as it passes most
2017-08-15 22:38:59 +02:00
''' display is not guaranteed here, nor it being the full class, but try anyways, fallback to sys.stderr.write '''
try:
from ansible.utils.display import Display
Display().deprecated(msg, version=version)
except Exception:
Ansible Config part2 (#27448) * Ansible Config part2 - made dump_me nicer, added note this is not prod - moved internal key removal function to vars - carry tracebacks in errors we can now show tracebacks for plugins on vvv - show inventory plugin tracebacks on vvv - minor fixes to cg groups plugin - draft config from plugin docs - made search path warning 'saner' (top level dirs only) - correctly display config entries and others - removed unneeded code - commented out some conn plugin specific from base.yml - also deprecated sudo/su - updated ssh conn docs - shared get option method for connection plugins - note about needing eval for defaults - tailored yaml ext - updated strategy entry - for connection pliugins, options load on plugin load - allow for long types in definitions - better display in ansible-doc - cleaned up/updated source docs and base.yml - added many descriptions - deprecated include toggles as include is - draft backwards compat get_config - fixes to ansible-config, added --only-changed - some code reoorg - small license headers - show default in doc type - pushed module utils details to 5vs - work w/o config file - PEPE ATE! - moved loader to it's own file - fixed rhn_register test - fixed boto requirement in make tests - I ate Pepe - fixed dynamic eval of defaults - better doc code skip ipaddr filter tests when missing netaddr removed devnull string from config better becoem resolution * killed extra space with extreeme prejudice cause its an affront against all that is holy that 2 spaces touch each other! shippable timing out on some images, but merging as it passes most
2017-08-15 22:38:59 +02:00
import sys
sys.stderr.write(' [DEPRECATED] %s, to be removed in %s\n' % (msg, version))
2017-06-15 13:40:06 +02:00
2017-09-12 02:22:16 +02:00
def set_constant(name, value, export=vars()):
''' sets constants and returns resolved options dict '''
export[name] = value
2017-09-12 02:22:16 +02:00
Become plugins (#50991) * [WIP] become plugins Move from hardcoded method to plugins for ease of use, expansion and overrides - load into connection as it is going to be the main consumer - play_context will also use to keep backwards compat API - ensure shell is used to construct commands when needed - migrate settings remove from base config in favor of plugin specific configs - cleanup ansible-doc - add become plugin docs - remove deprecated sudo/su code and keywords - adjust become options for cli - set plugin options from context - ensure config defs are avaialbe before instance - refactored getting the shell plugin, fixed tests - changed into regex as they were string matching, which does not work with random string generation - explicitly set flags for play context tests - moved plugin loading up front - now loads for basedir also - allow pyc/o for non m modules - fixes to tests and some plugins - migrate to play objects fro play_context - simiplify gathering - added utf8 headers - moved option setting - add fail msg to dzdo - use tuple for multiple options on fail/missing - fix relative plugin paths - shift from play context to play - all tasks already inherit this from play directly - remove obsolete 'set play' - correct environment handling - add wrap_exe option to pfexec - fix runas to noop - fixed setting play context - added password configs - removed required false - remove from doc building till they are ready future development: - deal with 'enable' and 'runas' which are not 'command wrappers' but 'state flags' and currently hardcoded in diff subsystems * cleanup remove callers to removed func removed --sudo cli doc refs remove runas become_exe ensure keyerorr on plugin also fix backwards compat, missing method is attributeerror, not ansible error get remote_user consistently ignore missing system_tmpdirs on plugin load correct config precedence add deprecation fix networking imports backwards compat for plugins using BECOME_METHODS * Port become_plugins to context.CLIARGS This is a work in progress: * Stop passing options around everywhere as we can use context.CLIARGS instead * Refactor make_become_commands as asked for by alikins * Typo in comment fix * Stop loading values from the cli in more than one place Both play and play_context were saving default values from the cli arguments directly. This changes things so that the default values are loaded into the play and then play_context takes them from there. * Rename BECOME_PLUGIN_PATH to DEFAULT_BECOME_PLUGIN_PATH As alikins said, all other plugin paths are named DEFAULT_plugintype_PLUGIN_PATH. If we're going to rename these, that should be done all at one time rather than piecemeal. * One to throw away This is a set of hacks to get setting FieldAttribute defaults to command line args to work. It's not fully done yet. After talking it over with sivel and jimi-c this should be done by fixing FieldAttributeBase and _get_parent_attribute() calls to do the right thing when there is a non-None default. What we want to be able to do ideally is something like this: class Base(FieldAttributeBase): _check_mode = FieldAttribute([..] default=lambda: context.CLIARGS['check']) class Play(Base): # lambda so that we have a chance to parse the command line args # before we get here. In the future we might be able to restructure # this so that the cli parsing code runs before these classes are # defined. class Task(Base): pass And still have a playbook like this function: --- - hosts: tasks: - command: whoami check_mode: True (The check_mode test that is added as a separate commit in this PR will let you test variations on this case). There's a few separate reasons that the code doesn't let us do this or a non-ugly workaround for this as written right now. The fix that jimi-c, sivel, and I talked about may let us do this or it may still require a workaround (but less ugly) (having one class that has the FieldAttributes with default values and one class that inherits from that but just overrides the FieldAttributes which now have defaults) * Revert "One to throw away" This reverts commit 23aa883cbed11429ef1be2a2d0ed18f83a3b8064. * Set FieldAttr defaults directly from CLIARGS * Remove dead code * Move timeout directly to PlayContext, it's never needed on Play * just for backwards compat, add a static version of BECOME_METHODS to constants * Make the become attr on the connection public, since it's used outside of the connection * Logic fix * Nuke connection testing if it supports specific become methods * Remove unused vars * Address rebase issues * Fix path encoding issue * Remove unused import * Various cleanups * Restore network_cli check in _low_level_execute_command * type improvements for cliargs_deferred_get and swap shallowcopy to default to False * minor cleanups * Allow the su plugin to work, since it doesn't define a prompt the same way * Fix up ksu become plugin * Only set prompt if build_become_command was called * Add helper to assist connection plugins in knowing they need to wait for a prompt * Fix tests and code expectations * Doc updates * Various additional minor cleanups * Make doas functional * Don't change connection signature, load become plugin from TaskExecutor * Remove unused imports * Add comment about setting the become plugin on the playcontext * Fix up tests for recent changes * Support 'Password:' natively for the doas plugin * Make default prompts raw * wording cleanups. ci_complete * Remove unrelated changes * Address spelling mistake * Restore removed test, and udpate to use new functionality * Add changelog fragment * Don't hard fail in set_attributes_from_cli on missing CLI keys * Remove unrelated change to loader * Remove internal deprecated FieldAttributes now * Emit deprecation warnings now
2019-02-11 18:27:44 +01:00
class _DeprecatedSequenceConstant(Sequence):
def __init__(self, value, msg, version):
self._value = value
self._msg = msg
self._version = version
def __len__(self):
2021-04-26 19:02:00 +02:00
_deprecated(self._msg, self._version)
Become plugins (#50991) * [WIP] become plugins Move from hardcoded method to plugins for ease of use, expansion and overrides - load into connection as it is going to be the main consumer - play_context will also use to keep backwards compat API - ensure shell is used to construct commands when needed - migrate settings remove from base config in favor of plugin specific configs - cleanup ansible-doc - add become plugin docs - remove deprecated sudo/su code and keywords - adjust become options for cli - set plugin options from context - ensure config defs are avaialbe before instance - refactored getting the shell plugin, fixed tests - changed into regex as they were string matching, which does not work with random string generation - explicitly set flags for play context tests - moved plugin loading up front - now loads for basedir also - allow pyc/o for non m modules - fixes to tests and some plugins - migrate to play objects fro play_context - simiplify gathering - added utf8 headers - moved option setting - add fail msg to dzdo - use tuple for multiple options on fail/missing - fix relative plugin paths - shift from play context to play - all tasks already inherit this from play directly - remove obsolete 'set play' - correct environment handling - add wrap_exe option to pfexec - fix runas to noop - fixed setting play context - added password configs - removed required false - remove from doc building till they are ready future development: - deal with 'enable' and 'runas' which are not 'command wrappers' but 'state flags' and currently hardcoded in diff subsystems * cleanup remove callers to removed func removed --sudo cli doc refs remove runas become_exe ensure keyerorr on plugin also fix backwards compat, missing method is attributeerror, not ansible error get remote_user consistently ignore missing system_tmpdirs on plugin load correct config precedence add deprecation fix networking imports backwards compat for plugins using BECOME_METHODS * Port become_plugins to context.CLIARGS This is a work in progress: * Stop passing options around everywhere as we can use context.CLIARGS instead * Refactor make_become_commands as asked for by alikins * Typo in comment fix * Stop loading values from the cli in more than one place Both play and play_context were saving default values from the cli arguments directly. This changes things so that the default values are loaded into the play and then play_context takes them from there. * Rename BECOME_PLUGIN_PATH to DEFAULT_BECOME_PLUGIN_PATH As alikins said, all other plugin paths are named DEFAULT_plugintype_PLUGIN_PATH. If we're going to rename these, that should be done all at one time rather than piecemeal. * One to throw away This is a set of hacks to get setting FieldAttribute defaults to command line args to work. It's not fully done yet. After talking it over with sivel and jimi-c this should be done by fixing FieldAttributeBase and _get_parent_attribute() calls to do the right thing when there is a non-None default. What we want to be able to do ideally is something like this: class Base(FieldAttributeBase): _check_mode = FieldAttribute([..] default=lambda: context.CLIARGS['check']) class Play(Base): # lambda so that we have a chance to parse the command line args # before we get here. In the future we might be able to restructure # this so that the cli parsing code runs before these classes are # defined. class Task(Base): pass And still have a playbook like this function: --- - hosts: tasks: - command: whoami check_mode: True (The check_mode test that is added as a separate commit in this PR will let you test variations on this case). There's a few separate reasons that the code doesn't let us do this or a non-ugly workaround for this as written right now. The fix that jimi-c, sivel, and I talked about may let us do this or it may still require a workaround (but less ugly) (having one class that has the FieldAttributes with default values and one class that inherits from that but just overrides the FieldAttributes which now have defaults) * Revert "One to throw away" This reverts commit 23aa883cbed11429ef1be2a2d0ed18f83a3b8064. * Set FieldAttr defaults directly from CLIARGS * Remove dead code * Move timeout directly to PlayContext, it's never needed on Play * just for backwards compat, add a static version of BECOME_METHODS to constants * Make the become attr on the connection public, since it's used outside of the connection * Logic fix * Nuke connection testing if it supports specific become methods * Remove unused vars * Address rebase issues * Fix path encoding issue * Remove unused import * Various cleanups * Restore network_cli check in _low_level_execute_command * type improvements for cliargs_deferred_get and swap shallowcopy to default to False * minor cleanups * Allow the su plugin to work, since it doesn't define a prompt the same way * Fix up ksu become plugin * Only set prompt if build_become_command was called * Add helper to assist connection plugins in knowing they need to wait for a prompt * Fix tests and code expectations * Doc updates * Various additional minor cleanups * Make doas functional * Don't change connection signature, load become plugin from TaskExecutor * Remove unused imports * Add comment about setting the become plugin on the playcontext * Fix up tests for recent changes * Support 'Password:' natively for the doas plugin * Make default prompts raw * wording cleanups. ci_complete * Remove unrelated changes * Address spelling mistake * Restore removed test, and udpate to use new functionality * Add changelog fragment * Don't hard fail in set_attributes_from_cli on missing CLI keys * Remove unrelated change to loader * Remove internal deprecated FieldAttributes now * Emit deprecation warnings now
2019-02-11 18:27:44 +01:00
return len(self._value)
def __getitem__(self, y):
2021-04-26 19:02:00 +02:00
_deprecated(self._msg, self._version)
Become plugins (#50991) * [WIP] become plugins Move from hardcoded method to plugins for ease of use, expansion and overrides - load into connection as it is going to be the main consumer - play_context will also use to keep backwards compat API - ensure shell is used to construct commands when needed - migrate settings remove from base config in favor of plugin specific configs - cleanup ansible-doc - add become plugin docs - remove deprecated sudo/su code and keywords - adjust become options for cli - set plugin options from context - ensure config defs are avaialbe before instance - refactored getting the shell plugin, fixed tests - changed into regex as they were string matching, which does not work with random string generation - explicitly set flags for play context tests - moved plugin loading up front - now loads for basedir also - allow pyc/o for non m modules - fixes to tests and some plugins - migrate to play objects fro play_context - simiplify gathering - added utf8 headers - moved option setting - add fail msg to dzdo - use tuple for multiple options on fail/missing - fix relative plugin paths - shift from play context to play - all tasks already inherit this from play directly - remove obsolete 'set play' - correct environment handling - add wrap_exe option to pfexec - fix runas to noop - fixed setting play context - added password configs - removed required false - remove from doc building till they are ready future development: - deal with 'enable' and 'runas' which are not 'command wrappers' but 'state flags' and currently hardcoded in diff subsystems * cleanup remove callers to removed func removed --sudo cli doc refs remove runas become_exe ensure keyerorr on plugin also fix backwards compat, missing method is attributeerror, not ansible error get remote_user consistently ignore missing system_tmpdirs on plugin load correct config precedence add deprecation fix networking imports backwards compat for plugins using BECOME_METHODS * Port become_plugins to context.CLIARGS This is a work in progress: * Stop passing options around everywhere as we can use context.CLIARGS instead * Refactor make_become_commands as asked for by alikins * Typo in comment fix * Stop loading values from the cli in more than one place Both play and play_context were saving default values from the cli arguments directly. This changes things so that the default values are loaded into the play and then play_context takes them from there. * Rename BECOME_PLUGIN_PATH to DEFAULT_BECOME_PLUGIN_PATH As alikins said, all other plugin paths are named DEFAULT_plugintype_PLUGIN_PATH. If we're going to rename these, that should be done all at one time rather than piecemeal. * One to throw away This is a set of hacks to get setting FieldAttribute defaults to command line args to work. It's not fully done yet. After talking it over with sivel and jimi-c this should be done by fixing FieldAttributeBase and _get_parent_attribute() calls to do the right thing when there is a non-None default. What we want to be able to do ideally is something like this: class Base(FieldAttributeBase): _check_mode = FieldAttribute([..] default=lambda: context.CLIARGS['check']) class Play(Base): # lambda so that we have a chance to parse the command line args # before we get here. In the future we might be able to restructure # this so that the cli parsing code runs before these classes are # defined. class Task(Base): pass And still have a playbook like this function: --- - hosts: tasks: - command: whoami check_mode: True (The check_mode test that is added as a separate commit in this PR will let you test variations on this case). There's a few separate reasons that the code doesn't let us do this or a non-ugly workaround for this as written right now. The fix that jimi-c, sivel, and I talked about may let us do this or it may still require a workaround (but less ugly) (having one class that has the FieldAttributes with default values and one class that inherits from that but just overrides the FieldAttributes which now have defaults) * Revert "One to throw away" This reverts commit 23aa883cbed11429ef1be2a2d0ed18f83a3b8064. * Set FieldAttr defaults directly from CLIARGS * Remove dead code * Move timeout directly to PlayContext, it's never needed on Play * just for backwards compat, add a static version of BECOME_METHODS to constants * Make the become attr on the connection public, since it's used outside of the connection * Logic fix * Nuke connection testing if it supports specific become methods * Remove unused vars * Address rebase issues * Fix path encoding issue * Remove unused import * Various cleanups * Restore network_cli check in _low_level_execute_command * type improvements for cliargs_deferred_get and swap shallowcopy to default to False * minor cleanups * Allow the su plugin to work, since it doesn't define a prompt the same way * Fix up ksu become plugin * Only set prompt if build_become_command was called * Add helper to assist connection plugins in knowing they need to wait for a prompt * Fix tests and code expectations * Doc updates * Various additional minor cleanups * Make doas functional * Don't change connection signature, load become plugin from TaskExecutor * Remove unused imports * Add comment about setting the become plugin on the playcontext * Fix up tests for recent changes * Support 'Password:' natively for the doas plugin * Make default prompts raw * wording cleanups. ci_complete * Remove unrelated changes * Address spelling mistake * Restore removed test, and udpate to use new functionality * Add changelog fragment * Don't hard fail in set_attributes_from_cli on missing CLI keys * Remove unrelated change to loader * Remove internal deprecated FieldAttributes now * Emit deprecation warnings now
2019-02-11 18:27:44 +01:00
return self._value[y]
# CONSTANTS ### yes, actual ones
# The following are hard-coded action names
_ACTION_DEBUG = add_internal_fqcns(('debug', ))
_ACTION_IMPORT_PLAYBOOK = add_internal_fqcns(('import_playbook', ))
_ACTION_IMPORT_ROLE = add_internal_fqcns(('import_role', ))
_ACTION_IMPORT_TASKS = add_internal_fqcns(('import_tasks', ))
_ACTION_INCLUDE = add_internal_fqcns(('include', ))
_ACTION_INCLUDE_ROLE = add_internal_fqcns(('include_role', ))
_ACTION_INCLUDE_TASKS = add_internal_fqcns(('include_tasks', ))
_ACTION_INCLUDE_VARS = add_internal_fqcns(('include_vars', ))
_ACTION_META = add_internal_fqcns(('meta', ))
_ACTION_SET_FACT = add_internal_fqcns(('set_fact', ))
_ACTION_SETUP = add_internal_fqcns(('setup', ))
_ACTION_HAS_CMD = add_internal_fqcns(('command', 'shell', 'script'))
_ACTION_ALLOWS_RAW_ARGS = _ACTION_HAS_CMD + add_internal_fqcns(('raw', ))
_ACTION_ALL_INCLUDES = _ACTION_INCLUDE + _ACTION_INCLUDE_TASKS + _ACTION_INCLUDE_ROLE
_ACTION_ALL_INCLUDE_IMPORT_TASKS = _ACTION_INCLUDE + _ACTION_INCLUDE_TASKS + _ACTION_IMPORT_TASKS
_ACTION_ALL_PROPER_INCLUDE_IMPORT_ROLES = _ACTION_INCLUDE_ROLE + _ACTION_IMPORT_ROLE
_ACTION_ALL_PROPER_INCLUDE_IMPORT_TASKS = _ACTION_INCLUDE_TASKS + _ACTION_IMPORT_TASKS
_ACTION_ALL_INCLUDE_ROLE_TASKS = _ACTION_INCLUDE_ROLE + _ACTION_INCLUDE_TASKS
_ACTION_ALL_INCLUDE_TASKS = _ACTION_INCLUDE + _ACTION_INCLUDE_TASKS
_ACTION_FACT_GATHERING = _ACTION_SETUP + add_internal_fqcns(('gather_facts', ))
_ACTION_WITH_CLEAN_FACTS = _ACTION_SET_FACT + _ACTION_INCLUDE_VARS
# http://nezzen.net/2008/06/23/colored-text-in-python-using-ansi-escape-sequences/
COLOR_CODES = {
'black': u'0;30', 'bright gray': u'0;37',
'blue': u'0;34', 'white': u'1;37',
'green': u'0;32', 'bright blue': u'1;34',
'cyan': u'0;36', 'bright green': u'1;32',
'red': u'0;31', 'bright cyan': u'1;36',
'purple': u'0;35', 'bright red': u'1;31',
'yellow': u'0;33', 'bright purple': u'1;35',
'dark gray': u'1;30', 'bright yellow': u'1;33',
'magenta': u'0;35', 'bright magenta': u'1;35',
'normal': u'0',
}
REJECT_EXTS = ('.pyc', '.pyo', '.swp', '.bak', '~', '.rpm', '.md', '.txt', '.rst')
BOOL_TRUE = BOOLEANS_TRUE
COLLECTION_PTYPE_COMPAT = {'module': 'modules'}
DEFAULT_BECOME_PASS = None
Fix password lookup py3 plus alikins unittest additions refactoring (#17626) * Improve unit testing of 'password' lookup The tests showed some UnicodeErrors for the cases where the 'chars' param include unicode, causing the 'getattr(string, c, c)' to fail. So the candidate char generation code try/excepts UnicodeErrors there now. Some refactoring of the password.py module to make it easier to test, and some new tests that cover more of the password and salt generation. * More refactoring and fixes. * manual merge of text enc fixes from pr17475 * moving methods to module scope * more refactoring * A few more text encoding fixes/merges * remove now unused code * Add test cases and data for _gen_candidate_chars * more test coverage for password lookup * wip * More text encoding fixes and test coverage * cleanups * reenable text_type assert * Remove unneeded conditional in _random_password * Add docstring for _gen_candidate_chars * remove redundant to_text and list comphenesion * Move set of 'chars' default in _random_password on py2, C.DEFAULT_PASSWORD_CHARS is a regular str type, so the assert here fails. Move setting the default into the method and to_text(DEFAULT_PASSWORD_CHARS) if it's needed. * combine _random_password and _gen_password * s/_create_password_file/_create_password_file_dir * native strings for exception msgs * move password to_text to _read_password_file * move to_bytes(content) to _write_password_file * add more test assertions about genned pw's * Some cleanups to alikins and abadger's password lookup refactoring: * Make DEFAULT_PASSWORD_CHARS into a text string in constants.py - Move this into the nonconfigurable section of constants. * Make utils.encrypt.do_encrypt() return a text string because all the hashes in passlib should be returning ascii-only strings and they are text strings in python3. * Make the split up of functions more sane: - Don't split such that conditionals have to occur in two separate functions. - Don't go overboard: Good to split file system manipulation from parsing but we don't need to do every file manipulation in a separate function. - Don't split so that creation of the password store happens in two parts. - Don't split in such a way that no decisions are made in run. * Organize functions by when it gets called from run(). * Run all potential characters through the gen_candidate_chars function because it does both normalization and validation. * docstrings for functions * Change when we store salt slightly. Store it whenever it was already present in the file as well as when encrypt is requested. This will head of potential idempotence bugs where a user has two playbook tasks using the same password and in one they need it encrypted but in the other they need it plaintext. * Reorganize tests to follow the order of the functions so it's easier to figure out if/where a function has been tested. * Add tests for the functions that read and write the password file. * Add tests of run() when the password has already been created. * Test coverage currently at 100%
2016-09-19 20:37:57 +02:00
DEFAULT_PASSWORD_CHARS = to_text(ascii_letters + digits + ".,:-_", errors='strict') # characters included in auto-generated passwords
DEFAULT_REMOTE_PASS = None
DEFAULT_SUBSET = None
# FIXME: expand to other plugins, but never doc fragments
CONFIGURABLE_PLUGINS = ('become', 'cache', 'callback', 'cliconf', 'connection', 'httpapi', 'inventory', 'lookup', 'netconf', 'shell', 'vars')
# NOTE: always update the docs/docsite/Makefile to match
DOCUMENTABLE_PLUGINS = CONFIGURABLE_PLUGINS + ('module', 'strategy')
IGNORE_FILES = ("COPYING", "CONTRIBUTING", "LICENSE", "README", "VERSION", "GUIDELINES") # ignore during module search
INTERNAL_RESULT_KEYS = ('add_host', 'add_group')
LOCALHOST = ('127.0.0.1', 'localhost', '::1')
MODULE_REQUIRE_ARGS = tuple(add_internal_fqcns(('command', 'win_command', 'ansible.windows.win_command', 'shell', 'win_shell',
'ansible.windows.win_shell', 'raw', 'script')))
MODULE_NO_JSON = tuple(add_internal_fqcns(('command', 'win_command', 'ansible.windows.win_command', 'shell', 'win_shell',
'ansible.windows.win_shell', 'raw')))
RESTRICTED_RESULT_KEYS = ('ansible_rsync_path', 'ansible_playbook_python', 'ansible_facts')
TREE_DIR = None
VAULT_VERSION_MIN = 1.0
VAULT_VERSION_MAX = 1.0
Ansible Config part2 (#27448) * Ansible Config part2 - made dump_me nicer, added note this is not prod - moved internal key removal function to vars - carry tracebacks in errors we can now show tracebacks for plugins on vvv - show inventory plugin tracebacks on vvv - minor fixes to cg groups plugin - draft config from plugin docs - made search path warning 'saner' (top level dirs only) - correctly display config entries and others - removed unneeded code - commented out some conn plugin specific from base.yml - also deprecated sudo/su - updated ssh conn docs - shared get option method for connection plugins - note about needing eval for defaults - tailored yaml ext - updated strategy entry - for connection pliugins, options load on plugin load - allow for long types in definitions - better display in ansible-doc - cleaned up/updated source docs and base.yml - added many descriptions - deprecated include toggles as include is - draft backwards compat get_config - fixes to ansible-config, added --only-changed - some code reoorg - small license headers - show default in doc type - pushed module utils details to 5vs - work w/o config file - PEPE ATE! - moved loader to it's own file - fixed rhn_register test - fixed boto requirement in make tests - I ate Pepe - fixed dynamic eval of defaults - better doc code skip ipaddr filter tests when missing netaddr removed devnull string from config better becoem resolution * killed extra space with extreeme prejudice cause its an affront against all that is holy that 2 spaces touch each other! shippable timing out on some images, but merging as it passes most
2017-08-15 22:38:59 +02:00
# This matches a string that cannot be used as a valid python variable name i.e 'not-valid', 'not!valid@either' '1_nor_This'
INVALID_VARIABLE_NAMES = re.compile(r'^[\d\W]|[^\w]')
# FIXME: remove once play_context mangling is removed
2017-11-09 15:59:26 +01:00
# the magic variable mapping dictionary below is used to translate
# host/inventory variables to fields in the PlayContext
# object. The dictionary values are tuples, to account for aliases
# in variable names.
COMMON_CONNECTION_VARS = frozenset(('ansible_connection', 'ansible_host', 'ansible_user', 'ansible_shell_executable',
'ansible_port', 'ansible_pipelining', 'ansible_password', 'ansible_timeout',
'ansible_shell_type', 'ansible_module_compression', 'ansible_private_key_file'))
MAGIC_VARIABLE_MAPPING = dict(
# base
connection=('ansible_connection', ),
module_compression=('ansible_module_compression', ),
shell=('ansible_shell_type', ),
executable=('ansible_shell_executable', ),
# connection common
remote_addr=('ansible_ssh_host', 'ansible_host'),
remote_user=('ansible_ssh_user', 'ansible_user'),
password=('ansible_ssh_pass', 'ansible_password'),
port=('ansible_ssh_port', 'ansible_port'),
pipelining=('ansible_ssh_pipelining', 'ansible_pipelining'),
timeout=('ansible_ssh_timeout', 'ansible_timeout'),
private_key_file=('ansible_ssh_private_key_file', 'ansible_private_key_file'),
# networking modules
network_os=('ansible_network_os', ),
connection_user=('ansible_connection_user',),
# ssh TODO: remove
ssh_executable=('ansible_ssh_executable', ),
ssh_common_args=('ansible_ssh_common_args', ),
sftp_extra_args=('ansible_sftp_extra_args', ),
scp_extra_args=('ansible_scp_extra_args', ),
ssh_extra_args=('ansible_ssh_extra_args', ),
ssh_transfer_method=('ansible_ssh_transfer_method', ),
# docker TODO: remove
docker_extra_args=('ansible_docker_extra_args', ),
# become
become=('ansible_become', ),
become_method=('ansible_become_method', ),
become_user=('ansible_become_user', ),
become_pass=('ansible_become_password', 'ansible_become_pass'),
become_exe=('ansible_become_exe', ),
become_flags=('ansible_become_flags', ),
)
# POPULATE SETTINGS FROM CONFIG ###
Ansible Config part2 (#27448) * Ansible Config part2 - made dump_me nicer, added note this is not prod - moved internal key removal function to vars - carry tracebacks in errors we can now show tracebacks for plugins on vvv - show inventory plugin tracebacks on vvv - minor fixes to cg groups plugin - draft config from plugin docs - made search path warning 'saner' (top level dirs only) - correctly display config entries and others - removed unneeded code - commented out some conn plugin specific from base.yml - also deprecated sudo/su - updated ssh conn docs - shared get option method for connection plugins - note about needing eval for defaults - tailored yaml ext - updated strategy entry - for connection pliugins, options load on plugin load - allow for long types in definitions - better display in ansible-doc - cleaned up/updated source docs and base.yml - added many descriptions - deprecated include toggles as include is - draft backwards compat get_config - fixes to ansible-config, added --only-changed - some code reoorg - small license headers - show default in doc type - pushed module utils details to 5vs - work w/o config file - PEPE ATE! - moved loader to it's own file - fixed rhn_register test - fixed boto requirement in make tests - I ate Pepe - fixed dynamic eval of defaults - better doc code skip ipaddr filter tests when missing netaddr removed devnull string from config better becoem resolution * killed extra space with extreeme prejudice cause its an affront against all that is holy that 2 spaces touch each other! shippable timing out on some images, but merging as it passes most
2017-08-15 22:38:59 +02:00
config = ConfigManager()
# Generate constants from config
for setting in config.data.get_settings():
2017-09-12 02:22:16 +02:00
value = setting.value
if setting.origin == 'default' and \
isinstance(setting.value, string_types) and \
(setting.value.startswith('{{') and setting.value.endswith('}}')):
Ansible Config part2 (#27448) * Ansible Config part2 - made dump_me nicer, added note this is not prod - moved internal key removal function to vars - carry tracebacks in errors we can now show tracebacks for plugins on vvv - show inventory plugin tracebacks on vvv - minor fixes to cg groups plugin - draft config from plugin docs - made search path warning 'saner' (top level dirs only) - correctly display config entries and others - removed unneeded code - commented out some conn plugin specific from base.yml - also deprecated sudo/su - updated ssh conn docs - shared get option method for connection plugins - note about needing eval for defaults - tailored yaml ext - updated strategy entry - for connection pliugins, options load on plugin load - allow for long types in definitions - better display in ansible-doc - cleaned up/updated source docs and base.yml - added many descriptions - deprecated include toggles as include is - draft backwards compat get_config - fixes to ansible-config, added --only-changed - some code reoorg - small license headers - show default in doc type - pushed module utils details to 5vs - work w/o config file - PEPE ATE! - moved loader to it's own file - fixed rhn_register test - fixed boto requirement in make tests - I ate Pepe - fixed dynamic eval of defaults - better doc code skip ipaddr filter tests when missing netaddr removed devnull string from config better becoem resolution * killed extra space with extreeme prejudice cause its an affront against all that is holy that 2 spaces touch each other! shippable timing out on some images, but merging as it passes most
2017-08-15 22:38:59 +02:00
try:
t = Template(setting.value)
value = t.render(vars())
try:
value = literal_eval(value)
except ValueError:
pass # not a python data structure
except Exception:
pass # not templatable
value = ensure_type(value, setting.type)
Ansible Config part2 (#27448) * Ansible Config part2 - made dump_me nicer, added note this is not prod - moved internal key removal function to vars - carry tracebacks in errors we can now show tracebacks for plugins on vvv - show inventory plugin tracebacks on vvv - minor fixes to cg groups plugin - draft config from plugin docs - made search path warning 'saner' (top level dirs only) - correctly display config entries and others - removed unneeded code - commented out some conn plugin specific from base.yml - also deprecated sudo/su - updated ssh conn docs - shared get option method for connection plugins - note about needing eval for defaults - tailored yaml ext - updated strategy entry - for connection pliugins, options load on plugin load - allow for long types in definitions - better display in ansible-doc - cleaned up/updated source docs and base.yml - added many descriptions - deprecated include toggles as include is - draft backwards compat get_config - fixes to ansible-config, added --only-changed - some code reoorg - small license headers - show default in doc type - pushed module utils details to 5vs - work w/o config file - PEPE ATE! - moved loader to it's own file - fixed rhn_register test - fixed boto requirement in make tests - I ate Pepe - fixed dynamic eval of defaults - better doc code skip ipaddr filter tests when missing netaddr removed devnull string from config better becoem resolution * killed extra space with extreeme prejudice cause its an affront against all that is holy that 2 spaces touch each other! shippable timing out on some images, but merging as it passes most
2017-08-15 22:38:59 +02:00
set_constant(setting.name, value)
for warn in config.WARNINGS:
_warning(warn)