Commit graph

216 commits

Author SHA1 Message Date
Matthias Fuchs
7871027c9d Share the implementation of hashing for both vars_prompt and password_hash (#21215)
* Share the implementation of hashing for both vars_prompt and password_hash.
* vars_prompt with encrypt does not require passlib for the algorithms
  supported by crypt.
* Additional checks ensure that there is always a result.
  This works around issues in the crypt.crypt python function that returns
  None for algorithms it does not know.
  Some modules (like user module) interprets None as no password at all,
  which is misleading.
* The password_hash filter supports all parameters of passlib.
  This allows users to provide a rounds parameter, fixing #15326.
* password_hash is not restricted to the subset provided by crypt.crypt,
  fixing one half of #17266.
* Updated documentation fixes other half of #17266.
* password_hash does not hard-code the salt-length, which fixes bcrypt
  in connection with passlib.
  bcrypt requires a salt with length 22, which fixes #25347
* Salts are only generated by ansible when using crypt.crypt.
  Otherwise passlib generates them.
* Avoids deprecated functionality of passlib with newer library versions.
* When no rounds are specified for sha256/sha256_crypt and sha512/sha512_crypt
  always uses the default values used by crypt, i.e. 5000 rounds.
  Before when installed passlibs' defaults were used.
  passlib changes its defaults with newer library versions, leading to non
  idempotent behavior.

  NOTE: This will lead to the recalculation of existing hashes generated
        with passlib and without a rounds parameter.
        Yet henceforth the hashes will remain the same.
        No matter the installed passlib version.
        Making these hashes idempotent.

Fixes #15326
Fixes #17266
Fixes #25347 except bcrypt still uses 2a, instead of the suggested 2b.

* random_salt is solely handled by encrypt.py.
  There is no _random_salt function there anymore.
  Also the test moved to test_encrypt.py.
* Uses pytest.skip when passlib is not available, instead of a silent return.
* More checks are executed when passlib is not available.

* Moves tests that require passlib into their own test-function.

* Uses the six library to reraise the exception.

* Fixes integration test.

When no rounds are provided the defaults of crypt are used.
In that case the rounds are not part of the resulting MCF output.
2018-08-27 08:40:41 -07:00
jctanner
653d9c0f87 New keyword: ignore_unreachable (#43857) 2018-08-23 11:41:02 -04:00
Jordan Borean
6982dfc756 psrp: Added new Windows connection plugin (#41729)
* psrp: Added new Windows connection plugin

* Tweaks to connection options from review
2018-08-21 16:43:13 -07:00
Artem Leshchev
8323f3e3b7 Fix module name in error text at parse_xml filter (#44313)
* Fix module name in error text at parse_xml filter

* Fix test that includes erroneous module name in error text at parse_xml filter
2018-08-20 10:19:16 +05:30
Zhikang Zhang
0971a342d8
Solve race condition in password lookup (#42529)
NOTE:
1. use os.open() with os.O_CREAT|os.O_EXCL to check existence
and create a lock file if not exists, it's an atomic operation
2. the fastest process will create the lock file and others will
wait until the lock file is removed
3. after the writer finished writing to the password file, all the reading
operations use built-in open so processes can read the file parallel
2018-08-15 15:10:52 -04:00
Lindsay Hill
5981a7489b new nos_command module (#43056) 2018-08-10 13:50:02 -07:00
Zhikang Zhang
3a3869f4c8
Clarify usage of encrypt arg (#43330)
According to the do_encrypt interface, encrypt arg should be the hash method name used for encrypting returning password. But in the doc and lookup code it's a boolean flag, correct it to string.
2018-08-02 14:03:49 -04:00
Joren Vrancken
b954917761 Surround top-level function and class definitions with two blank lines. 2018-07-31 12:06:56 -07:00
Fran Fitzpatrick
5928ec04ca Adds an ipmath filter (#41985) 2018-07-11 10:52:31 -07:00
Jordan Borean
940d4a0e89
win_reboot: fix 2.6 issues and better handle post reboot reboot (#42330)
* win_reboot: fix 2.6 issues and better handle post reboot reboot

* changed winrm _reset to reset

* Add handler to reset calls when .reset() throws an AnsibleError on older hosts

* Moving back to _reset to get the issue fixed
2018-07-11 09:12:29 +10:00
Jordan Borean
8bdd04c147 Fix remote_tmp when become with non admin user (#42396)
* Fix tmpdir on non root become

 - also avoid exception if tmpdir and remote_tmp are None
 - give 'None' on deescalation so tempfile will fallback to it's default behaviour
   and use system dirs
 - fix issue with bad tempdir (not existing/not createable/not writeable)
   i.e nobody and ~/.ansible/tmp
 - added tests for blockfile case

* Revert "Temporarily revert c119d54"

This reverts commit 5c614a59a6.

* changes based on PR feedback and changelog fragment

* changes based on the review

* Fix tmpdir when makedirs failed so we just use the system tmp

* Let missing remote_tmp fail

If remote_tmp is missing then there's something more basic wrong in the
communication from the controller to the module-side.  It's better to
be alerted in this case than to silently ignore it.

jborean and I have independently checked what happens if the user sets
ansible_remote_tmp to empty string and !!null and both cases work fine.
(null is turned into a default value controller-side.  empty string
triggers the warning because it is probably not a directory that the
become user is able to use).
2018-07-06 10:49:19 -07:00
Nathaniel Case
5850591d14
Base connection class for network-style connections (#41839)
* Create base class for network-style connections

* clean up some differences

* Move NetworkConnectionBase

* Tweak netconf for tests

* Tweak when network_os is checked to avoid failing tests

* Pull back exec_command
2018-07-02 08:41:00 -04:00
Jordan Borean
9b7b564d75
Stop displaying kinit pass input on a failure (#41882)
* Stop displaying kinit pass input on a failure

* Fixed up minor logic info and added tests
2018-06-26 05:49:19 +10:00
Brian Coca
27c43daab8
fix minor issues with debug and item labels (#41331)
* fix minor issues with debug and item labels

 - no more `item=None`, we always have a label now
 - debug should only show expected information, either msg= or the var in var=
 - also fixed method name, deprecated misleading _get_item
2018-06-21 16:14:57 -04:00
Jordan Borean
11bd3fd318 win_updates fix when win_updates is run with async (#41756) 2018-06-20 14:52:24 -07:00
Matt Davis
b01779ad18
skip winrm unit tests if winrm is not installed (#41596) 2018-06-15 10:23:15 -07:00
Jordan Borean
ad8e13e9f8
winrm: fix up unit tests (#41112) 2018-06-07 06:09:10 +08:00
Martin Krizek
77e2bd5f8e
Fix inserting noop tasks by using active state (#36056) 2018-06-06 12:34:25 +02:00
Ganesh Nalawade
ba4b12358c
Refactor ios cliconf plugin and ios_config module (#39695)
* Refactor ios cliconf plugin and ios_config module

*  Refactor ios cliconf plugin to support generic network_config module
*  Refactor ios_config module to work with cliconf api's
*  Enable command and response logging in cliconf pulgin
*  cliconf api documentation

* Fix unit test and other minor changes

* Doc update

* Fix CI failure

* Add default flag related changes

* Minor changes

* redact input command logging by default
2018-06-06 11:12:45 +05:30
Matt Martz
ad7ba91f75 Use _remote_is_local=True for local connection in synchronize (#40833)
* All instances of local connection should use _remote_is_local=True. Fixes #40551

* Switch to instance attribute for synchronize

* Add test that shows that synchronize _remote_is_local addresses tmpdir building
2018-05-30 13:28:11 -07:00
Sloane Hertel
cba64f5869 Fix inventory plugin cache + add tests (#38229)
* Fix setting the cache when refresh_cache or --flush-cache are used

* Use jsonify function that handles datetime objects in jsonfile cache plugin

* Don't access self._options directly

* Add initial integration tests for aws_ec2 inventory plugin

* Add CI alias

* Fix and add a few more unit tests

* Add integration tests for constructed

* Fix typo

* Use inventory config templates

* Collect all instances that are not terminated by default

* Create separate playbook for setting up the VPC, subnet, security group, and finding an image for the host

Create a separate playbook for removing the resources

* Allow easier grouping by region and add an example

* use a unified json encode/decode that can handle unsafe and vault
2018-05-24 15:34:54 -04:00
Jordan Borean
5e28e282a5 winrm: add better exception handling for krb5 auth with pexpect (#39930)
* winrm: add better exception handling for krb5 auth with pexpect

* Added changelog fragment

* Added exception handler in case kinit path isn't valid, added test cases

* fixed for Python 2 compatibility
2018-05-23 13:33:07 -07:00
Jordan Borean
457bccf540 win_updates: add scheduled tasks back in for older hosts (#38708)
* win_updates: add scheduled tasks back in for older hosts

* Fixed up typo in category name error message

* Fixed up some minor issues after merge

* added changelog fragment

* Default to become but add override to use scheduled tasks

* Added basic unit tests for win_updates

* fix minor typos
2018-05-23 13:21:01 -07:00
flowerysong
e2c1589201 Add cidr_merge filter (#36081) 2018-05-23 14:35:23 -05:00
Ken Celenza
dd02a4e943 create cisco type 5 filters (#39901) 2018-05-22 13:35:54 +01:00
Jordan Borean
1ac180c74d
winrm: source user from options than remote_user (#40467)
* winrm: source user from options than remote_user

* fixed up mock for kerberos import

* Added changelog fragment

* get hostname from option as well
2018-05-22 09:36:48 +10:00
Scott Buchanan
b12cf754f6 1Password lookup plugin (#37207)
* add pytest_cache to gitignore

* onepassword lookup plugin

* fix linter/style test complaints

* second pass at making pycodestyle happy

* use json module instead of jq

* update copyrights, license & version added

* fix python2 compatibility

* doh. fix spacing issue.

* use standard ansible exception

* remove potentially problematic stdin argument

* actually call assertion method

* add support for top-level fields

* make vault uuids pedantically consistent in fixture

* fix new style issues

* ability specify section & correct case handling

* improve error handling

* add onepassword_raw plugin

* Add maintainer info

* Move common code to module_utils/onepassword.py

* Load raw data JSON data for easier use in Ansible

* Put OnePass class back inside lookup plugin

There is no good place for sharing code across lookups currently.

* Remove debugging code in unit tests

* Patche proper module in raw unit tests

* Add changelog entry

Co-authored-by: Scott Buchanan <sbuchanan@ri.pn>
2018-05-17 17:29:47 -04:00
Ricardo Carrillo Cruz
62e1c14edc
Pull persistent connection parameters via get_option (#39367)
* WIP Pull persistent connection parameters via get_option

* Fix pep8

* Add use_persistent_connection setting to paramiko_ssh plugin

* Add vars section to persistent_command_timeout setting and prevail provider values over config manager

* Use persistent_command_timeout on network_cli instead of timeout

* Fix unit tests

If we don't call loader to get network_cli, then _load_name is never
set and we get KeyError.

* Pull persistent_command_timeout via config  manager for ios connection local

* Pull persistent_command_timeout via config manager on connection local
2018-05-16 14:59:01 +02:00
Jordan Borean
44ab948e5d
create module tmpdir based on remote_tmp (#39833)
* create module tmpdir based on remote_tmp

* Source remote_tmp from controller if possible

* Fixed sanity test and not use lambda

* Added expansion of env vars to the remote tmp

* Fixed sanity issues

* Added note around shell remote_tmp option

* Changed fallback tmp dir to ~/.ansible/tmp to make shell defaults
2018-05-15 09:31:21 +10:00
Brian Coca
4aac0f5f18
allow configuring sftp/scp executables (#36648)
* allow configuring sftp/scp executables

fixes #36616


also removed dupe test
2018-04-23 18:36:35 -04:00
Matthew Stone
5b1a8ee1e8 Adding slx_command module and supporting module_utils. (#38235)
* Adding slx_command module and supporting module_utils.

This commit adds the slx_command module and tests as well as the
required slxos module_utils.

* Update copyright in header

* Adding missing module init

* Cleaning up shebangs/licensing.

* Incorporating feedback

Removing reference to `waitfor` alias in `slxos_command` module.
Adding `Extreme Networks` to `short_description` of `slxos_command` module.

* Adding cliconf tests

* Fixing 3.X tests

* Adding docstrings to test methods for slxos cliconf tests

* Adding slxos terminal tests

* Adding slxos module_utils tests

* Adding Extreme Networks team members to BOTMETA.yml
2018-04-11 12:59:34 +01:00
AlanCoding
153c9bd539 Reduce recursion within group methods
This offers an optimization that allows loading larger
inventories of various structure by improving the
scaling laws involved for adding hosts and groups.

The primary speed benefit is the elimination of duplicate
recusion from traversing converging paths.
2018-04-05 18:38:44 -04:00
Izuke
e02c72d856 Fixes #32146 Fixes reconcile inventory to correctly remove hosts from ungrouped 2018-03-20 15:37:39 -04:00
Nathaniel Case
594840c1d6
Close & remove paramiko connection where appropriate (#37528)
* Close & remove paramiko connection where appropriate

* Update unit test
2018-03-16 13:28:04 -04:00
mikedlr
52f2057472 aws ssm parameter lookup - change to return Nones for missing variables (#36456)
* aws ssm parameter lookup - change to return Nones for missing variables

* aws ssm parameter lookup - fix error case message to dump response

* aws ssm parameter lookup - fix integration test cases
2018-02-21 06:50:00 -05:00
Brian Coca
a79378fccb remove extra fields from debug output
fixes #35493

updated tests
2018-02-20 08:46:05 -05:00
Toshio Kuratomi
06f73ad578
Normalize usage of temp and tmp on tmp (#36221)
* Normalize usage of temp and tmp on tmp
* Rename system_tmps system_tmpdirs
* Add ANSIBLE_REMOTE_TMP spelling of environment variables
2018-02-15 09:01:02 -08:00
Brian Coca
8743b20c8e skip aws inventory test if no boto 2018-02-13 11:04:37 -05:00
mikedlr
d31ded47fb Aws ssm multiple fixes (#35569)
* aws ssm parameter lookup test case - fails demonstrating no exception when parameter missing

* aws ssm parameter lookup - fail in case parameter doesn't exist

* aws ssm parameter lookup test case - failing case for nice return from path lookup

* aws ssm parameter lookup - convert incoming taglist to a key-value dictionary

* aws ssm parameter lookup - pep8 / style clean up

* aws_ssm lookup plugin rewrite for more standard interface

* aws_ssm module and lookup - introduce integration test and fix:

* aws_ssm module and lookup - error case integraton test and many PEP8 and other cleanups

* aws ssm parameter lookup - Various fixes in response to review + recursive fix & test

* aws ssm parameter lookup - more in response to review - shertel/abadger

* aws ssm parameter lookup unit test - move to mocker according to abadger

* aws ssm parameter lookup - integrate with new documentation fragment

* aws ssm parameter lookup - accept either aws_profile or boto_profile

* aws ssm parameter lookup - eliminate lookup document fragment until env vars are fixed later
2018-02-06 17:41:46 -05:00
Jason Vanderhoof
7c8e365dff Conjur Lookup Plugin (#34280)
* Imported lookup plugin from Role

* Plugin cleanup, including:
* Use existing Python YAML parsing
* Remove environment variables as connection options
* Added initial debugging information

* Reworked the lookup plugin using the Python Request library.  As it's available through Ansible, it makes communication with Conjur much more straight forward.

* Removed un-used libraries

* Fixed linting issues

* Standardized output on `format` and insure it works for 2.6, 2.7, and 3.x.

* Use quote_plus from the six library for improved python 2/3 behavior.

* Refactored identity & configuration to prefer user's file. This also includes a refactor to remove an un-needed dictionary merge method.

* Removed `requests` in favor of `ansible.module_utils.urls`.

* Refactored netrc loading to warn if host is not present.

* Tests and a refactor to support easier testing.

* Added reference to website

* Fixed two linting errors

* Fixed an extra line found by linting

* Updated file write to use binary to insure config files are written correctly

* Resolved linting issues

* Refactored config & identity loading to take advantage of plugin options

* Cleanup a bunch of small items caught by linting

* Removed extra line caught by linting

* Swapped in pytest and added some tests with mocked network responses

* Pushing to see if this approach works better...

* Refactored be open_url mocking based on feedback

* Fixed a couple linting issues & refactored mocking into each method to attempt to resolve a failing test

* Use a generic MagicMock for python 2.6

* Fixes doc typo

require -> required

* Use `type: path` in identity_file and config_file

Also removes `expanduser` calls below (which will now be called automatically on
paths.)

* Defines maintainers for conjur_variable plugin

* BOTMETA.yml:
** defines $team_cyberark_conjur as maintainers of Conjur Variable plugin
** adds myself and @jvanderhoof to that team

* Adds URLs to relevant documentation for Conjur Variable lookup plugin

* Clarifies "the server," "the machine" -> "controlling host"

The machine identity used is that of the Ansible controlling host, not any
server being provisioned or instructed. This documentation change aims to make
that relationship clear.

* Adds response code to exception message on authentication failure

* Enhances exception messages to specify the controlling host

These error messages are less likely to confuse a user as to which machine is
associated with the files, identities, and configurations being described.

* Adds ANSIBLE_METADATA for Conjur variable lookup plugin
2018-01-23 10:04:57 -06:00
Toshio Kuratomi
8c7bd8beb4 Set the tempdir on the shell mock object so that it is json serializable 2018-01-22 23:36:34 -08:00
Sloane Hertel
b94198f9de [cloud] New inventory plugin: aws_ec2 (#34416)
aws_ec2 is an inventory plugin intended to replace uses of the `contrib/inventory/ec2.py`
inventory script. 

For advanced grouping, please use the `constructed` inventory plugin.
2018-01-22 18:49:57 -05:00
Matt Martz
29c1d5cb5d Ensure that the become password is written on py3 in the ssh connection plugin. Fixes #34727 2018-01-18 09:32:22 -08:00
Brian Coca
bbd6b8bb42 Temporary (#31677)
* allow shells to have per host options, remote_tmp

added language to shell
removed module lang setting from general as  plugins have it now
use get to avoid bad powershell plugin
more resilient tmp discovery, fall back to `pwd`
add shell to docs
fixed options for when frags are only options
added shell set ops in t_e and fixed option frags
normalize tmp dir usag4e

- pass tmpdir/tmp/temp options as env var to commands, making it default for tempfile
- adjusted ansiballz tmpdir
- default local tempfile usage to the configured local tmp
- set env temp in action

add options to powershell
shift temporary to internal envvar/params
ensure tempdir is set if we pass var
ensure basic and url use expected tempdir
ensure localhost uses local tmp
give /var/tmp priority, less perms issues
more consistent tempfile mgmt for ansiballz
made async_dir configurable
better action handling, allow for finally rm tmp
fixed tmp issue and no more tempdir in ballz
hostvarize world readable and admin users
always set shell tempdir
added comment to discourage use of exception/flow control

* Mostly revert expand_user as it's not quite working.

This was an additional feature anyhow.

Kept the use of pwd as a fallback but moved it to a second ssh
connection.  This is not optimal but getting that to work in a single
ssh connection was part of the problem holding this up.

(cherry picked from commit 395b714120522f15e4c90a346f5e8e8d79213aca)

* fixed script and other action plugins

ensure tmpdir deletion
allow for connections that don't support new options (legacy, 3rd party)
fixed tests
2018-01-15 21:15:04 -08:00
James Cammarata
b107e397cb
Cache tasks as they are queued instead of en masse (#34752)
* Cache tasks as they are queued instead of en masse

This also moves the task caching from the PlayIterator to the
StrategyBase class, where it makes more sense (and makes it easier
to not have to change the strategy class methods leading to an API
change).

Fixes #31673

* Cleaning up unit tests due to 502ca780
2018-01-15 09:49:58 -06:00
Matt Martz
d1846425db
Provide a way to explicitly invoke the debugger (#34006)
* Provide a way to explicitly invoke the debugger with in the debug strategy

* Merge the debugger strategy into StrategyBase

* Fix some logic, pin to a single result

* Make redo also continue

* Make sure that if the debug closure doesn't need to process the result, that we still return it

* Fix failing tests for the strategy

* Clean up messages from debugger and exit code to match bin/ansible

* Move the FieldAttribute higher, to apply at different levels

* make debugger a string, expand logic

* Better host state rollbacks

* More explicit debugger prompt

* ENABLE_TASK_DEBUGGER should be boolean, and better docs

* No bare except, add pprint, alias h, vars to task_vars

* _validate_debugger can ignore non-string, that can be caught later

* Address issue if there were no previous tasks/state, and use the correct key

* Update docs for changes to the debugger

* Guard against a stat going negative through use of decrement

* Add a few notes about using the debugger on the free strategy

* Add changelog entry for task debugger

* Add a few versionadded indicators and a note about vars -> task_vars
2018-01-09 13:50:07 -06:00
Matt Martz
aece0818c4
Don't cache duplicate plugin names (#34420)
* Don't cache duplicate plugin names. Fixes #33484

* Add tests for duplicate plugin filenames, to showcase what happens
2018-01-04 16:47:23 -06:00
Pierre-Louis Bonicoli
737920cd89 YAML inventory unit test: add tests
Relates: #33878
2017-12-20 16:59:53 -05:00
Pierre-Louis Bonicoli
35a4fe8546 YAML inventory unit test: add tests 2017-12-20 16:59:53 -05:00
Pierre-Louis Bonicoli
8dff0b091e Fix YAML inventory unit test 2017-12-20 16:59:53 -05:00