Commit graph

518 commits

Author SHA1 Message Date
Ganesh Nalawade
6215922889 Add support for cliconf and netconf plugin (#25093)
* ansible-connection refactor and action plugin changes
* Add cliconf plugin for eos, ios, iosxr, junos, nxos, vyos
* Add netconf plugin for junos
* Add jsonrpc support
* Modify network_cli and netconf connection plugin
* Fix py3 unit test failure
* Fix review comment
* Minor fixes
* Fix ansible-connection review comments
* Fix CI issue
* platform_agnostic related changes
2017-06-06 13:56:25 +05:30
Brian Coca
8f97aef1a3 Transition inventory into plugins (#23001)
* draft new inventory plugin arch, yaml sample

 - split classes, moved out of init
 - extra debug statements
 - allow mulitple invenotry files
 - dont add hosts more than once
 - simplified host vars
 - since now we can have multiple, inventory_dir/file needs to be per host
 - ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
 - centralized localhost handling
 - added plugin docs
 - leaner meaner inventory (split to data + manager)
 - moved noop vars plugin
 - added 'postprocessing' inventory plugins
 - fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
 - grouphost_vars loading as inventory plugin (postprocessing)
 - playbook_dir allways full path
 - use bytes for file operations
 - better handling of empty/null sources
 - added test target that skips networking modules
 - now var manager loads play group/host_vars independant from inventory
 - centralized play setup repeat code
 - updated changelog with inv features
 - asperioribus verbis spatium album
 - fixed dataloader to new sig
 - made yaml plugin more resistant to bad data
 - nicer error msgs
 - fixed undeclared group detection
 - fixed 'ungrouping'
 - docs updated s/INI/file/ as its not only format
 - made behaviour of var merge a toggle
 - made 'source over group' path follow existing rule for var precedence
 - updated add_host/group from strategy
 - made host_list a plugin and added it to defaults
 - added advanced_host_list as example variation
 - refactored 'display' to be availbe by default in class inheritance
 - optimized implicit handling as per @pilou's feedback
 - removed unused code and tests
 - added inventory cache and vbox plugin now uses it
 - added _compose method for variable expressions in plugins
 - vbox plugin now uses 'compose'
 - require yaml extension for yaml
 - fix for plugin loader to always add original_path, even when not using all()
 - fix py3 issues
 - added --inventory as clearer option
 - return name when stringifying host objects
 - ajdust checks to code moving

* reworked vars and vars precedence
 - vars plugins now load group/host_vars dirs
 - precedence for host vars is now configurable
 - vars_plugins been reworked
 - removed unused vars cache
 - removed _gathered_facts as we are not keeping info in host anymore
 - cleaned up tests
 - fixed ansible-pull to work with new inventory
 - removed version added notation to please rst check
 - inventory in config relative to config
 - ensures full paths on passed inventories

* implicit localhost connection local
2017-05-23 17:16:49 -04:00
Ganesh Nalawade
5ec7f40196 Fix timeout issue in ansible-connection (#24556)
Fixes #24520 ansible-connection needs
to wait on timeout value of play-context
instead of ssh default timeout
2017-05-13 18:28:14 +05:30
Toshio Kuratomi
84a59e472b Be explicit about pickle protocol and encoding (#24454)
On Python3 and Python2 use pickle slightly differently so we need to be
explicit about some things.

If pickles could be shared between python2 and python3, as in
ansible-connection and the pickle cache, we need to specify the protocol
to use when dumping and the encoding to use for byte strings when
loading.

The dumping protocol needs to be no higher than 2 as python-2 only
supports up to protocol 2.  The encoding should usually be 'bytes' so
that python2 str type becomes python3 bytes type.  However, doing this
means that we must make sure that the objects being serialized properly
make their strings into text strings except when they're supposed to be
bytes.  If strings are improperly byte strings, they may cause
tracebacks on the receiving end
2017-05-12 18:04:48 -07:00
Toshio Kuratomi
d834412ead Fix for persistent connection plugin on Python3 (#24431)
Fix for persistent connection plugin on Python3.  Note that fixes are also needed to each terminal plugin.  This PR only fixes the ios terminal (as proof that this approach is workable.)  Future PRs can address the other terminal types.

* On Python3, pickle needs to work with byte strings, not text strings.
* Set the pickle protocol version to 0 because we're using a pty to feed data to the connection plugin.  A pty can't have control characters.  So we have to send ascii only.  That means
only using protocol=0 for pickling the data.
* ansible-connection isn't being used with py3 in the bug but it needs
several changes to work with python3.
* In python3, closing the pty too early causes no data to be sent.  So
leave stdin open until after we finish with the ansible-connection
process.
* Fix typo using traceback.format_exc()
* Cleanup unnecessary StringIO, BytesIO, and to_bytes calls
* Modify the network_cli and terminal plugins for py3 compat.  Lots of mixing of text and byte strings that needs to be straightened out to be compatible with python3
* Documentation for the bytes<=>text strategy for terminal plugins
* Update unittests for more bytes-oriented internals

Fixes #24355
2017-05-12 09:13:51 -07:00
Peter Sprygada
53c52cf65f updates log messages in ansible-connection (#23076) 2017-03-30 13:07:02 +01:00
Peter Sprygada
77ce83fe22 fixes ansible-connection working directory (#22964)
When ansible-connection forks the process, it changed the working
directory to /.  This patch will prevent ansible-connection from
changing the working directory in the forked process.
2017-03-25 00:19:48 -04:00
Peter Sprygada
4a8f2dde20 fixes exception raised from ansible-connection logging (#22836)
updates ansible-connection logging functions to use display instead of
log.
2017-03-21 09:15:23 -07:00
Peter Sprygada
e20ed8bc0c replaces logging with display for network connection plugins (#22819)
* updates network_cli and netconf connection plugins
* updates ansible-connection
2017-03-20 20:08:02 -07:00
Peter Sprygada
ed7cace425 implements python logging for network connection plugin (#22817)
* enables logging for network_cli and paramiko
* enables logging for ansible-connection
* enabled logging for netconf connection
2017-03-20 19:26:18 -07:00
Peter Sprygada
eed240797a adds connection established message to ansible-connection (#22492)
This will now print a log message that displays how long the ssh
connection took to establish to the remote device.  If the connection
established time is exceed it will also print a message to the log file
that the connection attempts have exceed.

Updates default connection retries to 30

fixes #22319
2017-03-14 10:31:02 -04:00
Toshio Kuratomi
9ff03e6c1e Handle downstream version additions (#22428)
Some downstreams want to ship multiple versions of ansible (Either to
have multiple ansible versions or to have a version that uses python3.X
and a version that uses python2.x).  When they do this, they append
a version number to the cli scripts in /usr/bin.  This patch will remove
those version numbers before trying to find the ansible python module to
import for this commandline
2017-03-10 12:01:11 -08:00
Peter Sprygada
6e9244a9e1 adds more logging output to network_cli and ansible-connection (#21716) 2017-02-21 09:21:41 -05:00
Peter Sprygada
e05b2b56f2 capture AnsibleConnectionFailure messages in ansible-connection (#21612)
Send the messages to the log file
2017-02-18 08:12:01 -05:00
Peter Sprygada
e77964f9da clean up pylint errors in ansible-connection (#21571) 2017-02-17 08:15:27 -05:00
Peter Sprygada
eaba067de9 fixes type that prevents socket clean up from being called (#21568) 2017-02-17 07:13:39 -05:00
Peter Sprygada
3ff2c471b2 fixes Bad file descriptor backtrace raised by ansible-connection (#21526)
This change will redirect stdout and stderr either to the log file
configured by log_path or to /dev/null if no log_path is specified.

fixes #21400
2017-02-16 13:59:47 -05:00
Peter Sprygada
138051540e updates the code path for network modules (#21193)
* replaces persistent connection digest with _create_control_path()
* adds _ansible_socket to _legal_inputs in basic.py
* adds connection_user to play_context
* maps remote_user to connection_user when connection is local
* maps ansible_socket in task_vars to module_args _ansible_socket if exists
2017-02-09 14:05:54 -05:00
Brian Coca
942ed146c3 errors to stderr 2017-02-08 09:47:10 -05:00
Peter Sprygada
b4d75f41f5 one more bug fix for ansible-connection (#20707)
The connection instance will now return a meaningful error if the
connection property returns False
2017-01-26 13:59:57 -05:00
Peter Sprygada
d668d53112 checks connected property to validate connection in Server (#20695)
fixes #20693
2017-01-26 14:30:38 +00:00
Peter Sprygada
6673673042 more logging and exception handling in ansible-connection (#20619)
adds more logging to handle display being called from plugins.  Also
rearranges some of the exception handling to better catch exceptions and
log to local syslog
2017-01-24 23:14:30 -05:00
Peter Sprygada
312328e22c adds more logging to ansible-connection (#20298)
* Server() instance will now add the entire traceback to syslog if it fail
  during init
2017-01-16 08:32:08 -05:00
Peter Sprygada
3f949358c7 adds some more logging to ansible-connection (#20205) 2017-01-12 19:20:25 -05:00
TaoBeier
6ec0369c26 fix indent (#20071) 2017-01-10 18:47:03 -08:00
Peter Sprygada
6fe9a5e40c functional updates to ansible-connection (#18574)
* sends the serialized play_context into an already established connection
* hooks the alarm_handler() method in the connection plugin if it exists
* added configuration options for connect interval and retries
* adds syslog logging to Server() instance

This update will send the updated play_context back into an already
established connection in case privilege escalation / descalation activities
need to be performed.  This change will also hook the alarm_handler() method
in the connection instance (if available) and call it in case of a
sigalarm raised.

This update adds two new configuration options

* PERSISTENT_CONNECT_INTERVAL - time to wait in between connection attempts
* PERSISTENT_CONNECT_RETRIES - max number of retries
2016-11-30 16:26:49 -05:00
James Cammarata
336f8d7233 Removing q debugging cruft from ansible-connection 2016-11-21 13:44:43 -06:00
James Cammarata
85064807af Removing test script for ansible-connection 2016-11-21 13:16:44 -06:00
James Cammarata
26ec2ecfce Adding a persistent connection utility 2016-11-21 12:35:27 -06:00
Brian Coca
18d3e0533f always log unexpected exceptions 2016-10-13 13:34:01 -04:00
Toshio Kuratomi
bf29961947 Fixes to the controller text model (#17527)
* Fixes to the controller text model

* Change command line args to text type
* Make display replace undecodable bytes with replacement chars.  This
  is only a problem on pyhton3 where surrogates can enter into the msg
  but sys.stdout doesn't know how to handle them.
* Remove a deprecated playbook syntax in unicode.yml

* Fix up run_cmd to change its parameters to byte string at appropriate times.
2016-09-12 12:57:41 -07:00
Toshio Kuratomi
4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
2016-09-06 22:54:17 -07:00
James Cammarata
37f721f315 Remove debug lock 2016-09-06 13:04:50 -05:00
Michael Scherer
f0023610e8 Fix bin/ansible to not make a double traceback on python3 (#15972) 2016-06-02 09:49:22 -07:00
Toshio Kuratomi
dcc5dfdf81 Controller-side module caching.
This makes our recursive, ast.parse performance measures as fast as
pre-ziploader baseline.

Since this unittest isn't testing that the returned module data is
correct we don't need to worry about os.rename not having any module
data.  Should devise a separate test for the module and caching code
2016-04-12 08:01:07 -07:00
Brian Coca
0eb2844cc6 moved display's debug lock to cli as global 2016-04-08 13:08:24 -04:00
Brian Coca
e74ab3ecdd draft 1st release of ansible-console
porting @dominis 's ansible-shell tool from 1.9 and integrating it into ansible
added verbosity control
made more resilitent to several errors
added highlight color, to configurable colors
more resilient on exception and interruptions
prompt coloring, goes red and changes to # when using become = true and root
become setting is now explicit and not a toggle
2016-03-07 20:25:21 -05:00
Toshio Kuratomi
f4d68b8860 Transform tracebacks into unicode before printing
Fixes #14042
2016-01-25 19:18:59 -08:00
James Cammarata
120b9a7ac6 Changing the way workers are forked 2015-12-11 23:35:07 -05:00
Toshio Kuratomi
318bfbb207 Migrate cli and dependencies to use global display 2015-11-11 10:44:22 -08:00
Brian Coca
10fd717a44 narrow the scope of import error and make sure non - aliases to ansible get their own error 2015-11-02 22:17:37 -05:00
Brian Coca
d0a22a7b45 made cli code more flexible to add new command line programs w/o need for updating bin/ansible 2015-11-02 14:35:06 -05:00
Brian Coca
56a2412dc2 now correctly traps unimplemented symlinks as subprograms 2015-11-02 12:46:33 -05:00
Toshio Kuratomi
7c10c16251 Revert "capture some install issues and give out a slightly nicer error"
This reverts commit e6b1dc45e1.

This is not ready yet and it's causing failures in some environments
(travis)
2015-10-27 23:23:25 -07:00
Brian Coca
e6b1dc45e1 capture some install issues and give out a slightly nicer error 2015-10-27 19:59:43 -04:00
Toshio Kuratomi
f34b55ac2b Add python3-compat boilerplate to all .py files in lib/ansible 2015-10-19 18:36:19 -07:00
James Cammarata
de792ba3c2 Improve handling of unicode errors
Fixes #12669
2015-10-08 10:04:15 -04:00
Brian Coca
a5e6845c8b Revert "Allow exceptions to pass through the program"
breaks handling constants.py errors gracefully

This reverts commit 59f96d713e.
2015-08-06 10:02:58 -04:00
Will Thames
59f96d713e Allow exceptions to pass through the program
This is useful when using `ipython --pdb -- $(which ansible-playbook)
...` for debugging.

Also show traceback when `ANSIBLE_DEBUG` is on
2015-08-06 21:11:38 +10:00
Brian Coca
e49575ff56 added wrap_text to 'last resort' as it fails if there is an error in callbacks 2015-07-24 11:25:25 -04:00
James Cammarata
469aa0c26a Don't wrap text on most errors 2015-07-23 15:26:30 -04:00
James Cammarata
6d50a261c5 Allow full exception tracebacks to be displayed with increased verbosity 2015-07-07 14:31:15 -04:00
Marc Abramowitz
314bae2a9e Don't wrap text for AnsibleParserError
This allows not messing up the wonderful error reporting that is
carefully created. Instead of:

    $ ansible-playbook foo.yml
     [ERROR]: ERROR! 'foo' is not a valid attribute for a Task  The error appears
    to have been in '/Users/marca/dev/git-repos/ansible/foo.yml': line 4, column 7,
    but may be elsewhere in the file depending on the exact syntax problem.  The
    offending line appears to be:    tasks:     - name: do something       ^ here

we get:

    $ ansible-playbook foo.yml
    ERROR! 'foo' is not a valid attribute for a Task

    The error appears to have been in '/Users/marca/dev/git-repos/ansible/foo.yml': line 4, column 7, but may
    be elsewhere in the file depending on the exact syntax problem.

    The offending line appears to be:

      tasks:
        - name: do something
          ^ here

which is much nicer.
2015-07-07 09:31:00 -07:00
Brian Coca
f42b6237d9 now has display of last resort
moved all display/color/err to use display.error
now also capture generic exceptions if they happen (never should!)
2015-07-05 17:46:51 -04:00
Brian Coca
6a75125f32 now traps exceptions on display instantiation 2015-07-05 17:24:15 -04:00
Brian Coca
fdeca37257 switched to argv[0] from __file__ as it is what we actually wanted 2015-06-09 17:29:46 -04:00
Brian Coca
b85ce38834 slight changes to error handling to align with v1 2015-05-13 11:15:32 -04:00
James Cammarata
ce3ef7f4c1 Making the switch to v2 2015-05-03 21:47:26 -05:00
Jesse Rusak
652cd6cd5e Fix --force-handlers, and allow it in plays and ansible.cfg
The --force-handlers command line argument was not correctly running
handlers on hosts which had tasks that later failed. This corrects that,
and also allows you to specify force_handlers in ansible.cfg or in a
play.
2015-04-10 19:38:59 -04:00
James Cammarata
317728f649 Allow ansible-galaxy to install symlinks 2015-03-23 09:21:12 -05:00
Steve Gargan
4ce791fe84 avoid path issues by determining the path of ansible-pull and using its path to run ansible and ansible-playbook 2015-03-17 21:25:45 +00:00
Brian Coca
1fd0a78b0e fix issue with ask pass signature 2015-03-11 10:28:27 -04:00
Brian Coca
5f6db0e164 preliminary privlege escalation unification + pbrun
- become constants inherit existing sudo/su ones
- become command line options, marked sudo/su as deprecated and moved sudo/su passwords to runas group
- changed method signatures as privlege escalation is collapsed to become
- added tests for su and become, diabled su for lack of support in local.py
- updated playbook,play and task objects to become
- added become to runner
- added whoami test for become/sudo/su
- added home override dir for plugins
- removed useless method from ask pass
- forced become pass to always be string also uses to_bytes
- fixed fakerunner for tests
- corrected reference in synchronize action plugin
- added pfexec (needs testing)
- removed unused sudo/su in runner init
- removed deprecated info
- updated pe tests to allow to run under sudo and not need root
- normalized become options into a funciton to avoid duplication and inconsistencies
- pushed suppored list to connection classs property
- updated all connection plugins to latest 'become' pe

- includes fixes from feedback (including typos)
- added draft docs
- stub of become_exe, leaving for future v2 fixes
2015-03-10 18:42:36 -04:00
Brian Coca
8267bb2ae1 added list-tags option and also show tags in list-tasks option 2015-03-09 00:07:14 -04:00
Brian Coca
4805fa6b0b fix ansible-doc listing with new docstring
fixes #10295
2015-03-04 09:39:06 -05:00
Brian Coca
9f83af51b7 Merge pull request #9129 from lotia/devel
Clean up template comment text in ansible-galaxy
2015-02-26 22:16:55 -05:00
Brian Coca
c81d981164 Merge pull request #10183 from bcoca/tag_control
adds complex tag management
2015-02-25 10:36:11 -05:00
Brian Coca
1e818f9c50 ansible-doc now shows return value docs 2015-02-19 13:18:05 -05:00
Brian Coca
65c08d1364 added tests
refactored task selection into method
list-tasks now uses common method of task selection
always is now skippable if specified in --skip-tags
2015-02-13 08:27:07 -05:00
Matt Martz
68358e31fb Allow ansible (ad-hoc) to support --extra-vars also 2015-02-11 19:34:01 -06:00
Monty Taylor
38e81083cd Add issue_tracker_url to the galaxy template
There is an optional issue_tracker_url setting that can be set in
meta/main.yml:galaxy_info. Include it in the generated template
commented out so that it can be filled out if desired.
2015-02-10 08:44:45 -05:00
Brian Coca
76a5dd7f62 Merge pull request #10189 from boxofrad/devel
Ignore errors from `shutil.rmtree` when removing temporary directory
2015-02-09 10:17:05 -05:00
Brian Coca
f384e0d8d3 Merge pull request #10174 from dago/solaris
Use termios instead of stty for compatibility
2015-02-09 09:24:35 -05:00
Daniel Upton
78fb13657a Ignore errors from shutil.rmtree when removing temporary directory 2015-02-09 13:33:48 +00:00
Dagobert Michelsen
84718156e3 Use termios instead of stty for compatibility 2015-02-06 13:43:47 +01:00
Toshio Kuratomi
5b279c1c15 Fix for v1 handling of unicode extra-vars 2015-02-02 14:12:16 -08:00
Max Riveiro
9e8dae7498
Fix #6346. Add key_file and accept_hostkey to ansible-pull
Signed-off-by: Max Riveiro <kavu13@gmail.com>
2015-01-31 18:40:20 +03:00
Brian Coca
540d13cf52 Merge pull request #9404 from mmoya/configure-retry-files
Configure retry file usage and location
2015-01-30 15:14:27 -05:00
Sören Tempel
773849d421 Only write actual errors to stderr 2015-01-30 15:55:37 +01:00
rpe-github
3e7bb0c081 ignore VERSION file in ansible-doc 2015-01-22 19:46:10 +01:00
Toshio Kuratomi
2e0472e03b Allow ansible-playbook to determine if an invalid limit is specified 2015-01-19 13:39:45 -08:00
Jacopo
8dcffaf56f Converted empty inventory error to warning 2015-01-17 09:20:40 +10:00
Jacopo
e1f2cb1d37 Removed early check for empty inventory 2014-12-30 20:17:17 +10:00
Toshio Kuratomi
f998e65701 Merge pull request #9809 from mscherer/fix_9695
Make sure ansible-doc work when stdin is not a tty
2014-12-16 13:02:47 -08:00
Toshio Kuratomi
fbadcfd44f Fix for not all platforms having optional libraries at the proper versions. 2014-12-16 09:20:11 -08:00
Brian Coca
88443d6dcf Merge pull request #7306 from abadger/feature/egg-info
Feature/egg info
2014-12-15 16:26:37 -05:00
Michael Scherer
caefc20f16 Use --version to see if less can be executed
(less) 2> /dev/null would fail if stdin is /dev/null.
Sinceless --version do not read anything from stdin,
it is perfect for seeing if the software exist or not.

Also replace the whole os system detection by directly
using subprocess ( as we use it elsewhere, we already
depend on it ).
2014-12-14 18:09:42 +01:00
Michael Scherer
f7ac012301 Do not assume that stdin is a tty
This can be used from another non interactive software, see #9695
for details.
2014-12-14 17:56:18 +01:00
Brian Coca
0ba2298fdd used del instead of pop, removed unused pprint import
Conflicts:
	lib/ansible/modules/core
	lib/ansible/modules/extras
2014-11-20 15:36:55 -05:00
Brian Coca
8250dba0f1 Merge pull request #9534 from mmicael1/devel
Add tags options [to ansible pull - ed]
2014-11-18 14:49:23 -05:00
Brian Coca
40caa11082 implemented info action for galaxy 2014-11-12 16:23:49 -05:00
mmicael1
a33dccfa61 Add tags options
Add -t OR --tags options to pass to ansible-playbook
2014-11-10 22:56:38 +01:00
Brian Coca
7a5e7db2df ansible doc now finds modules recursively
more intelligent about ignoring files that are clearly not modules
2014-11-01 01:17:42 -04:00
Brian Coca
0317e7b910 avoid modifying module var by just passing the substring to the append 2014-10-30 13:53:05 -04:00
Brian Coca
94a732fb1a fixed typo in comments 2014-10-30 13:50:34 -04:00
Brian Coca
0fb0548d0b removed no unused var that was not cleaned up properlly 2014-10-30 13:40:34 -04:00
Brian Coca
e41bcc41d3 removed 'column display' options as there is no code handling this
function
2014-10-30 08:56:41 -04:00
Brian Coca
1b70ef6cba Several changes to ansible-doc
added display of deprecated to ansible-doc
now it does better job of using tty columns
fixed indication truncation of desc with trailing ...
removed extension from module list, also fixed matching exlusion
blacklist
2014-10-29 22:33:31 -04:00
Brian Coca
b3b356480d added the ability to keep aliased and deprecated modules prefixed with
'_', they will be loaded after non prefixed modules are checked they can
be full modules or symlinks to existing ones (alias)

also updated ansible doc to ignore these, will eventually add selective
display
2014-10-27 15:54:41 -07:00
Ton Kersten
39bbe36fc7 Ignore compiled Python for documentation.
When running `ansible-doc --list` a lot of errors are
generated when `ansible-doc` tries to find documentation
strings in `.pyc` files.
2014-10-26 15:46:08 +01:00