Commit graph

470 commits

Author SHA1 Message Date
Brian Coca
498aea8acc Revert "Add log_only to debug messages (#25545)"
This reverts commit 9e8cc26720.
2017-06-13 09:29:18 -04:00
Ganesh Nalawade
9e8cc26720 Add log_only to debug messages (#25545)
Fixes #25544
When `debug` is enabled the debug messages triggered
from `bin/ansible-connection` should be logged only to file
and not on stdout.
2017-06-13 18:41:33 +05:30
Ganesh Nalawade
6215922889 Add support for cliconf and netconf plugin (#25093)
* ansible-connection refactor and action plugin changes
* Add cliconf plugin for eos, ios, iosxr, junos, nxos, vyos
* Add netconf plugin for junos
* Add jsonrpc support
* Modify network_cli and netconf connection plugin
* Fix py3 unit test failure
* Fix review comment
* Minor fixes
* Fix ansible-connection review comments
* Fix CI issue
* platform_agnostic related changes
2017-06-06 13:56:25 +05:30
Brian Coca
8f97aef1a3 Transition inventory into plugins (#23001)
* draft new inventory plugin arch, yaml sample

 - split classes, moved out of init
 - extra debug statements
 - allow mulitple invenotry files
 - dont add hosts more than once
 - simplified host vars
 - since now we can have multiple, inventory_dir/file needs to be per host
 - ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
 - centralized localhost handling
 - added plugin docs
 - leaner meaner inventory (split to data + manager)
 - moved noop vars plugin
 - added 'postprocessing' inventory plugins
 - fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
 - grouphost_vars loading as inventory plugin (postprocessing)
 - playbook_dir allways full path
 - use bytes for file operations
 - better handling of empty/null sources
 - added test target that skips networking modules
 - now var manager loads play group/host_vars independant from inventory
 - centralized play setup repeat code
 - updated changelog with inv features
 - asperioribus verbis spatium album
 - fixed dataloader to new sig
 - made yaml plugin more resistant to bad data
 - nicer error msgs
 - fixed undeclared group detection
 - fixed 'ungrouping'
 - docs updated s/INI/file/ as its not only format
 - made behaviour of var merge a toggle
 - made 'source over group' path follow existing rule for var precedence
 - updated add_host/group from strategy
 - made host_list a plugin and added it to defaults
 - added advanced_host_list as example variation
 - refactored 'display' to be availbe by default in class inheritance
 - optimized implicit handling as per @pilou's feedback
 - removed unused code and tests
 - added inventory cache and vbox plugin now uses it
 - added _compose method for variable expressions in plugins
 - vbox plugin now uses 'compose'
 - require yaml extension for yaml
 - fix for plugin loader to always add original_path, even when not using all()
 - fix py3 issues
 - added --inventory as clearer option
 - return name when stringifying host objects
 - ajdust checks to code moving

* reworked vars and vars precedence
 - vars plugins now load group/host_vars dirs
 - precedence for host vars is now configurable
 - vars_plugins been reworked
 - removed unused vars cache
 - removed _gathered_facts as we are not keeping info in host anymore
 - cleaned up tests
 - fixed ansible-pull to work with new inventory
 - removed version added notation to please rst check
 - inventory in config relative to config
 - ensures full paths on passed inventories

* implicit localhost connection local
2017-05-23 17:16:49 -04:00
Ganesh Nalawade
5ec7f40196 Fix timeout issue in ansible-connection (#24556)
Fixes #24520 ansible-connection needs
to wait on timeout value of play-context
instead of ssh default timeout
2017-05-13 18:28:14 +05:30
Toshio Kuratomi
84a59e472b Be explicit about pickle protocol and encoding (#24454)
On Python3 and Python2 use pickle slightly differently so we need to be
explicit about some things.

If pickles could be shared between python2 and python3, as in
ansible-connection and the pickle cache, we need to specify the protocol
to use when dumping and the encoding to use for byte strings when
loading.

The dumping protocol needs to be no higher than 2 as python-2 only
supports up to protocol 2.  The encoding should usually be 'bytes' so
that python2 str type becomes python3 bytes type.  However, doing this
means that we must make sure that the objects being serialized properly
make their strings into text strings except when they're supposed to be
bytes.  If strings are improperly byte strings, they may cause
tracebacks on the receiving end
2017-05-12 18:04:48 -07:00
Toshio Kuratomi
d834412ead Fix for persistent connection plugin on Python3 (#24431)
Fix for persistent connection plugin on Python3.  Note that fixes are also needed to each terminal plugin.  This PR only fixes the ios terminal (as proof that this approach is workable.)  Future PRs can address the other terminal types.

* On Python3, pickle needs to work with byte strings, not text strings.
* Set the pickle protocol version to 0 because we're using a pty to feed data to the connection plugin.  A pty can't have control characters.  So we have to send ascii only.  That means
only using protocol=0 for pickling the data.
* ansible-connection isn't being used with py3 in the bug but it needs
several changes to work with python3.
* In python3, closing the pty too early causes no data to be sent.  So
leave stdin open until after we finish with the ansible-connection
process.
* Fix typo using traceback.format_exc()
* Cleanup unnecessary StringIO, BytesIO, and to_bytes calls
* Modify the network_cli and terminal plugins for py3 compat.  Lots of mixing of text and byte strings that needs to be straightened out to be compatible with python3
* Documentation for the bytes<=>text strategy for terminal plugins
* Update unittests for more bytes-oriented internals

Fixes #24355
2017-05-12 09:13:51 -07:00
Peter Sprygada
53c52cf65f updates log messages in ansible-connection (#23076) 2017-03-30 13:07:02 +01:00
Peter Sprygada
77ce83fe22 fixes ansible-connection working directory (#22964)
When ansible-connection forks the process, it changed the working
directory to /.  This patch will prevent ansible-connection from
changing the working directory in the forked process.
2017-03-25 00:19:48 -04:00
Peter Sprygada
4a8f2dde20 fixes exception raised from ansible-connection logging (#22836)
updates ansible-connection logging functions to use display instead of
log.
2017-03-21 09:15:23 -07:00
Peter Sprygada
e20ed8bc0c replaces logging with display for network connection plugins (#22819)
* updates network_cli and netconf connection plugins
* updates ansible-connection
2017-03-20 20:08:02 -07:00
Peter Sprygada
ed7cace425 implements python logging for network connection plugin (#22817)
* enables logging for network_cli and paramiko
* enables logging for ansible-connection
* enabled logging for netconf connection
2017-03-20 19:26:18 -07:00
Peter Sprygada
eed240797a adds connection established message to ansible-connection (#22492)
This will now print a log message that displays how long the ssh
connection took to establish to the remote device.  If the connection
established time is exceed it will also print a message to the log file
that the connection attempts have exceed.

Updates default connection retries to 30

fixes #22319
2017-03-14 10:31:02 -04:00
Toshio Kuratomi
9ff03e6c1e Handle downstream version additions (#22428)
Some downstreams want to ship multiple versions of ansible (Either to
have multiple ansible versions or to have a version that uses python3.X
and a version that uses python2.x).  When they do this, they append
a version number to the cli scripts in /usr/bin.  This patch will remove
those version numbers before trying to find the ansible python module to
import for this commandline
2017-03-10 12:01:11 -08:00
Peter Sprygada
6e9244a9e1 adds more logging output to network_cli and ansible-connection (#21716) 2017-02-21 09:21:41 -05:00
Peter Sprygada
e05b2b56f2 capture AnsibleConnectionFailure messages in ansible-connection (#21612)
Send the messages to the log file
2017-02-18 08:12:01 -05:00
Peter Sprygada
e77964f9da clean up pylint errors in ansible-connection (#21571) 2017-02-17 08:15:27 -05:00
Peter Sprygada
eaba067de9 fixes type that prevents socket clean up from being called (#21568) 2017-02-17 07:13:39 -05:00
Peter Sprygada
3ff2c471b2 fixes Bad file descriptor backtrace raised by ansible-connection (#21526)
This change will redirect stdout and stderr either to the log file
configured by log_path or to /dev/null if no log_path is specified.

fixes #21400
2017-02-16 13:59:47 -05:00
Peter Sprygada
138051540e updates the code path for network modules (#21193)
* replaces persistent connection digest with _create_control_path()
* adds _ansible_socket to _legal_inputs in basic.py
* adds connection_user to play_context
* maps remote_user to connection_user when connection is local
* maps ansible_socket in task_vars to module_args _ansible_socket if exists
2017-02-09 14:05:54 -05:00
Brian Coca
942ed146c3 errors to stderr 2017-02-08 09:47:10 -05:00
Peter Sprygada
b4d75f41f5 one more bug fix for ansible-connection (#20707)
The connection instance will now return a meaningful error if the
connection property returns False
2017-01-26 13:59:57 -05:00
Peter Sprygada
d668d53112 checks connected property to validate connection in Server (#20695)
fixes #20693
2017-01-26 14:30:38 +00:00
Peter Sprygada
6673673042 more logging and exception handling in ansible-connection (#20619)
adds more logging to handle display being called from plugins.  Also
rearranges some of the exception handling to better catch exceptions and
log to local syslog
2017-01-24 23:14:30 -05:00
Peter Sprygada
312328e22c adds more logging to ansible-connection (#20298)
* Server() instance will now add the entire traceback to syslog if it fail
  during init
2017-01-16 08:32:08 -05:00
Peter Sprygada
3f949358c7 adds some more logging to ansible-connection (#20205) 2017-01-12 19:20:25 -05:00
TaoBeier
6ec0369c26 fix indent (#20071) 2017-01-10 18:47:03 -08:00
Peter Sprygada
6fe9a5e40c functional updates to ansible-connection (#18574)
* sends the serialized play_context into an already established connection
* hooks the alarm_handler() method in the connection plugin if it exists
* added configuration options for connect interval and retries
* adds syslog logging to Server() instance

This update will send the updated play_context back into an already
established connection in case privilege escalation / descalation activities
need to be performed.  This change will also hook the alarm_handler() method
in the connection instance (if available) and call it in case of a
sigalarm raised.

This update adds two new configuration options

* PERSISTENT_CONNECT_INTERVAL - time to wait in between connection attempts
* PERSISTENT_CONNECT_RETRIES - max number of retries
2016-11-30 16:26:49 -05:00
James Cammarata
336f8d7233 Removing q debugging cruft from ansible-connection 2016-11-21 13:44:43 -06:00
James Cammarata
85064807af Removing test script for ansible-connection 2016-11-21 13:16:44 -06:00
James Cammarata
26ec2ecfce Adding a persistent connection utility 2016-11-21 12:35:27 -06:00
Brian Coca
18d3e0533f always log unexpected exceptions 2016-10-13 13:34:01 -04:00
Toshio Kuratomi
bf29961947 Fixes to the controller text model (#17527)
* Fixes to the controller text model

* Change command line args to text type
* Make display replace undecodable bytes with replacement chars.  This
  is only a problem on pyhton3 where surrogates can enter into the msg
  but sys.stdout doesn't know how to handle them.
* Remove a deprecated playbook syntax in unicode.yml

* Fix up run_cmd to change its parameters to byte string at appropriate times.
2016-09-12 12:57:41 -07:00
Toshio Kuratomi
4ed88512e4 Move uses of to_bytes, to_text, to_native to use the module_utils version (#17423)
We couldn't copy to_unicode, to_bytes, to_str into module_utils because
of licensing.  So once created it we had two sets of functions that did
the same things but had different implementations.  To remedy that, this
change removes the ansible.utils.unicode versions of those functions.
2016-09-06 22:54:17 -07:00
James Cammarata
37f721f315 Remove debug lock 2016-09-06 13:04:50 -05:00
Michael Scherer
f0023610e8 Fix bin/ansible to not make a double traceback on python3 (#15972) 2016-06-02 09:49:22 -07:00
Toshio Kuratomi
dcc5dfdf81 Controller-side module caching.
This makes our recursive, ast.parse performance measures as fast as
pre-ziploader baseline.

Since this unittest isn't testing that the returned module data is
correct we don't need to worry about os.rename not having any module
data.  Should devise a separate test for the module and caching code
2016-04-12 08:01:07 -07:00
Brian Coca
0eb2844cc6 moved display's debug lock to cli as global 2016-04-08 13:08:24 -04:00
Brian Coca
e74ab3ecdd draft 1st release of ansible-console
porting @dominis 's ansible-shell tool from 1.9 and integrating it into ansible
added verbosity control
made more resilitent to several errors
added highlight color, to configurable colors
more resilient on exception and interruptions
prompt coloring, goes red and changes to # when using become = true and root
become setting is now explicit and not a toggle
2016-03-07 20:25:21 -05:00
Toshio Kuratomi
f4d68b8860 Transform tracebacks into unicode before printing
Fixes #14042
2016-01-25 19:18:59 -08:00
James Cammarata
120b9a7ac6 Changing the way workers are forked 2015-12-11 23:35:07 -05:00
Toshio Kuratomi
318bfbb207 Migrate cli and dependencies to use global display 2015-11-11 10:44:22 -08:00
Brian Coca
10fd717a44 narrow the scope of import error and make sure non - aliases to ansible get their own error 2015-11-02 22:17:37 -05:00
Brian Coca
d0a22a7b45 made cli code more flexible to add new command line programs w/o need for updating bin/ansible 2015-11-02 14:35:06 -05:00
Brian Coca
56a2412dc2 now correctly traps unimplemented symlinks as subprograms 2015-11-02 12:46:33 -05:00
Toshio Kuratomi
7c10c16251 Revert "capture some install issues and give out a slightly nicer error"
This reverts commit e6b1dc45e1.

This is not ready yet and it's causing failures in some environments
(travis)
2015-10-27 23:23:25 -07:00
Brian Coca
e6b1dc45e1 capture some install issues and give out a slightly nicer error 2015-10-27 19:59:43 -04:00
Toshio Kuratomi
f34b55ac2b Add python3-compat boilerplate to all .py files in lib/ansible 2015-10-19 18:36:19 -07:00
James Cammarata
de792ba3c2 Improve handling of unicode errors
Fixes #12669
2015-10-08 10:04:15 -04:00
Brian Coca
a5e6845c8b Revert "Allow exceptions to pass through the program"
breaks handling constants.py errors gracefully

This reverts commit 59f96d713e.
2015-08-06 10:02:58 -04:00