Commit graph

45 commits

Author SHA1 Message Date
Ganesh Nalawade
90cd87f950
Fix debug logs failing with persistent connection (#33049)
* Fix debug logs failing with persistent connection

Fixes #33047

*  As debug logs are written on stdout, it interrupts
   the communication between ansible-connection(background)
   process and main process. To avoid this add a string similar
   to exactly identify the response string.

*  Remove unwanted code in ansible-connection

*  Fix review comments

* Fix spurious log emitted on ansible-connection stdout issue

*  ansible-connection which runs as a background process sends a
   json string (contains response received from remote device)
   to foreground ansible-playbook process over stdout.

*  If in case debug flag is enabled the connection_loader api
   invoked from ansible-connection `ssh = connection_loader.get('ssh', class_only=True)`
   results in emitting debug logs on stdout. This  spurious log
   interfere with the actual response and results in failure while
   reading json string in ansible-playbook process

* To avoid this save stdout of ansible-connection and redirect it string
  buffer to accumulate all the logs emitted by core API's

* Add these logs in `result['messages']` which is send a json string after reinstating saved stdout

*  Remove unwanted code in ansible-connection

* Fix review comment
2018-01-25 02:48:45 +05:30
Ganesh Nalawade
2f932d8767
Add parent pid to persistent connection socket path hash (#33518)
* Add parent pid to persistent connection socket path hash

Fixes #33192

*  Add parent pid in persistent connection socket path hash
   to avoid using same socket path for multiple simultaneous
   connection to same remote host.

* Ensure unique persistent socket path for each ansible-playbook run

* Fix CI failures
2017-12-15 10:21:56 +05:30
Peter Sprygada
69575e25d0 shuts down persistent connections at end of play run (#32825)
This change will now track any created persistent connection and shut it
down at the end of the play run.  This change also includes an update to
properly honor the reset_connection meta handler.
2017-11-22 15:30:06 +00:00
Brian Coca
23b1dbacaf
Config continued (#31024)
* included inventory and callback in new config

allow inventory to be configurable
updated connection options settings
also updated winrm to work with new configs
removed now obsolete set_host_overrides
added notes for future bcoca, current one is just punting, it's future's problem
updated docs per feedback
added remove group/host methods to inv data
moved fact cache from data to constructed
cleaner/better options
fix when vars are added
extended ignore list to config dicts
updated paramiko connection docs
removed options from base that paramiko already handles
left the look option as it is used by other plugin types
resolve delegation
updated cache doc options
fixed test_script
better fragment merge for options
fixed proxy command
restore ini for proxy
normalized options
moved pipelining to class
updates for host_key_checking
restructured mixins

* fix typo
2017-11-16 13:49:57 -05:00
Nathaniel Case
21c8ab537b
getattr() with no default is an AttributeError if missing (#32946) 2017-11-15 15:40:32 -05:00
Nathaniel Case
a1517234f6
Fix authorize: yes and become_method: enable (#32864)
* Update connection play_context when socket exists

* Don't fail on connections other than network_cli

* Fix enable prompt detection on ios & eos

* Check against "Module not found" error code, defined in modules/jsonrpc.py
2017-11-14 15:51:14 -05:00
Nathaniel Case
9c0275a879
Connection plugins network_cli and netconf (#32521)
* implements jsonrpc message passing for ansible-connection

* implements more generic mechanism for persistent connections
* starts persistent connection in task_executor if enabled and supported
* supports using network_cli as top level connection plugin
* enhances logging for persistent connection to stdout

* Update action plugins

* Fix Python3 RPC

* Fix Junos bytes<-->str issues

* supports using netconf as top level connection plugin

* Error message when running netconf on an unsupported platform
* Update tests

* Fix `authorize: yes` for `connection: local`

* Handle potentially JSON data in terminal

* Add clarifying detail if possible on ConnectionError
2017-11-09 15:04:40 -05:00
Ganesh Nalawade
97d5e0d027 Remove log of command executed in persistent connection (#31581) 2017-10-11 19:03:21 +05:30
Ganesh Nalawade
cd8c1c1108 Create persistent socket path using port and connection type (#28492)
* Create persistent socket path using port and connection type

*  Use remote address, port, connection type and remote user
   to create a socket path.

* Fix review comment
2017-08-23 18:28:44 -04:00
Matt Clay
8e0b5800b7 Miscellaneous ansible-test fixes. (#28355)
* Include .github in test targets. Fix BOTMETA.yml.
* Include bin in compile tests.
* Exclude links from test targets.
* Include bin in pep8 and pylint tests.
* Fix pep8 and pylint issues in bin dir.
2017-08-17 12:04:43 -07:00
Brian Coca
f921369445 Ansible Config part2 (#27448)
* Ansible Config part2

- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code

skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution

* killed extra space with extreeme prejudice

cause its an affront against all that is holy that 2 spaces touch each other!

shippable timing out on some images, but merging as it passes most
2017-08-15 16:38:59 -04:00
Nathaniel Case
a3404418c0 Fix Python3 in ansible-connection (#27645)
* Fix Python3 in ansible-connection

* Add `errors='surrogate_or_strict'` to `to_bytes()`
2017-08-02 14:10:05 -04:00
Ganesh Nalawade
70ce394840 Persistent connection timer changes (#27272)
*  Add command_timeout timer that defines the amount
   of time to wait for a command or RPC call before
   timing out.
*  Remove connect_retries and connect_interval configuration
   varaible and replace it with connect_retry_timeout to control
   the timeout value of connection to local scoket.
*  Make required changes to netowrk action plugins and relevant
   network files in module_utils.
*  Required documentation changes.
2017-08-01 11:45:45 -06:00
Ganesh Nalawade
19fac707fa junos_netconf integration test failure fix (#27569)
*  Create socket using port value and not connection type
*  Correct error message in integration test task
2017-08-01 21:33:11 +05:30
Ganesh Nalawade
bb998a3cd2 Add fix to read correct socket path recieved from ansible-connection (#27560)
Currently socket path is send from `ansible-connection` (running as background
process) over stdout. This can conflict with debug logs that are also send on
stdout resulting in incorrect socket path received by the main process.

To avoid this add a socket path delimiter string which is recevied by
main process and socket path is retrieved based on delimiter string.

This implementation will change in future when ansible-connection
framework is made more robust.
2017-08-01 19:32:25 +05:30
Nathaniel Case
28c6b226c7 ansible-connection Python3 fix (#26441)
* Fix a number of unicode <-> bytes mismatches

* Return socket_path as text, not bytes

* Docstring run()
2017-07-05 14:07:26 -04:00
James Mighion
76cc19d0f2 Calling connection.close() instead of self.close() to actually close the connection. (#26257) 2017-06-30 07:46:45 +05:30
Brian Coca
9730d96517 added comment on why not os.chdir 2017-06-19 17:17:10 -04:00
Brian Coca
498aea8acc Revert "Add log_only to debug messages (#25545)"
This reverts commit 9e8cc26720.
2017-06-13 09:29:18 -04:00
Ganesh Nalawade
9e8cc26720 Add log_only to debug messages (#25545)
Fixes #25544
When `debug` is enabled the debug messages triggered
from `bin/ansible-connection` should be logged only to file
and not on stdout.
2017-06-13 18:41:33 +05:30
Ganesh Nalawade
6215922889 Add support for cliconf and netconf plugin (#25093)
* ansible-connection refactor and action plugin changes
* Add cliconf plugin for eos, ios, iosxr, junos, nxos, vyos
* Add netconf plugin for junos
* Add jsonrpc support
* Modify network_cli and netconf connection plugin
* Fix py3 unit test failure
* Fix review comment
* Minor fixes
* Fix ansible-connection review comments
* Fix CI issue
* platform_agnostic related changes
2017-06-06 13:56:25 +05:30
Ganesh Nalawade
5ec7f40196 Fix timeout issue in ansible-connection (#24556)
Fixes #24520 ansible-connection needs
to wait on timeout value of play-context
instead of ssh default timeout
2017-05-13 18:28:14 +05:30
Toshio Kuratomi
84a59e472b Be explicit about pickle protocol and encoding (#24454)
On Python3 and Python2 use pickle slightly differently so we need to be
explicit about some things.

If pickles could be shared between python2 and python3, as in
ansible-connection and the pickle cache, we need to specify the protocol
to use when dumping and the encoding to use for byte strings when
loading.

The dumping protocol needs to be no higher than 2 as python-2 only
supports up to protocol 2.  The encoding should usually be 'bytes' so
that python2 str type becomes python3 bytes type.  However, doing this
means that we must make sure that the objects being serialized properly
make their strings into text strings except when they're supposed to be
bytes.  If strings are improperly byte strings, they may cause
tracebacks on the receiving end
2017-05-12 18:04:48 -07:00
Toshio Kuratomi
d834412ead Fix for persistent connection plugin on Python3 (#24431)
Fix for persistent connection plugin on Python3.  Note that fixes are also needed to each terminal plugin.  This PR only fixes the ios terminal (as proof that this approach is workable.)  Future PRs can address the other terminal types.

* On Python3, pickle needs to work with byte strings, not text strings.
* Set the pickle protocol version to 0 because we're using a pty to feed data to the connection plugin.  A pty can't have control characters.  So we have to send ascii only.  That means
only using protocol=0 for pickling the data.
* ansible-connection isn't being used with py3 in the bug but it needs
several changes to work with python3.
* In python3, closing the pty too early causes no data to be sent.  So
leave stdin open until after we finish with the ansible-connection
process.
* Fix typo using traceback.format_exc()
* Cleanup unnecessary StringIO, BytesIO, and to_bytes calls
* Modify the network_cli and terminal plugins for py3 compat.  Lots of mixing of text and byte strings that needs to be straightened out to be compatible with python3
* Documentation for the bytes<=>text strategy for terminal plugins
* Update unittests for more bytes-oriented internals

Fixes #24355
2017-05-12 09:13:51 -07:00
Peter Sprygada
53c52cf65f updates log messages in ansible-connection (#23076) 2017-03-30 13:07:02 +01:00
Peter Sprygada
77ce83fe22 fixes ansible-connection working directory (#22964)
When ansible-connection forks the process, it changed the working
directory to /.  This patch will prevent ansible-connection from
changing the working directory in the forked process.
2017-03-25 00:19:48 -04:00
Peter Sprygada
4a8f2dde20 fixes exception raised from ansible-connection logging (#22836)
updates ansible-connection logging functions to use display instead of
log.
2017-03-21 09:15:23 -07:00
Peter Sprygada
e20ed8bc0c replaces logging with display for network connection plugins (#22819)
* updates network_cli and netconf connection plugins
* updates ansible-connection
2017-03-20 20:08:02 -07:00
Peter Sprygada
ed7cace425 implements python logging for network connection plugin (#22817)
* enables logging for network_cli and paramiko
* enables logging for ansible-connection
* enabled logging for netconf connection
2017-03-20 19:26:18 -07:00
Peter Sprygada
eed240797a adds connection established message to ansible-connection (#22492)
This will now print a log message that displays how long the ssh
connection took to establish to the remote device.  If the connection
established time is exceed it will also print a message to the log file
that the connection attempts have exceed.

Updates default connection retries to 30

fixes #22319
2017-03-14 10:31:02 -04:00
Peter Sprygada
6e9244a9e1 adds more logging output to network_cli and ansible-connection (#21716) 2017-02-21 09:21:41 -05:00
Peter Sprygada
e05b2b56f2 capture AnsibleConnectionFailure messages in ansible-connection (#21612)
Send the messages to the log file
2017-02-18 08:12:01 -05:00
Peter Sprygada
e77964f9da clean up pylint errors in ansible-connection (#21571) 2017-02-17 08:15:27 -05:00
Peter Sprygada
eaba067de9 fixes type that prevents socket clean up from being called (#21568) 2017-02-17 07:13:39 -05:00
Peter Sprygada
3ff2c471b2 fixes Bad file descriptor backtrace raised by ansible-connection (#21526)
This change will redirect stdout and stderr either to the log file
configured by log_path or to /dev/null if no log_path is specified.

fixes #21400
2017-02-16 13:59:47 -05:00
Peter Sprygada
138051540e updates the code path for network modules (#21193)
* replaces persistent connection digest with _create_control_path()
* adds _ansible_socket to _legal_inputs in basic.py
* adds connection_user to play_context
* maps remote_user to connection_user when connection is local
* maps ansible_socket in task_vars to module_args _ansible_socket if exists
2017-02-09 14:05:54 -05:00
Brian Coca
942ed146c3 errors to stderr 2017-02-08 09:47:10 -05:00
Peter Sprygada
b4d75f41f5 one more bug fix for ansible-connection (#20707)
The connection instance will now return a meaningful error if the
connection property returns False
2017-01-26 13:59:57 -05:00
Peter Sprygada
d668d53112 checks connected property to validate connection in Server (#20695)
fixes #20693
2017-01-26 14:30:38 +00:00
Peter Sprygada
6673673042 more logging and exception handling in ansible-connection (#20619)
adds more logging to handle display being called from plugins.  Also
rearranges some of the exception handling to better catch exceptions and
log to local syslog
2017-01-24 23:14:30 -05:00
Peter Sprygada
312328e22c adds more logging to ansible-connection (#20298)
* Server() instance will now add the entire traceback to syslog if it fail
  during init
2017-01-16 08:32:08 -05:00
Peter Sprygada
3f949358c7 adds some more logging to ansible-connection (#20205) 2017-01-12 19:20:25 -05:00
Peter Sprygada
6fe9a5e40c functional updates to ansible-connection (#18574)
* sends the serialized play_context into an already established connection
* hooks the alarm_handler() method in the connection plugin if it exists
* added configuration options for connect interval and retries
* adds syslog logging to Server() instance

This update will send the updated play_context back into an already
established connection in case privilege escalation / descalation activities
need to be performed.  This change will also hook the alarm_handler() method
in the connection instance (if available) and call it in case of a
sigalarm raised.

This update adds two new configuration options

* PERSISTENT_CONNECT_INTERVAL - time to wait in between connection attempts
* PERSISTENT_CONNECT_RETRIES - max number of retries
2016-11-30 16:26:49 -05:00
James Cammarata
336f8d7233 Removing q debugging cruft from ansible-connection 2016-11-21 13:44:43 -06:00
James Cammarata
26ec2ecfce Adding a persistent connection utility 2016-11-21 12:35:27 -06:00