tileradotorg/synapse
1
0
Fork 0
forked from MirrorHub/synapse
Code Pull requests Activity

Fix up docs

Browse source
This commit is contained in:
Erik Johnston 2017-03-31 11:19:24 +01:00
parent 63fcc42990
commit bfcf016714
4 changed files with 12 additions and 28 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
docs/tcp_replication.rst
View file

@ -1,20 +1,20 @@
TCP Replication TCP Replication
=============== ===============
This describes the TCP replication protocol that replaces the HTTP protocol.
Motivation Motivation
---------- ----------
The HTTP API used long poll from the workers to the master, this has the problem Previously the workers used an HTTP long poll mechanism to get updates from the
of causing a lot of duplicate work on the server. This TCP protocol aims to master, which had the problem of causing a lot of duplicate work on the server.
solve. This TCP protocol replaces those APIs with the aim of increased efficiency.
Overview Overview
-------- --------
The protocol is based on fire and forget, line based commands. An example flow The protocol is based on fire and forget, line based commands. An example flow
would be (where '>' indicates master->worker and '<' worker->master flows):: would be (where '>' indicates master to worker and '<' worker to master flows)::
> SERVER example.com > SERVER example.com
< REPLICATE events 53 < REPLICATE events 53
@ -24,7 +24,7 @@ would be (where '>' indicates master->worker and '<' worker->master flows)::
The example shows the server accepting a new connection and sending its identity The example shows the server accepting a new connection and sending its identity
with the ``SERVER`` command, followed by the client asking to subscribe to the with the ``SERVER`` command, followed by the client asking to subscribe to the
``events`` stream from the token ``53``. The server then periodically sends ``RDATA`` ``events`` stream from the token ``53``. The server then periodically sends ``RDATA``
commands which have the format ``RDATA <stream_name> <token> <row>```, where the commands which have the format ``RDATA <stream_name> <token> <row>``, where the
format of ``<row>`` is defined by the individual streams. format of ``<row>`` is defined by the individual streams.
Error reporting happens by either the client or server sending an `ERROR` Error reporting happens by either the client or server sending an `ERROR`
@ -125,7 +125,7 @@ recovers it can reconnect to the server and ask for missed messages.
Reliability Reliability
~~~~~~~~~~~ ~~~~~~~~~~~
In general the replication stream should be consisdered an unreliable transport In general the replication stream should be considered an unreliable transport
since e.g. commands are not resent if the connection disappears. since e.g. commands are not resent if the connection disappears.
The exception to that are the replication streams, i.e. RDATA commands, since The exception to that are the replication streams, i.e. RDATA commands, since

18
synapse/replication/tcp/__init__.py
View file

@ -16,22 +16,7 @@
"""This module implements the TCP replication protocol used by synapse to """This module implements the TCP replication protocol used by synapse to
communicate between the master process and its workers (when they're enabled). communicate between the master process and its workers (when they're enabled).
The protocol is based on fire and forget, line based commands. An example flow Further details can be found in docs/tcp_replication.rst
would be (where '>' indicates master->worker and '<' worker->master flows)::
> SERVER example.com
< REPLICATE events 53
> RDATA events 54 ["$foo1:bar.com", ...]
> RDATA events 55 ["$foo4:bar.com", ...]
The example shows the server accepting a new connection and sending its identity
with the `SERVER` command, followed by the client asking to subscribe to the
`events` stream from the token `53`. The server then periodically sends `RDATA`
commands which have the format `RDATA <stream_name> <token> <row>`, where the
format of `<row>` is defined by the individual streams.
Error reporting happens by either the client or server sending an `ERROR`
command, and usually the connection will be closed.
Structure of the module: Structure of the module:
@ -42,5 +27,4 @@ Structure of the module:
* resource.py - the server classes that accepts and handle client connections * resource.py - the server classes that accepts and handle client connections
* streams.py - the definitons of all the valid streams * streams.py - the definitons of all the valid streams
Further details can be found in docs/tcp_replication.rst
""" """

4
synapse/replication/tcp/streams.py
View file

@ -154,8 +154,8 @@ class Stream(object):
True then limit is provided, otherwise it's not. True then limit is provided, otherwise it's not.
Returns: Returns:
list(tuple): the first entry in the tuple is the token for that Deferred(list(tuple)): the first entry in the tuple is the token for
update, and the rest of the tuple gets used to construct that update, and the rest of the tuple gets used to construct
a ``ROW_TYPE`` instance a ``ROW_TYPE`` instance
""" """
raise NotImplementedError() raise NotImplementedError()

2
synapse/storage/pusher.py
View file

@ -139,7 +139,7 @@ class PusherStore(SQLBaseStore):
"""Get all the pushers that have changed between the given tokens. """Get all the pushers that have changed between the given tokens.
Returns: Returns:
list(tuple): each tuple consists of: Deferred(list(tuple)): each tuple consists of:
stream_id (str) stream_id (str)
user_id (str) user_id (str)
app_id (str) app_id (str)