Fix up docs
parent
63fcc42990
commit
bfcf016714
|
@ -1,20 +1,20 @@
|
|||
TCP Replication
|
||||
===============
|
||||
|
||||
This describes the TCP replication protocol that replaces the HTTP protocol.
|
||||
|
||||
Motivation
|
||||
----------
|
||||
|
||||
The HTTP API used long poll from the workers to the master, this has the problem
|
||||
of causing a lot of duplicate work on the server. This TCP protocol aims to
|
||||
solve.
|
||||
Previously the workers used an HTTP long poll mechanism to get updates from the
|
||||
master, which had the problem of causing a lot of duplicate work on the server.
|
||||
This TCP protocol replaces those APIs with the aim of increased efficiency.
|
||||
|
||||
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
||||
The protocol is based on fire and forget, line based commands. An example flow
|
||||
would be (where '>' indicates master->worker and '<' worker->master flows)::
|
||||
would be (where '>' indicates master to worker and '<' worker to master flows)::
|
||||
|
||||
> SERVER example.com
|
||||
< REPLICATE events 53
|
||||
|
@ -24,7 +24,7 @@ would be (where '>' indicates master->worker and '<' worker->master flows)::
|
|||
The example shows the server accepting a new connection and sending its identity
|
||||
with the ``SERVER`` command, followed by the client asking to subscribe to the
|
||||
``events`` stream from the token ``53``. The server then periodically sends ``RDATA``
|
||||
commands which have the format ``RDATA <stream_name> <token> <row>```, where the
|
||||
commands which have the format ``RDATA <stream_name> <token> <row>``, where the
|
||||
format of ``<row>`` is defined by the individual streams.
|
||||
|
||||
Error reporting happens by either the client or server sending an `ERROR`
|
||||
|
@ -125,7 +125,7 @@ recovers it can reconnect to the server and ask for missed messages.
|
|||
Reliability
|
||||
~~~~~~~~~~~
|
||||
|
||||
In general the replication stream should be consisdered an unreliable transport
|
||||
In general the replication stream should be considered an unreliable transport
|
||||
since e.g. commands are not resent if the connection disappears.
|
||||
|
||||
The exception to that are the replication streams, i.e. RDATA commands, since
|
||||
|
|
|
@ -16,22 +16,7 @@
|
|||
"""This module implements the TCP replication protocol used by synapse to
|
||||
communicate between the master process and its workers (when they're enabled).
|
||||
|
||||
The protocol is based on fire and forget, line based commands. An example flow
|
||||
would be (where '>' indicates master->worker and '<' worker->master flows)::
|
||||
|
||||
> SERVER example.com
|
||||
< REPLICATE events 53
|
||||
> RDATA events 54 ["$foo1:bar.com", ...]
|
||||
> RDATA events 55 ["$foo4:bar.com", ...]
|
||||
|
||||
The example shows the server accepting a new connection and sending its identity
|
||||
with the `SERVER` command, followed by the client asking to subscribe to the
|
||||
`events` stream from the token `53`. The server then periodically sends `RDATA`
|
||||
commands which have the format `RDATA <stream_name> <token> <row>`, where the
|
||||
format of `<row>` is defined by the individual streams.
|
||||
|
||||
Error reporting happens by either the client or server sending an `ERROR`
|
||||
command, and usually the connection will be closed.
|
||||
Further details can be found in docs/tcp_replication.rst
|
||||
|
||||
|
||||
Structure of the module:
|
||||
|
@ -42,5 +27,4 @@ Structure of the module:
|
|||
* resource.py - the server classes that accepts and handle client connections
|
||||
* streams.py - the definitons of all the valid streams
|
||||
|
||||
Further details can be found in docs/tcp_replication.rst
|
||||
"""
|
||||
|
|
|
@ -154,8 +154,8 @@ class Stream(object):
|
|||
True then limit is provided, otherwise it's not.
|
||||
|
||||
Returns:
|
||||
list(tuple): the first entry in the tuple is the token for that
|
||||
update, and the rest of the tuple gets used to construct
|
||||
Deferred(list(tuple)): the first entry in the tuple is the token for
|
||||
that update, and the rest of the tuple gets used to construct
|
||||
a ``ROW_TYPE`` instance
|
||||
"""
|
||||
raise NotImplementedError()
|
||||
|
|
|
@ -139,7 +139,7 @@ class PusherStore(SQLBaseStore):
|
|||
"""Get all the pushers that have changed between the given tokens.
|
||||
|
||||
Returns:
|
||||
list(tuple): each tuple consists of:
|
||||
Deferred(list(tuple)): each tuple consists of:
|
||||
stream_id (str)
|
||||
user_id (str)
|
||||
app_id (str)
|
||||
|
|
Loading…
Reference in New Issue