MatrixSynapse/synapse
Erik Johnston 52a50e8686
Use vector clocks for room stream tokens. (#8439)
Currently when using multiple event persisters we (in the worst case) don't tell clients about events until all event persisters have persisted new events after the original event. This is a suboptimal, especially if one of the event persisters goes down.

To handle this, we encode the position of each event persister in the room tokens so that we can send events to clients immediately. To reduce the size of the token we do two things:

1. We create a unique immutable persistent mapping between instance names and a generated small integer ID, which we can encode in the tokens instead of the instance name; and
2. We encode the "persisted upto position" of the room token and then only explicitly include instances that have positions strictly greater than that.

The new tokens look something like: `m3478~1.3488~2.3489`, where the first number is the min position, and the subsequent `-` separated pairs are the instance ID to positions map. (We use `.` and `~` as separators as they're URL safe and not already used by `StreamToken`).
2020-10-07 15:15:33 +01:00
..
_scripts
api
app
appservice
config
crypto
events
federation
groups
handlers
http
logging
metrics
module_api
push
replication
res/templates
rest
server_notices
spam_checker_api
state
static
storage Use vector clocks for room stream tokens. (#8439) 2020-10-07 15:15:33 +01:00
streams
util
__init__.py
event_auth.py
notifier.py
python_dependencies.py
secrets.py
server.py
types.py Use vector clocks for room stream tokens. (#8439) 2020-10-07 15:15:33 +01:00
visibility.py