MatrixSynapse/synapse
Erik Johnston 8de3703d21
Make event persisters periodically announce position over replication. (#8499)
Currently background proccesses stream the events stream use the "minimum persisted position" (i.e. `get_current_token()`) rather than the vector clock style tokens. This is broadly fine as it doesn't matter if the background processes lag a small amount. However, in extreme cases (i.e. SyTests) where we only write to one event persister the background processes will never make progress.

This PR changes it so that the `MultiWriterIDGenerator` keeps the current position of a given instance as up to date as possible (i.e using the latest token it sees if its not in the process of persisting anything), and then periodically announces that over replication. This then allows the "minimum persisted position" to advance, albeit with a small lag.
2020-10-12 15:51:41 +01:00
..
_scripts Remove obsolete __future__ imports (#8337) 2020-09-17 08:37:01 -04:00
api Do not assume that account data is of the correct form. (#8454) 2020-10-05 09:28:05 -04:00
app Move additional tasks to the background worker, part 3 (#8489) 2020-10-09 07:37:51 -04:00
appservice Add type hints to response cache. (#8507) 2020-10-09 11:35:11 -04:00
config Increase default max_upload_size from 10M to 50M (#8502) 2020-10-09 16:58:23 +01:00
crypto Fix occasional "Re-starting finished log context" from keyring (#8398) 2020-09-25 12:29:54 +01:00
events Combine `SpamCheckerApi` with the more generic `ModuleApi`. (#8464) 2020-10-07 12:03:26 +01:00
federation Add type hints to response cache. (#8507) 2020-10-09 11:35:11 -04:00
groups Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
handlers Add type hints to response cache. (#8507) 2020-10-09 11:35:11 -04:00
http Don't bother responding to client requests that have already disconnected (#8465) 2020-10-06 10:03:39 +01:00
logging Enable mypy checking for unreachable code and fix instances. (#8432) 2020-10-01 08:09:18 -04:00
metrics Rewrite BucketCollector 2020-09-30 16:49:15 +01:00
module_api Allow modules to create and send events into rooms (#8479) 2020-10-09 13:46:36 +01:00
push Enable mypy checking for unreachable code and fix instances. (#8432) 2020-10-01 08:09:18 -04:00
replication Make event persisters periodically announce position over replication. (#8499) 2020-10-12 15:51:41 +01:00
res/templates Convert additional templates to Jinja (#8444) 2020-10-02 11:15:53 +01:00
rest Remove the deprecated Handlers object (#8494) 2020-10-09 07:24:34 -04:00
server_notices Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
spam_checker_api Combine `SpamCheckerApi` with the more generic `ModuleApi`. (#8464) 2020-10-07 12:03:26 +01:00
state Enable mypy checking for unreachable code and fix instances. (#8432) 2020-10-01 08:09:18 -04:00
static Clean-up the fallback login code. (#7657) 2020-06-10 09:50:39 -04:00
storage Make event persisters periodically announce position over replication. (#8499) 2020-10-12 15:51:41 +01:00
streams Make token serializing/deserializing async (#8427) 2020-09-30 20:29:19 +01:00
util Add type hints to response cache. (#8507) 2020-10-09 11:35:11 -04:00
__init__.py 1.21.0rc3 2020-10-08 11:19:22 +01:00
event_auth.py Ensure that event.redacts is the proper type before handling it (#8457) 2020-10-05 10:24:17 -04:00
notifier.py Add type hints to some handlers (#8505) 2020-10-09 07:20:51 -04:00
python_dependencies.py Drop support for ancient prometheus_client (#8426) 2020-09-30 16:42:05 +01:00
secrets.py Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
server.py Remove the deprecated Handlers object (#8494) 2020-10-09 07:24:34 -04:00
types.py Use vector clocks for room stream tokens. (#8439) 2020-10-07 15:15:33 +01:00
visibility.py Do not assume that account data is of the correct form. (#8454) 2020-10-05 09:28:05 -04:00