Commit Graph

182 Commits (45b5fe91225e9188ccd2a3983b125d0c6901fbb9)

Author SHA1 Message Date
Erik Johnston a90c60912f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_event_push_actions 2018-02-23 11:26:31 +00:00
Erik Johnston 50e8657867
Merge pull request #2902 from matrix-org/erikj/split_events_store
Split out get_events and co into a worker store
2018-02-23 11:23:52 +00:00
Erik Johnston 1cf9e071dd
Merge pull request #2899 from matrix-org/erikj/split_pushers
Split PusherStore
2018-02-23 11:23:35 +00:00
Erik Johnston d0957753bf
Merge pull request #2898 from matrix-org/erikj/split_push_rules_store
Split PushRulesStore
2018-02-23 11:23:23 +00:00
Erik Johnston eba93b05bf Split EventsWorkerStore into separate file 2018-02-23 11:01:21 +00:00
Erik Johnston bf8a36e080 Update copyright 2018-02-23 10:52:10 +00:00
Erik Johnston c2ecfcc3a4 Update copyright 2018-02-23 10:41:34 +00:00
Erik Johnston 7e6cf89dc2 Update copyright 2018-02-23 10:39:19 +00:00
Erik Johnston 26d37f7a63 Update copyright 2018-02-23 10:33:55 +00:00
Erik Johnston bb73f55fc6 Use absolute imports 2018-02-23 10:31:16 +00:00
Erik Johnston 27b094f382 Split out get_events and co into a worker store 2018-02-21 11:41:48 +00:00
Erik Johnston d15d237b0d Split out EventPushActionWorkerStore 2018-02-21 11:01:13 +00:00
Erik Johnston 6f72765371 Split PusherStore 2018-02-21 10:54:21 +00:00
Erik Johnston cbaad969f9 Split PushRulesStore 2018-02-21 10:43:31 +00:00
Erik Johnston ca9b9d9703 Split AccountDataStore and TagStore 2018-02-21 10:15:04 +00:00
Erik Johnston 95e4cffd85 Fix comment 2018-02-20 17:58:40 +00:00
Erik Johnston e316bbb4c0 Use abstract base class to access stream IDs 2018-02-20 17:43:57 +00:00
Erik Johnston f5ac4dc2d4 Split ReceiptsStore 2018-02-20 16:28:28 +00:00
Erik Johnston 106906a65e Don't serialize current state over replication 2018-02-15 13:53:18 +00:00
Erik Johnston ef344b10e5 Don't log errors propogated from send_event 2018-02-15 11:03:49 +00:00
Erik Johnston 8ec2e638be Add event_creator worker 2018-02-07 10:32:32 +00:00
Erik Johnston 24dd73028a Add replication http endpoint for event sending 2018-02-07 10:32:32 +00:00
Erik Johnston 3d33eef6fc
Store state groups separately from events (#2784)
* Split state group persist into seperate storage func

* Add per database engine code for state group id gen

* Move store_state_group to StateReadStore

This allows other workers to use it, and so resolve state.

* Hook up store_state_group

* Fix tests

* Rename _store_mult_state_groups_txn

* Rename StateGroupReadStore

* Remove redundant _have_persisted_state_group_txn

* Update comments

* Comment compute_event_context

* Set start val for state_group_id_seq

... otherwise we try to recreate old state groups

* Update comments

* Don't store state for outliers

* Update comment

* Update docstring as state groups are ints
2018-02-06 14:31:24 +00:00
Richard van der Hoff 5c3c32f16f Metrics for number of RDATA commands received
I found myself wishing we had this.
2018-01-15 17:45:55 +00:00
Richard van der Hoff 0edf085b68 Fix some logcontext leaks in replication resource
The @measure_func annotations rely on the wrapped function respecting the
logcontext rules. Add the necessary yields to make this work.
2017-11-23 23:19:43 +00:00
Richard van der Hoff 35a4b63240 Pull out bits of StateStore to a mixin
... so that we don't need to secretly gut-wrench it for use in the slaved
stores. I haven't done the other stores yet, but we should. I'm tired of the
workers breaking every time we tweak the stores because I forgot to gut-wrench
the right method.

fixes https://github.com/matrix-org/synapse/issues/2655.
2017-11-14 11:43:58 +00:00
Richard van der Hoff 6cfee09be9 Make __init__ consitstent across Store heirarchy
Add db_conn parameters to the `__init__` methods of the *Store classes, so that
they are all consistent, which makes the multiple inheritance work correctly
(and so that we can later extract mixins which can be used in the slavedstores)
2017-11-13 10:46:07 +00:00
Richard van der Hoff eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
hera f807f7f804 log when we get an exception handling replication updates 2017-10-12 11:51:24 +01:00
Erik Johnston 2cc998fed8 Fix replication. And notify 2017-07-20 17:13:18 +01:00
Erik Johnston 925b3638ff Reduce log levels in tcp replication 2017-07-11 10:04:21 +01:00
Erik Johnston 27f26e48b7 Serialize user ip command as json 2017-06-27 16:25:38 +01:00
Erik Johnston 8c23221666 Fix up 2017-06-27 15:53:45 +01:00
Erik Johnston 78cefd78d6 Make workers report to master for user ip updates 2017-06-27 14:58:10 +01:00
Erik Johnston dae9a00a28 Initialise exclusive_user_regex 2017-06-21 14:19:33 +01:00
Erik Johnston 8177563ebe Fix for workers 2017-06-21 13:57:49 +01:00
Erik Johnston 6aa5bc8635 Initial worker impl 2017-06-16 11:47:11 +01:00
Erik Johnston d53fe399eb Add cache for is_host_joined 2017-06-13 09:56:18 +01:00
Erik Johnston a837765e8c Merge pull request #2266 from matrix-org/erikj/host_in_room
Change is_host_joined to use current_state table
2017-06-12 09:49:51 +01:00
Erik Johnston 8060974344 Fix replication 2017-06-09 16:40:52 +01:00
Erik Johnston 2cac7623a5 Add missing notifier 2017-06-09 11:24:41 +01:00
Erik Johnston 298d83b340 Fix replication 2017-06-09 11:01:28 +01:00
Erik Johnston dfbda5e025 Faster cache for get_joined_hosts 2017-05-25 17:24:44 +01:00
Erik Johnston f85a415279 Add missing storage function to slave store 2017-05-22 16:31:24 +01:00
Erik Johnston 9ac263ed1b Add new storage functions to slave store 2017-05-04 14:29:03 +01:00
Erik Johnston e4f3431116 Remove unused cache 2017-04-24 13:27:38 +01:00
Erik Johnston 247c736b9b Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl
Reduce federation replication traffic
2017-04-12 11:07:13 +01:00
Erik Johnston 9c712a366f Move get_presence_list_* to SlaveStore 2017-04-11 16:07:33 +01:00
Erik Johnston 28a4649785 Remove HTTP replication APIs 2017-04-11 09:52:11 +01:00
Erik Johnston 29574fd5b3 Reduce federation presence replication traffic
This is mainly done by moving the calculation of where to send presence
updates from the presence handler to the transaction queue, so we only
need to send the presence event (and not the destinations) across the
replication connection. Before we were duplicating by sending the full
state across once per destination.
2017-04-10 16:48:30 +01:00