Commit Graph

195 Commits (ebc5ed1296c433e97d4dcf1c8a5fc1477506e84e)

Author SHA1 Message Date
Erik Johnston 8e1ada9e6f Use the current_state_events.membership column 2019-07-18 14:16:01 +01:00
Erik Johnston 059d8c1a4e Track if current_state_events.membership is up to date 2019-07-18 14:16:01 +01:00
Erik Johnston c618a5d348 Add background update for current_state_events.membership column 2019-07-18 14:16:01 +01:00
Erik Johnston 6de09e07a6 Add membership column to current_state_events table.
It turns out that doing a join is surprisingly expensive for the DB to
do when room_membership table is larger than the disk cache.
2019-07-18 14:15:57 +01:00
Erik Johnston 8ee69f299c Add basic function to get all data for a user out of synapse 2019-07-02 12:09:04 +01:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Erik Johnston 04710cc2d7 Fetch membership counts all at once 2019-05-30 15:25:41 +01:00
Amber Brown 4a30e4acb4
Room Statistics (#4338) 2019-05-21 11:36:50 -05:00
Amber Brown 2e060774ad
Run `black` on some storage modules that the stats branch touches (#4959) 2019-03-29 00:37:16 +11:00
Erik Johnston b8082a5445 Use term 'out of band membership' instead 2019-01-24 17:33:19 +00:00
Erik Johnston 7c288c2250 Clarify the invite flows 2019-01-23 20:07:47 +00:00
Erik Johnston 886e5acc76 Store rejected remote invite events as outliers
Currently they're stored as non-outliers even though the server isn't in
the room, which can be problematic in places where the code assumes it
has the state for all non outlier events.

In particular, there is an edge case where persisting the leave event
triggers a state resolution, which requires looking up the room version
from state. Since the server doesn't have the state, this causes an
exception to be thrown.
2019-01-23 20:04:57 +00:00
Matthew Hodgson b041115415
Speed up lazy loading (#3827)
* speed up room summaries by pulling their data from room_memberships rather than room state
* disable LL for incr syncs, and log incr sync stats  (#3840)
2018-09-12 00:50:39 +01:00
Amber Brown b37c472419
Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678) 2018-08-10 23:50:21 +10:00
Richard van der Hoff 21e878ebb6 Make EventStore inherit from EventFederationStore
(since it uses methods therein)

Turns out that we had a bunch of things which were incorrectly importing
EventWorkerStore from events.py rather than events_worker.py, which broke once
I removed the import into events.py.
2018-07-26 12:48:51 +01:00
Erik Johnston 3188973857 Pull out did_forget to worker store 2018-07-24 16:49:14 +01:00
Richard van der Hoff 4f5cc8e4e7 Merge remote-tracking branch 'origin/develop' into rav/remove_who_forgot_in_room 2018-07-23 17:15:12 +01:00
Richard van der Hoff dae6dc1e77 Remove redundant checks on room forgottenness
Fixes #3550
2018-07-23 17:13:34 +01:00
Erik Johnston e42510ba63 Use new getters 2018-07-23 13:17:22 +01:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
Amber Brown 6350bf925e
Attempt to be more performant on PyPy (#3462) 2018-06-28 14:49:57 +01:00
Erik Johnston f8272813a9 Make _get_joined_hosts_cache cache non-iterable 2018-06-22 15:12:26 +01:00
Richard van der Hoff c1f4118bb6 Remove was_forgotten_at
This is unused. IT MUST DIE!!!1
̧̪͈̱̹̳͖͙H̵̰̤̰͕̖e̛ ͚͉̗̼̞w̶̩̥͉̮h̩̺̪̩͘ͅọ͎͉̟ ̜̩͔̦̘ͅW̪̫̩̣̲͔̳a͏͔̳͖i͖͜t͓̤̠͓͙s̘̰̩̥̙̝ͅ ̲̠̬̥Be̡̙̫̦h̰̩i̛̫͙͔̭̤̗̲n̳͞d̸ ͎̻͘T̛͇̝̲̹̠̗ͅh̫̦̝ͅe̩̫͟ ͓͖̼W͕̳͎͚̙̥ą̙l̘͚̺͔͞ͅl̳͍̙̤̤̮̳.̢
̟̺̜̙͉Z̤̲̙̙͎̥̝A͎̣͔̙͘L̥̻̗̳̻̳̳͢G͉̖̯͓̞̩̦O̹̹̺!̙͈͎̞̬ *
2018-06-01 18:21:49 +01:00
Adrian Tschira 6c16a4ec1b more iteritems 2018-05-24 20:19:06 +02:00
Richard van der Hoff 2fe3f848b9 Remove uses of events.content 2018-03-29 23:17:12 +01:00
Erik Johnston fa72803490 Merge branch 'master' of github.com:matrix-org/synapse into develop 2018-03-19 11:41:01 +00:00
Erik Johnston 926ba76e23 Replace ujson with simplejson 2018-03-15 23:43:31 +00:00
Erik Johnston 8cb44da4aa Fix race in sync when joining room
The race happens when the user joins a room at the same time as doing a
sync. We fetch the current token and then get the rooms the user is in.
If the join happens after the current token, but before we get the rooms
we end up sending down a partial room entry in the sync.

This is fixed by looking at the stream ordering of the membership
returned by get_rooms_for_user, and handling the case when that stream
ordering is after the current token.
2018-03-07 11:55:31 +00:00
Erik Johnston 493e25d554 Move storage functions for push calculations
This will allow push actions for an event to be calculated on workers.
2018-02-27 13:58:16 +00:00
Erik Johnston 70349872c2 Update copyright 2018-02-23 11:14:35 +00:00
Erik Johnston 3dec9c66b3 Split out RoomMemberStore 2018-02-21 12:07:26 +00:00
Richard van der Hoff 6cfee09be9 Make __init__ consitstent across Store heirarchy
Add db_conn parameters to the `__init__` methods of the *Store classes, so that
they are all consistent, which makes the multiple inheritance work correctly
(and so that we can later extract mixins which can be used in the slavedstores)
2017-11-13 10:46:07 +00:00
Richard van der Hoff eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Erik Johnston e8496efe84 Fix up comment 2017-10-09 15:17:34 +01:00
Erik Johnston e4ab96021e Update comments 2017-10-03 14:10:41 +01:00
Erik Johnston 30848c0fcd Ignore incoming events for rooms that we have left
When synapse receives an event for a room its not in over federation, it
double checks with the remote server to see if it is in fact in the
room. This is done so that if the server has forgotten about the room
(usually as a result of the database being dropped) it can recover from
it.

However, in the presence of state resets in large rooms, this can cause
a lot of work for servers that have legitimately left. As a hacky
solution that supports both cases we drop incoming events for rooms that
we have explicitly left.

This means that we no longer support the case of servers having
forgotten that they've rejoined a room, but that is sufficiently rare
that we're not going to support it for now.
2017-10-03 11:18:21 +01:00
Erik Johnston d53fe399eb Add cache for is_host_joined 2017-06-13 09:56:18 +01:00
Erik Johnston 935e588799 Tweak SQL 2017-06-09 13:01:23 +01:00
Erik Johnston 0185b75381 Change is_host_joined to use current_state table
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
2017-06-09 10:52:26 +01:00
Erik Johnston 6ba21bf2b8 Comments 2017-06-07 11:08:36 +01:00
Erik Johnston 6e2a7ee1bc Remove spurious log lines 2017-06-07 11:05:17 +01:00
Erik Johnston 619e8ecd0c Handle None state group correctly 2017-05-26 10:46:03 +01:00
Erik Johnston dfbda5e025 Faster cache for get_joined_hosts 2017-05-25 17:24:44 +01:00
Erik Johnston 85e8092cca Comment 2017-05-17 10:03:09 +01:00
Erik Johnston ad53fc3cf4 Short circuit when we have delta ids 2017-05-17 09:57:34 +01:00
Erik Johnston 13f540ef1b Speed up get_joined_hosts 2017-05-16 16:05:22 +01:00
Erik Johnston ffad4fe35b Don't update event cache hit ratio from get_joined_users
Otherwise the hit ration of plain get_events gets completely skewed by
calls to get_joined_users* functions.
2017-05-08 16:06:17 +01:00
Erik Johnston 66f371e8b8 Merge pull request #2176 from matrix-org/erikj/faster_get_joined
Make get_joined_users faster
2017-05-05 10:59:55 +01:00
Erik Johnston e3aa8a7aa8 Merge pull request #2185 from matrix-org/erikj/smaller_caches
Optimise caches for single key
2017-05-05 10:19:05 +01:00
Erik Johnston cf589f2c1e Fixes 2017-05-05 10:17:56 +01:00