d5e73cb6aa
Define StateMap as immutable and add a MutableStateMap type. ( #8183 )
2020-08-28 07:28:53 -04:00
5758dcf30c
Add type hints for state. ( #8140 )
2020-08-24 14:25:27 -04:00
f40645e60b
Convert events worker database to async/await. ( #8071 )
2020-08-18 16:20:49 -04:00
a7bdf98d01
Rename database classes to make some sense ( #8033 )
2020-08-05 21:38:57 +01:00
b975fa2e99
Convert state resolution to async/await ( #7942 )
2020-07-24 10:59:51 -04:00
831b31e563
Add another yield point to state res v2 ( #7746 )
2020-06-26 10:44:52 +01:00
0e0a2817a2
Yield during large v2 state res. ( #7735 )
...
State res v2 across large data sets can be very CPU intensive, and if
all the relevant events are in the cache the algorithm will run from
start to finish within a single reactor tick. This can result in
blocking the reactor tick for several seconds, which can have major
repercussions on other requests.
To fix this we simply add the occaisonal `sleep(0)` during iterations to
yield execution until the next reactor tick. The aim is to only do this
for large data sets so that we don't impact otherwise quick resolutions.=
2020-06-24 18:48:18 +01:00
91e886d615
Speed up state res v2 across large state differences. ( #7725 )
2020-06-19 13:56:35 +01:00
bd6dc17221
Replace iteritems/itervalues/iterkeys with native versions. ( #7692 )
2020-06-15 07:03:36 -04:00
7cb8b4bc67
Allow configuration of Synapse's cache without using synctl or environment variables ( #6391 )
2020-05-11 18:45:23 +01:00
4a17a647a9
Improve get auth chain difference algorithm. ( #7095 )
...
It was originally implemented by pulling the full auth chain of all
state sets out of the database and doing set comparison. However, that
can take a lot work if the state and auth chains are large.
Instead, lets try and fetch the auth chains at the same time and
calculate the difference on the fly, allowing us to bail early if all
the auth chains converge. Assuming that the auth chains do converge more
often than not, this should improve performance. Hopefully.
2020-03-18 16:46:41 +00:00
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 ( #6957 )
...
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
2b37eabca1
Reduce auth chains fetched during v2 state res. ( #6952 )
...
The state res v2 algorithm only cares about the difference between auth
chains, so we can pass in the known common state to the `get_auth_chain`
storage function so that it can ignore those events.
2020-02-19 15:04:47 +00:00
d7bf793cc1
s/get_room_version/get_room_version_id/
...
... to make way for a forthcoming get_room_version which returns a RoomVersion
object.
2020-01-31 10:06:21 +00:00
a8ce7aeb43
Pass room version object into event_auth.check and check_redaction ( #6788 )
...
These are easier to work with than the strings and we normally have one around.
This fixes `FederationHander._persist_auth_tree` which was passing a
RoomVersion object into event_auth.check instead of a string.
2020-01-28 14:18:29 +00:00
d386f2f339
Add StateMap type alias ( #6715 )
2020-01-16 13:31:22 +00:00
dc96943d51
Merge branch 'master' into develop
2019-12-31 11:01:06 +00:00
92eac974b9
Hacks to work around #6605 ( #6608 )
...
When we have an event which refers to non-existent auth_events, ignore said events rather than exploding in a ball of fire.
Fixes #6605 .
2019-12-31 10:41:44 +00:00
83895316d4
sanity-checking for events used in state res ( #6531 )
...
When we perform state resolution, check that all of the events involved are in
the right room.
2019-12-16 13:53:53 +00:00
1da15f05f5
sanity-checking for events used in state res ( #6531 )
...
When we perform state resolution, check that all of the events involved are in
the right room.
2019-12-13 12:55:32 +00:00
fc316a4894
Prevent redacted events from appearing in message search ( #6377 )
2019-12-11 13:39:47 +00:00
807ec3bd99
Fix bug which caused rejected events to be stored with the wrong room state ( #6320 )
...
Fixes a bug where rejected events were persisted with the wrong state group.
Also fixes an occasional internal-server-error when receiving events over
federation which are rejected and (possibly because they are
backwards-extremities) have no prev_group.
Fixes #6289 .
2019-11-06 10:01:39 +00:00
4086002827
Improve documentation for EventContext fields ( #6319 )
2019-11-05 13:23:25 +00:00
69f0054ce6
Port to use state storage
2019-10-30 14:46:54 +00:00
66537e10ce
add some metrics on the federation sender ( #6160 )
2019-10-03 17:47:20 +01:00
2a44782666
Remove double return statements ( #5962 )
...
Remove all the "double return" statements which were a result of us removing all the instances of
```
defer.returnValue(...)
return
```
statements when we switched to python3 fully.
2019-09-03 11:42:45 +01:00
4806651744
Replace returnValue with return ( #5736 )
2019-07-23 23:00:55 +10:00
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ ( #5606 )
2019-07-04 00:07:04 +10:00
45f28a9d2f
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/histogram_extremities
2019-06-20 11:59:14 +01:00
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
65787b0f7c
Add descriptions and remove redundant set(..)
2019-06-19 11:49:39 +01:00
16a3124b76
Only count non-cache state resolution
2019-06-18 13:02:06 +01:00
c9385dd238
Use consistent buckets
2019-06-18 12:43:41 +01:00
ff88d36dcb
Add metric fo number of state groups in resolution
2019-06-17 17:02:53 +01:00
4a4d5c4fd6
Fix grammar and document get_current_users_in_room ( #4998 )
2019-04-03 14:32:20 +01:00
54a87a7b08
Collect room-version variations into one place ( #4969 )
...
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
a1b0e1879b
Enable room version v3
2019-01-29 23:09:10 +00:00
47e2dd1994
Drop vdh support
2019-01-29 21:24:34 +00:00
ae2a957dba
Pass through room version to event auth
2019-01-25 18:31:41 +00:00
cd018e3543
Merge pull request #4307 from matrix-org/erikj/v2_rooms
...
Add v2 room version
2018-12-24 10:48:59 +01:00
df89f8afb8
Add v2 room version
2018-12-18 18:10:37 +00:00
c03324294d
Workaround for non-ascii event ids ( #4241 )
...
It turns out that we accept events with non-ascii IDs, which would later cause
an explosion during state res.
Fixes #4226
2018-12-03 21:47:48 +11:00
bc80b3f454
Add helpers for getting prev and auth events ( #4139 )
...
* Add helpers for getting prev and auth events
This is in preparation for allowing the event format to change between
room versions.
2018-11-06 00:35:15 +11:00
54aec35867
Fix None exception in state res v2
2018-11-02 10:29:19 +00:00
b3dd6fa981
Add STATE_V2_TEST room version
2018-11-01 11:43:46 +00:00
810715f79a
Rename resolve_events_with_factory
2018-10-24 09:44:22 +01:00
e238013c44
Add v2 state res algorithm.
...
We hook this up to the vdh test room version.
2018-10-16 16:16:13 +01:00
e70b4ce069
Logging improvements
...
Some logging tweaks to help with debugging incoming federation transactions
2018-09-26 17:36:14 +01:00
8601c24287
Fix some instances of ExpiringCache not expiring cache items
...
ExpiringCache required that `start()` be called before it would actually
start expiring entries. A number of places didn't do that.
This PR removes `start` from ExpiringCache, and automatically starts
backround reaping process on creation instead.
2018-09-21 14:19:46 +01:00
5c261107c9
Remove unnecessary resolve_events_with_state_map
...
We only ever used the synchronous resolve_events_with_state_map in one
place, which is trivial to replace with the async version.
2018-08-22 15:41:15 +01:00
Patrick Cloke