299b00d968
Prioritize outbound to-device over device list updates ( #13922 )
...
Otherwise device list changes for large accounts can temporarily delay to-device messages.
2022-09-27 15:17:41 +01:00
a7e506ddee
Reduce amount of state we pull out when attempting to send catchup PDUs. ( #12963 )
...
* Don't pull out state for catchup
* Newsfile
* Merge newsfile
2022-06-07 14:35:56 +01:00
c52abc1cfd
Additional constants for EDU types. ( #12884 )
...
Instead of hard-coding strings in many places.
2022-05-27 07:14:36 -04:00
b5707ceaba
Avoid attempting to delete push actions for remote users. ( #12879 )
...
Remote users will never have push actions, so we can avoid a database
round-trip/transaction completely.
2022-05-26 07:09:16 -04:00
423cca9efe
Spread out sending device lists to remote hosts ( #12132 )
2022-03-04 11:48:15 +00:00
e24ff8ebe3
Remove `HomeServer.get_datastore()` ( #12031 )
...
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.
Part of #11733
2022-02-23 11:04:02 +00:00
a85dde3445
Minor typing fixes ( #12034 )
...
These started failing in
https://github.com/matrix-org/synapse/pull/12031 ... I'm a bit mystified by how
they ever worked.
2022-02-21 18:37:04 +00:00
10a88ba91c
Use auto_attribs/native type hints for attrs classes. ( #11692 )
2022-01-13 13:49:28 +00:00
d2279f471b
Add most of the missing type hints to `synapse.federation`. ( #11483 )
...
This skips a few methods which are difficult to type.
2021-12-02 16:18:10 +00:00
aa2c027792
Remove unnecessary parentheses around tuples returned from methods ( #10889 )
2021-09-23 11:59:07 +01:00
ac5c221208
Stagger send presence to remotes ( #10398 )
...
This is to help with performance, where trying to connect to thousands
of hosts at once can consume a lot of CPU (due to TLS etc).
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-07-15 11:52:56 +01:00
bf72d10dbf
Use inline type hints in various other places (in `synapse/`) ( #10380 )
2021-07-15 11:02:43 +01:00
b378d98c8f
Add debug logging for issue #9533 ( #9959 )
...
Hopefully this will help us track down where to-device messages are getting
lost/delayed.
2021-05-11 11:04:03 +01:00
4e0fd35bc9
Revert "Experimental Federation Speedup ( #9702 )"
...
This reverts commit 05e8c70c05
2021-04-28 11:38:33 +01:00
05e8c70c05
Experimental Federation Speedup ( #9702 )
...
This basically speeds up federation by "squeezing" each individual dual database call (to destinations and destination_rooms), which previously happened per every event, into one call for an entire batch (100 max).
Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
2021-04-14 17:19:02 +01:00
4b965c862d
Remove redundant "coding: utf-8" lines ( #9786 )
...
Part of #9744
Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.
`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
33548f37aa
Improve tracing for to device messages ( #9686 )
2021-04-01 17:08:21 +01:00
c602ba8336
Fixed undefined variable error in catchup ( #9664 )
...
Broke in #9640
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-03-24 16:12:47 +00:00
dd71eb0f8a
Make federation catchup send last event from any server. ( #9640 )
...
Currently federation catchup will send the last *local* event that we
failed to send to the remote. This can cause issues for large rooms
where lots of servers have sent events while the remote server was down,
as when it comes back up again it'll be flooded with events from various
points in the DAG.
Instead, let's make it so that all the servers send the most recent
events, even if its not theirs. The remote should deduplicate the
events, so there shouldn't be much overhead in doing this.
Alternatively, the servers could only send local events if they were
also extremities and hope that the other server will send the event
over, but that is a bit risky.
2021-03-18 15:52:26 +00:00
026503fa3b
Don't go into federation catch up mode so easily ( #9561 )
...
Federation catch up mode is very inefficient if the number of events
that the remote server has missed is small, since handling gaps can be
very expensive, c.f. #9492 .
Instead of going into catch up mode whenever we see an error, we instead
do so only if we've backed off from trying the remote for more than an
hour (the assumption being that in such a case it is more than a
transient failure).
2021-03-15 14:42:40 +00:00
0a00b7ff14
Update black, and run auto formatting over the codebase ( #9381 )
...
- Update black version to the latest
- Run black auto formatting over the codebase
- Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md
2021-02-16 22:32:34 +00:00
f31f8e6319
Remove stream ordering from Metadata dict ( #8452 )
...
There's no need for it to be in the dict as well as the events table. Instead,
we store it in a separate attribute in the EventInternalMetadata object, and
populate that on load.
This means that we can rely on it being correctly populated for any event which
has been persited to the database.
2020-10-05 14:43:14 +01:00
3bd3707cb9
Fix malformed log line in new federation "catch up" logic ( #8442 )
2020-10-02 11:05:29 +01:00
576bc37d31
Catch-up after Federation Outage (split, 4): catch-up loop ( #8272 )
2020-09-15 09:07:19 +01:00
17fa4c7ca7
Catch up after Federation Outage (split, 2): Track last successful stream ordering after transmission ( #8247 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-09-04 15:06:51 +01:00
c619253db8
Stop sub-classing object ( #8249 )
2020-09-04 06:54:56 -04:00
4535e849d7
Remove obsolete order field in `send_new_transaction` ( #8245 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-09-03 19:23:07 +01:00
ad6190c925
Convert stream database to async/await. ( #8074 )
2020-08-17 07:24:46 -04:00
ff0e894656
Drop federation transmission queues during a significant remote outage. ( #7864 )
...
* Empty federation transmission queues when we are backing off.
Fixes #7828 .
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Address feedback
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Reword newsfile
2020-08-13 12:35:04 +01:00
2c1b9d6763
Update worker docs with recent enhancements ( #7969 )
2020-07-29 23:22:13 +01:00
649a7ead5c
Add ability to run multiple pusher instances ( #7855 )
...
This reuses the same scheme as federation sender sharding
2020-07-16 14:06:28 +01:00
f299441cc6
Add ability to shard the federation sender ( #7798 )
2020-07-10 18:26:36 +01:00
38e1fac886
Fix some spelling mistakes / typos. ( #7811 )
2020-07-09 09:52:58 -04:00
075375bbc9
add a comment
2020-05-21 13:25:41 +01:00
d5aa7d93ed
Fix catchup-on-reconnect for the Federation Stream ( #7374 )
...
looks like we managed to break this during the refactorathon.
2020-05-05 14:15:57 +01:00
b08b0a22d5
Add typing to synapse.federation.sender ( #6871 )
2020-02-07 13:56:38 +00:00
d386f2f339
Add StateMap type alias ( #6715 )
2020-01-16 13:31:22 +00:00
6f4bc6d01d
Merge branch 'develop' into cross-signing_federation
2019-10-31 22:38:21 -04:00
020add5099
Update black to 19.10b0 ( #6304 )
...
* update version of black and also fix the mypy config being overridden
2019-11-01 02:43:24 +11:00
bb6cec27a5
rename get_devices_by_remote to get_device_updates_by_remote
2019-10-30 14:57:34 -04:00
c40d7244f8
Merge branch 'develop' into cross-signing_federation
2019-10-24 22:31:25 -04:00
8d3542a64e
implement federation parts of cross-signing
2019-10-22 19:04:35 -04:00
c66a06ac6b
Move storage classes into a main "data store".
...
This is in preparation for having multiple data stores that offer
different functionality, e.g. splitting out state or event storage.
2019-10-21 16:05:06 +01:00
4806651744
Replace returnValue with return ( #5736 )
2019-07-23 23:00:55 +10:00
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
5c15039e06
Clean up code for sending federation EDUs. ( #5381 )
...
This code confused the hell out of me today. Split _get_new_device_messages
into its two (unrelated) parts.
2019-06-13 13:52:08 +01:00
2d1d7b7e6f
Prevent multiple device list updates from breaking a batch send ( #5156 )
...
fixes #5153
2019-06-06 23:54:00 +01:00
130f932cbc
Run `black` on per_destination_queue
...
... mostly to fix pep8 fails
2019-05-09 16:27:02 +01:00
11ea16777f
Limit the number of EDUs in transactions to 100 as expected by receiver ( #5138 )
...
Fixes #3951 .
2019-05-09 11:01:41 +01:00
a902d13180
Batch up outgoing read-receipts to reduce federation traffic. ( #4890 )
...
Rate-limit outgoing read-receipts as per #4730 .
2019-03-20 16:02:25 +00:00
Erik Johnston