29574fd5b3
Reduce federation presence replication traffic
...
This is mainly done by moving the calculation of where to send presence
updates from the presence handler to the transaction queue, so we only
need to send the presence event (and not the destinations) across the
replication connection. Before we were duplicating by sending the full
state across once per destination.
2017-04-10 16:48:30 +01:00
449d1297ca
Fix up federation SendQueue and document types
2017-04-07 11:48:33 +01:00
54f59bd7d4
Merge pull request #2107 from HarHarLinks/patch-1
...
fix typo in synctl help
2017-04-07 09:54:37 +01:00
98ce212093
Merge pull request #2103 from matrix-org/erikj/no-double-encode
...
Don't double encode replication data
2017-04-07 09:39:52 +01:00
8a1137ceab
fix typo in synctl help
2017-04-06 17:10:20 +02:00
391712a4f9
Comment
2017-04-06 13:35:00 +01:00
dbf87282d3
Docs
2017-04-06 13:11:21 +01:00
b5cb6347a4
Don't immediately notify the master about users whose syncs have gone away
2017-04-05 13:25:40 +01:00
96b9b6c127
Don't double json encode typing replication data
2017-04-05 11:34:20 +01:00
f10ce8944b
Don't double json encode federation replication data
2017-04-05 11:10:28 +01:00
a5c401bd12
Merge pull request #2097 from matrix-org/erikj/repl_tcp_client
...
Move to using TCP replication
2017-04-05 09:36:21 +01:00
d1d5362267
Add comment
2017-04-04 16:41:03 +01:00
ac66e11f2b
Add the appropriate amount of preserve_fn
2017-04-04 15:22:54 +01:00
3376f16012
Shuffle and comment synchrotron presence
2017-04-04 11:14:16 +01:00
6ce6bbedcb
Move where we ack federation
2017-04-04 11:02:44 +01:00
62b89daac6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server
2017-04-04 09:46:16 +01:00
36c28bc467
Update all the workers and master to use TCP replication
2017-04-03 15:35:52 +01:00
8a240e4f9c
Merge pull request #2078 from APwhitehat/assertuserfriendly
...
add user friendly report of assertion error in synctl.py
2017-03-31 14:41:49 +01:00
3ba2859e0c
Add tcp replication listener type and hook it up
2017-03-30 13:31:10 +01:00
6cdca71079
synctl.py: wait for synapse to stop before restarting ( #2020 )
2017-03-29 19:20:13 +01:00
305d16d612
add user friendly report of assertion error in synctl.py
...
Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2017-03-29 20:41:39 +05:30
48e7697911
Add slave transaction store
2017-03-24 13:05:30 +00:00
37a187bfab
Merge pull request #2033 from matrix-org/erikj/repl_speed
...
Don't send the full event json over replication
2017-03-21 13:11:15 +00:00
aac6d1fc9b
PEP8
2017-03-20 13:47:56 +00:00
067b00d49d
Run the reactor with the sentinel logcontext
...
This fixes a class of 'Unexpected logcontext' messages, which were happening
because the logcontext was somewhat arbitrarily swapping between the sentinel
and the `run` logcontext.
2017-03-18 00:07:43 +00:00
61f471f779
Don't send the full event json over replication
2017-03-17 15:50:01 +00:00
6957bfdca6
Don't recreate so many sets
2017-03-16 11:54:26 +00:00
d84bd51e95
Refactor logger config for workers
...
- to make it easier to add more config options.
2017-03-10 15:34:01 +00:00
3a46280ca3
Add db functions needed for room initial sync to slave
2017-02-13 11:16:53 +00:00
4ac363a168
Remove debug logging
2017-01-30 14:10:12 +00:00
252b503fc8
Hook device list updates to replication
2017-01-27 14:31:35 +00:00
edd6cdfc9a
Restore default bind address
2017-01-10 17:21:41 +00:00
8a0fddfd73
Remove spurious for..else..
2017-01-10 16:30:53 +00:00
702c020e58
Fix check for bind_address
...
The empty string is a valid setting for the bind_address option, so
explicitly check for None here instead.
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-20 01:37:50 +01:00
7dfd70fc83
Add support for specifying multiple bind addresses
...
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-18 21:51:56 +01:00
4d9b5c60f9
Comment
2016-11-23 11:11:41 +00:00
4c79a63fd7
Explicit federation ack
2016-11-23 10:40:44 +00:00
50934ce460
Comments
2016-11-21 16:55:23 +00:00
7c9cdb2245
Store federation stream positions in the database
2016-11-21 11:33:08 +00:00
f8ee66250a
Handle sending events and device messages over federation
2016-11-17 15:48:04 +00:00
ed787cf09e
Hook up the send queue and create a federation sender worker
2016-11-16 17:34:44 +00:00
24772ba56e
Respect use_frozen_dicts option in workers
2016-11-08 11:07:18 +00:00
1cc22da600
Set up the process collector during metrics __init__; that way all split-process workers have it
2016-10-27 18:09:34 +01:00
aac13b1f9a
Pass the Metrics group into the process collector instead of having it find its own one; this avoids it needing to import from synapse.metrics
2016-10-27 18:08:15 +01:00
3ae48a1f99
Move the process metrics collector code into its own file
2016-10-19 18:10:24 +01:00
a68ade6ed3
Merge pull request #1162 from larroy/master
...
Use sys.executable instead of hardcoded python. fixes #1161
2016-10-14 21:42:55 +01:00
c57f871184
Use sys.executable instead of hardcoded python. fixes #1161
2016-10-08 23:55:20 +02:00
748d8fdc7b
Reduce DB hits for replication
...
Some streams will occaisonally advance their positions without actually
having any new rows to send over federation. Currently this means that
the token will not advance on the workers, leading to them repeatedly
sending a slightly out of date token. This in turns requires the master
to hit the DB to check if there are any new rows, rather than hitting
the no op logic where we check if the given token matches the current
token.
This commit changes the API to always return an entry if the position
for a stream has changed, allowing workers to advance their tokens
correctly.
2016-09-23 16:49:21 +01:00
87528f0756
Support /initialSync in synchrotron worker
2016-09-21 11:46:28 +01:00
883df2e983
fix logger for client_reader worker
2016-09-17 14:12:04 +01:00
Erik Johnston