24772ba56e
Respect use_frozen_dicts option in workers
2016-11-08 11:07:18 +00:00
1cc22da600
Set up the process collector during metrics __init__; that way all split-process workers have it
2016-10-27 18:09:34 +01:00
aac13b1f9a
Pass the Metrics group into the process collector instead of having it find its own one; this avoids it needing to import from synapse.metrics
2016-10-27 18:08:15 +01:00
3ae48a1f99
Move the process metrics collector code into its own file
2016-10-19 18:10:24 +01:00
a68ade6ed3
Merge pull request #1162 from larroy/master
...
Use sys.executable instead of hardcoded python. fixes #1161
2016-10-14 21:42:55 +01:00
c57f871184
Use sys.executable instead of hardcoded python. fixes #1161
2016-10-08 23:55:20 +02:00
748d8fdc7b
Reduce DB hits for replication
...
Some streams will occaisonally advance their positions without actually
having any new rows to send over federation. Currently this means that
the token will not advance on the workers, leading to them repeatedly
sending a slightly out of date token. This in turns requires the master
to hit the DB to check if there are any new rows, rather than hitting
the no op logic where we check if the given token matches the current
token.
This commit changes the API to always return an entry if the position
for a stream has changed, allowing workers to advance their tokens
correctly.
2016-09-23 16:49:21 +01:00
87528f0756
Support /initialSync in synchrotron worker
2016-09-21 11:46:28 +01:00
883df2e983
fix logger for client_reader worker
2016-09-17 14:12:04 +01:00
4be85281f9
Enable state caches on workers
2016-09-15 14:31:22 +01:00
e0b6db29ed
Split out public room list into a worker process
2016-09-14 14:42:51 +01:00
44330a21e9
Comment
2016-09-09 17:22:07 +01:00
7ed5acacf4
Fix up the calls to the notifier for device messages
2016-09-01 18:08:40 +01:00
873f870e5a
Add new direct message storage functions to slave store
2016-08-30 09:40:32 +01:00
689f4cb914
Update comment
2016-08-22 16:17:31 +01:00
c8f9b45bc2
Add usage stats to prometheus monitoring
2016-08-22 15:34:38 +01:00
bcbd74dc5b
Remove log lines
2016-08-18 15:52:10 +01:00
07229bbdae
Add appservice worker
2016-08-18 14:59:55 +01:00
403ecd8a2c
Missed a s/federation reader/media repository/ in a log message
2016-08-18 10:26:15 +01:00
396624864a
Add a media repository worker
2016-08-18 09:38:42 +01:00
784a2d4f2c
Remove broken cache stuff
2016-08-15 11:25:48 +01:00
64e7e11853
Implement cache replication stream
2016-08-15 11:16:45 +01:00
4e1cebd56f
Make synchrotron accept /events
2016-08-12 15:31:44 +01:00
5b5148b7ec
Synced up synchrotron set_state with PresenceHandler set_state
2016-08-11 11:48:30 +01:00
24f36469bc
Add federation /version API
2016-08-05 16:36:07 +01:00
5aa024e501
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader
2016-07-29 11:24:56 +01:00
ec8b217722
Add destination retry to slave store
2016-07-28 17:35:53 +01:00
76b89d0edb
Add slace storage functions for public room list
2016-07-28 17:03:40 +01:00
05e7e5e972
Fix flake8 violation
...
Apparently flake8 v3 puts the error on a different line to v2. Easiest way to
make sure that happens is by putting the whole statement on one line :)
2016-07-26 11:59:08 +01:00
aede7248ab
Split out a FederationReader process
2016-07-21 17:37:44 +01:00
66868119dc
Add metrics for psutil derived memory usage
2016-07-20 16:00:21 +01:00
05f1a4596a
Merge branch 'master' into develop
2016-06-23 11:17:48 +01:00
5cc7564c5c
Optionally start or stop workers in synctl.
...
Optionally start or stop an individual worker by passing -w with
the path to the worker config.
Optionally start or stop every worker and the main synapse by
passing -a with a path to a directory containing worker configs.
The "-w" is intended to be used to bounce individual workers proceses.
THe "-a" is intended for when you want to restart all the workers
simultaneuously, for example when performing database upgrades.
2016-06-21 16:38:05 +01:00
13e334506c
Remove the legacy v0 content upload API.
...
The existing content can still be downloaded. The last upload to the
matrix.org server was in January 2015, so it is probably safe to remove
the upload API.
2016-06-21 11:47:39 +01:00
8c75040c25
Fix setting gc thresholds in the workers
2016-06-17 11:48:12 +01:00
a352b68acf
Use worker_ prefixes for worker config, use existing support for multiple config files
2016-06-16 17:29:50 +01:00
364d616792
Access the event_cache_size directly from the server object.
...
This means that the workers can override the event_cache_size
directly without clobbering the value in the main synapse config.
2016-06-16 12:53:15 +01:00
bde13833cb
Access replication_url from the worker config directly
2016-06-16 12:44:40 +01:00
885ee861f7
Inline the synchrotron and pusher configs into the main config
2016-06-16 11:06:12 +01:00
7dbb473339
Add function to load config without generating it
...
Renames ``load_config`` to ``load_or_generate_config``
Adds a method called ``load_config`` that just loads the
config.
The main synapse.app.homeserver will continue to use
``load_or_generate_config`` to retain backwards compat.
However new worker processes can use ``load_config`` to
load the config avoiding some of the cruft needed to generate
the config.
As the new ``load_config`` method is expected to be used by new
configs it removes support for the legacy commandline overrides
that ``load_or_generate_config`` supports
2016-06-09 18:50:38 +01:00
2d1d1025fa
Add gc_threshold to pusher and synchrotron
2016-06-07 16:26:25 +01:00
dded389ac1
Allow setting of gc.set_thresholds
2016-06-07 15:45:56 +01:00
b0932b34cb
Merge pull request #845 from matrix-org/markjh/synchrotron_presence
...
Fix a KeyError in the synchrotron presence
2016-06-06 16:52:27 +01:00
4a5bbb1941
Fix a KeyError in the synchrotron presence
2016-06-06 16:37:12 +01:00
5ef84da4f1
Yield on the sleeps intended to backoff replication
2016-06-06 16:05:28 +01:00
ac9716f154
Fix spelling
2016-06-03 18:10:00 +01:00
8f79084bd4
Add get_presence_list_accepted to the broken caches in synchrotron
2016-06-03 18:03:40 +01:00
48340e4f13
Clear the list of ongoing syncs on shutdown
2016-06-03 15:02:27 +01:00
da491e75b2
Appease flake8
2016-06-03 14:56:36 +01:00
0b3c80a234
Use ClientIpStore to record client ips
2016-06-03 14:55:01 +01:00
Erik Johnston