Commit Graph

8571 Commits (98ce212093bbe87c8cd2b5c92ee1587aba5cc1fa)

Author SHA1 Message Date
Erik Johnston 98ce212093 Merge pull request #2103 from matrix-org/erikj/no-double-encode
Don't double encode replication data
2017-04-07 09:39:52 +01:00
Erik Johnston 877c029c16 Use iteritems 2017-04-06 15:51:22 +01:00
Erik Johnston 944692ef69 Merge pull request #2106 from matrix-org/erikj/reduce_user_sync
Reduce rate of USER_SYNC repl commands
2017-04-06 13:35:31 +01:00
Erik Johnston 391712a4f9 Comment 2017-04-06 13:35:00 +01:00
Erik Johnston ad544c803a Document types of the replication streams 2017-04-06 13:28:52 +01:00
Erik Johnston dbf87282d3 Docs 2017-04-06 13:11:21 +01:00
Erik Johnston 69b3fd485d Fix incorrect type when using InvalidateCacheCommand 2017-04-06 09:36:38 +01:00
Erik Johnston fcc803b2bf Add log lines 2017-04-05 17:13:44 +01:00
Erik Johnston ea0152b132 Merge pull request #2104 from matrix-org/erikj/metrics_tcp
Rearrange TCP replication metrics
2017-04-05 14:24:06 +01:00
Erik Johnston 3f213d908d Rearrange metrics 2017-04-05 14:15:09 +01:00
Erik Johnston 1ca0e78ca1 Fix typo 2017-04-05 13:43:39 +01:00
Erik Johnston b43d3267e2 Fixup some metrics for tcp repl 2017-04-05 13:34:54 +01:00
Erik Johnston b5cb6347a4 Don't immediately notify the master about users whose syncs have gone away 2017-04-05 13:25:40 +01:00
Erik Johnston 96b9b6c127 Don't double json encode typing replication data 2017-04-05 11:34:20 +01:00
Erik Johnston f10ce8944b Don't double json encode federation replication data 2017-04-05 11:10:28 +01:00
Erik Johnston a5c401bd12 Merge pull request #2097 from matrix-org/erikj/repl_tcp_client
Move to using TCP replication
2017-04-05 09:36:21 +01:00
Erik Johnston b9caf4f726 Merge pull request #2099 from matrix-org/erikj/deviceinbox_reduce
Deduplicate new deviceinbox rows for replication
2017-04-05 09:35:59 +01:00
Erik Johnston d1d5362267 Add comment 2017-04-04 16:41:03 +01:00
Erik Johnston 9f26d3b75b Deduplicate new deviceinbox rows for replication 2017-04-04 16:21:21 +01:00
Erik Johnston a76886726b Merge pull request #2098 from matrix-org/erikj/repl_tcp_fix
Advance replication streams even if nothing is listening
2017-04-04 15:40:51 +01:00
Erik Johnston ac66e11f2b Add the appropriate amount of preserve_fn 2017-04-04 15:22:54 +01:00
Erik Johnston 4264ceb31c Fiddle tcp replication logging 2017-04-04 14:14:03 +01:00
Erik Johnston 023ee197be Advance replication streams even if nothing is listening
Otherwise the streams don't advance and steadily fall behind, so when a
worker does connect either a) they'll be streamed lots of old updates or
b) the connection will fail as the streams are too far behind.
2017-04-04 13:19:26 +01:00
Erik Johnston d1605794ad Remove unused worker config option 2017-04-04 11:17:00 +01:00
Erik Johnston 3376f16012 Shuffle and comment synchrotron presence 2017-04-04 11:14:16 +01:00
Erik Johnston 6ce6bbedcb Move where we ack federation 2017-04-04 11:02:44 +01:00
Erik Johnston 27cc627e42 Merge pull request #2082 from matrix-org/erikj/repl_tcp_server
Replace HTTP replication with TCP replication (Server side part)
2017-04-04 10:07:57 +01:00
Erik Johnston 62b89daac6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server 2017-04-04 09:46:16 +01:00
Richard van der Hoff 773e64cc1a Merge pull request #2095 from matrix-org/rav/cull_log_preserves
Cull spurious PreserveLoggingContexts
2017-04-03 17:02:25 +01:00
Richard van der Hoff 2d05eb3cf5 Merge remote-tracking branch 'origin/release-v0.20.0' into develop 2017-04-03 16:23:23 +01:00
Richard van der Hoff ac63b92b64 Merge pull request #2094 from matrix-org/rav/fix_federation_join
Accept join events from all servers
2017-04-03 16:17:17 +01:00
Richard van der Hoff 30bcbf775a Accept join events from all servers
Make sure that we accept join events from any server, rather than just the
origin server, to make the federation join dance work correctly.

(Fixes #1893).
2017-04-03 15:58:07 +01:00
Richard van der Hoff 7eb9f34cc3 Remove spurious yield
In `MessageHandler`, remove `yield` on call to `Notifier.on_new_room_event`:
it doesn't return anything anyway.
2017-04-03 15:44:19 +01:00
Richard van der Hoff 0b08c48fc5 Remove more spurious `PreserveLoggingContext`s
Remove `PreserveLoggingContext` around calls to `Notifier.on_new_room_event`;
there is no problem if the logcontext is set when calling it.
2017-04-03 15:43:37 +01:00
Richard van der Hoff 65e1683680 Remove spurious PreserveLoggingContext
In `on_new_room_event`, remove `PreserveLoggingContext` - we can call its
subroutines with the logcontext set.
2017-04-03 15:42:38 +01:00
Richard van der Hoff feb496056e preserve_fn some deferred-returning things
In `Notifier._on_new_room_event`, `preserve_fn` around its subroutines which
return deferreds, so that it is safe to call it with an active logcontext.
2017-04-03 15:41:17 +01:00
Richard van der Hoff e2eebf1696 Fix fixme in preserve_fn
`preserve_fn` is no longer used as a decorator anywhere, so we can safely fix a
fixme therein.
2017-04-03 15:38:02 +01:00
Erik Johnston 36c28bc467 Update all the workers and master to use TCP replication 2017-04-03 15:35:52 +01:00
Erik Johnston 3a1f3f8388 Change slave storage to use new replication interface
As the TCP replication uses a slightly different API and streams than
the HTTP replication.

This breaks HTTP replication.
2017-04-03 15:34:19 +01:00
Erik Johnston 52bfa604e1 Add basic replication client handler and factory 2017-04-03 15:34:13 +01:00
Erik Johnston 0a6a966e2b Always advance stream tokens 2017-04-03 15:22:56 +01:00
Richard van der Hoff 773e1c6d68 Remove spurious @preserve_fn decorators
Remove `@preserve_fn` decorators on `on_new_room_event`,
`_notify_pending_new_room_events`, `_on_new_room_event`, `on_new_event`, and
`on_new_replication_data` - none of these functions return a deferred, and the
decorator does nothing unless the wrapped function returns a deferred, so the
decorator was a no-op.
2017-04-03 15:14:11 +01:00
Erik Johnston 0d1c85e643 Merge branch 'release-v0.20.0' of github.com:matrix-org/synapse into develop 2017-04-03 14:58:14 +01:00
Erik Johnston 1df7c28661 Use callbacks to notify tcp replication rather than deferreds 2017-03-31 15:42:51 +01:00
Erik Johnston 36d2b66f90 Add a timestamp to USER_SYNC command
This timestamp is used to indicate when the user last sync'd
2017-03-31 15:42:22 +01:00
Erik Johnston 8a240e4f9c Merge pull request #2078 from APwhitehat/assertuserfriendly
add user friendly report of assertion error in synctl.py
2017-03-31 14:41:49 +01:00
Erik Johnston ec039e6790 Merge pull request #1984 from RyanBreaker/patch-1
Add missing package to CentOS section
2017-03-31 14:39:32 +01:00
Erik Johnston 142b6b4abf Merge pull request #2011 from matrix-org/matthew/turn_allow_guests
add setting (on by default) to support TURN for guests
2017-03-31 14:37:09 +01:00
Erik Johnston 2a06b44be2 Merge pull request #1986 from matrix-org/matthew/enable_guest_3p
enable guest access for the 3pl/3pid APIs
2017-03-31 14:36:03 +01:00
Erik Johnston 2dc57e7413 Merge pull request #2024 from jerrykan/db_port_schema
Don't assume postgres tables are in the public schema during db port
2017-03-31 14:14:30 +01:00