Commit Graph

11628 Commits (a2070a2c4e008ceff6decce3d569f984d5e0f902)

Author SHA1 Message Date
Erik Johnston a2070a2c4e Remove unused 'stream' param of REPLICATE and update docs 2020-03-23 14:56:22 +00:00
Erik Johnston 4f2a803c66 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/catchup_on_worker 2020-03-23 14:49:07 +00:00
Richard van der Hoff a564b92d37
Convert `*StreamRow` classes to inner classes (#7116)
This just helps keep the rows closer to their streams, so that it's easier to
see what the format of each stream is.
2020-03-23 13:59:11 +00:00
Richard van der Hoff 5126cb1253 Merge branch 'master' into develop 2020-03-23 13:54:29 +00:00
Richard van der Hoff 2fa55c0cc6 1.12.0 2020-03-23 12:13:09 +00:00
Richard van der Hoff b3cee0ce67
Fix processing of `groups` stream, and use symbolic names for streams (#7117)
`groups` != `receipts`

Introduced in #6964
2020-03-23 11:39:36 +00:00
Dionysis Grigoropoulos 96071eea8f
Set Referrer-Policy to no-referrer for media (#7009) 2020-03-23 09:48:28 +00:00
Patrick Cloke 477c4f5b1c
Clean-up some auth/login REST code (#7115) 2020-03-20 16:22:47 -04:00
Erik Johnston 32c656865a Always subscribe to all streams.
This already happens since the worker merge.
2020-03-20 15:31:52 +00:00
Erik Johnston 8734b75ca8 Remove unused token param from REPLICATE cmd 2020-03-20 15:31:51 +00:00
Erik Johnston 1f83255de1 Move stream catchup to workers. 2020-03-20 15:31:49 +00:00
Erik Johnston ba90596687 Add ability to catchup on stream by talking to master. 2020-03-20 15:31:47 +00:00
Erik Johnston 811d2ecf2e Don't panic if streams get behind.
The catchup will in future happen on workers, so master process won't
need to protect itself by dropping the connection.
2020-03-20 15:31:45 +00:00
Erik Johnston 7233d38690 Move stream fetch DB queries to worker stores. 2020-03-20 15:31:43 +00:00
Richard van der Hoff c165c1233b
Improve database configuration docs (#6988)
Attempts to clarify the sample config for databases, and add some stuff about
tcp keepalives to `postgres.md`.
2020-03-20 15:24:22 +00:00
Erik Johnston fdb1344716
Remove concept of a non-limited stream. (#7011) 2020-03-20 14:40:47 +00:00
Patrick Cloke caec7d4fa0
Convert some of the media REST code to async/await (#7110) 2020-03-20 07:20:02 -04:00
Patrick Cloke c2db6599c8
Fix a bug in the federation API which could cause occasional "Failed to get PDU" errors (#7089). 2020-03-19 08:22:56 -04:00
Erik Johnston a319cb1dd1
Change device list streams to have one row per ID (#7010)
* Add 'device_lists_outbound_pokes' as extra table.

This makes sure we check all the relevant tables to get the current max
stream ID.

Currently not doing so isn't problematic as the max stream ID in
`device_lists_outbound_pokes` is the same as in `device_lists_stream`,
however that will change.

* Change device lists stream to have one row per id.

This will make it possible to process the streams more incrementally,
avoiding having to process large chunks at once.

* Change device list replication to match new semantics.

Instead of sending down batches of user ID/host tuples, send down a row
per entity (user ID or host).

* Newsfile

* Remove handling of multiple rows per ID

* Fix worker handling

* Comments from review
2020-03-19 11:36:53 +00:00
Richard van der Hoff e43e78b985 1.12.0rc1 2020-03-19 11:07:16 +00:00
Richard van der Hoff e913823a22
Fix concurrent modification errors in pusher metrics (#7106)
add a lock to try to make this metric actually work
2020-03-19 10:28:49 +00:00
Richard van der Hoff 8c75667ad7
Add prometheus metrics for the number of active pushers (#7103) 2020-03-19 10:00:24 +00:00
Richard van der Hoff 443162e577
Move pusherpool startup into _base.setup (#7104)
This should be safe to do on all workers/masters because it is guarded by
a config option which will ensure it is only actually done on the worker
assigned as a pusher.
2020-03-19 09:48:45 +00:00
Erik Johnston 4a17a647a9
Improve get auth chain difference algorithm. (#7095)
It was originally implemented by pulling the full auth chain of all
state sets out of the database and doing set comparison. However, that
can take a lot work if the state and auth chains are large.

Instead, lets try and fetch the auth chains at the same time and
calculate the difference on the fly, allowing us to bail early if all
the auth chains converge. Assuming that the auth chains do converge more
often than not, this should improve performance. Hopefully.
2020-03-18 16:46:41 +00:00
Patrick Cloke 88b41986db
Add an option to the set password API to choose whether to logout other devices. (#7085) 2020-03-18 07:50:00 -04:00
Erik Johnston 6e6476ef07 Comments from review 2020-03-18 10:13:55 +00:00
Richard van der Hoff c37db0211e
Share SSL contexts for non-federation requests (#7094)
Extends #5794 etc to the SimpleHttpClient so that it also applies to non-federation requests.

Fixes #7092.
2020-03-17 21:32:25 +00:00
The Stranjer 5e477c1deb
Set charset to utf-8 when adding headers for certain text content types (#7044)
Fixes #7043
2020-03-17 13:29:09 +00:00
Patrick Cloke 7581d30e9f
Remove unused federation endpoint (`query_auth`) (#7026) 2020-03-17 08:04:49 -04:00
Patrick Cloke 60724c46b7
Remove special casing of `m.room.aliases` events (#7034) 2020-03-17 07:37:04 -04:00
Richard van der Hoff 6a35046363 Revert "Add options to disable setting profile info for prevent changes. (#7053)"
This reverts commit 54dd28621b, reversing
changes made to 6640460d05.
2020-03-17 11:25:01 +00:00
Brendan Abolivier 7df04ca0e6
Populate the room version from state events (#7070)
Fixes #7065 

This is basically the same as https://github.com/matrix-org/synapse/pull/6847 except it tries to populate events from `state_events` rather than `current_state_events`, since the latter might have been cleared from the state of some rooms too early, leaving them with a `NULL` room version.
2020-03-16 22:31:47 +00:00
Brendan Abolivier beb19cf61a
Fix buggy condition in account validity handler (#7074) 2020-03-16 12:16:30 +00:00
Brendan Abolivier d8d91983bc
Merge pull request #7067 from matrix-org/babolivier/saml_error_moar
Move the default SAML2 error HTML to a dedicated file
2020-03-13 19:53:19 +00:00
Brendan Abolivier ebfcbbff9c
Use innerText instead of innerHTML 2020-03-13 19:09:22 +00:00
Patrick Cloke 77d0a4507b
Add type annotations and comments to auth handler (#7063) 2020-03-12 11:36:27 -04:00
Brendan Abolivier 0de9f9486a
Lint 2020-03-11 20:39:18 +00:00
Brendan Abolivier f9e98176bf
Put the file in the templates directory 2020-03-11 20:31:42 +00:00
Brendan Abolivier 900bca9707
Update wording and config 2020-03-11 19:40:30 +00:00
Brendan Abolivier b8cfe79ffc
Move the default SAML2 error HTML to a dedicated file
Also add some JS to it to process any error we might have in the URI
(see #6893).
2020-03-11 19:33:16 +00:00
Brendan Abolivier 8120a238a4
Refactor a bit 2020-03-11 18:49:41 +00:00
Brendan Abolivier 37a9873f63
Also don't fail on aliases events in this case 2020-03-11 18:43:41 +00:00
Brendan Abolivier e38c44b418
Lint 2020-03-11 18:06:07 +00:00
Brendan Abolivier 2dce68c651
Also don't filter out events sent by ignored users when checking state visibility 2020-03-11 17:53:22 +00:00
Brendan Abolivier 9c0775e86a
Fix condition 2020-03-11 17:53:18 +00:00
Brendan Abolivier 69ce55c510
Don't filter out dummy events when we're checking the visibility of state 2020-03-11 17:52:54 +00:00
Brendan Abolivier 54dd28621b
Add options to disable setting profile info for prevent changes. (#7053) 2020-03-10 22:23:01 +00:00
Dirk Klimpel 42ac4ca477
Update synapse/config/registration.py
Co-Authored-By: Brendan Abolivier <github@brendanabolivier.com>
2020-03-10 21:26:55 +01:00
Brendan Abolivier 8f826f98ac
Rephrase default message 2020-03-10 17:22:45 +00:00
Brendan Abolivier dc6fb56c5f
Hopefully mypy is happy now 2020-03-10 14:40:28 +00:00