f455b0e420
GHA: reintroduce an env var for `$GITHUB_HEAD_REF` ( #10659 )
...
This should ensure GHA runs synapse against the same-named sytest branch
2021-09-20 17:35:16 +01:00
b3590614da
Require type hints in the handlers module. ( #10831 )
...
Adds missing type hints to methods in the synapse.handlers
module and requires all methods to have type hints there.
This also removes the unused construct_auth_difference method
from the FederationHandler.
2021-09-20 08:56:23 -04:00
437961744c
Fix remove_stale_pushers job on SQLite. ( #10843 )
2021-09-20 10:26:13 +01:00
6b6bb81b23
Fix #10837 by adding JSON encoding/decoding to the Module API example… ( #10845 )
2021-09-17 18:04:37 +01:00
bfb4b858a9
Create a constant for a small png image in tests. ( #10834 )
...
To avoid duplicating it between a few tests.
2021-09-16 12:01:14 -04:00
3eba047d38
Add type hints to state database module. ( #10823 )
2021-09-15 09:54:13 -04:00
b93259082c
Add missing type hints to non-client REST servlets. ( #10817 )
...
Including admin, consent, key, synapse, and media. All REST servlets
(the synapse.rest module) now require typed method definitions.
2021-09-15 08:45:32 -04:00
8c7a531e27
Use direct references for some configuration variables (part 2) ( #10812 )
2021-09-15 08:34:52 -04:00
145c006ef7
Verify `?chunk_id` actually corresponds to an insertion event that exists (MSC2716) ( #10776 )
2021-09-15 09:34:30 +01:00
1c555527b3
Split out `/batch_send` meta events to their own fields (MSC2716) ( #10777 )
2021-09-15 09:30:58 +01:00
8eb7cb2e0d
Make StateFilter frozen so we can hash it ( #10816 )
...
Also enables Mypy for related tests.
2021-09-14 16:35:53 +01:00
14b8c0476f
Prevent logging context going missing on federation request timeout ( #10810 )
...
In `MatrixFederationHttpClient._send_request()`, we make a HTTP request
using an `Agent`, wrap that request in a timeout and await the resulting
`Deferred`. On its own, the `Agent` performing the HTTP request
correctly stashes and restores the logging context while waiting.
The addition of the timeout introduces a path where the logging context
is not restored when execution resumes.
To address this, we wrap the timeout `Deferred` in a
`make_deferred_yieldable()` to stash the logging context and restore it
on completion of the `await`. However this is not sufficient, since by
the time we construct the timeout `Deferred`, the `Agent` has already
stashed and cleared the logging context when using
`make_deferred_yieldable()` to produce its `Deferred` for the request.
Hence, we wrap the `Agent` request in a `run_in_background()` to "fork"
and preserve the logging context so that we can stash and restore it
when `await`ing the timeout `Deferred`.
This approach is similar to the one used with `defer.gatherResults`.
Note that the code is still not fully correct. When a timeout occurs,
the request remains running in the background (existing behavior which
is nothing to do with the new call to `run_in_background`) and may
re-start the logging context after it has finished.
2021-09-14 13:01:30 +01:00
51e1b96d04
Synapse 1.43.0rc1 (2021-09-14)
...
This release drops support for the deprecated, unstable API for [MSC2858](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2858-Multiple-SSO-Identity-Providers.md#unstable-prefix ), as well as the undocumented `experimental.msc2858_enabled` config option. Client authors should update their clients to use the stable API, available since Synapse 1.30.
Features
--------
- Allow room creators to send historical events specified by [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) in existing room versions. ([\#10566](https://github.com/matrix-org/synapse/issues/10566 ))
- Add config option to use non-default manhole password and keys. ([\#10643](https://github.com/matrix-org/synapse/issues/10643 ))
- Skip final GC at shutdown to improve restart performance. ([\#10712](https://github.com/matrix-org/synapse/issues/10712 ))
- Allow configuration of the oEmbed URLs used for URL previews. ([\#10714](https://github.com/matrix-org/synapse/issues/10714 ), [\#10759](https://github.com/matrix-org/synapse/issues/10759 ))
- Prefer [room version 9](https://github.com/matrix-org/matrix-doc/pull/3375 ) for restricted rooms per the [room version capabilities](https://github.com/matrix-org/matrix-doc/pull/3244 ) API. ([\#10772](https://github.com/matrix-org/synapse/issues/10772 ))
Bugfixes
--------
- Fix a long-standing bug where room avatars were not included in email notifications. ([\#10658](https://github.com/matrix-org/synapse/issues/10658 ))
- Fix a bug where the ordering algorithm was skipping the `origin_server_ts` step in the spaces summary resulting in unstable room orderings. ([\#10730](https://github.com/matrix-org/synapse/issues/10730 ))
- Fix edge case when persisting events into a room where there are multiple events we previously hadn't calculated auth chains for (and hadn't marked as needing to be calculated). ([\#10743](https://github.com/matrix-org/synapse/issues/10743 ))
- Fix a bug which prevented calls to `/createRoom` that included the `room_alias_name` parameter from being handled by worker processes. ([\#10757](https://github.com/matrix-org/synapse/issues/10757 ))
- Fix a bug which prevented user registration via SSO to require consent tracking for SSO mapping providers that don't prompt for Matrix ID selection. Contributed by @AndrewFerr. ([\#10733](https://github.com/matrix-org/synapse/issues/10733 ))
- Only return the stripped state events for the `m.space.child` events in a room for the spaces summary from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946 ). ([\#10760](https://github.com/matrix-org/synapse/issues/10760 ))
- Properly handle room upgrades of spaces. ([\#10774](https://github.com/matrix-org/synapse/issues/10774 ))
- Fix a bug which generated invalid homeserver config when the `frontend_proxy` worker type was passed to the Synapse Worker-based Complement image. ([\#10783](https://github.com/matrix-org/synapse/issues/10783 ))
Improved Documentation
----------------------
- Minor fix to the `media_repository` developer documentation. Contributed by @cuttingedge1109. ([\#10556](https://github.com/matrix-org/synapse/issues/10556 ))
- Update the documentation to note that the `/spaces` and `/hierarchy` endpoints can be routed to workers. ([\#10648](https://github.com/matrix-org/synapse/issues/10648 ))
- Clarify admin API documentation on undoing room deletions. ([\#10735](https://github.com/matrix-org/synapse/issues/10735 ))
- Split up the modules documentation and add examples for module developers. ([\#10758](https://github.com/matrix-org/synapse/issues/10758 ))
- Correct 2 typographical errors in the [Log Contexts documentation](https://matrix-org.github.io/synapse/latest/log_contexts.html ). ([\#10795](https://github.com/matrix-org/synapse/issues/10795 ))
- Fix a wording mistake in the sample configuration. Contributed by @bramvdnheuvel:nltrix.net. ([\#10804](https://github.com/matrix-org/synapse/issues/10804 ))
Deprecations and Removals
-------------------------
- Remove the [unstable MSC2858 API](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2858-Multiple-SSO-Identity-Providers.md#unstable-prefix ), including the undocumented `experimental.msc2858_enabled` config option. The unstable API has been deprecated since Synapse 1.35. Client authors should update their clients to use the stable API introduced in Synapse 1.30 if they have not already done so. ([\#10693](https://github.com/matrix-org/synapse/issues/10693 ))
Internal Changes
----------------
- Add OpenTracing logging to help debug stuck messages (as described by issue [#9424 ](https://github.com/matrix-org/synapse/issues/9424 )). ([\#10704](https://github.com/matrix-org/synapse/issues/10704 ))
- Add type annotations to the `synapse.util` package. ([\#10601](https://github.com/matrix-org/synapse/issues/10601 ))
- Ensure `rooms.creator` field is always populated for easy lookup in [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) usage later. ([\#10697](https://github.com/matrix-org/synapse/issues/10697 ))
- Add missing type hints to REST servlets. ([\#10707](https://github.com/matrix-org/synapse/issues/10707 ), [\#10728](https://github.com/matrix-org/synapse/issues/10728 ), [\#10736](https://github.com/matrix-org/synapse/issues/10736 ))
- Do not include rooms with unknown room versions in the spaces summary results. ([\#10727](https://github.com/matrix-org/synapse/issues/10727 ))
- Additional error checking for the `preset` field when creating a room. ([\#10738](https://github.com/matrix-org/synapse/issues/10738 ))
- Clean up some of the federation event authentication code for clarity. ([\#10744](https://github.com/matrix-org/synapse/issues/10744 ), [\#10745](https://github.com/matrix-org/synapse/issues/10745 ), [\#10746](https://github.com/matrix-org/synapse/issues/10746 ), [\#10771](https://github.com/matrix-org/synapse/issues/10771 ), [\#10773](https://github.com/matrix-org/synapse/issues/10773 ), [\#10781](https://github.com/matrix-org/synapse/issues/10781 ))
- Add an index to `presence_stream` to hopefully speed up startups a little. ([\#10748](https://github.com/matrix-org/synapse/issues/10748 ))
- Refactor event size checking code to simplify searching the codebase for the origins of certain error strings that are occasionally emitted. ([\#10750](https://github.com/matrix-org/synapse/issues/10750 ))
- Move tests relating to rooms having encryption out of the user directory tests. ([\#10752](https://github.com/matrix-org/synapse/issues/10752 ))
- Use `attrs` internally for the URL preview code & update documentation. ([\#10753](https://github.com/matrix-org/synapse/issues/10753 ))
- Minor speed ups when joining large rooms over federation. ([\#10754](https://github.com/matrix-org/synapse/issues/10754 ), [\#10755](https://github.com/matrix-org/synapse/issues/10755 ), [\#10756](https://github.com/matrix-org/synapse/issues/10756 ), [\#10780](https://github.com/matrix-org/synapse/issues/10780 ), [\#10784](https://github.com/matrix-org/synapse/issues/10784 ))
- Add a constant for `m.federate`. ([\#10775](https://github.com/matrix-org/synapse/issues/10775 ))
- Add a script to update the Debian changelog in a Docker container for systems that are not Debian-based. ([\#10778](https://github.com/matrix-org/synapse/issues/10778 ))
- Change the format of authenticated users in logs when a user is being puppeted by and admin user. ([\#10779](https://github.com/matrix-org/synapse/issues/10779 ))
- Remove fixed and flakey tests from the Sytest blacklist. ([\#10788](https://github.com/matrix-org/synapse/issues/10788 ))
- Improve internal details of the user directory code. ([\#10789](https://github.com/matrix-org/synapse/issues/10789 ))
- Use direct references to config flags. ([\#10798](https://github.com/matrix-org/synapse/issues/10798 ))
- Ensure the Rust reporter passes type checking with jaeger-client 4.7's type annotations. ([\#10799](https://github.com/matrix-org/synapse/issues/10799 ))
-----BEGIN PGP SIGNATURE-----
iQJHBAABCAAxFiEEgQG31Z317NrSMt0QiISIDS7+X/QFAmFAgvcTHGFuZHJld0Bh
bW9yZ2FuLnh5egAKCRCIhIgNLv5f9M6REACpRcTVbsp9Z3c6cajrShHWZr7o4z9y
9gjs38mELM7aysP1rcePsYWgsJtG66YCvmErEjuBGBKOSpuQCTRs+2fDC0m5bpia
Av+LzLyWM8lK2vaeg93xP5aBK7Nspkst3XW7Ststkcu25rZFkR/6gU1dE0+YXVbH
wlohqRGA085cZy2xGy+PalwuY8YWcMI2Vw2T+4pCxM7Awly6uK5ieghppKPZ4Xns
Z6rklnkrsf2yaCHzMZYq6TkCiZaJeXyGAxiBEPCrvOvBgTcfX53ZMa2YESOROgil
b9U2YUFa6d0V2vl1KZoSri/fafhY3LC1H+GJvxNeUFFOo1spKKOluNWPaxvZXQDk
+7MKdIzYp2UHyqn2F9rAf213RmGFvh3QDMxXkjUqlJby/mcxzMYsshTBPExcOR44
/XOOhrQRuOAbYttikXRldEta5R6oST9TKJ6ft0HZyYu0VFS2n5RM/HjPXfCZDPBk
0R6l1GjRf3YYYNu+9XZ1o1nZ49RgOHe3p+Ar5istzgbCL3/7nVo7lnCljldFz6ol
xQUdPCnGA0fOWins7o84gQw1fzvcbEHDlL/45VvwR6y5r3ZvbxFa17Vw1s1eKYhT
AglZvUhy8nXzYrasSagQ/a3xWMOcdpiyHyLCEHvpYdYWEbblYNmihivOrzffJsqd
DaRSoFyMrMI53A==
=fq25
-----END PGP SIGNATURE-----
Merge tag 'v1.43.0rc1' into develop
Synapse 1.43.0rc1 (2021-09-14)
This release drops support for the deprecated, unstable API for [MSC2858](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2858-Multiple-SSO-Identity-Providers.md#unstable-prefix ), as well as the undocumented `experimental.msc2858_enabled` config option. Client authors should update their clients to use the stable API, available since Synapse 1.30.
Features
--------
- Allow room creators to send historical events specified by [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) in existing room versions. ([\#10566](https://github.com/matrix-org/synapse/issues/10566 ))
- Add config option to use non-default manhole password and keys. ([\#10643](https://github.com/matrix-org/synapse/issues/10643 ))
- Skip final GC at shutdown to improve restart performance. ([\#10712](https://github.com/matrix-org/synapse/issues/10712 ))
- Allow configuration of the oEmbed URLs used for URL previews. ([\#10714](https://github.com/matrix-org/synapse/issues/10714 ), [\#10759](https://github.com/matrix-org/synapse/issues/10759 ))
- Prefer [room version 9](https://github.com/matrix-org/matrix-doc/pull/3375 ) for restricted rooms per the [room version capabilities](https://github.com/matrix-org/matrix-doc/pull/3244 ) API. ([\#10772](https://github.com/matrix-org/synapse/issues/10772 ))
Bugfixes
--------
- Fix a long-standing bug where room avatars were not included in email notifications. ([\#10658](https://github.com/matrix-org/synapse/issues/10658 ))
- Fix a bug where the ordering algorithm was skipping the `origin_server_ts` step in the spaces summary resulting in unstable room orderings. ([\#10730](https://github.com/matrix-org/synapse/issues/10730 ))
- Fix edge case when persisting events into a room where there are multiple events we previously hadn't calculated auth chains for (and hadn't marked as needing to be calculated). ([\#10743](https://github.com/matrix-org/synapse/issues/10743 ))
- Fix a bug which prevented calls to `/createRoom` that included the `room_alias_name` parameter from being handled by worker processes. ([\#10757](https://github.com/matrix-org/synapse/issues/10757 ))
- Fix a bug which prevented user registration via SSO to require consent tracking for SSO mapping providers that don't prompt for Matrix ID selection. Contributed by @AndrewFerr. ([\#10733](https://github.com/matrix-org/synapse/issues/10733 ))
- Only return the stripped state events for the `m.space.child` events in a room for the spaces summary from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946 ). ([\#10760](https://github.com/matrix-org/synapse/issues/10760 ))
- Properly handle room upgrades of spaces. ([\#10774](https://github.com/matrix-org/synapse/issues/10774 ))
- Fix a bug which generated invalid homeserver config when the `frontend_proxy` worker type was passed to the Synapse Worker-based Complement image. ([\#10783](https://github.com/matrix-org/synapse/issues/10783 ))
Improved Documentation
----------------------
- Minor fix to the `media_repository` developer documentation. Contributed by @cuttingedge1109. ([\#10556](https://github.com/matrix-org/synapse/issues/10556 ))
- Update the documentation to note that the `/spaces` and `/hierarchy` endpoints can be routed to workers. ([\#10648](https://github.com/matrix-org/synapse/issues/10648 ))
- Clarify admin API documentation on undoing room deletions. ([\#10735](https://github.com/matrix-org/synapse/issues/10735 ))
- Split up the modules documentation and add examples for module developers. ([\#10758](https://github.com/matrix-org/synapse/issues/10758 ))
- Correct 2 typographical errors in the [Log Contexts documentation](https://matrix-org.github.io/synapse/latest/log_contexts.html ). ([\#10795](https://github.com/matrix-org/synapse/issues/10795 ))
- Fix a wording mistake in the sample configuration. Contributed by @bramvdnheuvel:nltrix.net. ([\#10804](https://github.com/matrix-org/synapse/issues/10804 ))
Deprecations and Removals
-------------------------
- Remove the [unstable MSC2858 API](https://github.com/matrix-org/matrix-doc/blob/master/proposals/2858-Multiple-SSO-Identity-Providers.md#unstable-prefix ), including the undocumented `experimental.msc2858_enabled` config option. The unstable API has been deprecated since Synapse 1.35. Client authors should update their clients to use the stable API introduced in Synapse 1.30 if they have not already done so. ([\#10693](https://github.com/matrix-org/synapse/issues/10693 ))
Internal Changes
----------------
- Add OpenTracing logging to help debug stuck messages (as described by issue [#9424 ](https://github.com/matrix-org/synapse/issues/9424 )). ([\#10704](https://github.com/matrix-org/synapse/issues/10704 ))
- Add type annotations to the `synapse.util` package. ([\#10601](https://github.com/matrix-org/synapse/issues/10601 ))
- Ensure `rooms.creator` field is always populated for easy lookup in [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) usage later. ([\#10697](https://github.com/matrix-org/synapse/issues/10697 ))
- Add missing type hints to REST servlets. ([\#10707](https://github.com/matrix-org/synapse/issues/10707 ), [\#10728](https://github.com/matrix-org/synapse/issues/10728 ), [\#10736](https://github.com/matrix-org/synapse/issues/10736 ))
- Do not include rooms with unknown room versions in the spaces summary results. ([\#10727](https://github.com/matrix-org/synapse/issues/10727 ))
- Additional error checking for the `preset` field when creating a room. ([\#10738](https://github.com/matrix-org/synapse/issues/10738 ))
- Clean up some of the federation event authentication code for clarity. ([\#10744](https://github.com/matrix-org/synapse/issues/10744 ), [\#10745](https://github.com/matrix-org/synapse/issues/10745 ), [\#10746](https://github.com/matrix-org/synapse/issues/10746 ), [\#10771](https://github.com/matrix-org/synapse/issues/10771 ), [\#10773](https://github.com/matrix-org/synapse/issues/10773 ), [\#10781](https://github.com/matrix-org/synapse/issues/10781 ))
- Add an index to `presence_stream` to hopefully speed up startups a little. ([\#10748](https://github.com/matrix-org/synapse/issues/10748 ))
- Refactor event size checking code to simplify searching the codebase for the origins of certain error strings that are occasionally emitted. ([\#10750](https://github.com/matrix-org/synapse/issues/10750 ))
- Move tests relating to rooms having encryption out of the user directory tests. ([\#10752](https://github.com/matrix-org/synapse/issues/10752 ))
- Use `attrs` internally for the URL preview code & update documentation. ([\#10753](https://github.com/matrix-org/synapse/issues/10753 ))
- Minor speed ups when joining large rooms over federation. ([\#10754](https://github.com/matrix-org/synapse/issues/10754 ), [\#10755](https://github.com/matrix-org/synapse/issues/10755 ), [\#10756](https://github.com/matrix-org/synapse/issues/10756 ), [\#10780](https://github.com/matrix-org/synapse/issues/10780 ), [\#10784](https://github.com/matrix-org/synapse/issues/10784 ))
- Add a constant for `m.federate`. ([\#10775](https://github.com/matrix-org/synapse/issues/10775 ))
- Add a script to update the Debian changelog in a Docker container for systems that are not Debian-based. ([\#10778](https://github.com/matrix-org/synapse/issues/10778 ))
- Change the format of authenticated users in logs when a user is being puppeted by and admin user. ([\#10779](https://github.com/matrix-org/synapse/issues/10779 ))
- Remove fixed and flakey tests from the Sytest blacklist. ([\#10788](https://github.com/matrix-org/synapse/issues/10788 ))
- Improve internal details of the user directory code. ([\#10789](https://github.com/matrix-org/synapse/issues/10789 ))
- Use direct references to config flags. ([\#10798](https://github.com/matrix-org/synapse/issues/10798 ))
- Ensure the Rust reporter passes type checking with jaeger-client 4.7's type annotations. ([\#10799](https://github.com/matrix-org/synapse/issues/10799 ))
2021-09-14 12:59:53 +01:00
b996782df5
Convert media repo's FileInfo to attrs. ( #10785 )
...
This is mostly an internal change, but improves type hints in the
media code.
2021-09-14 07:09:38 -04:00
d725e0956f
1.43.0rc1
2021-09-14 11:47:11 +01:00
319b8b6bef
Name the type of token in "Invalid token" messages ( #10815 )
...
I had one of these error messages yesterday and assumed it was an
invalid auth token (because that was an HTTP query parameter in the
test) I was working on. In fact, it was an invalid next batch token for
syncing.
2021-09-14 11:25:05 +01:00
01c88a09cd
Use direct references for some configuration variables ( #10798 )
...
Instead of proxying through the magic getter of the RootConfig
object. This should be more performant (and is more explicit).
2021-09-13 13:07:12 -04:00
9f111075e8
Fix copy-paste error in the password section of the sample-config. ( #10804 )
2021-09-13 08:58:34 -04:00
003846d68a
Use the BaseReporter super-class for _WrappedRustReporter. ( #10799 )
...
This fixes mypy errors with jaeger-client >= 4.7.0 and should be a no-op
for versions before that.
2021-09-13 08:54:01 -04:00
524b8ead77
Add types to synapse.util. ( #10601 )
2021-09-10 17:03:18 +01:00
ceab5a4bfa
Fix 2 typos in docs/log_contexts.md ( #10795 )
2021-09-10 16:33:36 +01:00
63f28e4a0c
Handle room upgrades for spaces ( #10774 )
...
By copying the `room_type` field of the create event and
migrating any non-empty `m.space.child` events to the
new room that is created.
2021-09-10 07:30:05 -04:00
318162f5de
Easy refactors of the user directory ( #10789 )
...
No functional changes here. This came out as I was working to tackle #5677
2021-09-10 10:54:38 +01:00
c6f5fb5477
Remove fixed and flakey tests from the sytest blacklist ( #10788 )
...
We want to blacklist only known, consistent failures. We should deflake tests rather than ignoring them.
2021-09-10 10:53:04 +01:00
0c0da36a68
Ask consent on SSO registration with default mxid ( #10733 )
...
Fixes #10732 : consent flow skipped during SSO user registration if username is left at default
Signed-off-by: Andrew Ferrazzutti fair@miscworks.net
2021-09-10 10:36:45 +01:00
7f0565e029
Don't needlessly batch in `add_event_to_cache` ( #10784 )
...
We've already batched up the events previously, and assume in other
places in the events.py file that we have. Removing this makes it easier
to adjust the batch sizes in one place.
2021-09-10 10:16:52 +01:00
273b6861f2
Remove unstable MSC2858 API, including `experimental.msc2858_enabled` config option ( #10693 )
...
Signed-off-by: Sean Quah <seanq@element.io>
2021-09-09 17:59:59 +01:00
a621ba0259
Prefer room v9 for restricted rooms. ( #10772 )
...
Hint to clients via the room capabilities API (MSC3244) that
room version 9 should be preferred for creating a room with
restricted join rules (instead of room version 8).
2021-09-09 07:37:33 -04:00
abedf7d77f
Get rid of `_auth_and_persist_event` ( #10781 )
...
This is only called in two places, and the code seems much clearer without it.
2021-09-08 19:03:08 +01:00
03caba6577
Improve the modules doc ( #10758 )
...
* Split up the documentation in several files rather than one huge one
* Add examples for each callback category
* Other niceties like fixing https://github.com/matrix-org/synapse/issues/10632
* Add titles to callbacks so they're easier to find in the navigation panels and link to
2021-09-08 17:14:54 +00:00
01df612e1e
Fix frontend_proxy jinja script in docker workers ( #10783 )
2021-09-08 17:24:53 +01:00
5154afc00d
Add a script to update the debian changelog for non-Debian systems ( #10778 )
...
When releasing 1.42.0 with @Azrenbeth and talking with @clokep yesterday I realised doing the dch incantations related to releasing Synapse wasn't trivial on eg a macOS system, so this is a script to run in a Debian container to make things a bit easier.
2021-09-08 17:03:25 +02:00
1fdf2cf8e8
Document that /spaces & /hierarchy can be routed to workers. ( #10648 )
...
Also refactors some of the registration of endpoints on workers.
2021-09-08 11:02:31 -04:00
74f01e11c9
Skip handling of push actions for outlier events ( #10780 )
...
Outlier events don't ever have push actions associated with them, so we
can skip some expensive queries during event persistence.
2021-09-08 15:18:35 +01:00
0288e6033b
Add a constant for m.federate. ( #10775 )
2021-09-08 10:00:43 -04:00
66d72b7e17
Change logging of puppeted requests to better differentiate users ( #10779 )
...
This used to be a comma and got accidentally changed to a period in #9654 , but a pipe character is more easier to parse visually.
2021-09-08 12:59:15 +00:00
580a15e039
Request JSON for oEmbed requests (and ignore XML only providers). ( #10759 )
...
This adds the format to the request arguments / URL to
ensure that JSON data is returned (which is all that
Synapse supports).
This also adds additional error checking / filtering to the
configuration file to ignore XML-only providers.
2021-09-08 07:17:52 -04:00
aacdce8fc0
Add some assertions about outliers ( #10773 )
...
I think I have finally teased apart the codepaths which handle outliers, and those that handle non-outliers.
Let's add some assertions to demonstrate my newfound knowledge.
2021-09-08 10:41:13 +01:00
5724883ac2
Persist auth events before the events that rely on them ( #10771 )
...
If we're persisting an event E which has auth_events A1, A2, then we ought to make sure that we correctly auth
and persist A1 and A2, before we blindly accept E.
This PR does part of that - it persists the auth events first - but it does not fully solve the problem, because we
still don't check that the auth events weren't rejected.
2021-09-08 10:37:50 +01:00
89ba834818
Use attrs internally for the URL preview code & add documentation. ( #10753 )
2021-09-07 13:10:34 +00:00
a23f3abb9b
Return stripped m.space.child events via the space summary. ( #10760 )
...
The full event content cannot be trusted from this API (as no auth
chain, etc.) is processed over federation. Returning the full event
content was a bug as MSC2946 specifies that only the stripped
state should be returned.
This also avoids calculating aggregations / annotations which go
unused.
2021-09-07 08:43:54 -04:00
f30c9745ab
Underscore-prefix private fields in `FederationEventHandler` ( #10746 )
2021-09-07 11:15:51 +01:00
f1c6b76418
Add logging to help debug #9424 ( #10704 )
2021-09-06 16:08:25 +01:00
6e895366ea
Add config option to use non-default manhole password and keys ( #10643 )
2021-09-06 16:08:03 +01:00
b298de780a
Stop using BaseHandler in `FederationEventHandler` ( #10745 )
...
It's now only used in a couple of places, so we can drop it altogether.
2021-09-06 14:49:33 +01:00
40a1fddd1b
Allow `room_alias_name` parameter to be handled by /createRoom calls on workers ( #10757 )
2021-09-06 14:37:15 +01:00
7bb3673f37
Ease searching for M_TOO_LARGE-related error codes ( #10750 )
2021-09-06 14:35:56 +01:00
e1641b46d1
Doc: Clarifying undoing room shutdowns ( #10480 )
2021-09-06 14:24:31 +01:00
56e2a30634
Move `maybe_kick_guest_users` out of `BaseHandler` ( #10744 )
...
This is part of my ongoing war against BaseHandler. I've moved kick_guest_users into RoomMemberHandler (since it calls out to that handler anyway), and split maybe_kick_guest_users into the two places it is called.
2021-09-06 12:17:16 +01:00
5e9b382505
Pull out encrypted_by_default tests from user_directory tests ( #10752 )
2021-09-06 11:37:54 +01:00
2ca0d64854
Speed up persisting redacted events ( #10756 )
2021-09-06 10:14:07 +01:00
1ca70fd312
Allow room creator to send MSC2716 related events in existing room versions ( #10566 )
...
* Allow room creator to send MSC2716 related events in existing room versions
Discussed at https://github.com/matrix-org/matrix-doc/pull/2716/#discussion_r682474869
Restoring `get_create_event_for_room_txn` from,
44bb3f0cf5https://matrix.to/#/!UytJQHLQYfvYWsGrGY:jki.re/$p2fKESoFst038x6pOOmsY0C49S2gLKMr0jhNMz_JJz0?via=jki.re&via=matrix.org ), my plan is to use `rooms.creator`. But currently, we don't fill in `creator` for remote rooms when a user is invited to a room for example. So we need to add some code to fill in `creator` wherever we add to the `rooms` table. And also add a background update to fill in the rows missing `creator` (we can use the same logic that `get_create_event_for_room_txn` is doing by looking in the state events to get the `creator`).
>
> https://github.com/matrix-org/synapse/pull/10566#issuecomment-901616642
* Remove and switch away from get_create_event_for_room_txn
* Fix no create event being found because no state events persisted yet
* Fix and add tests for rooms creator bg update
* Populate rooms.creator field for easy lookup
Part of https://github.com/matrix-org/synapse/pull/10566
- Fill in creator whenever we insert into the rooms table
- Add background update to backfill any missing creator values
* Add changelog
* Fix usage
* Remove extra delta already included in #10697
* Don't worry about setting creator for invite
* Only iterate over rows missing the creator
See https://github.com/matrix-org/synapse/pull/10697#discussion_r695940898
* Use constant to fetch room creator field
See https://github.com/matrix-org/synapse/pull/10697#discussion_r696803029
* More protection from other random types
See https://github.com/matrix-org/synapse/pull/10697#discussion_r696806853
* Move new background update to end of list
See https://github.com/matrix-org/synapse/pull/10697#discussion_r696814181
* Fix query casing
* Fix ambiguity iterating over cursor instead of list
Fix `psycopg2.ProgrammingError: no results to fetch` error
when tests run with Postgres.
```
SYNAPSE_POSTGRES=1 SYNAPSE_TEST_LOG_LEVEL=INFO python -m twisted.trial tests.storage.databases.main.test_room
```
---
We use `txn.fetchall` because it will return the results as a
list or an empty list when there are no results.
Docs:
> `cursor` objects are iterable, so, instead of calling explicitly fetchone() in a loop, the object itself can be used:
>
> https://www.psycopg.org/docs/cursor.html#cursor-iterable
And I'm guessing iterating over a raw cursor does something weird when there are no results.
---
Test CI failure: https://github.com/matrix-org/synapse/pull/10697/checks?check_run_id=3468916530
```
tests.test_visibility.FilterEventsForServerTestCase.test_large_room
===============================================================================
[FAIL]
Traceback (most recent call last):
File "/home/runner/work/synapse/synapse/tests/storage/databases/main/test_room.py", line 85, in test_background_populate_rooms_creator_column
self.get_success(
File "/home/runner/work/synapse/synapse/tests/unittest.py", line 500, in get_success
return self.successResultOf(d)
File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/trial/_synctest.py", line 700, in successResultOf
self.fail(
twisted.trial.unittest.FailTest: Success result expected on <Deferred at 0x7f4022f3eb50 current result: None>, found failure result instead:
Traceback (most recent call last):
File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/internet/defer.py", line 701, in errback
self._startRunCallbacks(fail)
File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/internet/defer.py", line 764, in _startRunCallbacks
self._runCallbacks()
File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/internet/defer.py", line 858, in _runCallbacks
current.result = callback( # type: ignore[misc]
File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/internet/defer.py", line 1751, in gotResult
current_context.run(_inlineCallbacks, r, gen, status)
--- <exception caught here> ---
File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/internet/defer.py", line 1657, in _inlineCallbacks
result = current_context.run(
File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/python/failure.py", line 500, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "/home/runner/work/synapse/synapse/synapse/storage/background_updates.py", line 224, in do_next_background_update
await self._do_background_update(desired_duration_ms)
File "/home/runner/work/synapse/synapse/synapse/storage/background_updates.py", line 261, in _do_background_update
items_updated = await update_handler(progress, batch_size)
File "/home/runner/work/synapse/synapse/synapse/storage/databases/main/room.py", line 1399, in _background_populate_rooms_creator_column
end = await self.db_pool.runInteraction(
File "/home/runner/work/synapse/synapse/synapse/storage/database.py", line 686, in runInteraction
result = await self.runWithConnection(
File "/home/runner/work/synapse/synapse/synapse/storage/database.py", line 791, in runWithConnection
return await make_deferred_yieldable(
File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/internet/defer.py", line 858, in _runCallbacks
current.result = callback( # type: ignore[misc]
File "/home/runner/work/synapse/synapse/tests/server.py", line 425, in <lambda>
d.addCallback(lambda x: function(*args, **kwargs))
File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/enterprise/adbapi.py", line 293, in _runWithConnection
compat.reraise(excValue, excTraceback)
File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/python/deprecate.py", line 298, in deprecatedFunction
return function(*args, **kwargs)
File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/python/compat.py", line 404, in reraise
raise exception.with_traceback(traceback)
File "/home/runner/work/synapse/synapse/.tox/py/lib/python3.9/site-packages/twisted/enterprise/adbapi.py", line 284, in _runWithConnection
result = func(conn, *args, **kw)
File "/home/runner/work/synapse/synapse/synapse/storage/database.py", line 786, in inner_func
return func(db_conn, *args, **kwargs)
File "/home/runner/work/synapse/synapse/synapse/storage/database.py", line 554, in new_transaction
r = func(cursor, *args, **kwargs)
File "/home/runner/work/synapse/synapse/synapse/storage/databases/main/room.py", line 1375, in _background_populate_rooms_creator_column_txn
for room_id, event_json in txn:
psycopg2.ProgrammingError: no results to fetch
```
* Move code not under the MSC2716 room version underneath an experimental config option
See https://github.com/matrix-org/synapse/pull/10566#issuecomment-906437909
* Add ordering to rooms creator background update
See https://github.com/matrix-org/synapse/pull/10697#discussion_r696815277
* Add comment to better document constant
See https://github.com/matrix-org/synapse/pull/10697#discussion_r699674458
* Use constant field
2021-09-04 00:58:49 -05:00
92b6ac31b2
Speed up MultiWriterIdGenerator when lots of IDs are in flight. ( #10755 )
2021-09-03 18:23:46 +01:00
924276f482
Add a partial index to `presence_stream` to speed up startups ( #10748 )
...
Signed-off-by: Sean Quah <seanq@element.io>
2021-09-03 17:16:56 +01:00
0eae330a26
Use `execute_values` more in PostgreSQL ( #10754 )
...
`execute_values` is a faster version of `execute_batch`.
2021-09-03 15:35:49 +00:00
2cb85bdf75
Raise an error if an unknown preset is used to create a room. ( #10738 )
...
Raises a 400 error instead of a 500 if an unknown preset is passed
from a client to create a room.
2021-09-03 13:46:18 +00:00
ecbfa4fe4f
Additional type hints for client REST servlets (part 5) ( #10736 )
...
Additionally this enforce type hints on all function signatures inside
of the synapse.rest.client package.
2021-09-03 09:22:22 -04:00
f58d202e3f
Fix bug with reusing 'txn' when persisting event. ( #10743 )
...
This will only happen when a server has multiple out of band membership
events in a single room.
2021-09-03 10:59:25 +01:00
00640ee71a
Fix documentation of directory name for remote thumbnails ( #10556 )
2021-09-02 14:07:53 +01:00
c586d6803a
Ignore rooms with unknown room versions in the spaces summary. ( #10727 )
...
This avoids breaking the entire endpoint if a room with
an unsupported room version is encountered.
2021-09-01 17:01:08 +00:00
6258730ebe
Consider the `origin_server_ts` of the `m.space.child` event when ordering rooms. ( #10730 )
...
This updates the ordering of the returned events from the spaces
summary API to that defined in MSC2946 (which updates MSC1772).
Previously a step was skipped causing ordering to be inconsistent with
clients.
2021-09-01 12:59:52 -04:00
d1f1b46c2c
Additional type hints for client REST servlets (part 4) ( #10728 )
2021-09-01 11:59:32 -04:00
dc75fb7f05
Populate `rooms.creator` field for easy lookup ( #10697 )
...
Part of https://github.com/matrix-org/synapse/pull/10566
- Fill in creator whenever we insert into the rooms table
- Add background update to backfill any missing creator values
2021-09-01 16:27:58 +01:00
e059094119
Synapse 1.42.0rc1 (2021-09-01)
...
==============================
Server administrators are reminded to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).
Features
--------
- Add support for [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231 ): Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. ([\#10142](https://github.com/matrix-org/synapse/issues/10142 ))
- Add support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283 ): Expose `enable_set_displayname` in capabilities. ([\#10452](https://github.com/matrix-org/synapse/issues/10452 ))
- Port the `PresenceRouter` module interface to the new generic interface. ([\#10524](https://github.com/matrix-org/synapse/issues/10524 ))
- Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946 ). ([\#10613](https://github.com/matrix-org/synapse/issues/10613 ), [\#10725](https://github.com/matrix-org/synapse/issues/10725 ))
Bugfixes
--------
- Validate new `m.room.power_levels` events. Contributed by @aaronraimist. ([\#10232](https://github.com/matrix-org/synapse/issues/10232 ))
- Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown. ([\#10561](https://github.com/matrix-org/synapse/issues/10561 ))
- Remove pushers when deleting an e-mail address from an account. Pushers for old unlinked emails will also be deleted. ([\#10581](https://github.com/matrix-org/synapse/issues/10581 ), [\#10734](https://github.com/matrix-org/synapse/issues/10734 ))
- Reject Client-Server `/keys/query` requests which provide `device_ids` incorrectly. ([\#10593](https://github.com/matrix-org/synapse/issues/10593 ))
- Rooms with unsupported room versions are no longer returned via `/sync`. ([\#10644](https://github.com/matrix-org/synapse/issues/10644 ))
- Enforce the maximum length for per-room display names and avatar URLs. ([\#10654](https://github.com/matrix-org/synapse/issues/10654 ))
- Fix a bug which caused the `synapse_user_logins_total` Prometheus metric not to be correctly initialised on restart. ([\#10677](https://github.com/matrix-org/synapse/issues/10677 ))
- Improve `ServerNoticeServlet` to avoid duplicate requests and add unit tests. ([\#10679](https://github.com/matrix-org/synapse/issues/10679 ))
- Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating. ([\#10684](https://github.com/matrix-org/synapse/issues/10684 ))
- Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang. ([\#10703](https://github.com/matrix-org/synapse/issues/10703 ))
- Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library. ([\#10713](https://github.com/matrix-org/synapse/issues/10713 ))
Improved Documentation
----------------------
- Add documentation on how to connect Django with Synapse using OpenID Connect and django-oauth-toolkit. Contributed by @HugoDelval. ([\#10192](https://github.com/matrix-org/synapse/issues/10192 ))
- Advertise https://matrix-org.github.io/synapse documentation in the `README` and `CONTRIBUTING` files. ([\#10595](https://github.com/matrix-org/synapse/issues/10595 ))
- Fix some of the titles not rendering in the OpenID Connect documentation. ([\#10639](https://github.com/matrix-org/synapse/issues/10639 ))
- Minor clarifications to the documentation for reverse proxies. ([\#10708](https://github.com/matrix-org/synapse/issues/10708 ))
- Remove table of contents from the top of installation and contributing documentation pages. ([\#10711](https://github.com/matrix-org/synapse/issues/10711 ))
Deprecations and Removals
-------------------------
- Remove deprecated Shutdown Room and Purge Room Admin API. ([\#8830](https://github.com/matrix-org/synapse/issues/8830 ))
Internal Changes
----------------
- Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel. ([\#10608](https://github.com/matrix-org/synapse/issues/10608 ))
- Clean up some of the federation event authentication code for clarity. ([\#10614](https://github.com/matrix-org/synapse/issues/10614 ), [\#10615](https://github.com/matrix-org/synapse/issues/10615 ), [\#10624](https://github.com/matrix-org/synapse/issues/10624 ), [\#10640](https://github.com/matrix-org/synapse/issues/10640 ))
- Add a comment asking developers to leave a reason when bumping the database schema version. ([\#10621](https://github.com/matrix-org/synapse/issues/10621 ))
- Remove not needed database updates in modify user admin API. ([\#10627](https://github.com/matrix-org/synapse/issues/10627 ))
- Convert room member storage tuples to `attrs` classes. ([\#10629](https://github.com/matrix-org/synapse/issues/10629 ), [\#10642](https://github.com/matrix-org/synapse/issues/10642 ))
- Use auto-attribs for the attrs classes used in sync. ([\#10630](https://github.com/matrix-org/synapse/issues/10630 ))
- Make `backfill` and `get_missing_events` use the same codepath. ([\#10645](https://github.com/matrix-org/synapse/issues/10645 ))
- Improve the performance of the `/hierarchy` API (from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946 )) by caching responses received over federation. ([\#10647](https://github.com/matrix-org/synapse/issues/10647 ))
- Run a nightly CI build against Twisted trunk. ([\#10651](https://github.com/matrix-org/synapse/issues/10651 ), [\#10672](https://github.com/matrix-org/synapse/issues/10672 ))
- Do not print out stack traces for network errors when fetching data over federation. ([\#10662](https://github.com/matrix-org/synapse/issues/10662 ))
- Simplify tests for device admin rest API. ([\#10664](https://github.com/matrix-org/synapse/issues/10664 ))
- Add missing type hints to REST servlets. ([\#10665](https://github.com/matrix-org/synapse/issues/10665 ), [\#10666](https://github.com/matrix-org/synapse/issues/10666 ), [\#10674](https://github.com/matrix-org/synapse/issues/10674 ))
- Flatten the `tests.synapse.rests` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10667](https://github.com/matrix-org/synapse/issues/10667 ))
- Update `complement.sh` to rebuild the base Docker image when run with workers. ([\#10686](https://github.com/matrix-org/synapse/issues/10686 ))
- Split the event-processing methods in `FederationHandler` into a separate `FederationEventHandler`. ([\#10692](https://github.com/matrix-org/synapse/issues/10692 ))
- Remove unused `compare_digest` function. ([\#10706](https://github.com/matrix-org/synapse/issues/10706 ))
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEE8SRSDO7gYkSP4chELS76LzL74EcFAmEvd3cACgkQLS76LzL7
4EcF0hAAvOjDeH2hn0eI4YX03/BTCesJ5pCTbEJeC9Hsui3SbWamyYarH0Ae0gM6
BP9x5/XkCdsZhKT2OJedtZQUQ0cUws5FdJP87NnE1m7KBW9djbTxmkue3JKaHNTT
ajvEoLe8UjOFbjvyMc+188uWVgtKQ6KLcetwllTpYY3FJ3+yk9CQP1ydYzwfHcHe
hXW3b20WoRdAmH0B27mj1oQzCeXclbj1kCthcBcxH5RytA8bBc1itBtpAP2FFaeS
Ml6paoXSs0wZaXXL5T1MhQtcVOOG5Yh7EPsemdyfqkwhIk9QtV3nmGfC7K5iEeq0
DldTgBNE0wLO+ycdgEZ2oCvitCLT4WDoKkVqKPA91wpuvCMeK5lnDdqbFUSN3hfd
l8Oz+ydhjsCx7+pBBopnsyGrPIHNaN/vySYGF3lPa3ANKS2J0IVlAaJ59bZqzg2C
vzdyMBCnn0fhnXMkADPLnIYqtebvC09JlPBRGo5BKmwmlM8Ea1dLQ+Lcy5vXalcw
+GaYw3U805c03EvsWQxWchjwq6IwMeP06YpK/1ruMjlrCWYJp3V7f/4hXjlTdYm5
OEWc/kHOnlLfH2cA4aS0sV5WsnEuftotQIzBXuXWw5hSMdFmH7eAviPg0vXgfeA/
i0FOkOgEYR8O3RMh4Cip8+h44t8SPNjYUFJK9ip3DmNateeUCH0=
=XxMc
-----END PGP SIGNATURE-----
Merge tag 'v1.42.0rc1' into develop
Synapse 1.42.0rc1 (2021-09-01)
==============================
Server administrators are reminded to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).
Features
--------
- Add support for [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231 ): Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. ([\#10142](https://github.com/matrix-org/synapse/issues/10142 ))
- Add support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283 ): Expose `enable_set_displayname` in capabilities. ([\#10452](https://github.com/matrix-org/synapse/issues/10452 ))
- Port the `PresenceRouter` module interface to the new generic interface. ([\#10524](https://github.com/matrix-org/synapse/issues/10524 ))
- Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946 ). ([\#10613](https://github.com/matrix-org/synapse/issues/10613 ), [\#10725](https://github.com/matrix-org/synapse/issues/10725 ))
Bugfixes
--------
- Validate new `m.room.power_levels` events. Contributed by @aaronraimist. ([\#10232](https://github.com/matrix-org/synapse/issues/10232 ))
- Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown. ([\#10561](https://github.com/matrix-org/synapse/issues/10561 ))
- Remove pushers when deleting an e-mail address from an account. Pushers for old unlinked emails will also be deleted. ([\#10581](https://github.com/matrix-org/synapse/issues/10581 ), [\#10734](https://github.com/matrix-org/synapse/issues/10734 ))
- Reject Client-Server `/keys/query` requests which provide `device_ids` incorrectly. ([\#10593](https://github.com/matrix-org/synapse/issues/10593 ))
- Rooms with unsupported room versions are no longer returned via `/sync`. ([\#10644](https://github.com/matrix-org/synapse/issues/10644 ))
- Enforce the maximum length for per-room display names and avatar URLs. ([\#10654](https://github.com/matrix-org/synapse/issues/10654 ))
- Fix a bug which caused the `synapse_user_logins_total` Prometheus metric not to be correctly initialised on restart. ([\#10677](https://github.com/matrix-org/synapse/issues/10677 ))
- Improve `ServerNoticeServlet` to avoid duplicate requests and add unit tests. ([\#10679](https://github.com/matrix-org/synapse/issues/10679 ))
- Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating. ([\#10684](https://github.com/matrix-org/synapse/issues/10684 ))
- Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang. ([\#10703](https://github.com/matrix-org/synapse/issues/10703 ))
- Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library. ([\#10713](https://github.com/matrix-org/synapse/issues/10713 ))
Improved Documentation
----------------------
- Add documentation on how to connect Django with Synapse using OpenID Connect and django-oauth-toolkit. Contributed by @HugoDelval. ([\#10192](https://github.com/matrix-org/synapse/issues/10192 ))
- Advertise https://matrix-org.github.io/synapse documentation in the `README` and `CONTRIBUTING` files. ([\#10595](https://github.com/matrix-org/synapse/issues/10595 ))
- Fix some of the titles not rendering in the OpenID Connect documentation. ([\#10639](https://github.com/matrix-org/synapse/issues/10639 ))
- Minor clarifications to the documentation for reverse proxies. ([\#10708](https://github.com/matrix-org/synapse/issues/10708 ))
- Remove table of contents from the top of installation and contributing documentation pages. ([\#10711](https://github.com/matrix-org/synapse/issues/10711 ))
Deprecations and Removals
-------------------------
- Remove deprecated Shutdown Room and Purge Room Admin API. ([\#8830](https://github.com/matrix-org/synapse/issues/8830 ))
Internal Changes
----------------
- Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel. ([\#10608](https://github.com/matrix-org/synapse/issues/10608 ))
- Clean up some of the federation event authentication code for clarity. ([\#10614](https://github.com/matrix-org/synapse/issues/10614 ), [\#10615](https://github.com/matrix-org/synapse/issues/10615 ), [\#10624](https://github.com/matrix-org/synapse/issues/10624 ), [\#10640](https://github.com/matrix-org/synapse/issues/10640 ))
- Add a comment asking developers to leave a reason when bumping the database schema version. ([\#10621](https://github.com/matrix-org/synapse/issues/10621 ))
- Remove not needed database updates in modify user admin API. ([\#10627](https://github.com/matrix-org/synapse/issues/10627 ))
- Convert room member storage tuples to `attrs` classes. ([\#10629](https://github.com/matrix-org/synapse/issues/10629 ), [\#10642](https://github.com/matrix-org/synapse/issues/10642 ))
- Use auto-attribs for the attrs classes used in sync. ([\#10630](https://github.com/matrix-org/synapse/issues/10630 ))
- Make `backfill` and `get_missing_events` use the same codepath. ([\#10645](https://github.com/matrix-org/synapse/issues/10645 ))
- Improve the performance of the `/hierarchy` API (from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946 )) by caching responses received over federation. ([\#10647](https://github.com/matrix-org/synapse/issues/10647 ))
- Run a nightly CI build against Twisted trunk. ([\#10651](https://github.com/matrix-org/synapse/issues/10651 ), [\#10672](https://github.com/matrix-org/synapse/issues/10672 ))
- Do not print out stack traces for network errors when fetching data over federation. ([\#10662](https://github.com/matrix-org/synapse/issues/10662 ))
- Simplify tests for device admin rest API. ([\#10664](https://github.com/matrix-org/synapse/issues/10664 ))
- Add missing type hints to REST servlets. ([\#10665](https://github.com/matrix-org/synapse/issues/10665 ), [\#10666](https://github.com/matrix-org/synapse/issues/10666 ), [\#10674](https://github.com/matrix-org/synapse/issues/10674 ))
- Flatten the `tests.synapse.rests` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10667](https://github.com/matrix-org/synapse/issues/10667 ))
- Update `complement.sh` to rebuild the base Docker image when run with workers. ([\#10686](https://github.com/matrix-org/synapse/issues/10686 ))
- Split the event-processing methods in `FederationHandler` into a separate `FederationEventHandler`. ([\#10692](https://github.com/matrix-org/synapse/issues/10692 ))
- Remove unused `compare_digest` function. ([\#10706](https://github.com/matrix-org/synapse/issues/10706 ))
2021-09-01 14:58:14 +01:00
d9069388f3
Correctly include room avatars in email notifications ( #10658 )
...
Judging by the template, this was intended ages ago, but we never
actually passed an avatar URL to the template. So let's provide one.
Closes #1546 .
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-09-01 13:48:41 +01:00
f8bf83b811
Skip the final GC on shutdown to improve restart times ( #10712 )
...
Use `gc.freeze()` on exit to exclude all existing objects from the final GC.
In testing, this sped up shutdown by up to a few seconds.
`gc.freeze()` runs in constant time, so there is little chance of performance
regression.
Signed-off-by: Sean Quah <seanq@element.io>
2021-09-01 11:55:31 +01:00
6b2aca473a
1.42.0rc1
2021-09-01 11:47:24 +01:00
3693ea61f5
Fix iteration in _remove_deleted_email_pushers background job. ( #10734 )
2021-09-01 09:13:01 +00:00
e2481dbe93
Allow configuration of the oEmbed URLs. ( #10714 )
...
This adds configuration options (under an `oembed` section) to
configure which URLs are matched to use oEmbed for URL
previews.
2021-08-31 18:37:07 -04:00
287918e2d4
Additional type hints for the client REST servlets (part 3). ( #10707 )
2021-08-31 17:22:29 +00:00
78e590d473
Move the sessions delta to the latest schema version. ( #10725 )
...
This was erroneously put under schema version 62 instead of 63.
2021-08-31 16:38:43 +00:00
5d9e7e0c71
Merge branch 'master' into develop
2021-08-31 14:09:48 +01:00
8c26f16c76
Fix up unit tests ( #10723 )
...
These were broken in an incorrect merge of GHSA-jj53-8fmw-f2w2 (cb35df9
2021-08-31 12:56:22 +01:00
46ff99ef95
Advertise matrix-org.github.io/synapse docs ( #10595 )
...
Point to the book where possible, and use hyperlinks to github to refer to files not included in the book.
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-08-31 11:29:27 +01:00
e3abc0a5cc
Merge remote-tracking branch 'origin/release-v1.41' into develop
2021-08-27 16:35:02 +01:00
8f98260552
Fix incompatibility with Twisted < 21. ( #10713 )
...
Turns out that the functionality added in #10546 to skip TLS was incompatible
with older Twisted versions, so we need to be a bit more inventive.
Also, add a test to (hopefully) not break this in future. Sadly, testing TLS is
really hard.
2021-08-27 16:33:41 +01:00
54aa7047eb
Removed page summaries from the top of installation and contributing doc pages ( #10711 )
...
- Removed page summaries from CONTRIBUTING and installation pages as
this information was already in the table of contents on the right hand side
- Fixed some broken links in CONTRIBUTING
- Added margin-right tag for when table of contents is being shown
(otherwise the text in the page sometimes overlaps with it)
2021-08-27 14:19:17 +00:00
051ddac53b
Clarifications to reverse_proxy.md ( #10708 )
...
* Update reverse_proxy.md
* Create 10708.doc
2021-08-27 12:54:21 +01:00
029b7ad7b9
Remove unused `compare_digest` function. ( #10706 )
2021-08-27 07:08:02 -04:00
e62cdbef1a
Improve ServerNoticeServlet to avoid duplicate requests ( #10679 )
...
Fixes : #9544
2021-08-27 09:16:40 +00:00
c4fa4f37cb
Fix perf of fetching the same events many times. ( #10703 )
...
The code to deduplicate repeated fetches of the same set of events was
N^2 (over the number of events requested), which could lead to a process
being completely wedged.
The main fix is to deduplicate the returned deferreds so we only await
on a deferred once rather than many times. Seperately, when handling the
returned events from the defrered we only add the events we care about
to the event map to be returned (so that we don't pay the price of
inserting extraneous events into the dict).
2021-08-27 09:15:50 +00:00
1800aabfc2
Split `FederationHandler` in half ( #10692 )
...
The idea here is to take anything to do with incoming events and move it out to a separate handler, as a way of making FederationHandler smaller.
2021-08-26 21:41:44 +01:00
96715d7633
Make `backfill` and `get_missing_events` use the same codepath ( #10645 )
...
Given that backfill and get_missing_events are basically the same thing, it's somewhat crazy that we have entirely separate code paths for them. This makes backfill use the existing get_missing_events code, and then clears up all the unused code.
2021-08-26 18:34:57 +01:00
40f619eaa5
Validate new m.room.power_levels events ( #10232 )
...
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-08-26 17:07:58 +01:00
ad17fbd20e
Remove pushers when deleting 3pid from account ( #10581 )
...
When a user deletes an email from their account it will
now also remove all pushers for that email and that user
(even if these pushers were created by a different client)
2021-08-26 13:53:57 +01:00
1aa0dad021
Additional type hints for REST servlets (part 2). ( #10674 )
...
Applies the changes from #10665 to additional modules.
2021-08-26 11:53:52 +00:00
5548fe0978
Cache the result of fetching the room hierarchy over federation. ( #10647 )
2021-08-26 07:16:53 -04:00
b45cc1530b
Make a note to leave a summary when one is bumping the schema version ( #10621 )
...
I found this easy to miss (and evidently, it looks like it was missed for schema version 62).
2021-08-25 17:00:44 +01:00
882539e423
Ensure the base Docker image is rebuilt when running complement with workers. ( #10686 )
...
We now always rebuild the matrixdotorg/synapse image, then
build the matrixdotorg/synapse-workers image on top of it.
2021-08-25 10:18:23 -04:00
7367473f96
Fix error when selecting between thumbnails with the same quality ( #10684 )
...
Fixes #10318
2021-08-25 09:51:08 +00:00
6f77a3d433
1.41.0
2021-08-24 15:31:55 +01:00
d12ba52f17
Persist room hierarchy pagination sessions to the database. ( #10613 )
2021-08-24 08:14:03 -04:00
15db8b7c7f
Correctly initialise the `synapse_user_logins` metric. ( #10677 )
...
Fix a bug where the prometheus metrics for SSO logins wouldn't be initialised
until the first user logged in with a given auth provider.
2021-08-24 09:17:51 +00:00
86415f162d
doc: add django-oauth-toolkit to oidc doc ( #10192 )
...
Signed-off-by: Hugo Delval <hugo.delval@gmail.com>
2021-08-23 17:12:36 +00:00
0c1d6f65d7
Enforce the max length for per-room display names / avatar URLs. ( #10654 )
...
To match the maximum lengths allowed for profile data.
2021-08-23 11:25:33 -04:00
3e83f97154
Fix the titles in the OIDC documentation ( #10639 )
...
* Fix the titles in the OIDC documentation
Having them as links broke the table-of-contents rendering in mdbook.
Plus there's no reason for only some of the provider titles to be links.
* Changelog
* Add link to google idp docs
2021-08-23 14:58:31 +01:00
2efc838f05
Avoid duplicate issues from Twisted trunk failures ( #10672 )
...
Setting `update_existing: true` in the `create-an-issue` GitHub Action
will avoid opening duplicate issues if an open issue already exists with
an identical title.
If no open issues match the title, then a new issue will be created.
This helps avoid spamming our issue tracker should there be a failure
when testing against Twisted's trunk.
This PR also pins the SHA of the `create-an-issue` action to mitigate
the risk of a malicious actor gaining access to JasonEtco's account.
See GitHub's page on security hardening third party actions for more:
https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions#using-third-party-actions
Signed-off-by: Dan Callahan <danc@element.io>
2021-08-23 13:06:49 +00:00
bd7d398b05
Additional type hints for the sync REST servlet. ( #10666 )
2021-08-23 08:14:42 -04:00
2af6d31b78
Addtional type hints for the REST servlets. ( #10665 )
2021-08-23 08:14:17 -04:00
31dac7ffee
Do not include stack traces for known exceptions when trying multiple federation destinations. ( #10662 )
2021-08-23 08:00:25 -04:00
David Robertson