Erik Johnston
dd8da8c5f6
Precompute joined hosts and store in Redis ( #9198 )
2021-01-26 13:57:31 +00:00
Patrick Cloke
f81d02d75b
Synapse 1.26.0rc1 (2021-01-20)
...
==============================
This release brings a new schema version for Synapse and rolling back to a previous
verious is not trivial. Please review [UPGRADE.rst](UPGRADE.rst) for more details
on these changes and for general upgrade guidance.
Features
--------
- Add support for multiple SSO Identity Providers. ([\#9015](https://github.com/matrix-org/synapse/issues/9015 ), [\#9017](https://github.com/matrix-org/synapse/issues/9017 ), [\#9036](https://github.com/matrix-org/synapse/issues/9036 ), [\#9067](https://github.com/matrix-org/synapse/issues/9067 ), [\#9081](https://github.com/matrix-org/synapse/issues/9081 ), [\#9082](https://github.com/matrix-org/synapse/issues/9082 ), [\#9105](https://github.com/matrix-org/synapse/issues/9105 ), [\#9107](https://github.com/matrix-org/synapse/issues/9107 ), [\#9109](https://github.com/matrix-org/synapse/issues/9109 ), [\#9110](https://github.com/matrix-org/synapse/issues/9110 ), [\#9127](https://github.com/matrix-org/synapse/issues/9127 ), [\#9153](https://github.com/matrix-org/synapse/issues/9153 ), [\#9154](https://github.com/matrix-org/synapse/issues/9154 ), [\#9177](https://github.com/matrix-org/synapse/issues/9177 ))
- During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. ([\#9091](https://github.com/matrix-org/synapse/issues/9091 ))
- Give the `public_baseurl` a default value, if it is not explicitly set in the configuration file. ([\#9159](https://github.com/matrix-org/synapse/issues/9159 ))
- Improve performance when calculating ignored users in large rooms. ([\#9024](https://github.com/matrix-org/synapse/issues/9024 ))
- Implement [MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176 ) in an experimental room version. ([\#8984](https://github.com/matrix-org/synapse/issues/8984 ))
- Add an admin API for protecting local media from quarantine. ([\#9086](https://github.com/matrix-org/synapse/issues/9086 ))
- Remove a user's avatar URL and display name when deactivated with the Admin API. ([\#8932](https://github.com/matrix-org/synapse/issues/8932 ))
- Update `/_synapse/admin/v1/users/<user_id>/joined_rooms` to work for both local and remote users. ([\#8948](https://github.com/matrix-org/synapse/issues/8948 ))
- Add experimental support for handling to-device messages on worker processes. ([\#9042](https://github.com/matrix-org/synapse/issues/9042 ), [\#9043](https://github.com/matrix-org/synapse/issues/9043 ), [\#9044](https://github.com/matrix-org/synapse/issues/9044 ), [\#9130](https://github.com/matrix-org/synapse/issues/9130 ))
- Add experimental support for handling `/keys/claim` and `/room_keys` APIs on worker processes. ([\#9068](https://github.com/matrix-org/synapse/issues/9068 ))
- Add experimental support for handling `/devices` API on worker processes. ([\#9092](https://github.com/matrix-org/synapse/issues/9092 ))
- Add experimental support for moving off receipts and account data persistence off master. ([\#9104](https://github.com/matrix-org/synapse/issues/9104 ), [\#9166](https://github.com/matrix-org/synapse/issues/9166 ))
Bugfixes
--------
- Fix a long-standing issue where an internal server error would occur when requesting a profile over federation that did not include a display name / avatar URL. ([\#9023](https://github.com/matrix-org/synapse/issues/9023 ))
- Fix a long-standing bug where some caches could grow larger than configured. ([\#9028](https://github.com/matrix-org/synapse/issues/9028 ))
- Fix error handling during insertion of client IPs into the database. ([\#9051](https://github.com/matrix-org/synapse/issues/9051 ))
- Fix bug where we didn't correctly record CPU time spent in `on_new_event` block. ([\#9053](https://github.com/matrix-org/synapse/issues/9053 ))
- Fix a minor bug which could cause confusing error messages from invalid configurations. ([\#9054](https://github.com/matrix-org/synapse/issues/9054 ))
- Fix incorrect exit code when there is an error at startup. ([\#9059](https://github.com/matrix-org/synapse/issues/9059 ))
- Fix `JSONDecodeError` spamming the logs when sending transactions to remote servers. ([\#9070](https://github.com/matrix-org/synapse/issues/9070 ))
- Fix "Failed to send request" errors when a client provides an invalid room alias. ([\#9071](https://github.com/matrix-org/synapse/issues/9071 ))
- Fix bugs in federation catchup logic that caused outbound federation to be delayed for large servers after start up. Introduced in v1.8.0 and v1.21.0. ([\#9114](https://github.com/matrix-org/synapse/issues/9114 ), [\#9116](https://github.com/matrix-org/synapse/issues/9116 ))
- Fix corruption of `pushers` data when a postgres bouncer is used. ([\#9117](https://github.com/matrix-org/synapse/issues/9117 ))
- Fix minor bugs in handling the `clientRedirectUrl` parameter for SSO login. ([\#9128](https://github.com/matrix-org/synapse/issues/9128 ))
- Fix "Unhandled error in Deferred: BodyExceededMaxSize" errors when .well-known files that are too large. ([\#9108](https://github.com/matrix-org/synapse/issues/9108 ))
- Fix "UnboundLocalError: local variable 'length' referenced before assignment" errors when the response body exceeds the expected size. This bug was introduced in v1.25.0. ([\#9145](https://github.com/matrix-org/synapse/issues/9145 ))
- Fix a long-standing bug "ValueError: invalid literal for int() with base 10" when `/publicRooms` is requested with an invalid `server` parameter. ([\#9161](https://github.com/matrix-org/synapse/issues/9161 ))
Improved Documentation
----------------------
- Add some extra docs for getting Synapse running on macOS. ([\#8997](https://github.com/matrix-org/synapse/issues/8997 ))
- Correct a typo in the `systemd-with-workers` documentation. ([\#9035](https://github.com/matrix-org/synapse/issues/9035 ))
- Correct a typo in `INSTALL.md`. ([\#9040](https://github.com/matrix-org/synapse/issues/9040 ))
- Add missing `user_mapping_provider` configuration to the Keycloak OIDC example. Contributed by @chris-ruecker. ([\#9057](https://github.com/matrix-org/synapse/issues/9057 ))
- Quote `pip install` packages when extras are used to avoid shells interpreting bracket characters. ([\#9151](https://github.com/matrix-org/synapse/issues/9151 ))
Deprecations and Removals
-------------------------
- Remove broken and unmaintained `demo/webserver.py` script. ([\#9039](https://github.com/matrix-org/synapse/issues/9039 ))
Internal Changes
----------------
- Improve efficiency of large state resolutions. ([\#8868](https://github.com/matrix-org/synapse/issues/8868 ), [\#9029](https://github.com/matrix-org/synapse/issues/9029 ), [\#9115](https://github.com/matrix-org/synapse/issues/9115 ), [\#9118](https://github.com/matrix-org/synapse/issues/9118 ), [\#9124](https://github.com/matrix-org/synapse/issues/9124 ))
- Various clean-ups to the structured logging and logging context code. ([\#8939](https://github.com/matrix-org/synapse/issues/8939 ))
- Ensure rejected events get added to some metadata tables. ([\#9016](https://github.com/matrix-org/synapse/issues/9016 ))
- Ignore date-rotated homeserver logs saved to disk. ([\#9018](https://github.com/matrix-org/synapse/issues/9018 ))
- Remove an unused column from `access_tokens` table. ([\#9025](https://github.com/matrix-org/synapse/issues/9025 ))
- Add a `-noextras` factor to `tox.ini`, to support running the tests with no optional dependencies. ([\#9030](https://github.com/matrix-org/synapse/issues/9030 ))
- Fix running unit tests when optional dependencies are not installed. ([\#9031](https://github.com/matrix-org/synapse/issues/9031 ))
- Allow bumping schema version when using split out state database. ([\#9033](https://github.com/matrix-org/synapse/issues/9033 ))
- Configure the linters to run on a consistent set of files. ([\#9038](https://github.com/matrix-org/synapse/issues/9038 ))
- Various cleanups to device inbox store. ([\#9041](https://github.com/matrix-org/synapse/issues/9041 ))
- Drop unused database tables. ([\#9055](https://github.com/matrix-org/synapse/issues/9055 ))
- Remove unused `SynapseService` class. ([\#9058](https://github.com/matrix-org/synapse/issues/9058 ))
- Remove unnecessary declarations in the tests for the admin API. ([\#9063](https://github.com/matrix-org/synapse/issues/9063 ))
- Remove `SynapseRequest.get_user_agent`. ([\#9069](https://github.com/matrix-org/synapse/issues/9069 ))
- Remove redundant `Homeserver.get_ip_from_request` method. ([\#9080](https://github.com/matrix-org/synapse/issues/9080 ))
- Add type hints to media repository. ([\#9093](https://github.com/matrix-org/synapse/issues/9093 ))
- Fix the wrong arguments being passed to `BlacklistingAgentWrapper` from `MatrixFederationAgent`. Contributed by Timothy Leung. ([\#9098](https://github.com/matrix-org/synapse/issues/9098 ))
- Reduce the scope of caught exceptions in `BlacklistingAgentWrapper`. ([\#9106](https://github.com/matrix-org/synapse/issues/9106 ))
- Improve `UsernamePickerTestCase`. ([\#9112](https://github.com/matrix-org/synapse/issues/9112 ))
- Remove dependency on `distutils`. ([\#9125](https://github.com/matrix-org/synapse/issues/9125 ))
- Enforce that replication HTTP clients are called with keyword arguments only. ([\#9144](https://github.com/matrix-org/synapse/issues/9144 ))
- Fix the Python 3.5 / old dependencies build in CI. ([\#9146](https://github.com/matrix-org/synapse/issues/9146 ))
- Replace the old `perspectives` option in the Synapse docker config file template with `trusted_key_servers`. ([\#9157](https://github.com/matrix-org/synapse/issues/9157 ))
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAmAIVFkACgkQM/xY9qcR
MEhLfQ//Z4xnP5Icx732k2xpdR+JaXX6heNVFm/UuLNd6wq+uXjO/Dgc9IKQ5LXD
kfG+/OnNGDr/eUk4mQ5s/ccBvaTrOid2IIemJd4lUBy/Q5LvPvfKIp08QFr72WJT
7Y4Ma54nNSSMUAdESgj/aFAHjATbfHhxPOZ5OdQHGLAuJ/OUwR2ksasVNCuINbgh
8vrxrzjeYn1Zl3UTBrsdCcat7AFqYjxK/Y0i6JxfSgRwDZjUbW4M2C4OGWZJPUrU
2YTPHangxd+22+HObE2KDEzaHDdCR/Kj4dhCiJBynZ5iucuHSS+mgf39RtbjqWML
01cmXXUV+FH7rnCilmF9rYdmHZ6L9BJxtrQDQU4dB106BX9G7hKR6rYHc9Z6tjOh
3tg0cKWYMXOsTNnvRqiFNh2q0yBKS228HRAwyGosW/6NiYj43ArtYP6knX7dQyBX
0mYNkfry2dyX6Kj5el0i9MOTHYQxfc8apsBm2M3OTNYukgQKA0NhbAquibaAxEEW
2qIqbSp4CUlGPvM7+u+ZGTu0hEbVKKNjqMfXhuY8A4/BWiV2mSqKcEirR5cE/LqY
mq5Ac7vbO4Uh+1xiRw/G9ITi1dqAjYIzhBawlhdUPOh+aINWTUikYYzjUSmm4PbN
H9BTPLA9iAdVRfuWJ9um2G0DdS8Qpx/aIRh3MScXVly6cGCC6Do=
=kgqn
-----END PGP SIGNATURE-----
Merge tag 'v1.26.0rc1' into develop
Synapse 1.26.0rc1 (2021-01-20)
==============================
This release brings a new schema version for Synapse and rolling back to a previous
verious is not trivial. Please review [UPGRADE.rst](UPGRADE.rst) for more details
on these changes and for general upgrade guidance.
Features
--------
- Add support for multiple SSO Identity Providers. ([\#9015](https://github.com/matrix-org/synapse/issues/9015 ), [\#9017](https://github.com/matrix-org/synapse/issues/9017 ), [\#9036](https://github.com/matrix-org/synapse/issues/9036 ), [\#9067](https://github.com/matrix-org/synapse/issues/9067 ), [\#9081](https://github.com/matrix-org/synapse/issues/9081 ), [\#9082](https://github.com/matrix-org/synapse/issues/9082 ), [\#9105](https://github.com/matrix-org/synapse/issues/9105 ), [\#9107](https://github.com/matrix-org/synapse/issues/9107 ), [\#9109](https://github.com/matrix-org/synapse/issues/9109 ), [\#9110](https://github.com/matrix-org/synapse/issues/9110 ), [\#9127](https://github.com/matrix-org/synapse/issues/9127 ), [\#9153](https://github.com/matrix-org/synapse/issues/9153 ), [\#9154](https://github.com/matrix-org/synapse/issues/9154 ), [\#9177](https://github.com/matrix-org/synapse/issues/9177 ))
- During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. ([\#9091](https://github.com/matrix-org/synapse/issues/9091 ))
- Give the `public_baseurl` a default value, if it is not explicitly set in the configuration file. ([\#9159](https://github.com/matrix-org/synapse/issues/9159 ))
- Improve performance when calculating ignored users in large rooms. ([\#9024](https://github.com/matrix-org/synapse/issues/9024 ))
- Implement [MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176 ) in an experimental room version. ([\#8984](https://github.com/matrix-org/synapse/issues/8984 ))
- Add an admin API for protecting local media from quarantine. ([\#9086](https://github.com/matrix-org/synapse/issues/9086 ))
- Remove a user's avatar URL and display name when deactivated with the Admin API. ([\#8932](https://github.com/matrix-org/synapse/issues/8932 ))
- Update `/_synapse/admin/v1/users/<user_id>/joined_rooms` to work for both local and remote users. ([\#8948](https://github.com/matrix-org/synapse/issues/8948 ))
- Add experimental support for handling to-device messages on worker processes. ([\#9042](https://github.com/matrix-org/synapse/issues/9042 ), [\#9043](https://github.com/matrix-org/synapse/issues/9043 ), [\#9044](https://github.com/matrix-org/synapse/issues/9044 ), [\#9130](https://github.com/matrix-org/synapse/issues/9130 ))
- Add experimental support for handling `/keys/claim` and `/room_keys` APIs on worker processes. ([\#9068](https://github.com/matrix-org/synapse/issues/9068 ))
- Add experimental support for handling `/devices` API on worker processes. ([\#9092](https://github.com/matrix-org/synapse/issues/9092 ))
- Add experimental support for moving off receipts and account data persistence off master. ([\#9104](https://github.com/matrix-org/synapse/issues/9104 ), [\#9166](https://github.com/matrix-org/synapse/issues/9166 ))
Bugfixes
--------
- Fix a long-standing issue where an internal server error would occur when requesting a profile over federation that did not include a display name / avatar URL. ([\#9023](https://github.com/matrix-org/synapse/issues/9023 ))
- Fix a long-standing bug where some caches could grow larger than configured. ([\#9028](https://github.com/matrix-org/synapse/issues/9028 ))
- Fix error handling during insertion of client IPs into the database. ([\#9051](https://github.com/matrix-org/synapse/issues/9051 ))
- Fix bug where we didn't correctly record CPU time spent in `on_new_event` block. ([\#9053](https://github.com/matrix-org/synapse/issues/9053 ))
- Fix a minor bug which could cause confusing error messages from invalid configurations. ([\#9054](https://github.com/matrix-org/synapse/issues/9054 ))
- Fix incorrect exit code when there is an error at startup. ([\#9059](https://github.com/matrix-org/synapse/issues/9059 ))
- Fix `JSONDecodeError` spamming the logs when sending transactions to remote servers. ([\#9070](https://github.com/matrix-org/synapse/issues/9070 ))
- Fix "Failed to send request" errors when a client provides an invalid room alias. ([\#9071](https://github.com/matrix-org/synapse/issues/9071 ))
- Fix bugs in federation catchup logic that caused outbound federation to be delayed for large servers after start up. Introduced in v1.8.0 and v1.21.0. ([\#9114](https://github.com/matrix-org/synapse/issues/9114 ), [\#9116](https://github.com/matrix-org/synapse/issues/9116 ))
- Fix corruption of `pushers` data when a postgres bouncer is used. ([\#9117](https://github.com/matrix-org/synapse/issues/9117 ))
- Fix minor bugs in handling the `clientRedirectUrl` parameter for SSO login. ([\#9128](https://github.com/matrix-org/synapse/issues/9128 ))
- Fix "Unhandled error in Deferred: BodyExceededMaxSize" errors when .well-known files that are too large. ([\#9108](https://github.com/matrix-org/synapse/issues/9108 ))
- Fix "UnboundLocalError: local variable 'length' referenced before assignment" errors when the response body exceeds the expected size. This bug was introduced in v1.25.0. ([\#9145](https://github.com/matrix-org/synapse/issues/9145 ))
- Fix a long-standing bug "ValueError: invalid literal for int() with base 10" when `/publicRooms` is requested with an invalid `server` parameter. ([\#9161](https://github.com/matrix-org/synapse/issues/9161 ))
Improved Documentation
----------------------
- Add some extra docs for getting Synapse running on macOS. ([\#8997](https://github.com/matrix-org/synapse/issues/8997 ))
- Correct a typo in the `systemd-with-workers` documentation. ([\#9035](https://github.com/matrix-org/synapse/issues/9035 ))
- Correct a typo in `INSTALL.md`. ([\#9040](https://github.com/matrix-org/synapse/issues/9040 ))
- Add missing `user_mapping_provider` configuration to the Keycloak OIDC example. Contributed by @chris-ruecker. ([\#9057](https://github.com/matrix-org/synapse/issues/9057 ))
- Quote `pip install` packages when extras are used to avoid shells interpreting bracket characters. ([\#9151](https://github.com/matrix-org/synapse/issues/9151 ))
Deprecations and Removals
-------------------------
- Remove broken and unmaintained `demo/webserver.py` script. ([\#9039](https://github.com/matrix-org/synapse/issues/9039 ))
Internal Changes
----------------
- Improve efficiency of large state resolutions. ([\#8868](https://github.com/matrix-org/synapse/issues/8868 ), [\#9029](https://github.com/matrix-org/synapse/issues/9029 ), [\#9115](https://github.com/matrix-org/synapse/issues/9115 ), [\#9118](https://github.com/matrix-org/synapse/issues/9118 ), [\#9124](https://github.com/matrix-org/synapse/issues/9124 ))
- Various clean-ups to the structured logging and logging context code. ([\#8939](https://github.com/matrix-org/synapse/issues/8939 ))
- Ensure rejected events get added to some metadata tables. ([\#9016](https://github.com/matrix-org/synapse/issues/9016 ))
- Ignore date-rotated homeserver logs saved to disk. ([\#9018](https://github.com/matrix-org/synapse/issues/9018 ))
- Remove an unused column from `access_tokens` table. ([\#9025](https://github.com/matrix-org/synapse/issues/9025 ))
- Add a `-noextras` factor to `tox.ini`, to support running the tests with no optional dependencies. ([\#9030](https://github.com/matrix-org/synapse/issues/9030 ))
- Fix running unit tests when optional dependencies are not installed. ([\#9031](https://github.com/matrix-org/synapse/issues/9031 ))
- Allow bumping schema version when using split out state database. ([\#9033](https://github.com/matrix-org/synapse/issues/9033 ))
- Configure the linters to run on a consistent set of files. ([\#9038](https://github.com/matrix-org/synapse/issues/9038 ))
- Various cleanups to device inbox store. ([\#9041](https://github.com/matrix-org/synapse/issues/9041 ))
- Drop unused database tables. ([\#9055](https://github.com/matrix-org/synapse/issues/9055 ))
- Remove unused `SynapseService` class. ([\#9058](https://github.com/matrix-org/synapse/issues/9058 ))
- Remove unnecessary declarations in the tests for the admin API. ([\#9063](https://github.com/matrix-org/synapse/issues/9063 ))
- Remove `SynapseRequest.get_user_agent`. ([\#9069](https://github.com/matrix-org/synapse/issues/9069 ))
- Remove redundant `Homeserver.get_ip_from_request` method. ([\#9080](https://github.com/matrix-org/synapse/issues/9080 ))
- Add type hints to media repository. ([\#9093](https://github.com/matrix-org/synapse/issues/9093 ))
- Fix the wrong arguments being passed to `BlacklistingAgentWrapper` from `MatrixFederationAgent`. Contributed by Timothy Leung. ([\#9098](https://github.com/matrix-org/synapse/issues/9098 ))
- Reduce the scope of caught exceptions in `BlacklistingAgentWrapper`. ([\#9106](https://github.com/matrix-org/synapse/issues/9106 ))
- Improve `UsernamePickerTestCase`. ([\#9112](https://github.com/matrix-org/synapse/issues/9112 ))
- Remove dependency on `distutils`. ([\#9125](https://github.com/matrix-org/synapse/issues/9125 ))
- Enforce that replication HTTP clients are called with keyword arguments only. ([\#9144](https://github.com/matrix-org/synapse/issues/9144 ))
- Fix the Python 3.5 / old dependencies build in CI. ([\#9146](https://github.com/matrix-org/synapse/issues/9146 ))
- Replace the old `perspectives` option in the Synapse docker config file template with `trusted_key_servers`. ([\#9157](https://github.com/matrix-org/synapse/issues/9157 ))
2021-01-20 11:27:39 -05:00
Richard van der Hoff
0cd2938bc8
Support icons for Identity Providers ( #9154 )
2021-01-20 08:15:14 -05:00
Patrick Cloke
620ecf13b0
Various improvements to the federation client. ( #9129 )
...
* Type hints for `FederationClient`.
* Using `async` functions instead of returning `Awaitable` instances.
2021-01-20 07:59:18 -05:00
Erik Johnston
5e99a94502
Support routing edu's to multiple instances ( #9042 )
...
This is in preparation for moving `SendToDeviceServlet` off master
2021-01-07 18:07:28 +00:00
Patrick Cloke
1c9a850562
Add type hints to the crypto module. ( #8999 )
2021-01-04 10:04:50 -05:00
David Teller
f14428b25c
Allow spam-checker modules to be provide async methods. ( #8890 )
...
Spam checker modules can now provide async methods. This is implemented
in a backwards-compatible manner.
2020-12-11 14:05:15 -05:00
Richard van der Hoff
ed5172852a
Merge pull request #8858 from matrix-org/rav/sso_uia
...
UIA: offer only available auth flows
2020-12-02 20:06:53 +00:00
Richard van der Hoff
7ea85302f3
fix up various test cases
...
A few test cases were relying on being able to mount non-client servlets on the
test resource. it's better to give them their own Resources.
2020-12-02 16:30:01 +00:00
Patrick Cloke
30fba62108
Apply an IP range blacklist to push and key revocation requests. ( #8821 )
...
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:
* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events
The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
2020-12-02 11:09:24 -05:00
Richard van der Hoff
950bb0305f
Consistently use room_id from federation request body ( #8776 )
...
* Consistently use room_id from federation request body
Some federation APIs have a redundant `room_id` path param (see
https://github.com/matrix-org/matrix-doc/issues/2330 ). We should make sure we
consistently use either the path param or the body param, and the body param is
easier.
* Kill off some references to "context"
Once upon a time, "rooms" were known as "contexts". I think this kills of the
last references to "contexts".
2020-11-19 10:05:33 +00:00
Erik Johnston
f21e24ffc2
Add ability for access tokens to belong to one user but grant access to another user. ( #8616 )
...
We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't).
A future PR will add an API for creating such a token.
When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
2020-10-29 15:58:44 +00:00
Erik Johnston
921a3f8a59
Fix not sending events over federation when using sharded event persisters ( #8536 )
...
* Fix outbound federaion with multiple event persisters.
We incorrectly notified federation senders that the minimum persisted
stream position had advanced when we got an `RDATA` from an event
persister.
Notifying of federation senders already correctly happens in the
notifier, so we just delete the offending line.
* Change some interfaces to use RoomStreamToken.
By enforcing use of `RoomStreamTokens` we make it less likely that
people pass in random ints that they got from somewhere random.
2020-10-14 13:27:51 +01:00
Patrick Cloke
1781bbe319
Add type hints to response cache. ( #8507 )
2020-10-09 11:35:11 -04:00
Patrick Cloke
c9c0ad5e20
Remove the deprecated Handlers object ( #8494 )
...
All handlers now available via get_*_handler() methods on the HomeServer.
2020-10-09 07:24:34 -04:00
Patrick Cloke
a93f3121f8
Add type hints to some handlers ( #8505 )
2020-10-09 07:20:51 -04:00
Richard van der Hoff
f31f8e6319
Remove stream ordering from Metadata dict ( #8452 )
...
There's no need for it to be in the dict as well as the events table. Instead,
we store it in a separate attribute in the EventInternalMetadata object, and
populate that on load.
This means that we can rely on it being correctly populated for any event which
has been persited to the database.
2020-10-05 14:43:14 +01:00
Richard van der Hoff
462e681c79
Synapse 1.21.0rc2 (2020-10-02)
...
==============================
Features
--------
- Convert additional templates from inline HTML to Jinja2 templates. ([\#8444](https://github.com/matrix-org/synapse/issues/8444 ))
Bugfixes
--------
- Fix a regression in v1.21.0rc1 which broke thumbnails of remote media. ([\#8438](https://github.com/matrix-org/synapse/issues/8438 ))
- Do not expose the experimental `uk.half-shot.msc2778.login.application_service` flow in the login API, which caused a compatibility problem with Element iOS. ([\#8440](https://github.com/matrix-org/synapse/issues/8440 ))
- Fix malformed log line in new federation "catch up" logic. ([\#8442](https://github.com/matrix-org/synapse/issues/8442 ))
- Fix DB query on startup for negative streams which caused long start up times. Introduced in [\#8374](https://github.com/matrix-org/synapse/issues/8374 ). ([\#8447](https://github.com/matrix-org/synapse/issues/8447 ))
-----BEGIN PGP SIGNATURE-----
iQEzBAABCAAdFiEEv27Axt/F4vrTL/8QOSor00I9eP8FAl93FccACgkQOSor00I9
eP9/Egf7B4YOF6tniyAXxZvmvFOwV1WNw4sbFmF+czUKHBTAwS/Ij9MbutulD4OB
+yqHAvu15qUCQR/G+KGjyHBDtESEUtn5SRy8znLYlR2n3qfEdEpd5y6LJSq4s7sr
NjFVNVI1g5L8PmbvvWCINfpPm2JSm8zyOdyxy4KZifex1B+8YgPILeQOB59sWL/H
1maFbHCgepqO3jotsA8PUXQZx5oScABmqYYe92b4sLna00uFBq2NWp0NA654dRqK
VRFlGzId1fZNWTy1jzfOY2sJKpBCy4cMrtfGJ/eqMtryHqbnBFT6hgB8FyTNg0h0
oew+BLV/mcJLcvB0ALRMFS7xZHdoxQ==
=+3N3
-----END PGP SIGNATURE-----
Merge tag 'v1.21.0rc2' into develop
Synapse 1.21.0rc2 (2020-10-02)
==============================
Features
--------
- Convert additional templates from inline HTML to Jinja2 templates. ([\#8444](https://github.com/matrix-org/synapse/issues/8444 ))
Bugfixes
--------
- Fix a regression in v1.21.0rc1 which broke thumbnails of remote media. ([\#8438](https://github.com/matrix-org/synapse/issues/8438 ))
- Do not expose the experimental `uk.half-shot.msc2778.login.application_service` flow in the login API, which caused a compatibility problem with Element iOS. ([\#8440](https://github.com/matrix-org/synapse/issues/8440 ))
- Fix malformed log line in new federation "catch up" logic. ([\#8442](https://github.com/matrix-org/synapse/issues/8442 ))
- Fix DB query on startup for negative streams which caused long start up times. Introduced in [\#8374](https://github.com/matrix-org/synapse/issues/8374 ). ([\#8447](https://github.com/matrix-org/synapse/issues/8447 ))
2020-10-02 12:59:17 +01:00
Richard van der Hoff
3bd3707cb9
Fix malformed log line in new federation "catch up" logic ( #8442 )
2020-10-02 11:05:29 +01:00
Patrick Cloke
4ff0201e62
Enable mypy checking for unreachable code and fix instances. ( #8432 )
2020-10-01 08:09:18 -04:00
Richard van der Hoff
c1ef579b63
Add prometheus metrics to track federation delays ( #8430 )
...
Add a pair of federation metrics to track the delays in sending PDUs to/from
particular servers.
2020-10-01 11:09:12 +01:00
Richard van der Hoff
2649d545a5
Mypy fixes for `synapse.handlers.federation` ( #8422 )
...
For some reason, an apparently unrelated PR upset mypy about this module. Here are a number of little fixes.
2020-09-29 15:57:36 +01:00
Patrick Cloke
00db7786de
Synapse 1.20.0rc5 (2020-09-18)
...
==============================
In addition to the below, Synapse 1.20.0rc5 also includes the bug fix that was included in 1.19.3.
Features
--------
- Add flags to the `/versions` endpoint for whether new rooms default to using E2EE. ([\#8343](https://github.com/matrix-org/synapse/issues/8343 ))
Bugfixes
--------
- Fix rate limiting of federation `/send` requests. ([\#8342](https://github.com/matrix-org/synapse/issues/8342 ))
- Fix a longstanding bug where back pagination over federation could get stuck if it failed to handle a received event. ([\#8349](https://github.com/matrix-org/synapse/issues/8349 ))
Internal Changes
----------------
- Blacklist [MSC2753](https://github.com/matrix-org/matrix-doc/pull/2753 ) SyTests until it is implemented. ([\#8285](https://github.com/matrix-org/synapse/issues/8285 ))
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl9kzk8ACgkQM/xY9qcR
MEim6A//aERkhyLGRlGpLd37lCyFQCeffTMH1rTvu04iIBQBaUZ6g7CYWOpK43zT
U8kt379+5OShjdAXs/X4XP+ucdHVbrwsRSP3hBS/fFLiDT0fJgP8uiSf5QqO6NnT
OqDyXYjcXvj/c6tMKglVtsdh8u4hFwNZjGPMGG68IzJu14uEhnD100cL9jSB9bLB
ongWpsQzzdGBpJPSFRjv9dCUSeRbzyUdl1t0uqzrNqyN9s/JnzFTn7ZYo6y3lnSS
dHGVMMo/12M2PkbBHnbJVvDY5Q/R7ZxyXlpz0gvSNOQIw8FqYFnuB0Niy5dQhXSR
Sy5h4qbczLxqbql1x+lmzeQm4ZMORsW/Tl4C3z6yK6OYaOCJHIf9en4DplTSTqp1
t+85JxWR2wH10d99YHBpaYKmkVovpwgchrO4YWrtXljUFAhhavzf+YiAdOHYT52s
RDsDLsvjMbxEHsz4cHfycmshYhjzjb340wkoDXuQpj0zrO99d+Zd83xdK8pS0UQn
OaljLRAd/5iBjTSyZPSrB1U5141OzlM3QZVJzaYAnP12yhR9eaX2twSCk+lPYOWd
nhLJjNnj1B1XSGArthuE5NLyEiCPz6KyN2RhO0EOx5YjZN9TwH7LS9upyNFe1nN1
GIhO5gz+jWLuBZE3xzRNjJyCx/I/LolpCwGMvKDu6638rpsbrPs=
=tT5/
-----END PGP SIGNATURE-----
Merge tag 'v1.20.0rc5' into develop
Synapse 1.20.0rc5 (2020-09-18)
==============================
In addition to the below, Synapse 1.20.0rc5 also includes the bug fix that was included in 1.19.3.
Features
--------
- Add flags to the `/versions` endpoint for whether new rooms default to using E2EE. ([\#8343](https://github.com/matrix-org/synapse/issues/8343 ))
Bugfixes
--------
- Fix rate limiting of federation `/send` requests. ([\#8342](https://github.com/matrix-org/synapse/issues/8342 ))
- Fix a longstanding bug where back pagination over federation could get stuck if it failed to handle a received event. ([\#8349](https://github.com/matrix-org/synapse/issues/8349 ))
Internal Changes
----------------
- Blacklist [MSC2753](https://github.com/matrix-org/matrix-doc/pull/2753 ) SyTests until it is implemented. ([\#8285](https://github.com/matrix-org/synapse/issues/8285 ))
2020-09-18 11:17:58 -04:00
Patrick Cloke
88e67d1adb
1.19.3
...
Synapse 1.19.3 (2020-09-18)
===========================
Bugfixes
--------
- Partially mitigate bug where newly joined servers couldn't get past
events in a room when there is a malformed event.
([\#8350](https://github.com/matrix-org/synapse/issues/8350 ))
-----BEGIN PGP SIGNATURE-----
iQJHBAABCAAxFiEEgQG31Z317NrSMt0QiISIDS7+X/QFAl9kxN4THGFuZHJld0Bh
bW9yZ2FuLnh5egAKCRCIhIgNLv5f9LznEACSm7ZL0GjVDcDjGEu+QjKIi3KUiFq3
i8EXEZWT3w5NNNER0Jey5BHxaBKtnPsvon0k3U4bp5KKOA6BGa9L4NDGZYJa3p0k
A1uc3DCgGG1aVazpIzfhWRA0va13T+zRKdz52GdjzksH0WGl6w3UoJhloWOmHyxz
K4UxGwOqJMSBxseBHOFcXdomPtsNYUqsrOcZYWjh3hWN0GMV6H+WrcbKVYl49V0F
6aVHuaxit35iAGYER41mnTA34ZNuC1Qkp83mAaE+Z8i39qBWPMRErUNAyZQ/mCKz
QrF98p7F2kFgSzDagtZiUPZj3w3XwfZf05bqnyd9cxBEQdIYFLAL0lokEXcoY1os
q7gKwGuwicuvYEQrt+gSFlkoUaSvy7/b4cmFqvT0NGnBNZoYl6MX4MXP2CNHuaFk
yljZoTecKEmhInY10S4uy+Hp0JNHuZWEOYGKy7CrQaqRo8MhBLk5LWBPjUOayPLP
uvDNv6MShQ8SpCiKvsoCBiX9G3LEo1yHPo5oX57nOr+IHawH0PPkXVKL3b+K+7s0
eXah/9n/wQYO5K+ReqTFd9ZCegN0/hW/NAT9aX/gEYASkS4ANvGALWwXbZSOG5IG
2glXiewbJSOaVutPRpIVI3XGDSdm3/8VpO+cAKotZ+pR1V6nsxtVwLRmAhxqhNFD
3AULCLMt2yKzDw==
=a9VC
-----END PGP SIGNATURE-----
Merge tag 'v1.19.3' into release-v1.20.0
1.19.3
Synapse 1.19.3 (2020-09-18)
===========================
Bugfixes
--------
- Partially mitigate bug where newly joined servers couldn't get past
events in a room when there is a malformed event.
([\#8350](https://github.com/matrix-org/synapse/issues/8350 ))
2020-09-18 10:53:01 -04:00
reivilibre
36efbcaf51
Catch-up after Federation Outage (bonus): Catch-up on Synapse Startup ( #8322 )
...
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
* Fix _set_destination_retry_timings
This came about because the code assumed that retry_interval
could not be NULL — which has been challenged by catch-up.
2020-09-18 14:59:13 +01:00
Patrick Cloke
8a4a4186de
Simplify super() calls to Python 3 syntax. ( #8344 )
...
This converts calls like super(Foo, self) -> super().
Generated with:
sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Andrew Morgan
27c1abc7b8
Use _check_sigs_and_hash_and_fetch to validate backfill requests ( #8350 )
...
This is a bit of a hack, as `_check_sigs_and_hash_and_fetch` is intended
for attempting to pull an event from the database/(re)pull it from the
server that originally sent the event if checking the signature of the
event fails.
During backfill we *know* that we won't have the event in our database,
however it is still useful to be able to query the original sending
server as the server we're backfilling from may be acting maliciously.
The main benefit and reason for this change however is that
`_check_sigs_and_hash_and_fetch` will drop an event during backfill if
it cannot be successfully validated, whereas the current code will
simply fail the backfill request - resulting in the client's /messages
request silently being dropped.
This is a quick patch to fix backfilling rooms that contain malformed
events. A better implementation in planned in future.
2020-09-18 14:51:11 +01:00
Erik Johnston
5e42e61609
Merge remote-tracking branch 'origin/release-v1.20.0' into develop
2020-09-18 10:50:04 +01:00
Erik Johnston
14b5b48a22
Fix ratelimiting for federation `/send` requests. ( #8342 )
...
c.f. #8295 for rationale
2020-09-18 10:49:29 +01:00
Erik Johnston
5dd051acae
Merge branch 'erikj/fix_origin_check' into develop
2020-09-16 12:43:02 +01:00
Erik Johnston
9c8ef134fe
Merge branch 'erikj/fix_origin_check' into release-v1.20.0
2020-09-16 12:42:42 +01:00
Erik Johnston
c570f24acc
Don't assume that an event has an origin field
...
This fixes #8319 .
2020-09-16 11:56:23 +01:00
reivilibre
576bc37d31
Catch-up after Federation Outage (split, 4): catch-up loop ( #8272 )
2020-09-15 09:07:19 +01:00
reivilibre
17fa4c7ca7
Catch up after Federation Outage (split, 2): Track last successful stream ordering after transmission ( #8247 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-09-04 15:06:51 +01:00
reivilibre
58f61f10f7
Catch-up after Federation Outage (split, 1) ( #8230 )
...
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2020-09-04 12:22:23 +01:00
Patrick Cloke
c619253db8
Stop sub-classing object ( #8249 )
2020-09-04 06:54:56 -04:00
reivilibre
4535e849d7
Remove obsolete order field in `send_new_transaction` ( #8245 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-09-03 19:23:07 +01:00
Patrick Cloke
5c03134d0f
Convert additional database code to async/await. ( #8195 )
2020-08-28 07:54:27 -04:00
Patrick Cloke
9b7ac03af3
Convert calls of async database methods to async ( #8166 )
2020-08-27 13:38:41 -04:00
Patrick Cloke
5758dcf30c
Add type hints for state. ( #8140 )
2020-08-24 14:25:27 -04:00
Patrick Cloke
eebf52be06
Be stricter about JSON that is accepted by Synapse ( #8106 )
2020-08-19 07:26:03 -04:00
Patrick Cloke
ad6190c925
Convert stream database to async/await. ( #8074 )
2020-08-17 07:24:46 -04:00
reivilibre
ff0e894656
Drop federation transmission queues during a significant remote outage. ( #7864 )
...
* Empty federation transmission queues when we are backing off.
Fixes #7828 .
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Address feedback
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Reword newsfile
2020-08-13 12:35:04 +01:00
Erik Johnston
9d1e4942ab
Fix typing for notifier ( #8064 )
2020-08-12 14:03:08 +01:00
Olivier Wilkinson (reivilibre)
3aa36b782c
Merge branch 'master' into develop
2020-07-30 15:18:36 +01:00
Patrick Cloke
c978f6c451
Convert federation client to async/await. ( #7975 )
2020-07-30 08:01:33 -04:00
Erik Johnston
2c1b9d6763
Update worker docs with recent enhancements ( #7969 )
2020-07-29 23:22:13 +01:00
Patrick Cloke
b975fa2e99
Convert state resolution to async/await ( #7942 )
2020-07-24 10:59:51 -04:00
Patrick Cloke
fefe9943ef
Convert presence handler helpers to async/await. ( #7939 )
2020-07-23 16:47:36 -04:00
Richard van der Hoff
7078866969
Put a cache on `/state_ids` ( #7931 )
...
If we send out an event which refers to `prev_events` which other servers in
the federation are missing, then (after a round or two of backfill attempts),
they will end up asking us for `/state_ids` at a particular point in the DAG.
As per https://github.com/matrix-org/synapse/issues/7893 , this is quite
expensive, and we tend to see lots of very similar requests around the same
time.
We can therefore handle this much more efficiently by using a cache, which (a)
ensures that if we see the same request from multiple servers (or even the same
server, multiple times), then they share the result, and (b) any other servers
that miss the initial excitement can also benefit from the work.
[It's interesting to note that `/state` has a cache for exactly this
reason. `/state` is now essentially unused and replaced with `/state_ids`, but
evidently when we replaced it we forgot to add a cache to the new endpoint.]
2020-07-23 18:38:19 +01:00
Richard van der Hoff
4876af06dd
Abort federation requests if the client disconnects early ( #7930 )
...
For inbound federation requests, if a given remote server makes too many
requests at once, we start stacking them up rather than processing them
immediatedly.
However, that means that there is a fair chance that the requesting server will
disconnect before we start processing the request. In that case, if it was a
read-only request (ie, a GET request), there is absolutely no point in
building a response (and some requests are quite expensive to handle).
Even in the case of a POST request, one of two things will happen:
* Most likely, the requesting server will retry the request and we'll get the
information anyway.
* Even if it doesn't, the requesting server has to assume that we didn't get
the memo, and act accordingly.
In short, we're better off aborting the request at this point rather than
ploughing on with what might be a quite expensive request.
2020-07-23 16:52:33 +01:00
Patrick Cloke
de119063f2
Convert room list handler to async/await. ( #7912 )
2020-07-21 07:51:48 -04:00
Patrick Cloke
00e57b755c
Convert synapse.app to async/await. ( #7868 )
2020-07-17 07:08:56 -04:00
Erik Johnston
f2e38ca867
Allow moving typing off master ( #7869 )
2020-07-16 15:12:54 +01:00
Erik Johnston
649a7ead5c
Add ability to run multiple pusher instances ( #7855 )
...
This reuses the same scheme as federation sender sharding
2020-07-16 14:06:28 +01:00
Olivier Wilkinson (reivilibre)
12528dc42f
Remove obsolete comment.
...
It was correct at the time of our friend Jorik writing it (checking
git blame), but the world has moved now and it is no longer a
generator.
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2020-07-16 11:12:48 +01:00
Patrick Cloke
35450519de
Ensure that calls to `json.dumps` are compatible with the standard library json. ( #7836 )
2020-07-15 13:40:54 -04:00
Patrick Cloke
66a4af8d96
Do not use canonicaljson to magically handle decoding bytes from JSON. ( #7802 )
2020-07-10 14:30:08 -04:00
Erik Johnston
f299441cc6
Add ability to shard the federation sender ( #7798 )
2020-07-10 18:26:36 +01:00
Patrick Cloke
38e1fac886
Fix some spelling mistakes / typos. ( #7811 )
2020-07-09 09:52:58 -04:00
Richard van der Hoff
67593b1728
Add `HomeServer.signing_key` property ( #7805 )
...
... instead of duplicating `config.signing_key[0]` everywhere
2020-07-08 17:51:56 +01:00
Erik Johnston
5cdca53aa0
Merge different Resource implementation classes ( #7732 )
2020-07-03 19:02:19 +01:00
Erik Johnston
1e03513f9a
Fix new metric where we used ms instead of seconds ( #7771 )
...
Introduced in #7755 , not yet released.
2020-07-01 15:23:58 +01:00
Erik Johnston
a99658074d
Add some metrics for inbound and outbound federation processing times ( #7755 )
2020-06-30 16:58:06 +01:00
Dagfinn Ilmari Mannsåker
a3f11567d9
Replace all remaining six usage with native Python 3 equivalents ( #7704 )
2020-06-16 08:51:47 -04:00
Patrick Cloke
bd6dc17221
Replace iteritems/itervalues/iterkeys with native versions. ( #7692 )
2020-06-15 07:03:36 -04:00
Richard van der Hoff
075375bbc9
add a comment
2020-05-21 13:25:41 +01:00
Patrick Cloke
56b66db78a
Strictly enforce canonicaljson requirements in a new room version ( #7381 )
2020-05-14 13:24:01 -04:00
Richard van der Hoff
16b1a34e80
Fix typing annotations in synapse/federation ( #7382 )
...
We're pretty close to having mypy working for `synapse.federation`, so let's
finish the job.
2020-05-05 14:27:13 +01:00
Richard van der Hoff
d5aa7d93ed
Fix catchup-on-reconnect for the Federation Stream ( #7374 )
...
looks like we managed to break this during the refactorathon.
2020-05-05 14:15:57 +01:00
Andrew Morgan
6b22921b19
async/await is_server_admin ( #7363 )
2020-05-01 15:15:36 +01:00
Andrew Morgan
2e8955f4a6
Further improvements to requesting the public rooms list on a homeserver which has it set to private ( #7368 )
2020-05-01 15:15:08 +01:00
Richard van der Hoff
2aa5bf13c8
Merge branch 'release-v1.12.4' into develop
2020-04-22 13:09:23 +01:00
Andrew Morgan
f89ad3b6df
Query missing cross-signing keys on local sig upload ( #7289 )
2020-04-22 12:29:36 +01:00
Richard van der Hoff
461f01ad43
Merge branch 'release-v1.12.4' into develop
2020-04-20 17:55:39 +01:00
Richard van der Hoff
d41c8f6d4d
Revert "Query missing cross-signing keys on local sig upload"
...
This was incorrectly merged to the release branch before it was ready.
This reverts commit 72fe2affb6
.
2020-04-20 17:54:35 +01:00
Richard van der Hoff
a46ff43319
Merge branch 'release-v1.12.4' into develop
2020-04-20 17:42:27 +01:00
Andrew Morgan
72fe2affb6
Query missing cross-signing keys on local sig upload
...
Add changelog
Save retrieved keys to the db
lint
Fix and de-brittle remote result dict processing
Use query_user_devices instead, assume only master, self_signing key types
Make changelog more useful
Remove very specific exception handling
Wrap get_verify_key_from_cross_signing_key in a try/except
Note that _get_e2e_cross_signing_verify_key can raise a SynapseError
lint
Add comment explaining why this is useful
Only fetch master and self_signing key types
Fix log statements, docstrings
Remove extraneous items from remote query try/except
lint
Factor key retrieval out into a separate function
Send device updates, modeled after SigningKeyEduUpdater._handle_signing_key_updates
Update method docstring
2020-04-17 15:47:49 +01:00
Patrick Cloke
d78cb31588
Add typing information to federation_server. ( #7219 )
2020-04-07 15:03:23 -04:00
Erik Johnston
4cff617df1
Move catchup of replication streams to worker. ( #7024 )
...
This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
2020-03-25 14:54:01 +00:00
Richard van der Hoff
39230d2171
Clean up some LoggingContext stuff ( #7120 )
...
* Pull Sentinel out of LoggingContext
... and drop a few unnecessary references to it
* Factor out LoggingContext.current_context
move `current_context` and `set_context` out to top-level functions.
Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.
* move copy-to-parent into `stop`
this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.
* Replace `LoggingContext.alive` with `finished`
Turn `alive` into `finished` and make it a bit better defined.
2020-03-24 14:45:33 +00:00
Richard van der Hoff
a564b92d37
Convert `*StreamRow` classes to inner classes ( #7116 )
...
This just helps keep the rows closer to their streams, so that it's easier to
see what the format of each stream is.
2020-03-23 13:59:11 +00:00
Patrick Cloke
c2db6599c8
Fix a bug in the federation API which could cause occasional "Failed to get PDU" errors ( #7089 ).
2020-03-19 08:22:56 -04:00
Patrick Cloke
7581d30e9f
Remove unused federation endpoint (`query_auth`) ( #7026 )
2020-03-17 08:04:49 -04:00
Patrick Cloke
12d4259000
Add some type annotations to the federation base & client classes ( #6995 )
2020-02-28 07:31:07 -05:00
Patrick Cloke
ab0073a6c0
Merge remote-tracking branch 'origin/release-v1.11.1' into develop
2020-02-27 13:47:44 -05:00
Patrick Cloke
380122866f
Cast a coroutine into a Deferred in the federation base ( #6996 )
...
Properly convert a coroutine into a Deferred in federation_base to fix an error when joining a room.
2020-02-26 11:32:13 -05:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 ( #6957 )
...
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Erik Johnston
21db35f77e
Add support for putting fed user query API on workers ( #6873 )
2020-02-07 15:45:39 +00:00
Richard van der Hoff
799001f2c0
Add a `make_event_from_dict` method ( #6858 )
...
... and use it in places where it's trivial to do so.
This will make it easier to pass room versions into the FrozenEvent
constructors.
2020-02-07 15:30:04 +00:00
Erik Johnston
b08b0a22d5
Add typing to synapse.federation.sender ( #6871 )
2020-02-07 13:56:38 +00:00
Richard van der Hoff
928edef979
Pass room_version into `event_from_pdu_json`
...
It's called from all over the shop, so this one's a bit messy.
2020-02-06 16:08:27 +00:00
Richard van der Hoff
b0c8bdd49d
pass room version into FederationClient.send_join ( #6854 )
...
... which allows us to sanity-check the create event.
2020-02-06 15:50:39 +00:00
Erik Johnston
ed630ea17c
Reduce amount of logging at INFO level. ( #6862 )
...
A lot of the things we log at INFO are now a bit superfluous, so lets
make them DEBUG logs to reduce the amount we log by default.
Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
2020-02-06 13:31:05 +00:00
Richard van der Hoff
ee0525b2b2
Simplify `room_version` handling in `FederationClient.send_invite`
2020-02-05 17:43:57 +00:00
Richard van der Hoff
f84700fba8
Pass room version object into `FederationClient.get_pdu`
2020-02-05 17:25:46 +00:00
Richard van der Hoff
6bbd890f05
make FederationClient._do_send_invite async
2020-02-05 15:50:31 +00:00
Richard van der Hoff
146fec0820
Apply suggestions from code review
...
Co-Authored-By: Erik Johnston <erik@matrix.org>
2020-02-05 15:47:00 +00:00
Richard van der Hoff
ea23210b2d
make FederationClient.send_invite async
2020-02-03 22:29:49 +00:00
Richard van der Hoff
6deeefb68c
make FederationClient.get_missing_events async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
abadf44eb2
make FederationClient._do_send_leave async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
e88b90aaeb
make FederationClient.send_leave.send_request async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
638001116d
make FederationClient._do_send_join async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
3960527c2e
make FederationClient.send_join.send_request async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
ad09ee9262
make FederationClient.make_membership_event.send_request async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
1330c311b7
make FederationClient._try_destination_list async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
a46fabf17b
make FederationClient.send_leave async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
8af9f11bea
make FederationClient.send_join async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
3f11cbb404
make FederationClient.make_membership_event async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
24d814ca23
make FederationClient.get_event_auth async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
d73683c363
make FederationClient.get_room_state_ids async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
0cb0c7bcd5
make FederationClient.get_pdu async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
0536d0c9be
make FederationClient.backfill async
2020-02-03 22:28:45 +00:00
Richard van der Hoff
b0d112e78b
Fix `room_version` in `on_invite_request` flow ( #6827 )
...
I messed this up a bit in #6805 , but fortunately we weren't actually doing
anything with the room_version so it didn't matter that it was a str not a RoomVersion.
2020-02-03 13:15:23 +00:00
Richard van der Hoff
a5afdd15e5
Merge pull request #6806 from matrix-org/rav/redact_changes/3
...
Pass room_version into add_hashes_and_signatures
2020-01-31 10:57:03 +00:00
Richard van der Hoff
d7bf793cc1
s/get_room_version/get_room_version_id/
...
... to make way for a forthcoming get_room_version which returns a RoomVersion
object.
2020-01-31 10:06:21 +00:00
Richard van der Hoff
54f3f369bd
Pass room_version into create_local_event_from_event_dict
2020-01-30 22:15:50 +00:00
Richard van der Hoff
46a446828d
pass room version into FederationHandler.on_invite_request ( #6805 )
2020-01-30 22:13:02 +00:00
Erik Johnston
c3d4ad8afd
Fix sending server up commands from workers ( #6811 )
...
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2020-01-30 16:42:11 +00:00
Erik Johnston
a855b7c3a8
Remove unused DeviceRow class ( #6800 )
2020-01-29 12:06:31 +00:00
Erik Johnston
6b9e1014cf
Fix race in federation sender that delayed device updates. ( #6799 )
...
We were sending device updates down both the federation stream and
device streams. This mean there was a race if the federation sender
worker processed the federation stream first, as when the sender checked
if there were new device updates the slaved ID generator hadn't been
updated with the new stream IDs and so returned nothing.
This situation is correctly handled by events/receipts/etc by not
sending updates down the federation stream and instead having the
federation sender worker listen on the other streams and poke the
transaction queues as appropriate.
2020-01-29 11:23:01 +00:00
Erik Johnston
8df862e45d
Add `rooms.room_version` column ( #6729 )
...
This is so that we don't have to rely on pulling it out from `current_state_events` table.
2020-01-27 14:30:57 +00:00
Erik Johnston
a8a50f5b57
Wake up transaction queue when remote server comes back online ( #6706 )
...
This will be used to retry outbound transactions to a remote server if
we think it might have come back up.
2020-01-17 10:27:19 +00:00
Erik Johnston
d386f2f339
Add StateMap type alias ( #6715 )
2020-01-16 13:31:22 +00:00
Erik Johnston
48c3a96886
Port synapse.replication.tcp to async/await ( #6666 )
...
* Port synapse.replication.tcp to async/await
* Newsfile
* Correctly document type of on_<FOO> functions as async
* Don't be overenthusiastic with the asyncing....
2020-01-16 09:16:12 +00:00
Erik Johnston
b5ce7f5874
Process EDUs in parallel with PDUs. ( #6697 )
...
This means that things like to device messages don't get blocked behind
processing PDUs, which can potentially take *ages*.
2020-01-14 14:08:35 +00:00
Brendan Abolivier
54ae52ba96
Merge pull request #6349 from matrix-org/babolivier/msc1802
...
Implement v2 APIs for send_join and send_leave
2019-12-11 11:41:47 +00:00
Richard van der Hoff
f8bc2ae883
Move get_state methods into FederationHandler ( #6503 )
...
This is a non-functional refactor as a precursor to some other work.
2019-12-10 17:42:46 +00:00
Richard van der Hoff
c3dda2874d
Refactor get_events_from_store_or_dest to return a dict ( #6501 )
...
There was a bunch of unnecessary conversion back and forth between dict and
list going on here. We can simplify a bunch of the code.
2019-12-10 16:22:00 +00:00
Richard van der Hoff
e519489fc4
Remove fallback for missing /federation/v1/state_ids API ( #6488 )
...
This API was added way back in 0.17.0; the code here is annoying to maintain
and entirely redundant.
2019-12-09 11:37:26 +00:00
Brendan Abolivier
e126d83f74
Merge branch 'develop' into babolivier/msc1802
2019-12-05 21:00:43 +00:00
Amber Brown
0f87b912ab
Implementation of MSC2314 ( #6176 )
2019-11-28 08:54:07 +11:00
Andrew Morgan
3916e1b97a
Clean up newline quote marks around the codebase ( #6362 )
2019-11-21 12:00:14 +00:00
Andrew Morgan
bc29a19731
Replace instance variations of homeserver with correct case/spacing
2019-11-12 13:08:12 +00:00
Brendan Abolivier
94cdd6fffe
Lint
2019-11-11 16:56:55 +00:00
Brendan Abolivier
edc4c7d4c5
Lint
2019-11-11 16:51:54 +00:00
Brendan Abolivier
5e18dc7955
Fix prefix for v2/send_leave
2019-11-11 16:46:09 +00:00
Brendan Abolivier
74897de01f
Add server-side support to the v2 API
2019-11-11 16:40:45 +00:00
Brendan Abolivier
1e202a90f1
Implement v2 API for send_leave
2019-11-11 16:26:53 +00:00
Brendan Abolivier
92527d7b21
Add missing yield
2019-11-11 16:20:53 +00:00
Brendan Abolivier
4c131b2c78
Implement v2 API for send_join
2019-11-11 15:47:47 +00:00
Andrew Morgan
ace947e8da
Depublish a room from the public rooms list when it is upgraded ( #6232 )
2019-11-01 10:28:09 +00:00
Hubert Chathi
6f4bc6d01d
Merge branch 'develop' into cross-signing_federation
2019-10-31 22:38:21 -04:00
Amber Brown
020add5099
Update black to 19.10b0 ( #6304 )
...
* update version of black and also fix the mypy config being overridden
2019-11-01 02:43:24 +11:00
Andrew Morgan
54fef094b3
Remove usage of deprecated logger.warn method from codebase ( #6271 )
...
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Hubert Chathi
bb6cec27a5
rename get_devices_by_remote to get_device_updates_by_remote
2019-10-30 14:57:34 -04:00
Hubert Chathi
7d7eac61be
Merge branch 'develop' into cross-signing_federation
2019-10-30 10:17:10 -04:00
Andrew Morgan
7955abeaac
Fix small typo in comment ( #6269 )
2019-10-30 11:16:19 +00:00
Erik Johnston
770d1ef673
Merge pull request #6280 from matrix-org/erikj/receipts_async_await
...
Port receipt and read markers to async/wait
2019-10-30 11:44:18 +01:00
Erik Johnston
a287f1e804
Don't return coroutines
2019-10-29 16:36:46 +00:00