74d09a43d9
Always communicate device OTK counts to clients ( #10485 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-07-27 14:36:38 +01:00
2d16e69b4b
Show all joinable rooms in the spaces summary. ( #10298 )
...
Previously only world-readable rooms were shown. This means that
rooms which are public, knockable, or invite-only with a pending invitation,
are included in a space summary. It also applies the same logic to
the experimental room version from MSC3083 -- if a user has access
to the proper allowed rooms then it is shown in the spaces summary.
This change is made per MSC3173 allowing stripped state of a room to
be shown to any potential room joiner.
2021-07-13 08:59:27 -04:00
89cfc3dd98
[pyupgrade] `tests/` ( #10347 )
2021-07-13 11:43:15 +01:00
19d0401c56
Additional unit tests for spaces summary. ( #10305 )
2021-07-12 11:21:04 -04:00
f6767abc05
Remove functionality associated with unused historical stats tables ( #9721 )
...
Fixes #9602
2021-07-08 16:57:13 +01:00
47e28b4031
Ignore EDUs for rooms we're not in ( #10317 )
2021-07-06 14:31:13 +01:00
8d609435c0
Move methods involving event authentication to EventAuthHandler. ( #10268 )
...
Instead of mixing them with user authentication methods.
2021-07-01 14:25:37 -04:00
f55836929d
Do not recurse into non-spaces in the spaces summary. ( #10256 )
...
Previously m.child.room events in non-space rooms would be
treated as part of the room graph, but this is no longer
supported.
2021-06-29 12:00:04 -04:00
6e8fb42be7
Improve validation for `send_{join,leave,knock}` ( #10225 )
...
The idea here is to stop people sending things that aren't joins/leaves/knocks through these endpoints: previously you could send anything you liked through them. I wasn't able to find any security holes from doing so, but it doesn't sound like a good thing.
2021-06-24 15:30:49 +01:00
bd4919fb72
MSC2918 Refresh tokens implementation ( #9450 )
...
This implements refresh tokens, as defined by MSC2918
This MSC has been implemented client side in Hydrogen Web: vector-im/hydrogen-web#235
The basics of the MSC works: requesting refresh tokens on login, having the access tokens expire, and using the refresh token to get a new one.
Signed-off-by: Quentin Gliech <quentingliech@gmail.com>
2021-06-24 14:33:20 +01:00
763dba77ef
Synapse 1.37.0rc1 (2021-06-24)
...
==============================
This release deprecates the current spam checker interface. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#deprecation-of-the-current-spam-checker-interface ) for more information on how to update to the new generic module interface.
This release also removes support for fetching and renewing TLS certificates using the ACME v1 protocol, which has been fully decommissioned by Let's Encrypt on June 1st 2021. Admins previously using this feature should use a [reverse proxy](https://matrix-org.github.io/synapse/develop/reverse_proxy.html ) to handle TLS termination, or use an external ACME client (such as [certbot](https://certbot.eff.org/ )) to retrieve a certificate and key and provide them to Synapse using the `tls_certificate_path` and `tls_private_key_path` configuration settings.
Features
--------
- Implement "room knocking" as per [MSC2403](https://github.com/matrix-org/matrix-doc/pull/2403 ). Contributed by @Sorunome and anoa. ([\#6739](https://github.com/matrix-org/synapse/issues/6739 ), [\#9359](https://github.com/matrix-org/synapse/issues/9359 ), [\#10167](https://github.com/matrix-org/synapse/issues/10167 ), [\#10212](https://github.com/matrix-org/synapse/issues/10212 ), [\#10227](https://github.com/matrix-org/synapse/issues/10227 ))
- Add experimental support for backfilling history into rooms ([MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 )). ([\#9247](https://github.com/matrix-org/synapse/issues/9247 ))
- Implement a generic interface for third-party plugin modules. ([\#10062](https://github.com/matrix-org/synapse/issues/10062 ), [\#10206](https://github.com/matrix-org/synapse/issues/10206 ))
- Implement config option `sso.update_profile_information` to sync SSO users' profile information with the identity provider each time they login. Currently only displayname is supported. ([\#10108](https://github.com/matrix-org/synapse/issues/10108 ))
- Ensure that errors during startup are written to the logs and the console. ([\#10191](https://github.com/matrix-org/synapse/issues/10191 ))
Bugfixes
--------
- Fix a bug introduced in Synapse v1.25.0 that prevented the `ip_range_whitelist` configuration option from working for federation and identity servers. Contributed by @mikure. ([\#10115](https://github.com/matrix-org/synapse/issues/10115 ))
- Remove a broken import line in Synapse's `admin_cmd` worker. Broke in Synapse v1.33.0. ([\#10154](https://github.com/matrix-org/synapse/issues/10154 ))
- Fix a bug introduced in Synapse v1.21.0 which could cause `/sync` to return immediately with an empty response. ([\#10157](https://github.com/matrix-org/synapse/issues/10157 ), [\#10158](https://github.com/matrix-org/synapse/issues/10158 ))
- Fix a minor bug in the response to `/_matrix/client/r0/user/{user}/openid/request_token` causing `expires_in` to be a float instead of an integer. Contributed by @lukaslihotzki. ([\#10175](https://github.com/matrix-org/synapse/issues/10175 ))
- Always require users to re-authenticate for dangerous operations: deactivating an account, modifying an account password, and adding 3PIDs. ([\#10184](https://github.com/matrix-org/synapse/issues/10184 ))
- Fix a bug introduced in Synpase v1.7.2 where remote server count metrics collection would be incorrectly delayed on startup. Found by @heftig. ([\#10195](https://github.com/matrix-org/synapse/issues/10195 ))
- Fix a bug introduced in Synapse v1.35.1 where an `allow` key of a `m.room.join_rules` event could be applied for incorrect room versions and configurations. ([\#10208](https://github.com/matrix-org/synapse/issues/10208 ))
- Fix performance regression in responding to user key requests over federation. Introduced in Synapse v1.34.0rc1. ([\#10221](https://github.com/matrix-org/synapse/issues/10221 ))
Improved Documentation
----------------------
- Add a new guide to decoding request logs. ([\#8436](https://github.com/matrix-org/synapse/issues/8436 ))
- Mention in the sample homeserver config that you may need to configure max upload size in your reverse proxy. Contributed by @aaronraimist. ([\#10122](https://github.com/matrix-org/synapse/issues/10122 ))
- Fix broken links in documentation. ([\#10180](https://github.com/matrix-org/synapse/issues/10180 ))
- Deploy a snapshot of the documentation website upon each new Synapse release. ([\#10198](https://github.com/matrix-org/synapse/issues/10198 ))
Deprecations and Removals
-------------------------
- The current spam checker interface is deprecated in favour of a new generic modules system. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#deprecation-of-the-current-spam-checker-interface ) for more information on how to update to the new system. ([\#10062](https://github.com/matrix-org/synapse/issues/10062 ), [\#10210](https://github.com/matrix-org/synapse/issues/10210 ), [\#10238](https://github.com/matrix-org/synapse/issues/10238 ))
- Stop supporting the unstable spaces prefixes from MSC1772. ([\#10161](https://github.com/matrix-org/synapse/issues/10161 ))
- Remove Synapse's support for automatically fetching and renewing certificates using the ACME v1 protocol. This protocol has been fully turned off by Let's Encrypt for existing installations on June 1st 2021. Admins previously using this feature should use a [reverse proxy](https://matrix-org.github.io/synapse/develop/reverse_proxy.html ) to handle TLS termination, or use an external ACME client (such as [certbot](https://certbot.eff.org/ )) to retrieve a certificate and key and provide them to Synapse using the `tls_certificate_path` and `tls_private_key_path` configuration settings. ([\#10194](https://github.com/matrix-org/synapse/issues/10194 ))
Internal Changes
----------------
- Update the database schema versioning to support gradual migration away from legacy tables. ([\#9933](https://github.com/matrix-org/synapse/issues/9933 ))
- Add type hints to the federation servlets. ([\#10080](https://github.com/matrix-org/synapse/issues/10080 ))
- Improve OpenTracing for event persistence. ([\#10134](https://github.com/matrix-org/synapse/issues/10134 ), [\#10193](https://github.com/matrix-org/synapse/issues/10193 ))
- Clean up the interface for injecting OpenTracing over HTTP. ([\#10143](https://github.com/matrix-org/synapse/issues/10143 ))
- Limit the number of in-flight `/keys/query` requests from a single device. ([\#10144](https://github.com/matrix-org/synapse/issues/10144 ))
- Refactor EventPersistenceQueue. ([\#10145](https://github.com/matrix-org/synapse/issues/10145 ))
- Document `SYNAPSE_TEST_LOG_LEVEL` to see the logger output when running tests. ([\#10148](https://github.com/matrix-org/synapse/issues/10148 ))
- Update the Complement build tags in GitHub Actions to test currently experimental features. ([\#10155](https://github.com/matrix-org/synapse/issues/10155 ))
- Add a `synapse_federation_soft_failed_events_total` metric to track how often events are soft failed. ([\#10156](https://github.com/matrix-org/synapse/issues/10156 ))
- Fetch the corresponding complement branch when performing CI. ([\#10160](https://github.com/matrix-org/synapse/issues/10160 ))
- Add some developer documentation about boolean columns in database schemas. ([\#10164](https://github.com/matrix-org/synapse/issues/10164 ))
- Add extra logging fields to better debug where events are being soft failed. ([\#10168](https://github.com/matrix-org/synapse/issues/10168 ))
- Add debug logging for when we enter and exit `Measure` blocks. ([\#10183](https://github.com/matrix-org/synapse/issues/10183 ))
- Improve comments in structured logging code. ([\#10188](https://github.com/matrix-org/synapse/issues/10188 ))
- Update [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083 ) support with modifications from the MSC. ([\#10189](https://github.com/matrix-org/synapse/issues/10189 ))
- Remove redundant DNS lookup limiter. ([\#10190](https://github.com/matrix-org/synapse/issues/10190 ))
- Upgrade `black` linting tool to 21.6b0. ([\#10197](https://github.com/matrix-org/synapse/issues/10197 ))
- Expose OpenTracing trace id in response headers. ([\#10199](https://github.com/matrix-org/synapse/issues/10199 ))
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEdVkXOgzrGzds0jtrHgFcFF8ZFs0FAmDUW7UACgkQHgFcFF8Z
Fs1ZfQ//bSyzg9mB7F43JdVBH9cpGups3Ur6nmsqE608/i/OEtomx9kTZsoN/9AO
j6WoM1bhJ3j9hoE63w4DIPVhBrojtwmYWeFrRvpRVlwTzZPhfij1nIH9pPrNVwd6
/LE0uA1kW5sDZCTiOhrz84BBBz6knjBH4otl+MDDdrszYixoP3i3SrrZVJAEYyGz
Dpw2OudpoOtMdGyzFMr2ewf2mQm5UFEPs9MT0AqhQJMjyM1lJYzrq3Obv1oTdCJq
BdLx6kMhK8+MbZPrcqIMXgSMHLYeQYeJOrdTzuCozFo0DSblThpW//aa21+sZtPx
dnIE0PpjAsm2Qg1II3EUX+YycYpI5KPhMq0ydEmUD7vTuGgJCKqAV3FBRwMlF2VE
9SOr8Mjd1ip03dHz2vlg38OlrFJ52P0e1RDZwlDtWfeJjZGZ4W6VjMhT7H/Z5USg
2J1FRPTD8E9w8wBOkjVcl76Lj6AY1KU36/ZLoYUbtrt58XYfkD4OzZmAxdJrqbMg
na5T9bd9iM8vWditU+vncxU+c7Bkn55GFyDVMQYl8QtXzBvnbv8gQilGwJLkncZJ
MreQ3OIBPbLNcHml56nmmY+ab+AtrIxuFl0UcZu+LGW8+lN+UZ9G9q0NFO3HbAWN
/LuhCVpO42SLAuLlIhnO3mcHM5jzZodKIa6oXpcMS2jv0nylQWo=
=MtOo
-----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEdVkXOgzrGzds0jtrHgFcFF8ZFs0FAmDUXG8ACgkQHgFcFF8Z
Fs2buxAAg7X4mYKHwlWLn2n2W6bUKY24HAnjtzMrNW76xIHXBJJiJMnhQD3MSKVd
uiUdolc+Tt3drJYOJyurP8R3YI0OuEVbSL0Krs8f3l6jNOU0Gy7YQzc6XUHnA42J
rb6mU8gMdCusk9RHHr1r+sBefwzmxTk4naCkw2n5i+r4ZfNparJK7H9yXCPoedUN
VBZi9f09jaUIm1SiHgK24sW9XfE6ifjdVg+E/r599v2eHJxtCTHUasH9TCAFD8KL
g17NtAP95+SQWPGwUXJFZYcVee+pGbylXuwP0RXXmMvyOWVKloI0LE6phGyLQ5cH
rMRagIkvVLMdYagaX8Tf47hNLVCXRJpuryxMgY3+5fx0O7lCMXlC3mDfs9zze+Fj
KXmJtVy/vpd+zpjQw+HHPnL1dDzsKnDqLFXFCj9Sjv6A/ruckxwWofuh0z83dYtv
1Abp8k7gVTVGZqLGPo8gRoqI2jQhWDvmJd+crb05qp86SzCz1sCma2ujWQtv3c1t
6aQpfmGMhFPp+hS2lS8AcyxK4vq+GudtXUOgzdqCNvIs+eJAV5cq6uufLIy6c2XR
FAR7UowoAoDvqTdD7+i5nHEjbudjjN5KgPXlO3a9+gG5jOR3IsOLZ8FVN0ML6qIJ
wzl0wM9YIlxyleMlWb0l4CFNDKrGJVhWgOupruHS+D5jfZQsZ3g=
=AjRf
-----END PGP SIGNATURE-----
Merge tag 'v1.37.0rc1' into develop
Synapse 1.37.0rc1 (2021-06-24)
==============================
This release deprecates the current spam checker interface. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#deprecation-of-the-current-spam-checker-interface ) for more information on how to update to the new generic module interface.
This release also removes support for fetching and renewing TLS certificates using the ACME v1 protocol, which has been fully decommissioned by Let's Encrypt on June 1st 2021. Admins previously using this feature should use a [reverse proxy](https://matrix-org.github.io/synapse/develop/reverse_proxy.html ) to handle TLS termination, or use an external ACME client (such as [certbot](https://certbot.eff.org/ )) to retrieve a certificate and key and provide them to Synapse using the `tls_certificate_path` and `tls_private_key_path` configuration settings.
Features
--------
- Implement "room knocking" as per [MSC2403](https://github.com/matrix-org/matrix-doc/pull/2403 ). Contributed by @Sorunome and anoa. ([\#6739](https://github.com/matrix-org/synapse/issues/6739 ), [\#9359](https://github.com/matrix-org/synapse/issues/9359 ), [\#10167](https://github.com/matrix-org/synapse/issues/10167 ), [\#10212](https://github.com/matrix-org/synapse/issues/10212 ), [\#10227](https://github.com/matrix-org/synapse/issues/10227 ))
- Add experimental support for backfilling history into rooms ([MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 )). ([\#9247](https://github.com/matrix-org/synapse/issues/9247 ))
- Implement a generic interface for third-party plugin modules. ([\#10062](https://github.com/matrix-org/synapse/issues/10062 ), [\#10206](https://github.com/matrix-org/synapse/issues/10206 ))
- Implement config option `sso.update_profile_information` to sync SSO users' profile information with the identity provider each time they login. Currently only displayname is supported. ([\#10108](https://github.com/matrix-org/synapse/issues/10108 ))
- Ensure that errors during startup are written to the logs and the console. ([\#10191](https://github.com/matrix-org/synapse/issues/10191 ))
Bugfixes
--------
- Fix a bug introduced in Synapse v1.25.0 that prevented the `ip_range_whitelist` configuration option from working for federation and identity servers. Contributed by @mikure. ([\#10115](https://github.com/matrix-org/synapse/issues/10115 ))
- Remove a broken import line in Synapse's `admin_cmd` worker. Broke in Synapse v1.33.0. ([\#10154](https://github.com/matrix-org/synapse/issues/10154 ))
- Fix a bug introduced in Synapse v1.21.0 which could cause `/sync` to return immediately with an empty response. ([\#10157](https://github.com/matrix-org/synapse/issues/10157 ), [\#10158](https://github.com/matrix-org/synapse/issues/10158 ))
- Fix a minor bug in the response to `/_matrix/client/r0/user/{user}/openid/request_token` causing `expires_in` to be a float instead of an integer. Contributed by @lukaslihotzki. ([\#10175](https://github.com/matrix-org/synapse/issues/10175 ))
- Always require users to re-authenticate for dangerous operations: deactivating an account, modifying an account password, and adding 3PIDs. ([\#10184](https://github.com/matrix-org/synapse/issues/10184 ))
- Fix a bug introduced in Synpase v1.7.2 where remote server count metrics collection would be incorrectly delayed on startup. Found by @heftig. ([\#10195](https://github.com/matrix-org/synapse/issues/10195 ))
- Fix a bug introduced in Synapse v1.35.1 where an `allow` key of a `m.room.join_rules` event could be applied for incorrect room versions and configurations. ([\#10208](https://github.com/matrix-org/synapse/issues/10208 ))
- Fix performance regression in responding to user key requests over federation. Introduced in Synapse v1.34.0rc1. ([\#10221](https://github.com/matrix-org/synapse/issues/10221 ))
Improved Documentation
----------------------
- Add a new guide to decoding request logs. ([\#8436](https://github.com/matrix-org/synapse/issues/8436 ))
- Mention in the sample homeserver config that you may need to configure max upload size in your reverse proxy. Contributed by @aaronraimist. ([\#10122](https://github.com/matrix-org/synapse/issues/10122 ))
- Fix broken links in documentation. ([\#10180](https://github.com/matrix-org/synapse/issues/10180 ))
- Deploy a snapshot of the documentation website upon each new Synapse release. ([\#10198](https://github.com/matrix-org/synapse/issues/10198 ))
Deprecations and Removals
-------------------------
- The current spam checker interface is deprecated in favour of a new generic modules system. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#deprecation-of-the-current-spam-checker-interface ) for more information on how to update to the new system. ([\#10062](https://github.com/matrix-org/synapse/issues/10062 ), [\#10210](https://github.com/matrix-org/synapse/issues/10210 ), [\#10238](https://github.com/matrix-org/synapse/issues/10238 ))
- Stop supporting the unstable spaces prefixes from MSC1772. ([\#10161](https://github.com/matrix-org/synapse/issues/10161 ))
- Remove Synapse's support for automatically fetching and renewing certificates using the ACME v1 protocol. This protocol has been fully turned off by Let's Encrypt for existing installations on June 1st 2021. Admins previously using this feature should use a [reverse proxy](https://matrix-org.github.io/synapse/develop/reverse_proxy.html ) to handle TLS termination, or use an external ACME client (such as [certbot](https://certbot.eff.org/ )) to retrieve a certificate and key and provide them to Synapse using the `tls_certificate_path` and `tls_private_key_path` configuration settings. ([\#10194](https://github.com/matrix-org/synapse/issues/10194 ))
Internal Changes
----------------
- Update the database schema versioning to support gradual migration away from legacy tables. ([\#9933](https://github.com/matrix-org/synapse/issues/9933 ))
- Add type hints to the federation servlets. ([\#10080](https://github.com/matrix-org/synapse/issues/10080 ))
- Improve OpenTracing for event persistence. ([\#10134](https://github.com/matrix-org/synapse/issues/10134 ), [\#10193](https://github.com/matrix-org/synapse/issues/10193 ))
- Clean up the interface for injecting OpenTracing over HTTP. ([\#10143](https://github.com/matrix-org/synapse/issues/10143 ))
- Limit the number of in-flight `/keys/query` requests from a single device. ([\#10144](https://github.com/matrix-org/synapse/issues/10144 ))
- Refactor EventPersistenceQueue. ([\#10145](https://github.com/matrix-org/synapse/issues/10145 ))
- Document `SYNAPSE_TEST_LOG_LEVEL` to see the logger output when running tests. ([\#10148](https://github.com/matrix-org/synapse/issues/10148 ))
- Update the Complement build tags in GitHub Actions to test currently experimental features. ([\#10155](https://github.com/matrix-org/synapse/issues/10155 ))
- Add a `synapse_federation_soft_failed_events_total` metric to track how often events are soft failed. ([\#10156](https://github.com/matrix-org/synapse/issues/10156 ))
- Fetch the corresponding complement branch when performing CI. ([\#10160](https://github.com/matrix-org/synapse/issues/10160 ))
- Add some developer documentation about boolean columns in database schemas. ([\#10164](https://github.com/matrix-org/synapse/issues/10164 ))
- Add extra logging fields to better debug where events are being soft failed. ([\#10168](https://github.com/matrix-org/synapse/issues/10168 ))
- Add debug logging for when we enter and exit `Measure` blocks. ([\#10183](https://github.com/matrix-org/synapse/issues/10183 ))
- Improve comments in structured logging code. ([\#10188](https://github.com/matrix-org/synapse/issues/10188 ))
- Update [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083 ) support with modifications from the MSC. ([\#10189](https://github.com/matrix-org/synapse/issues/10189 ))
- Remove redundant DNS lookup limiter. ([\#10190](https://github.com/matrix-org/synapse/issues/10190 ))
- Upgrade `black` linting tool to 21.6b0. ([\#10197](https://github.com/matrix-org/synapse/issues/10197 ))
- Expose OpenTracing trace id in response headers. ([\#10199](https://github.com/matrix-org/synapse/issues/10199 ))
2021-06-24 11:20:28 +01:00
c955e37868
Fix wrapping of legacy check_registration_for_spam ( #10238 )
...
Fixes #10234
2021-06-23 17:22:08 +02:00
e19e3d452d
Improve the reliability of auto-joining remote rooms ( #10237 )
...
If a room is remote and we don't have a user in it, always try to join it. It might fail if the room is invite-only, but we don't have a user to invite with, so at this point it's the best we can do.
Fixes #10233 (at least to some extent)
2021-06-23 16:14:52 +02:00
96f6293de5
Add endpoints for backfilling history (MSC2716) ( #9247 )
...
Work on https://github.com/matrix-org/matrix-doc/pull/2716
2021-06-22 10:02:53 +01:00
0bd968921c
Fix a missing await when in the spaces summary. ( #10208 )
...
This could cause a minor data leak if someone defined a non-restricted join rule
with an allow key or used a restricted join rule in an older room version, but this is
unlikely.
Additionally this starts adding unit tests to the spaces summary handler.
2021-06-18 18:41:33 +01:00
1b3e398bea
Standardise the module interface ( #10062 )
...
This PR adds a common configuration section for all modules (see docs). These modules are then loaded at startup by the homeserver. Modules register their hooks and web resources using the new `register_[...]_callbacks` and `register_web_resource` methods of the module API.
2021-06-18 12:15:52 +01:00
8070b893db
update black to 21.6b0 ( #10197 )
...
Reformat all files with the new version.
Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
2021-06-17 15:20:06 +01:00
11846dff8c
Limit the number of in-flight /keys/query requests from a single device. ( #10144 )
2021-06-09 07:05:32 -04:00
8942e23a69
Always update AS last_pos, even on no events ( #10107 )
...
Fixes #1834 .
`get_new_events_for_appservice` internally calls `get_events_as_list`, which will filter out any rejected events. If all returned events are filtered out, `_notify_interested_services` will return without updating the last handled stream position. If there are 100 consecutive such events, processing will halt altogether.
Breaking the loop is now done by checking whether we're up-to-date with `current_max` in the loop condition, instead of relying on an empty `events` list.
Signed-off-by: Willem Mulder <14mRh4X0r@gmail.com>
2021-06-07 15:42:05 +01:00
3e831f24ff
Don't hammer the database for destination retry timings every ~5mins ( #10036 )
2021-05-21 17:57:08 +01:00
21bd230831
Add a test for update_presence ( #10033 )
...
https://github.com/matrix-org/synapse/issues/9962 uncovered that we accidentally removed all but one of the presence updates that we store in the database when persisting multiple updates. This could cause users' presence state to be stale.
The bug was fixed in #10014 , and this PR just adds a test that failed on the old code, and was used to initially verify the bug.
The test attempts to insert some presence into the database in a batch using `PresenceStore.update_presence`, and then simply pulls it out again.
2021-05-21 17:29:14 +01:00
7562d887e1
Change the format of access tokens away from macaroons ( #5588 )
2021-05-12 15:04:51 +01:00
27c375f812
Sort child events according to MSC1772 for the spaces summary API. ( #9954 )
...
This should help ensure that equivalent results are achieved between
homeservers querying for the summary of a space.
This implements modified MSC1772 rules, according to MSC2946.
The different is that the origin_server_ts of the m.room.create event
is not used as a tie-breaker since this might not be known if the
homeserver is not part of the room.
2021-05-11 12:57:39 -04:00
37623e3382
Increase perf of handling presence when joining large rooms. ( #9916 )
2021-05-05 17:27:05 +01:00
e4ab8676b4
Fix tight loop handling presence replication. ( #9900 )
...
Only affects workers. Introduced in #9819 .
Fixes #9899 .
2021-04-28 14:42:50 +01:00
683d6f75af
Rename handler and config modules which end in handler/config. ( #9816 )
2021-04-20 14:55:20 -04:00
de0d088adc
Add presence federation stream ( #9819 )
2021-04-20 14:11:24 +01:00
495b214f4f
Fix (final) Bugbear violations ( #9838 )
2021-04-20 11:50:49 +01:00
4b965c862d
Remove redundant "coding: utf-8" lines ( #9786 )
...
Part of #9744
Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.
`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
0b3112123d
Use mock from the stdlib. ( #9772 )
2021-04-09 13:44:38 -04:00
04819239ba
Add a Synapse Module for configuring presence update routing ( #9491 )
...
At the moment, if you'd like to share presence between local or remote users, those users must be sharing a room together. This isn't always the most convenient or useful situation though.
This PR adds a module to Synapse that will allow deployments to set up extra logic on where presence updates should be routed. The module must implement two methods, `get_users_for_states` and `get_interested_users`. These methods are given presence updates or user IDs and must return information that Synapse will use to grant passing presence updates around.
A method is additionally added to `ModuleApi` which allows triggering a set of users to receive the current, online presence information for all users they are considered interested in. This is the equivalent of that user receiving presence information during an initial sync.
The goal of this module is to be fairly generic and useful for a variety of applications, with hard requirements being:
* Sending state for a specific set or all known users to a defined set of local and remote users.
* The ability to trigger an initial sync for specific users, so they receive all current state.
2021-04-06 14:38:30 +01:00
405aeb0b2c
Implement MSC3026: busy presence state
2021-03-18 16:34:47 +01:00
dd5e5dc1d6
Add SSO attribute requirements for OIDC providers ( #9609 )
...
Allows limiting who can login using OIDC via the claims
made from the IdP.
2021-03-16 11:46:07 -04:00
847ecdd8fa
Pass SSO IdP information to spam checker's registration function ( #9626 )
...
Fixes https://github.com/matrix-org/synapse/issues/9572
When a SSO user logs in for the first time, we create a local Matrix user for them. This goes through the register_user flow, which ends up triggering the spam checker. Spam checker modules don't currently have any way to differentiate between a user trying to sign up initially, versus an SSO user (whom has presumably already been approved elsewhere) trying to log in for the first time.
This PR passes `auth_provider_id` as an argument to the `check_registration_for_spam` function. This argument will contain an ID of an SSO provider (`"saml"`, `"cas"`, etc.) if one was used, else `None`.
2021-03-16 12:41:41 +00:00
eaada74075
JWT OIDC secrets for Sign in with Apple ( #9549 )
...
Apple had to be special. They want a client secret which is generated from an EC key.
Fixes #9220 . Also fixes #9212 while I'm here.
2021-03-09 15:03:37 +00:00
7eb6e39a8f
Record the SSO Auth Provider in the login token ( #9510 )
...
This great big stack of commits is a a whole load of hoop-jumping to make it easier to store additional values in login tokens, and then to actually store the SSO Identity Provider in the login token. (Making use of that data will follow in a subsequent PR.)
2021-03-04 14:44:22 +00:00
c4a55ac4a4
Fix style checking due to updated black.
2021-02-19 08:19:54 -05:00
8bcfc2eaad
Be smarter about which hosts to send presence to when processing room joins ( #9402 )
...
This PR attempts to eliminate unnecessary presence sending work when your local server joins a room, or when a remote server joins a room your server is participating in by processing state deltas in chunks rather than individually.
---
When your server joins a room for the first time, it requests the historical state as well. This chunk of new state is passed to the presence handler which, after filtering that state down to only membership joins, will send presence updates to homeservers for each join processed.
It turns out that we were being a bit naive and processing each event individually, and sending out presence updates for every one of those joins. Even if many different joins were users on the same server (hello IRC bridges), we'd send presence to that same homeserver for every remote user join we saw.
This PR attempts to deduplicate all of that by processing the entire batch of state deltas at once, instead of only doing each join individually. We process the joins and note down which servers need which presence:
* If it was a local user join, send that user's latest presence to all servers in the room
* If it was a remote user join, send the presence for all local users in the room to that homeserver
We deduplicate by inserting all of those pending updates into a dictionary of the form:
```
{
server_name1: {presence_update1, ...},
server_name2: {presence_update1, presence_update2, ...}
}
```
Only after building this dict do we then start sending out presence updates.
2021-02-19 11:37:29 +00:00
13e9029f44
Add a config option to prioritise local users in user directory search results ( #9383 )
...
This PR adds a homeserver config option, `user_directory.prefer_local_users`, that when enabled will show local users higher in user directory search results than remote users. This option is off by default.
Note that turning this on doesn't necessarily mean that remote users will always be put below local users, but they should be assuming all other ranking factors (search query match, profile information present etc) are identical.
This is useful for, say, University networks that are openly federating, but want to prioritise local students and staff in the user directory over other random users.
2021-02-19 11:02:03 +00:00
9bc74743d5
Add configs to make profile data more private ( #9203 )
...
Add off-by-default configuration settings to:
- disable putting an invitee's profile info in invite events
- disable profile lookup via federation
Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
2021-02-19 09:50:41 +00:00
e1071fd625
Support for form_post in OIDC responses ( #9376 )
...
Apple want to POST the OIDC auth response back to us rather than using query-params; add the necessary support to make that work.
2021-02-17 10:15:14 +00:00
0a00b7ff14
Update black, and run auto formatting over the codebase ( #9381 )
...
- Update black version to the latest
- Run black auto formatting over the codebase
- Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md
2021-02-16 22:32:34 +00:00
3b754aea27
Clean up caching/locking of OIDC metadata load ( #9362 )
...
Ensure that we lock correctly to prevent multiple concurrent metadata load
requests, and generally clean up the way we construct the metadata cache.
2021-02-16 16:27:38 +00:00
74af356baf
Convert additional test-cases to homeserver test case. ( #9396 )
...
And convert some inlineDeferreds to async-friendly functions.
2021-02-16 08:04:15 -05:00
8a33d217bd
Convert some test cases to use HomeserverTestCase. ( #9377 )
...
This has the side-effect of being able to remove use of `inlineCallbacks`
in the test-cases for cleaner tracebacks.
2021-02-11 10:29:09 -05:00
6dade80048
Combine the CAS & SAML implementations for required attributes. ( #9326 )
2021-02-11 10:05:15 -05:00
ff55300b91
Honour ratelimit flag for application services for invite ratelimiting ( #9302 )
2021-02-03 10:17:37 +00:00
846b9d3df0
Put OIDC callback URI under /_synapse/client. ( #9288 )
2021-02-01 22:56:01 +00:00
18ab35284a
Merge branch 'social_login' into develop
2021-02-01 17:28:37 +00:00
8aed29dc61
Improve styling and wording of SSO redirect confirm template ( #9272 )
2021-02-01 15:50:56 +00:00
Andrew Morgan