Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes
Erik Johnston
commit
fd1b6334f0
|
|
@ -377,7 +377,7 @@ jobs:
|
|||
# Run Complement
|
||||
- run: |
|
||||
set -o pipefail
|
||||
go test -v -json -p 1 -tags synapse_blacklist,msc2403,msc2716,msc3030 ./tests/... 2>&1 | gotestfmt
|
||||
go test -v -json -tags synapse_blacklist,msc2403,msc2716,msc3030 ./tests/... 2>&1 | gotestfmt
|
||||
shell: bash
|
||||
name: Run Complement Tests
|
||||
env:
|
||||
|
|
|
|||
37
CHANGES.md
37
CHANGES.md
|
|
@ -1,10 +1,43 @@
|
|||
Synapse 1.55.0rc1 (2022-03-15)
|
||||
==============================
|
||||
Synapse 1.55.2 (2022-03-24)
|
||||
===========================
|
||||
|
||||
This patch version reverts the earlier fixes from Synapse 1.55.1, which could cause problems in certain deployments, and instead adds a cap to the version of Jinja to be installed. Again, this is to fix an incompatibility with version 3.1.0 of the [Jinja](https://pypi.org/project/Jinja2/) library, and again, deployments of Synapse using the `matrixdotorg/synapse` Docker image or Debian packages from packages.matrix.org are not affected.
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Pin Jinja to <3.1.0, as Synapse fails to start with Jinja 3.1.0. ([\#12297](https://github.com/matrix-org/synapse/issues/12297))
|
||||
- Revert changes from 1.55.1 as they caused problems with older versions of Jinja ([\#12296](https://github.com/matrix-org/synapse/issues/12296))
|
||||
|
||||
|
||||
Synapse 1.55.1 (2022-03-24)
|
||||
===========================
|
||||
|
||||
This is a patch release that fixes an incompatibility with version 3.1.0 of the [Jinja](https://pypi.org/project/Jinja2/) library, released on March 24th, 2022. Deployments of Synapse using the `matrixdotorg/synapse` Docker image or Debian packages from packages.matrix.org are not affected.
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Remove uses of the long-deprecated `jinja2.Markup` which would prevent Synapse from starting with Jinja 3.1.0 or above installed. ([\#12289](https://github.com/matrix-org/synapse/issues/12289))
|
||||
|
||||
|
||||
Synapse 1.55.0 (2022-03-22)
|
||||
===========================
|
||||
|
||||
This release removes a workaround introduced in Synapse 1.50.0 for Mjolnir compatibility. **This breaks compatibility with Mjolnir 1.3.1 and earlier. ([\#11700](https://github.com/matrix-org/synapse/issues/11700))**; Mjolnir users should upgrade Mjolnir before upgrading Synapse to this version.
|
||||
|
||||
This release also moves the location of the `synctl` script; see the [upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#synctl-script-has-been-moved) for more details.
|
||||
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Tweak copy for default Single Sign-On account details template to better adhere to mobile app store guidelines. ([\#12265](https://github.com/matrix-org/synapse/issues/12265), [\#12260](https://github.com/matrix-org/synapse/issues/12260))
|
||||
|
||||
|
||||
Synapse 1.55.0rc1 (2022-03-15)
|
||||
==============================
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1 @@
|
|||
Rename `shared_rooms` to `mutual_rooms` (MSC2666), as per proposal changes.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Remove check on `update_user_directory` for shared rooms handler (MSC2666), and update/expand documentation.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) instead of abusing `auth_event_ids`.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Fix a long-standing bug which caused the `/_matrix/federation/v1/state` and `.../state_ids` endpoints to return incorrect or invalid data when called for an event which we have stored as an "outlier".
|
||||
|
|
@ -0,0 +1 @@
|
|||
Refuse to start if registration is enabled without email, captcha, or token-based verification unless new config flag `enable_registration_without_verification` is set.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Allow modules to store already existing 3PID associations.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Add tests for database transaction callbacks.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Handle cancellation in `DatabasePool.runInteraction()`.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Add missing type hints for cache storage.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Clean-up logic around rebasing URLs for URL image previews.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Add type hints to tests files.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Use the `ignored_users` table in additional places instead of re-parsing the account data.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Fix a long-standing bug where events from ignored users were still considered for relations.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Fix a bug introduced in v1.53.0 where an unnecessary query could be performed when fetching bundled aggregations for threads.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Fix the link to the module documentation in the legacy spam checker warning message.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Fix a long-standing bug where events from ignored users were still considered for relations.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Refactor the relations endpoints to add a `RelationsHandler`.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Add type hints to tests files.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Generate announcement links in the release script.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Remove incorrect prefixes in the worker documentation for some endpoints.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Improve error message when dependencies check finds a broken installation.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Correct `check_username_for_spam` annotations and docs.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Add missing type hints for storage.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Allow registering admin users using the module API. Contributed by Famedly.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Add missing type hints for storage.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Add type hints to tests files.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Compress metrics HTTP resource when enabled. Contributed by Nick @ Beeper.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Reword 'Choose your user name' as 'Choose your account name' in the SSO registration template, in order to comply with SIWA guidelines.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Fix a bug introduced in Synapse 1.52 where admins could not deactivate and GDPR-erase a user if Synapse was configured with limits on avatars.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Refuse to start if DB has non-`C` locale, unless config flag `allow_unsafe_db_locale` is set to true.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Tweak copy for default sso account details template to better adhere to mobile app store guidelines.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Optionally include account validity expiration information to experimental [MSC3720](https://github.com/matrix-org/matrix-doc/pull/3720) account status responses.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Use type stubs for `psycopg2`.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Add a new cache `_get_membership_from_event_id` to speed up push rule calculations in large rooms.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Corrected Authentik OpenID typo, added helpful note for troubleshooting. Contributed by @IronTooch.
|
||||
|
|
@ -0,0 +1 @@
|
|||
HAProxy reverse proxy guide update to stop sending IPv4-mapped address to homeserver. Contributed by @villepeh.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Re-enable Complement concurrency in CI.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Fix a long-standing bug where events from ignored users were still considered for relations.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Refuse to start if DB has non-`C` locale, unless config flag `allow_unsafe_db_locale` is set to true.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Remove unused test utilities.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Enhance logging for inbound federation events.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Refactor `create_new_client_event` to use a new parameter, `state_event_ids`, which accurately describes the usage with [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) instead of abusing `auth_event_ids`.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Improve type annotations for `execute_values`.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Fix compatibility with the recently-released Jinja 3.1.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Avoid trying to calculate the state at outlier events.
|
||||
|
|
@ -1,3 +1,21 @@
|
|||
matrix-synapse-py3 (1.55.2) stable; urgency=medium
|
||||
|
||||
* New synapse release 1.55.2.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Thu, 24 Mar 2022 19:07:11 +0000
|
||||
|
||||
matrix-synapse-py3 (1.55.1) stable; urgency=medium
|
||||
|
||||
* New synapse release 1.55.1.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Thu, 24 Mar 2022 17:44:23 +0000
|
||||
|
||||
matrix-synapse-py3 (1.55.0) stable; urgency=medium
|
||||
|
||||
* New synapse release 1.55.0.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Tue, 22 Mar 2022 13:59:26 +0000
|
||||
|
||||
matrix-synapse-py3 (1.55.0~rc1) stable; urgency=medium
|
||||
|
||||
* New synapse release 1.55.0~rc1.
|
||||
|
|
|
|||
|
|
@ -38,6 +38,7 @@ for port in 8080 8081 8082; do
|
|||
printf '\n\n# Customisation made by demo/start.sh\n\n'
|
||||
echo "public_baseurl: http://localhost:$port/"
|
||||
echo 'enable_registration: true'
|
||||
echo 'enable_registration_without_verification: true'
|
||||
echo ''
|
||||
|
||||
# Warning, this heredoc depends on the interaction of tabs and spaces.
|
||||
|
|
|
|||
|
|
@ -172,7 +172,7 @@ any of the subsequent implementations of this callback.
|
|||
_First introduced in Synapse v1.37.0_
|
||||
|
||||
```python
|
||||
async def check_username_for_spam(user_profile: Dict[str, str]) -> bool
|
||||
async def check_username_for_spam(user_profile: synapse.module_api.UserProfile) -> bool
|
||||
```
|
||||
|
||||
Called when computing search results in the user directory. The module must return a
|
||||
|
|
@ -182,9 +182,11 @@ search results; otherwise return `False`.
|
|||
|
||||
The profile is represented as a dictionary with the following keys:
|
||||
|
||||
* `user_id`: The Matrix ID for this user.
|
||||
* `display_name`: The user's display name.
|
||||
* `avatar_url`: The `mxc://` URL to the user's avatar.
|
||||
* `user_id: str`. The Matrix ID for this user.
|
||||
* `display_name: Optional[str]`. The user's display name, or `None` if this user
|
||||
has not set a display name.
|
||||
* `avatar_url: Optional[str]`. The `mxc://` URL to the user's avatar, or `None`
|
||||
if this user has not set an avatar.
|
||||
|
||||
The module is given a copy of the original dictionary, so modifying it from within the
|
||||
module cannot modify a user's profile when included in user directory search results.
|
||||
|
|
|
|||
|
|
@ -225,6 +225,8 @@ oidc_providers:
|
|||
3. Create an application for synapse in Authentik and link it to the provider.
|
||||
4. Note the slug of your application, Client ID and Client Secret.
|
||||
|
||||
Note: RSA keys must be used for signing for Authentik, ECC keys do not work.
|
||||
|
||||
Synapse config:
|
||||
```yaml
|
||||
oidc_providers:
|
||||
|
|
@ -240,7 +242,7 @@ oidc_providers:
|
|||
- "email"
|
||||
user_mapping_provider:
|
||||
config:
|
||||
localpart_template: "{{ user.preferred_username }}}"
|
||||
localpart_template: "{{ user.preferred_username }}"
|
||||
display_name_template: "{{ user.preferred_username|capitalize }}" # TO BE FILLED: If your users have names in Authentik and you want those in Synapse, this should be replaced with user.name|capitalize.
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -234,12 +234,13 @@ host all all ::1/128 ident
|
|||
### Fixing incorrect `COLLATE` or `CTYPE`
|
||||
|
||||
Synapse will refuse to set up a new database if it has the wrong values of
|
||||
`COLLATE` and `CTYPE` set, and will log warnings on existing databases. Using
|
||||
different locales can cause issues if the locale library is updated from
|
||||
`COLLATE` and `CTYPE` set. Synapse will also refuse to start an existing database with incorrect values
|
||||
of `COLLATE` and `CTYPE` unless the config flag `allow_unsafe_locale`, found in the
|
||||
`database` section of the config, is set to true. Using different locales can cause issues if the locale library is updated from
|
||||
underneath the database, or if a different version of the locale is used on any
|
||||
replicas.
|
||||
|
||||
The safest way to fix the issue is to dump the database and recreate it with
|
||||
If you have a databse with an unsafe locale, the safest way to fix the issue is to dump the database and recreate it with
|
||||
the correct locale parameter (as shown above). It is also possible to change the
|
||||
parameters on a live database and run a `REINDEX` on the entire database,
|
||||
however extreme care must be taken to avoid database corruption.
|
||||
|
|
|
|||
|
|
@ -182,7 +182,7 @@ matrix.example.com {
|
|||
|
||||
```
|
||||
frontend https
|
||||
bind :::443 v4v6 ssl crt /etc/ssl/haproxy/ strict-sni alpn h2,http/1.1
|
||||
bind *:443,[::]:443 ssl crt /etc/ssl/haproxy/ strict-sni alpn h2,http/1.1
|
||||
http-request set-header X-Forwarded-Proto https if { ssl_fc }
|
||||
http-request set-header X-Forwarded-Proto http if !{ ssl_fc }
|
||||
http-request set-header X-Forwarded-For %[src]
|
||||
|
|
@ -195,7 +195,7 @@ frontend https
|
|||
use_backend matrix if matrix-host matrix-path
|
||||
|
||||
frontend matrix-federation
|
||||
bind :::8448 v4v6 ssl crt /etc/ssl/haproxy/synapse.pem alpn h2,http/1.1
|
||||
bind *:8448,[::]:8448 ssl crt /etc/ssl/haproxy/synapse.pem alpn h2,http/1.1
|
||||
http-request set-header X-Forwarded-Proto https if { ssl_fc }
|
||||
http-request set-header X-Forwarded-Proto http if !{ ssl_fc }
|
||||
http-request set-header X-Forwarded-For %[src]
|
||||
|
|
|
|||
|
|
@ -783,6 +783,12 @@ caches:
|
|||
# 'txn_limit' gives the maximum number of transactions to run per connection
|
||||
# before reconnecting. Defaults to 0, which means no limit.
|
||||
#
|
||||
# 'allow_unsafe_locale' is an option specific to Postgres. Under the default behavior, Synapse will refuse to
|
||||
# start if the postgres db is set to a non-C locale. You can override this behavior (which is *not* recommended)
|
||||
# by setting 'allow_unsafe_locale' to true. Note that doing so may corrupt your database. You can find more information
|
||||
# here: https://matrix-org.github.io/synapse/latest/postgres.html#fixing-incorrect-collate-or-ctype and here:
|
||||
# https://wiki.postgresql.org/wiki/Locale_data_changes
|
||||
#
|
||||
# 'args' gives options which are passed through to the database engine,
|
||||
# except for options starting 'cp_', which are used to configure the Twisted
|
||||
# connection pool. For a reference to valid arguments, see:
|
||||
|
|
@ -1212,10 +1218,18 @@ oembed:
|
|||
# Registration can be rate-limited using the parameters in the "Ratelimiting"
|
||||
# section of this file.
|
||||
|
||||
# Enable registration for new users.
|
||||
# Enable registration for new users. Defaults to 'false'. It is highly recommended that if you enable registration,
|
||||
# you use either captcha, email, or token-based verification to verify that new users are not bots. In order to enable registration
|
||||
# without any verification, you must also set `enable_registration_without_verification`, found below.
|
||||
#
|
||||
#enable_registration: false
|
||||
|
||||
# Enable registration without email or captcha verification. Note: this option is *not* recommended,
|
||||
# as registration without verification is a known vector for spam and abuse. Defaults to false. Has no effect
|
||||
# unless `enable_registration` is also enabled.
|
||||
#
|
||||
#enable_registration_without_verification: true
|
||||
|
||||
# Time that a user's session remains valid for, after they log in.
|
||||
#
|
||||
# Note that this is not currently compatible with guest logins.
|
||||
|
|
|
|||
|
|
@ -99,8 +99,21 @@ experimental_features:
|
|||
groups_enabled: false
|
||||
```
|
||||
|
||||
## Change in behaviour for PostgreSQL databases with unsafe locale
|
||||
|
||||
Synapse now refuses to start when using PostgreSQL with non-`C` values for `COLLATE` and
|
||||
`CTYPE` unless the config flag `allow_unsafe_locale`, found in the database section of
|
||||
the configuration file, is set to `true`. See the [PostgreSQL documentation](https://matrix-org.github.io/synapse/latest/postgres.html#fixing-incorrect-collate-or-ctype)
|
||||
for more information and instructions on how to fix a database with incorrect values.
|
||||
|
||||
# Upgrading to v1.55.0
|
||||
|
||||
## Open registration without verification is now disabled by default
|
||||
|
||||
Synapse will refuse to start if registration is enabled without email, captcha, or token-based verification unless the new config
|
||||
flag `enable_registration_without_verification` is set to "true".
|
||||
|
||||
|
||||
## `synctl` script has been moved
|
||||
|
||||
The `synctl` script
|
||||
|
|
|
|||
|
|
@ -185,8 +185,8 @@ worker: refer to the [stream writers](#stream-writers) section below for further
|
|||
information.
|
||||
|
||||
# Sync requests
|
||||
^/_matrix/client/(v2_alpha|r0|v3)/sync$
|
||||
^/_matrix/client/(api/v1|v2_alpha|r0|v3)/events$
|
||||
^/_matrix/client/(r0|v3)/sync$
|
||||
^/_matrix/client/(api/v1|r0|v3)/events$
|
||||
^/_matrix/client/(api/v1|r0|v3)/initialSync$
|
||||
^/_matrix/client/(api/v1|r0|v3)/rooms/[^/]+/initialSync$
|
||||
|
||||
|
|
@ -200,13 +200,9 @@ information.
|
|||
^/_matrix/federation/v1/query/
|
||||
^/_matrix/federation/v1/make_join/
|
||||
^/_matrix/federation/v1/make_leave/
|
||||
^/_matrix/federation/v1/send_join/
|
||||
^/_matrix/federation/v2/send_join/
|
||||
^/_matrix/federation/v1/send_leave/
|
||||
^/_matrix/federation/v2/send_leave/
|
||||
^/_matrix/federation/v1/invite/
|
||||
^/_matrix/federation/v2/invite/
|
||||
^/_matrix/federation/v1/query_auth/
|
||||
^/_matrix/federation/(v1|v2)/send_join/
|
||||
^/_matrix/federation/(v1|v2)/send_leave/
|
||||
^/_matrix/federation/(v1|v2)/invite/
|
||||
^/_matrix/federation/v1/event_auth/
|
||||
^/_matrix/federation/v1/exchange_third_party_invite/
|
||||
^/_matrix/federation/v1/user/devices/
|
||||
|
|
@ -274,6 +270,8 @@ information.
|
|||
Additionally, the following REST endpoints can be handled for GET requests:
|
||||
|
||||
^/_matrix/federation/v1/groups/
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/pushrules/
|
||||
^/_matrix/client/(r0|v3|unstable)/groups/
|
||||
|
||||
Pagination requests can also be handled, but all requests for a given
|
||||
room must be routed to the same instance. Additionally, care must be taken to
|
||||
|
|
@ -397,23 +395,23 @@ the stream writer for the `typing` stream:
|
|||
The following endpoints should be routed directly to the worker configured as
|
||||
the stream writer for the `to_device` stream:
|
||||
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/sendToDevice/
|
||||
^/_matrix/client/(r0|v3|unstable)/sendToDevice/
|
||||
|
||||
##### The `account_data` stream
|
||||
|
||||
The following endpoints should be routed directly to the worker configured as
|
||||
the stream writer for the `account_data` stream:
|
||||
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/.*/tags
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/.*/account_data
|
||||
^/_matrix/client/(r0|v3|unstable)/.*/tags
|
||||
^/_matrix/client/(r0|v3|unstable)/.*/account_data
|
||||
|
||||
##### The `receipts` stream
|
||||
|
||||
The following endpoints should be routed directly to the worker configured as
|
||||
the stream writer for the `receipts` stream:
|
||||
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/receipt
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/read_markers
|
||||
^/_matrix/client/(r0|v3|unstable)/rooms/.*/receipt
|
||||
^/_matrix/client/(r0|v3|unstable)/rooms/.*/read_markers
|
||||
|
||||
##### The `presence` stream
|
||||
|
||||
|
|
@ -528,19 +526,28 @@ Note that if a reverse proxy is used , then `/_matrix/media/` must be routed for
|
|||
Handles searches in the user directory. It can handle REST endpoints matching
|
||||
the following regular expressions:
|
||||
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/user_directory/search$
|
||||
^/_matrix/client/(r0|v3|unstable)/user_directory/search$
|
||||
|
||||
When using this worker you must also set `update_user_directory: False` in the
|
||||
When using this worker you must also set `update_user_directory: false` in the
|
||||
shared configuration file to stop the main synapse running background
|
||||
jobs related to updating the user directory.
|
||||
|
||||
Above endpoint is not *required* to be routed to this worker. By default,
|
||||
`update_user_directory` is set to `true`, which means the main process
|
||||
will handle updates. All workers configured with `client` can handle the above
|
||||
endpoint as long as either this worker or the main process are configured to
|
||||
handle it, and are online.
|
||||
|
||||
If `update_user_directory` is set to `false`, and this worker is not running,
|
||||
the above endpoint may give outdated results.
|
||||
|
||||
### `synapse.app.frontend_proxy`
|
||||
|
||||
Proxies some frequently-requested client endpoints to add caching and remove
|
||||
load from the main synapse. It can handle REST endpoints matching the following
|
||||
regular expressions:
|
||||
|
||||
^/_matrix/client/(api/v1|r0|v3|unstable)/keys/upload
|
||||
^/_matrix/client/(r0|v3|unstable)/keys/upload
|
||||
|
||||
If `use_presence` is False in the homeserver config, it can also handle REST
|
||||
endpoints matching the following regular expressions:
|
||||
|
|
|
|||
20
mypy.ini
20
mypy.ini
|
|
@ -38,17 +38,11 @@ exclude = (?x)
|
|||
|synapse/_scripts/update_synapse_database.py
|
||||
|
||||
|synapse/storage/databases/__init__.py
|
||||
|synapse/storage/databases/main/__init__.py
|
||||
|synapse/storage/databases/main/cache.py
|
||||
|synapse/storage/databases/main/devices.py
|
||||
|synapse/storage/databases/main/event_federation.py
|
||||
|synapse/storage/databases/main/group_server.py
|
||||
|synapse/storage/databases/main/metrics.py
|
||||
|synapse/storage/databases/main/monthly_active_users.py
|
||||
|synapse/storage/databases/main/push_rule.py
|
||||
|synapse/storage/databases/main/receipts.py
|
||||
|synapse/storage/databases/main/roommember.py
|
||||
|synapse/storage/databases/main/search.py
|
||||
|synapse/storage/databases/main/state.py
|
||||
|synapse/storage/schema/
|
||||
|
||||
|
|
@ -66,14 +60,6 @@ exclude = (?x)
|
|||
|tests/federation/test_federation_server.py
|
||||
|tests/federation/transport/test_knocking.py
|
||||
|tests/federation/transport/test_server.py
|
||||
|tests/handlers/test_cas.py
|
||||
|tests/handlers/test_directory.py
|
||||
|tests/handlers/test_e2e_keys.py
|
||||
|tests/handlers/test_federation.py
|
||||
|tests/handlers/test_oidc.py
|
||||
|tests/handlers/test_presence.py
|
||||
|tests/handlers/test_profile.py
|
||||
|tests/handlers/test_saml.py
|
||||
|tests/handlers/test_typing.py
|
||||
|tests/http/federation/test_matrix_federation_agent.py
|
||||
|tests/http/federation/test_srv_resolver.py
|
||||
|
|
@ -85,7 +71,6 @@ exclude = (?x)
|
|||
|tests/logging/test_terse_json.py
|
||||
|tests/module_api/test_api.py
|
||||
|tests/push/test_email.py
|
||||
|tests/push/test_http.py
|
||||
|tests/push/test_presentable_names.py
|
||||
|tests/push/test_push_rule_evaluator.py
|
||||
|tests/rest/client/test_transactions.py
|
||||
|
|
@ -94,12 +79,7 @@ exclude = (?x)
|
|||
|tests/server.py
|
||||
|tests/server_notices/test_resource_limits_server_notices.py
|
||||
|tests/state/test_v2.py
|
||||
|tests/storage/test_background_update.py
|
||||
|tests/storage/test_base.py
|
||||
|tests/storage/test_client_ips.py
|
||||
|tests/storage/test_database.py
|
||||
|tests/storage/test_event_federation.py
|
||||
|tests/storage/test_id_generators.py
|
||||
|tests/storage/test_roommember.py
|
||||
|tests/test_metrics.py
|
||||
|tests/test_phone_home.py
|
||||
|
|
|
|||
|
|
@ -66,11 +66,15 @@ def cli():
|
|||
|
||||
./scripts-dev/release.py tag
|
||||
|
||||
# ... wait for asssets to build ...
|
||||
# ... wait for assets to build ...
|
||||
|
||||
./scripts-dev/release.py publish
|
||||
./scripts-dev/release.py upload
|
||||
|
||||
# Optional: generate some nice links for the announcement
|
||||
|
||||
./scripts-dev/release.py upload
|
||||
|
||||
If the env var GH_TOKEN (or GITHUB_TOKEN) is set, or passed into the
|
||||
`tag`/`publish` command, then a new draft release will be created/published.
|
||||
"""
|
||||
|
|
@ -415,6 +419,41 @@ def upload():
|
|||
)
|
||||
|
||||
|
||||
@cli.command()
|
||||
def announce():
|
||||
"""Generate markdown to announce the release."""
|
||||
|
||||
current_version, _, _ = parse_version_from_module()
|
||||
tag_name = f"v{current_version}"
|
||||
|
||||
click.echo(
|
||||
f"""
|
||||
Hi everyone. Synapse {current_version} has just been released.
|
||||
|
||||
[notes](https://github.com/matrix-org/synapse/releases/tag/{tag_name}) |\
|
||||
[docker](https://hub.docker.com/r/matrixdotorg/synapse/tags?name={tag_name}) | \
|
||||
[debs](https://packages.matrix.org/debian/) | \
|
||||
[pypi](https://pypi.org/project/matrix-synapse/{current_version}/)"""
|
||||
)
|
||||
|
||||
if "rc" in tag_name:
|
||||
click.echo(
|
||||
"""
|
||||
Announce the RC in
|
||||
- #homeowners:matrix.org (Synapse Announcements)
|
||||
- #synapse-dev:matrix.org"""
|
||||
)
|
||||
else:
|
||||
click.echo(
|
||||
"""
|
||||
Announce the release in
|
||||
- #homeowners:matrix.org (Synapse Announcements), bumping the version in the topic
|
||||
- #synapse:matrix.org (Synapse Admins), bumping the version in the topic
|
||||
- #synapse-dev:matrix.org
|
||||
- #synapse-package-maintainers:matrix.org"""
|
||||
)
|
||||
|
||||
|
||||
def parse_version_from_module() -> Tuple[
|
||||
version.Version, redbaron.RedBaron, redbaron.Node
|
||||
]:
|
||||
|
|
|
|||
1
setup.py
1
setup.py
|
|
@ -108,6 +108,7 @@ CONDITIONAL_REQUIREMENTS["mypy"] = [
|
|||
"types-jsonschema>=3.2.0",
|
||||
"types-opentracing>=2.4.2",
|
||||
"types-Pillow>=8.3.4",
|
||||
"types-psycopg2>=2.9.9",
|
||||
"types-pyOpenSSL>=20.0.7",
|
||||
"types-PyYAML>=5.4.10",
|
||||
"types-requests>=2.26.0",
|
||||
|
|
|
|||
|
|
@ -68,7 +68,7 @@ try:
|
|||
except ImportError:
|
||||
pass
|
||||
|
||||
__version__ = "1.55.0rc1"
|
||||
__version__ = "1.55.2"
|
||||
|
||||
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
|
||||
# We import here so that we don't have to install a bunch of deps when
|
||||
|
|
|
|||
|
|
@ -261,7 +261,10 @@ class SynapseHomeServer(HomeServer):
|
|||
resources[SERVER_KEY_V2_PREFIX] = KeyApiV2Resource(self)
|
||||
|
||||
if name == "metrics" and self.config.metrics.enable_metrics:
|
||||
resources[METRICS_PREFIX] = MetricsResource(RegistryProxy)
|
||||
metrics_resource: Resource = MetricsResource(RegistryProxy)
|
||||
if compress:
|
||||
metrics_resource = gz_wrap(metrics_resource)
|
||||
resources[METRICS_PREFIX] = metrics_resource
|
||||
|
||||
if name == "replication":
|
||||
resources[REPLICATION_PREFIX] = ReplicationRestResource(self)
|
||||
|
|
@ -348,6 +351,23 @@ def setup(config_options: List[str]) -> SynapseHomeServer:
|
|||
if config.server.gc_seconds:
|
||||
synapse.metrics.MIN_TIME_BETWEEN_GCS = config.server.gc_seconds
|
||||
|
||||
if (
|
||||
config.registration.enable_registration
|
||||
and not config.registration.enable_registration_without_verification
|
||||
):
|
||||
if (
|
||||
not config.captcha.enable_registration_captcha
|
||||
and not config.registration.registrations_require_3pid
|
||||
and not config.registration.registration_requires_token
|
||||
):
|
||||
|
||||
raise ConfigError(
|
||||
"You have enabled open registration without any verification. This is a known vector for "
|
||||
"spam and abuse. If you would like to allow public registration, please consider adding email, "
|
||||
"captcha, or token-based verification. Otherwise this check can be removed by setting the "
|
||||
"`enable_registration_without_verification` config option to `true`."
|
||||
)
|
||||
|
||||
hs = SynapseHomeServer(
|
||||
config.server.server_name,
|
||||
config=config,
|
||||
|
|
|
|||
|
|
@ -37,6 +37,12 @@ DEFAULT_CONFIG = """\
|
|||
# 'txn_limit' gives the maximum number of transactions to run per connection
|
||||
# before reconnecting. Defaults to 0, which means no limit.
|
||||
#
|
||||
# 'allow_unsafe_locale' is an option specific to Postgres. Under the default behavior, Synapse will refuse to
|
||||
# start if the postgres db is set to a non-C locale. You can override this behavior (which is *not* recommended)
|
||||
# by setting 'allow_unsafe_locale' to true. Note that doing so may corrupt your database. You can find more information
|
||||
# here: https://matrix-org.github.io/synapse/latest/postgres.html#fixing-incorrect-collate-or-ctype and here:
|
||||
# https://wiki.postgresql.org/wiki/Locale_data_changes
|
||||
#
|
||||
# 'args' gives options which are passed through to the database engine,
|
||||
# except for options starting 'cp_', which are used to configure the Twisted
|
||||
# connection pool. For a reference to valid arguments, see:
|
||||
|
|
|
|||
|
|
@ -33,6 +33,10 @@ class RegistrationConfig(Config):
|
|||
str(config["disable_registration"])
|
||||
)
|
||||
|
||||
self.enable_registration_without_verification = strtobool(
|
||||
str(config.get("enable_registration_without_verification", False))
|
||||
)
|
||||
|
||||
self.registrations_require_3pid = config.get("registrations_require_3pid", [])
|
||||
self.allowed_local_3pids = config.get("allowed_local_3pids", [])
|
||||
self.enable_3pid_lookup = config.get("enable_3pid_lookup", True)
|
||||
|
|
@ -207,10 +211,18 @@ class RegistrationConfig(Config):
|
|||
# Registration can be rate-limited using the parameters in the "Ratelimiting"
|
||||
# section of this file.
|
||||
|
||||
# Enable registration for new users.
|
||||
# Enable registration for new users. Defaults to 'false'. It is highly recommended that if you enable registration,
|
||||
# you use either captcha, email, or token-based verification to verify that new users are not bots. In order to enable registration
|
||||
# without any verification, you must also set `enable_registration_without_verification`, found below.
|
||||
#
|
||||
#enable_registration: false
|
||||
|
||||
# Enable registration without email or captcha verification. Note: this option is *not* recommended,
|
||||
# as registration without verification is a known vector for spam and abuse. Defaults to false. Has no effect
|
||||
# unless `enable_registration` is also enabled.
|
||||
#
|
||||
#enable_registration_without_verification: true
|
||||
|
||||
# Time that a user's session remains valid for, after they log in.
|
||||
#
|
||||
# Note that this is not currently compatible with guest logins.
|
||||
|
|
|
|||
|
|
@ -676,6 +676,10 @@ class ServerConfig(Config):
|
|||
):
|
||||
raise ConfigError("'custom_template_directory' must be a string")
|
||||
|
||||
self.use_account_validity_in_account_status: bool = (
|
||||
config.get("use_account_validity_in_account_status") or False
|
||||
)
|
||||
|
||||
def has_tls_listener(self) -> bool:
|
||||
return any(listener.tls for listener in self.listeners)
|
||||
|
||||
|
|
|
|||
|
|
@ -25,8 +25,8 @@ logger = logging.getLogger(__name__)
|
|||
LEGACY_SPAM_CHECKER_WARNING = """
|
||||
This server is using a spam checker module that is implementing the deprecated spam
|
||||
checker interface. Please check with the module's maintainer to see if a new version
|
||||
supporting Synapse's generic modules system is available.
|
||||
For more information, please see https://matrix-org.github.io/synapse/latest/modules.html
|
||||
supporting Synapse's generic modules system is available. For more information, please
|
||||
see https://matrix-org.github.io/synapse/latest/modules/index.html
|
||||
---------------------------------------------------------------------------------------"""
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -21,7 +21,6 @@ from typing import (
|
|||
Awaitable,
|
||||
Callable,
|
||||
Collection,
|
||||
Dict,
|
||||
List,
|
||||
Optional,
|
||||
Tuple,
|
||||
|
|
@ -31,7 +30,7 @@ from typing import (
|
|||
from synapse.rest.media.v1._base import FileInfo
|
||||
from synapse.rest.media.v1.media_storage import ReadableFileWrapper
|
||||
from synapse.spam_checker_api import RegistrationBehaviour
|
||||
from synapse.types import RoomAlias
|
||||
from synapse.types import RoomAlias, UserProfile
|
||||
from synapse.util.async_helpers import maybe_awaitable
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
|
@ -50,7 +49,7 @@ USER_MAY_SEND_3PID_INVITE_CALLBACK = Callable[[str, str, str, str], Awaitable[bo
|
|||
USER_MAY_CREATE_ROOM_CALLBACK = Callable[[str], Awaitable[bool]]
|
||||
USER_MAY_CREATE_ROOM_ALIAS_CALLBACK = Callable[[str, RoomAlias], Awaitable[bool]]
|
||||
USER_MAY_PUBLISH_ROOM_CALLBACK = Callable[[str, str], Awaitable[bool]]
|
||||
CHECK_USERNAME_FOR_SPAM_CALLBACK = Callable[[Dict[str, str]], Awaitable[bool]]
|
||||
CHECK_USERNAME_FOR_SPAM_CALLBACK = Callable[[UserProfile], Awaitable[bool]]
|
||||
LEGACY_CHECK_REGISTRATION_FOR_SPAM_CALLBACK = Callable[
|
||||
[
|
||||
Optional[dict],
|
||||
|
|
@ -383,7 +382,7 @@ class SpamChecker:
|
|||
|
||||
return True
|
||||
|
||||
async def check_username_for_spam(self, user_profile: Dict[str, str]) -> bool:
|
||||
async def check_username_for_spam(self, user_profile: UserProfile) -> bool:
|
||||
"""Checks if a user ID or display name are considered "spammy" by this server.
|
||||
|
||||
If the server considers a username spammy, then it will not be included in
|
||||
|
|
|
|||
|
|
@ -38,8 +38,8 @@ from synapse.util.frozenutils import unfreeze
|
|||
from . import EventBase
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.handlers.relations import BundledAggregations
|
||||
from synapse.server import HomeServer
|
||||
from synapse.storage.databases.main.relations import BundledAggregations
|
||||
|
||||
|
||||
# Split strings on "." but not "\." This uses a negative lookbehind assertion for '\'
|
||||
|
|
|
|||
|
|
@ -22,7 +22,6 @@ from typing import (
|
|||
Callable,
|
||||
Collection,
|
||||
Dict,
|
||||
Iterable,
|
||||
List,
|
||||
Optional,
|
||||
Tuple,
|
||||
|
|
@ -577,10 +576,10 @@ class FederationServer(FederationBase):
|
|||
async def _on_context_state_request_compute(
|
||||
self, room_id: str, event_id: Optional[str]
|
||||
) -> Dict[str, list]:
|
||||
pdus: Collection[EventBase]
|
||||
if event_id:
|
||||
pdus: Iterable[EventBase] = await self.handler.get_state_for_pdu(
|
||||
room_id, event_id
|
||||
)
|
||||
event_ids = await self.handler.get_state_ids_for_pdu(room_id, event_id)
|
||||
pdus = await self.store.get_events_as_list(event_ids)
|
||||
else:
|
||||
pdus = (await self.state.get_current_state(room_id)).values()
|
||||
|
||||
|
|
@ -1093,7 +1092,7 @@ class FederationServer(FederationBase):
|
|||
# has started processing).
|
||||
while True:
|
||||
async with lock:
|
||||
logger.info("handling received PDU: %s", event)
|
||||
logger.info("handling received PDU in room %s: %s", room_id, event)
|
||||
try:
|
||||
with nested_logging_context(event.event_id):
|
||||
await self._federation_event_handler.on_receive_pdu(
|
||||
|
|
|
|||
|
|
@ -26,6 +26,10 @@ class AccountHandler:
|
|||
self._main_store = hs.get_datastores().main
|
||||
self._is_mine = hs.is_mine
|
||||
self._federation_client = hs.get_federation_client()
|
||||
self._use_account_validity_in_account_status = (
|
||||
hs.config.server.use_account_validity_in_account_status
|
||||
)
|
||||
self._account_validity_handler = hs.get_account_validity_handler()
|
||||
|
||||
async def get_account_statuses(
|
||||
self,
|
||||
|
|
@ -106,6 +110,13 @@ class AccountHandler:
|
|||
"deactivated": userinfo.is_deactivated,
|
||||
}
|
||||
|
||||
if self._use_account_validity_in_account_status:
|
||||
status[
|
||||
"org.matrix.expired"
|
||||
] = await self._account_validity_handler.is_user_expired(
|
||||
user_id.to_string()
|
||||
)
|
||||
|
||||
return status
|
||||
|
||||
async def _get_remote_account_statuses(
|
||||
|
|
|
|||
|
|
@ -950,54 +950,35 @@ class FederationHandler:
|
|||
|
||||
return event
|
||||
|
||||
async def get_state_for_pdu(self, room_id: str, event_id: str) -> List[EventBase]:
|
||||
"""Returns the state at the event. i.e. not including said event."""
|
||||
|
||||
event = await self.store.get_event(event_id, check_room_id=room_id)
|
||||
|
||||
state_groups = await self.state_store.get_state_groups(room_id, [event_id])
|
||||
|
||||
if state_groups:
|
||||
_, state = list(state_groups.items()).pop()
|
||||
results = {(e.type, e.state_key): e for e in state}
|
||||
|
||||
if event.is_state():
|
||||
# Get previous state
|
||||
if "replaces_state" in event.unsigned:
|
||||
prev_id = event.unsigned["replaces_state"]
|
||||
if prev_id != event.event_id:
|
||||
prev_event = await self.store.get_event(prev_id)
|
||||
results[(event.type, event.state_key)] = prev_event
|
||||
else:
|
||||
del results[(event.type, event.state_key)]
|
||||
|
||||
res = list(results.values())
|
||||
return res
|
||||
else:
|
||||
return []
|
||||
|
||||
async def get_state_ids_for_pdu(self, room_id: str, event_id: str) -> List[str]:
|
||||
"""Returns the state at the event. i.e. not including said event."""
|
||||
event = await self.store.get_event(event_id, check_room_id=room_id)
|
||||
if event.internal_metadata.outlier:
|
||||
raise NotFoundError("State not known at event %s" % (event_id,))
|
||||
|
||||
state_groups = await self.state_store.get_state_groups_ids(room_id, [event_id])
|
||||
|
||||
if state_groups:
|
||||
_, state = list(state_groups.items()).pop()
|
||||
results = state
|
||||
# get_state_groups_ids should return exactly one result
|
||||
assert len(state_groups) == 1
|
||||
|
||||
if event.is_state():
|
||||
# Get previous state
|
||||
if "replaces_state" in event.unsigned:
|
||||
prev_id = event.unsigned["replaces_state"]
|
||||
if prev_id != event.event_id:
|
||||
results[(event.type, event.state_key)] = prev_id
|
||||
else:
|
||||
results.pop((event.type, event.state_key), None)
|
||||
state_map = next(iter(state_groups.values()))
|
||||
|
||||
return list(results.values())
|
||||
else:
|
||||
return []
|
||||
state_key = event.get_state_key()
|
||||
if state_key is not None:
|
||||
# the event was not rejected (get_event raises a NotFoundError for rejected
|
||||
# events) so the state at the event should include the event itself.
|
||||
assert (
|
||||
state_map.get((event.type, state_key)) == event.event_id
|
||||
), "State at event did not include event itself"
|
||||
|
||||
# ... but we need the state *before* that event
|
||||
if "replaces_state" in event.unsigned:
|
||||
prev_id = event.unsigned["replaces_state"]
|
||||
state_map[(event.type, state_key)] = prev_id
|
||||
else:
|
||||
del state_map[(event.type, state_key)]
|
||||
|
||||
return list(state_map.values())
|
||||
|
||||
async def on_backfill_request(
|
||||
self, origin: str, room_id: str, pdu_list: List[str], limit: int
|
||||
|
|
|
|||
|
|
@ -493,6 +493,7 @@ class EventCreationHandler:
|
|||
allow_no_prev_events: bool = False,
|
||||
prev_event_ids: Optional[List[str]] = None,
|
||||
auth_event_ids: Optional[List[str]] = None,
|
||||
state_event_ids: Optional[List[str]] = None,
|
||||
require_consent: bool = True,
|
||||
outlier: bool = False,
|
||||
historical: bool = False,
|
||||
|
|
@ -527,6 +528,15 @@ class EventCreationHandler:
|
|||
|
||||
If non-None, prev_event_ids must also be provided.
|
||||
|
||||
state_event_ids:
|
||||
The full state at a given event. This is used particularly by the MSC2716
|
||||
/batch_send endpoint. One use case is with insertion events which float at
|
||||
the beginning of a historical batch and don't have any `prev_events` to
|
||||
derive from; we add all of these state events as the explicit state so the
|
||||
rest of the historical batch can inherit the same state and state_group.
|
||||
This should normally be left as None, which will cause the auth_event_ids
|
||||
to be calculated based on the room state at the prev_events.
|
||||
|
||||
require_consent: Whether to check if the requester has
|
||||
consented to the privacy policy.
|
||||
|
||||
|
|
@ -612,6 +622,7 @@ class EventCreationHandler:
|
|||
allow_no_prev_events=allow_no_prev_events,
|
||||
prev_event_ids=prev_event_ids,
|
||||
auth_event_ids=auth_event_ids,
|
||||
state_event_ids=state_event_ids,
|
||||
depth=depth,
|
||||
)
|
||||
|
||||
|
|
@ -771,7 +782,7 @@ class EventCreationHandler:
|
|||
event_dict: dict,
|
||||
allow_no_prev_events: bool = False,
|
||||
prev_event_ids: Optional[List[str]] = None,
|
||||
auth_event_ids: Optional[List[str]] = None,
|
||||
state_event_ids: Optional[List[str]] = None,
|
||||
ratelimit: bool = True,
|
||||
txn_id: Optional[str] = None,
|
||||
ignore_shadow_ban: bool = False,
|
||||
|
|
@ -795,12 +806,14 @@ class EventCreationHandler:
|
|||
The event IDs to use as the prev events.
|
||||
Should normally be left as None to automatically request them
|
||||
from the database.
|
||||
auth_event_ids:
|
||||
The event ids to use as the auth_events for the new event.
|
||||
Should normally be left as None, which will cause them to be calculated
|
||||
based on the room state at the prev_events.
|
||||
|
||||
If non-None, prev_event_ids must also be provided.
|
||||
state_event_ids:
|
||||
The full state at a given event. This is used particularly by the MSC2716
|
||||
/batch_send endpoint. One use case is with insertion events which float at
|
||||
the beginning of a historical batch and don't have any `prev_events` to
|
||||
derive from; we add all of these state events as the explicit state so the
|
||||
rest of the historical batch can inherit the same state and state_group.
|
||||
This should normally be left as None, which will cause the auth_event_ids
|
||||
to be calculated based on the room state at the prev_events.
|
||||
ratelimit: Whether to rate limit this send.
|
||||
txn_id: The transaction ID.
|
||||
ignore_shadow_ban: True if shadow-banned users should be allowed to
|
||||
|
|
@ -856,8 +869,9 @@ class EventCreationHandler:
|
|||
requester,
|
||||
event_dict,
|
||||
txn_id=txn_id,
|
||||
allow_no_prev_events=allow_no_prev_events,
|
||||
prev_event_ids=prev_event_ids,
|
||||
auth_event_ids=auth_event_ids,
|
||||
state_event_ids=state_event_ids,
|
||||
outlier=outlier,
|
||||
historical=historical,
|
||||
depth=depth,
|
||||
|
|
@ -893,6 +907,7 @@ class EventCreationHandler:
|
|||
allow_no_prev_events: bool = False,
|
||||
prev_event_ids: Optional[List[str]] = None,
|
||||
auth_event_ids: Optional[List[str]] = None,
|
||||
state_event_ids: Optional[List[str]] = None,
|
||||
depth: Optional[int] = None,
|
||||
) -> Tuple[EventBase, EventContext]:
|
||||
"""Create a new event for a local client
|
||||
|
|
@ -915,6 +930,15 @@ class EventCreationHandler:
|
|||
Should normally be left as None, which will cause them to be calculated
|
||||
based on the room state at the prev_events.
|
||||
|
||||
state_event_ids:
|
||||
The full state at a given event. This is used particularly by the MSC2716
|
||||
/batch_send endpoint. One use case is with insertion events which float at
|
||||
the beginning of a historical batch and don't have any `prev_events` to
|
||||
derive from; we add all of these state events as the explicit state so the
|
||||
rest of the historical batch can inherit the same state and state_group.
|
||||
This should normally be left as None, which will cause the auth_event_ids
|
||||
to be calculated based on the room state at the prev_events.
|
||||
|
||||
depth: Override the depth used to order the event in the DAG.
|
||||
Should normally be set to None, which will cause the depth to be calculated
|
||||
based on the prev_events.
|
||||
|
|
@ -922,31 +946,26 @@ class EventCreationHandler:
|
|||
Returns:
|
||||
Tuple of created event, context
|
||||
"""
|
||||
# Strip down the auth_event_ids to only what we need to auth the event.
|
||||
# Strip down the state_event_ids to only what we need to auth the event.
|
||||
# For example, we don't need extra m.room.member that don't match event.sender
|
||||
full_state_ids_at_event = None
|
||||
if auth_event_ids is not None:
|
||||
# If auth events are provided, prev events must be also.
|
||||
if state_event_ids is not None:
|
||||
# Do a quick check to make sure that prev_event_ids is present to
|
||||
# make the type-checking around `builder.build` happy.
|
||||
# prev_event_ids could be an empty array though.
|
||||
assert prev_event_ids is not None
|
||||
|
||||
# Copy the full auth state before it stripped down
|
||||
full_state_ids_at_event = auth_event_ids.copy()
|
||||
|
||||
temp_event = await builder.build(
|
||||
prev_event_ids=prev_event_ids,
|
||||
auth_event_ids=auth_event_ids,
|
||||
auth_event_ids=state_event_ids,
|
||||
depth=depth,
|
||||
)
|
||||
auth_events = await self.store.get_events_as_list(auth_event_ids)
|
||||
state_events = await self.store.get_events_as_list(state_event_ids)
|
||||
# Create a StateMap[str]
|
||||
auth_event_state_map = {
|
||||
(e.type, e.state_key): e.event_id for e in auth_events
|
||||
}
|
||||
# Actually strip down and use the necessary auth events
|
||||
state_map = {(e.type, e.state_key): e.event_id for e in state_events}
|
||||
# Actually strip down and only use the necessary auth events
|
||||
auth_event_ids = self._event_auth_handler.compute_auth_events(
|
||||
event=temp_event,
|
||||
current_state_ids=auth_event_state_map,
|
||||
current_state_ids=state_map,
|
||||
for_verification=False,
|
||||
)
|
||||
|
||||
|
|
@ -989,12 +1008,16 @@ class EventCreationHandler:
|
|||
context = EventContext.for_outlier()
|
||||
elif (
|
||||
event.type == EventTypes.MSC2716_INSERTION
|
||||
and full_state_ids_at_event
|
||||
and state_event_ids
|
||||
and builder.internal_metadata.is_historical()
|
||||
):
|
||||
# Add explicit state to the insertion event so it has state to derive
|
||||
# from even though it's floating with no `prev_events`. The rest of
|
||||
# the batch can derive from this state and state_group.
|
||||
#
|
||||
# TODO(faster_joins): figure out how this works, and make sure that the
|
||||
# old state is complete.
|
||||
old_state = await self.store.get_events_as_list(full_state_ids_at_event)
|
||||
old_state = await self.store.get_events_as_list(state_event_ids)
|
||||
context = await self.state.compute_event_context(event, old_state=old_state)
|
||||
else:
|
||||
context = await self.state.compute_event_context(event)
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@
|
|||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
import logging
|
||||
from typing import TYPE_CHECKING, Any, Collection, Dict, List, Optional, Set
|
||||
from typing import TYPE_CHECKING, Collection, Dict, List, Optional, Set
|
||||
|
||||
import attr
|
||||
|
||||
|
|
@ -134,6 +134,7 @@ class PaginationHandler:
|
|||
self.clock = hs.get_clock()
|
||||
self._server_name = hs.hostname
|
||||
self._room_shutdown_handler = hs.get_room_shutdown_handler()
|
||||
self._relations_handler = hs.get_relations_handler()
|
||||
|
||||
self.pagination_lock = ReadWriteLock()
|
||||
# IDs of rooms in which there currently an active purge *or delete* operation.
|
||||
|
|
@ -422,7 +423,7 @@ class PaginationHandler:
|
|||
pagin_config: PaginationConfig,
|
||||
as_client_event: bool = True,
|
||||
event_filter: Optional[Filter] = None,
|
||||
) -> Dict[str, Any]:
|
||||
) -> JsonDict:
|
||||
"""Get messages in a room.
|
||||
|
||||
Args:
|
||||
|
|
@ -431,6 +432,7 @@ class PaginationHandler:
|
|||
pagin_config: The pagination config rules to apply, if any.
|
||||
as_client_event: True to get events in client-server format.
|
||||
event_filter: Filter to apply to results or None
|
||||
|
||||
Returns:
|
||||
Pagination API results
|
||||
"""
|
||||
|
|
@ -538,7 +540,9 @@ class PaginationHandler:
|
|||
state_dict = await self.store.get_events(list(state_ids.values()))
|
||||
state = state_dict.values()
|
||||
|
||||
aggregations = await self.store.get_bundled_aggregations(events, user_id)
|
||||
aggregations = await self._relations_handler.get_bundled_aggregations(
|
||||
events, user_id
|
||||
)
|
||||
|
||||
time_now = self.clock.time_msec()
|
||||
|
||||
|
|
|
|||
|
|
@ -336,12 +336,18 @@ class ProfileHandler:
|
|||
"""Check that the size and content type of the avatar at the given MXC URI are
|
||||
within the configured limits.
|
||||
|
||||
If the given `mxc` is empty, no checks are performed. (Users are always able to
|
||||
unset their avatar.)
|
||||
|
||||
Args:
|
||||
mxc: The MXC URI at which the avatar can be found.
|
||||
|
||||
Returns:
|
||||
A boolean indicating whether the file can be allowed to be set as an avatar.
|
||||
"""
|
||||
if mxc == "":
|
||||
return True
|
||||
|
||||
if not self.max_avatar_size and not self.allowed_avatar_mimetypes:
|
||||
return True
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,271 @@
|
|||
# Copyright 2021 The Matrix.org Foundation C.I.C.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
import logging
|
||||
from typing import TYPE_CHECKING, Dict, Iterable, Optional, cast
|
||||
|
||||
import attr
|
||||
from frozendict import frozendict
|
||||
|
||||
from synapse.api.constants import RelationTypes
|
||||
from synapse.api.errors import SynapseError
|
||||
from synapse.events import EventBase
|
||||
from synapse.types import JsonDict, Requester, StreamToken
|
||||
from synapse.visibility import filter_events_for_client
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from synapse.server import HomeServer
|
||||
from synapse.storage.databases.main import DataStore
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@attr.s(slots=True, frozen=True, auto_attribs=True)
|
||||
class _ThreadAggregation:
|
||||
# The latest event in the thread.
|
||||
latest_event: EventBase
|
||||
# The latest edit to the latest event in the thread.
|
||||
latest_edit: Optional[EventBase]
|
||||
# The total number of events in the thread.
|
||||
count: int
|
||||
# True if the current user has sent an event to the thread.
|
||||
current_user_participated: bool
|
||||
|
||||
|
||||
@attr.s(slots=True, auto_attribs=True)
|
||||
class BundledAggregations:
|
||||
"""
|
||||
The bundled aggregations for an event.
|
||||
|
||||
Some values require additional processing during serialization.
|
||||
"""
|
||||
|
||||
annotations: Optional[JsonDict] = None
|
||||
references: Optional[JsonDict] = None
|
||||
replace: Optional[EventBase] = None
|
||||
thread: Optional[_ThreadAggregation] = None
|
||||
|
||||
def __bool__(self) -> bool:
|
||||
return bool(self.annotations or self.references or self.replace or self.thread)
|
||||
|
||||
|
||||
class RelationsHandler:
|
||||
def __init__(self, hs: "HomeServer"):
|
||||
self._main_store = hs.get_datastores().main
|
||||
self._storage = hs.get_storage()
|
||||
self._auth = hs.get_auth()
|
||||
self._clock = hs.get_clock()
|
||||
self._event_handler = hs.get_event_handler()
|
||||
self._event_serializer = hs.get_event_client_serializer()
|
||||
|
||||
async def get_relations(
|
||||
self,
|
||||
requester: Requester,
|
||||
event_id: str,
|
||||
room_id: str,
|
||||
relation_type: Optional[str] = None,
|
||||
event_type: Optional[str] = None,
|
||||
aggregation_key: Optional[str] = None,
|
||||
limit: int = 5,
|
||||
direction: str = "b",
|
||||
from_token: Optional[StreamToken] = None,
|
||||
to_token: Optional[StreamToken] = None,
|
||||
) -> JsonDict:
|
||||
"""Get related events of a event, ordered by topological ordering.
|
||||
|
||||
TODO Accept a PaginationConfig instead of individual pagination parameters.
|
||||
|
||||
Args:
|
||||
requester: The user requesting the relations.
|
||||
event_id: Fetch events that relate to this event ID.
|
||||
room_id: The room the event belongs to.
|
||||
relation_type: Only fetch events with this relation type, if given.
|
||||
event_type: Only fetch events with this event type, if given.
|
||||
aggregation_key: Only fetch events with this aggregation key, if given.
|
||||
limit: Only fetch the most recent `limit` events.
|
||||
direction: Whether to fetch the most recent first (`"b"`) or the
|
||||
oldest first (`"f"`).
|
||||
from_token: Fetch rows from the given token, or from the start if None.
|
||||
to_token: Fetch rows up to the given token, or up to the end if None.
|
||||
|
||||
Returns:
|
||||
The pagination chunk.
|
||||
"""
|
||||
|
||||
user_id = requester.user.to_string()
|
||||
|
||||
# TODO Properly handle a user leaving a room.
|
||||
(_, member_event_id) = await self._auth.check_user_in_room_or_world_readable(
|
||||
room_id, user_id, allow_departed_users=True
|
||||
)
|
||||
|
||||
# This gets the original event and checks that a) the event exists and
|
||||
# b) the user is allowed to view it.
|
||||
event = await self._event_handler.get_event(requester.user, room_id, event_id)
|
||||
if event is None:
|
||||
raise SynapseError(404, "Unknown parent event.")
|
||||
|
||||
pagination_chunk = await self._main_store.get_relations_for_event(
|
||||
event_id=event_id,
|
||||
event=event,
|
||||
room_id=room_id,
|
||||
relation_type=relation_type,
|
||||
event_type=event_type,
|
||||
aggregation_key=aggregation_key,
|
||||
limit=limit,
|
||||
direction=direction,
|
||||
from_token=from_token,
|
||||
to_token=to_token,
|
||||
)
|
||||
|
||||
events = await self._main_store.get_events_as_list(
|
||||
[c["event_id"] for c in pagination_chunk.chunk]
|
||||
)
|
||||
|
||||
events = await filter_events_for_client(
|
||||
self._storage, user_id, events, is_peeking=(member_event_id is None)
|
||||
)
|
||||
|
||||
now = self._clock.time_msec()
|
||||
# Do not bundle aggregations when retrieving the original event because
|
||||
# we want the content before relations are applied to it.
|
||||
original_event = self._event_serializer.serialize_event(
|
||||
event, now, bundle_aggregations=None
|
||||
)
|
||||
# The relations returned for the requested event do include their
|
||||
# bundled aggregations.
|
||||
aggregations = await self.get_bundled_aggregations(
|
||||
events, requester.user.to_string()
|
||||
)
|
||||
serialized_events = self._event_serializer.serialize_events(
|
||||
events, now, bundle_aggregations=aggregations
|
||||
)
|
||||
|
||||
return_value = await pagination_chunk.to_dict(self._main_store)
|
||||
return_value["chunk"] = serialized_events
|
||||
return_value["original_event"] = original_event
|
||||
|
||||
return return_value
|
||||
|
||||
async def _get_bundled_aggregation_for_event(
|
||||
self, event: EventBase, user_id: str
|
||||
) -> Optional[BundledAggregations]:
|
||||
"""Generate bundled aggregations for an event.
|
||||
|
||||
Note that this does not use a cache, but depends on cached methods.
|
||||
|
||||
Args:
|
||||
event: The event to calculate bundled aggregations for.
|
||||
user_id: The user requesting the bundled aggregations.
|
||||
|
||||
Returns:
|
||||
The bundled aggregations for an event, if bundled aggregations are
|
||||
enabled and the event can have bundled aggregations.
|
||||
"""
|
||||
|
||||
# Do not bundle aggregations for an event which represents an edit or an
|
||||
# annotation. It does not make sense for them to have related events.
|
||||
relates_to = event.content.get("m.relates_to")
|
||||
if isinstance(relates_to, (dict, frozendict)):
|
||||
relation_type = relates_to.get("rel_type")
|
||||
if relation_type in (RelationTypes.ANNOTATION, RelationTypes.REPLACE):
|
||||
return None
|
||||
|
||||
event_id = event.event_id
|
||||
room_id = event.room_id
|
||||
|
||||
# The bundled aggregations to include, a mapping of relation type to a
|
||||
# type-specific value. Some types include the direct return type here
|
||||
# while others need more processing during serialization.
|
||||
aggregations = BundledAggregations()
|
||||
|
||||
annotations = await self._main_store.get_aggregation_groups_for_event(
|
||||
event_id, room_id
|
||||
)
|
||||
if annotations.chunk:
|
||||
aggregations.annotations = await annotations.to_dict(
|
||||
cast("DataStore", self)
|
||||
)
|
||||
|
||||
references = await self._main_store.get_relations_for_event(
|
||||
event_id, event, room_id, RelationTypes.REFERENCE, direction="f"
|
||||
)
|
||||
if references.chunk:
|
||||
aggregations.references = await references.to_dict(cast("DataStore", self))
|
||||
|
||||
# Store the bundled aggregations in the event metadata for later use.
|
||||
return aggregations
|
||||
|
||||
async def get_bundled_aggregations(
|
||||
self, events: Iterable[EventBase], user_id: str
|
||||
) -> Dict[str, BundledAggregations]:
|
||||
"""Generate bundled aggregations for events.
|
||||
|
||||
Args:
|
||||
events: The iterable of events to calculate bundled aggregations for.
|
||||
user_id: The user requesting the bundled aggregations.
|
||||
|
||||
Returns:
|
||||
A map of event ID to the bundled aggregation for the event. Not all
|
||||
events may have bundled aggregations in the results.
|
||||
"""
|
||||
# De-duplicate events by ID to handle the same event requested multiple times.
|
||||
#
|
||||
# State events do not get bundled aggregations.
|
||||
events_by_id = {
|
||||
event.event_id: event for event in events if not event.is_state()
|
||||
}
|
||||
|
||||
# event ID -> bundled aggregation in non-serialized form.
|
||||
results: Dict[str, BundledAggregations] = {}
|
||||
|
||||
# Fetch other relations per event.
|
||||
for event in events_by_id.values():
|
||||
event_result = await self._get_bundled_aggregation_for_event(event, user_id)
|
||||
if event_result:
|
||||
results[event.event_id] = event_result
|
||||
|
||||
# Fetch any edits (but not for redacted events).
|
||||
edits = await self._main_store.get_applicable_edits(
|
||||
[
|
||||
event_id
|
||||
for event_id, event in events_by_id.items()
|
||||
if not event.internal_metadata.is_redacted()
|
||||
]
|
||||
)
|
||||
for event_id, edit in edits.items():
|
||||
results.setdefault(event_id, BundledAggregations()).replace = edit
|
||||
|
||||
# Fetch thread summaries.
|
||||
summaries = await self._main_store.get_thread_summaries(events_by_id.keys())
|
||||
# Only fetch participated for a limited selection based on what had
|
||||
# summaries.
|
||||
participated = await self._main_store.get_threads_participated(
|
||||
[event_id for event_id, summary in summaries.items() if summary], user_id
|
||||
)
|
||||
for event_id, summary in summaries.items():
|
||||
if summary:
|
||||
thread_count, latest_thread_event, edit = summary
|
||||
results.setdefault(
|
||||
event_id, BundledAggregations()
|
||||
).thread = _ThreadAggregation(
|
||||
latest_event=latest_thread_event,
|
||||
latest_edit=edit,
|
||||
count=thread_count,
|
||||
# If there's a thread summary it must also exist in the
|
||||
# participated dictionary.
|
||||
current_user_participated=participated[event_id],
|
||||
)
|
||||
|
||||
return results
|
||||
|
|
@ -60,8 +60,8 @@ from synapse.events import EventBase
|
|||
from synapse.events.utils import copy_power_levels_contents
|
||||
from synapse.federation.federation_client import InvalidResponseError
|
||||
from synapse.handlers.federation import get_domains_from_state
|
||||
from synapse.handlers.relations import BundledAggregations
|
||||
from synapse.rest.admin._base import assert_user_is_admin
|
||||
from synapse.storage.databases.main.relations import BundledAggregations
|
||||
from synapse.storage.state import StateFilter
|
||||
from synapse.streams import EventSource
|
||||
from synapse.types import (
|
||||
|
|
@ -1118,6 +1118,7 @@ class RoomContextHandler:
|
|||
self.store = hs.get_datastores().main
|
||||
self.storage = hs.get_storage()
|
||||
self.state_store = self.storage.state
|
||||
self._relations_handler = hs.get_relations_handler()
|
||||
|
||||
async def get_event_context(
|
||||
self,
|
||||
|
|
@ -1190,7 +1191,7 @@ class RoomContextHandler:
|
|||
event = filtered[0]
|
||||
|
||||
# Fetch the aggregations.
|
||||
aggregations = await self.store.get_bundled_aggregations(
|
||||
aggregations = await self._relations_handler.get_bundled_aggregations(
|
||||
itertools.chain(events_before, (event,), events_after),
|
||||
user.to_string(),
|
||||
)
|
||||
|
|
|
|||
|
|
@ -121,12 +121,11 @@ class RoomBatchHandler:
|
|||
|
||||
return create_requester(user_id, app_service=app_service)
|
||||
|
||||
async def get_most_recent_auth_event_ids_from_event_id_list(
|
||||
async def get_most_recent_full_state_ids_from_event_id_list(
|
||||
self, event_ids: List[str]
|
||||
) -> List[str]:
|
||||
"""Find the most recent auth event ids (derived from state events) that
|
||||
allowed that message to be sent. We will use this as a base
|
||||
to auth our historical messages against.
|
||||
"""Find the most recent event_id and grab the full state at that event.
|
||||
We will use this as a base to auth our historical messages against.
|
||||
|
||||
Args:
|
||||
event_ids: List of event ID's to look at
|
||||
|
|
@ -136,38 +135,37 @@ class RoomBatchHandler:
|
|||
"""
|
||||
|
||||
(
|
||||
most_recent_prev_event_id,
|
||||
most_recent_event_id,
|
||||
_,
|
||||
) = await self.store.get_max_depth_of(event_ids)
|
||||
# mapping from (type, state_key) -> state_event_id
|
||||
prev_state_map = await self.state_store.get_state_ids_for_event(
|
||||
most_recent_prev_event_id
|
||||
most_recent_event_id
|
||||
)
|
||||
# List of state event ID's
|
||||
prev_state_ids = list(prev_state_map.values())
|
||||
auth_event_ids = prev_state_ids
|
||||
full_state_ids = list(prev_state_map.values())
|
||||
|
||||
return auth_event_ids
|
||||
return full_state_ids
|
||||
|
||||
async def persist_state_events_at_start(
|
||||
self,
|
||||
state_events_at_start: List[JsonDict],
|
||||
room_id: str,
|
||||
initial_auth_event_ids: List[str],
|
||||
initial_state_event_ids: List[str],
|
||||
app_service_requester: Requester,
|
||||
) -> List[str]:
|
||||
"""Takes all `state_events_at_start` event dictionaries and creates/persists
|
||||
them as floating state events which don't resolve into the current room state.
|
||||
They are floating because they reference a fake prev_event which doesn't connect
|
||||
to the normal DAG at all.
|
||||
them in a floating state event chain which don't resolve into the current room
|
||||
state. They are floating because they reference no prev_events and are marked
|
||||
as outliers which disconnects them from the normal DAG.
|
||||
|
||||
Args:
|
||||
state_events_at_start:
|
||||
room_id: Room where you want the events persisted in.
|
||||
initial_auth_event_ids: These will be the auth_events for the first
|
||||
state event created. Each event created afterwards will be
|
||||
added to the list of auth events for the next state event
|
||||
created.
|
||||
initial_state_event_ids:
|
||||
The base set of state for the historical batch which the floating
|
||||
state chain will derive from. This should probably be the state
|
||||
from the `prev_event` defined by `/batch_send?prev_event_id=$abc`.
|
||||
app_service_requester: The requester of an application service.
|
||||
|
||||
Returns:
|
||||
|
|
@ -176,7 +174,7 @@ class RoomBatchHandler:
|
|||
assert app_service_requester.app_service
|
||||
|
||||
state_event_ids_at_start = []
|
||||
auth_event_ids = initial_auth_event_ids.copy()
|
||||
state_event_ids = initial_state_event_ids.copy()
|
||||
|
||||
# Make the state events float off on their own by specifying no
|
||||
# prev_events for the first one in the chain so we don't have a bunch of
|
||||
|
|
@ -189,9 +187,7 @@ class RoomBatchHandler:
|
|||
)
|
||||
|
||||
logger.debug(
|
||||
"RoomBatchSendEventRestServlet inserting state_event=%s, auth_event_ids=%s",
|
||||
state_event,
|
||||
auth_event_ids,
|
||||
"RoomBatchSendEventRestServlet inserting state_event=%s", state_event
|
||||
)
|
||||
|
||||
event_dict = {
|
||||
|
|
@ -217,16 +213,26 @@ class RoomBatchHandler:
|
|||
room_id=room_id,
|
||||
action=membership,
|
||||
content=event_dict["content"],
|
||||
# Mark as an outlier to disconnect it from the normal DAG
|
||||
# and not show up between batches of history.
|
||||
outlier=True,
|
||||
historical=True,
|
||||
# Only the first event in the chain should be floating.
|
||||
# Only the first event in the state chain should be floating.
|
||||
# The rest should hang off each other in a chain.
|
||||
allow_no_prev_events=index == 0,
|
||||
prev_event_ids=prev_event_ids_for_state_chain,
|
||||
# Since each state event is marked as an outlier, the
|
||||
# `EventContext.for_outlier()` won't have any `state_ids`
|
||||
# set and therefore can't derive any state even though the
|
||||
# prev_events are set. Also since the first event in the
|
||||
# state chain is floating with no `prev_events`, it can't
|
||||
# derive state from anywhere automatically. So we need to
|
||||
# set some state explicitly.
|
||||
#
|
||||
# Make sure to use a copy of this list because we modify it
|
||||
# later in the loop here. Otherwise it will be the same
|
||||
# reference and also update in the event when we append later.
|
||||
auth_event_ids=auth_event_ids.copy(),
|
||||
state_event_ids=state_event_ids.copy(),
|
||||
)
|
||||
else:
|
||||
# TODO: Add some complement tests that adds state that is not member joins
|
||||
|
|
@ -240,21 +246,31 @@ class RoomBatchHandler:
|
|||
state_event["sender"], app_service_requester.app_service
|
||||
),
|
||||
event_dict,
|
||||
# Mark as an outlier to disconnect it from the normal DAG
|
||||
# and not show up between batches of history.
|
||||
outlier=True,
|
||||
historical=True,
|
||||
# Only the first event in the chain should be floating.
|
||||
# Only the first event in the state chain should be floating.
|
||||
# The rest should hang off each other in a chain.
|
||||
allow_no_prev_events=index == 0,
|
||||
prev_event_ids=prev_event_ids_for_state_chain,
|
||||
# Since each state event is marked as an outlier, the
|
||||
# `EventContext.for_outlier()` won't have any `state_ids`
|
||||
# set and therefore can't derive any state even though the
|
||||
# prev_events are set. Also since the first event in the
|
||||
# state chain is floating with no `prev_events`, it can't
|
||||
# derive state from anywhere automatically. So we need to
|
||||
# set some state explicitly.
|
||||
#
|
||||
# Make sure to use a copy of this list because we modify it
|
||||
# later in the loop here. Otherwise it will be the same
|
||||
# reference and also update in the event when we append later.
|
||||
auth_event_ids=auth_event_ids.copy(),
|
||||
state_event_ids=state_event_ids.copy(),
|
||||
)
|
||||
event_id = event.event_id
|
||||
|
||||
state_event_ids_at_start.append(event_id)
|
||||
auth_event_ids.append(event_id)
|
||||
state_event_ids.append(event_id)
|
||||
# Connect all the state in a floating chain
|
||||
prev_event_ids_for_state_chain = [event_id]
|
||||
|
||||
|
|
@ -265,7 +281,7 @@ class RoomBatchHandler:
|
|||
events_to_create: List[JsonDict],
|
||||
room_id: str,
|
||||
inherited_depth: int,
|
||||
auth_event_ids: List[str],
|
||||
initial_state_event_ids: List[str],
|
||||
app_service_requester: Requester,
|
||||
) -> List[str]:
|
||||
"""Create and persists all events provided sequentially. Handles the
|
||||
|
|
@ -281,8 +297,10 @@ class RoomBatchHandler:
|
|||
room_id: Room where you want the events persisted in.
|
||||
inherited_depth: The depth to create the events at (you will
|
||||
probably by calling inherit_depth_from_prev_ids(...)).
|
||||
auth_event_ids: Define which events allow you to create the given
|
||||
event in the room.
|
||||
initial_state_event_ids:
|
||||
This is used to set explicit state for the insertion event at
|
||||
the start of the historical batch since it's floating with no
|
||||
prev_events to derive state from automatically.
|
||||
app_service_requester: The requester of an application service.
|
||||
|
||||
Returns:
|
||||
|
|
@ -290,6 +308,11 @@ class RoomBatchHandler:
|
|||
"""
|
||||
assert app_service_requester.app_service
|
||||
|
||||
# We expect the first event in a historical batch to be an insertion event
|
||||
assert events_to_create[0]["type"] == EventTypes.MSC2716_INSERTION
|
||||
# We expect the last event in a historical batch to be an batch event
|
||||
assert events_to_create[-1]["type"] == EventTypes.MSC2716_BATCH
|
||||
|
||||
# Make the historical event chain float off on its own by specifying no
|
||||
# prev_events for the first event in the chain which causes the HS to
|
||||
# ask for the state at the start of the batch later.
|
||||
|
|
@ -321,11 +344,16 @@ class RoomBatchHandler:
|
|||
ev["sender"], app_service_requester.app_service
|
||||
),
|
||||
event_dict,
|
||||
# Only the first event in the chain should be floating.
|
||||
# The rest should hang off each other in a chain.
|
||||
# Only the first event (which is the insertion event) in the
|
||||
# chain should be floating. The rest should hang off each other
|
||||
# in a chain.
|
||||
allow_no_prev_events=index == 0,
|
||||
prev_event_ids=event_dict.get("prev_events"),
|
||||
auth_event_ids=auth_event_ids,
|
||||
# Since the first event (which is the insertion event) in the
|
||||
# chain is floating with no `prev_events`, it can't derive state
|
||||
# from anywhere automatically. So we need to set some state
|
||||
# explicitly.
|
||||
state_event_ids=initial_state_event_ids if index == 0 else None,
|
||||
historical=True,
|
||||
depth=inherited_depth,
|
||||
)
|
||||
|
|
@ -343,10 +371,9 @@ class RoomBatchHandler:
|
|||
)
|
||||
|
||||
logger.debug(
|
||||
"RoomBatchSendEventRestServlet inserting event=%s, prev_event_ids=%s, auth_event_ids=%s",
|
||||
"RoomBatchSendEventRestServlet inserting event=%s, prev_event_ids=%s",
|
||||
event,
|
||||
prev_event_ids,
|
||||
auth_event_ids,
|
||||
)
|
||||
|
||||
events_to_persist.append((event, context))
|
||||
|
|
@ -376,12 +403,12 @@ class RoomBatchHandler:
|
|||
room_id: str,
|
||||
batch_id_to_connect_to: str,
|
||||
inherited_depth: int,
|
||||
auth_event_ids: List[str],
|
||||
initial_state_event_ids: List[str],
|
||||
app_service_requester: Requester,
|
||||
) -> Tuple[List[str], str]:
|
||||
"""
|
||||
Handles creating and persisting all of the historical events as well
|
||||
as insertion and batch meta events to make the batch navigable in the DAG.
|
||||
Handles creating and persisting all of the historical events as well as
|
||||
insertion and batch meta events to make the batch navigable in the DAG.
|
||||
|
||||
Args:
|
||||
events_to_create: List of historical events to create in JSON
|
||||
|
|
@ -391,8 +418,13 @@ class RoomBatchHandler:
|
|||
want this batch to connect to.
|
||||
inherited_depth: The depth to create the events at (you will
|
||||
probably by calling inherit_depth_from_prev_ids(...)).
|
||||
auth_event_ids: Define which events allow you to create the given
|
||||
event in the room.
|
||||
initial_state_event_ids:
|
||||
This is used to set explicit state for the insertion event at
|
||||
the start of the historical batch since it's floating with no
|
||||
prev_events to derive state from automatically. This should
|
||||
probably be the state from the `prev_event` defined by
|
||||
`/batch_send?prev_event_id=$abc` plus the outcome of
|
||||
`persist_state_events_at_start`
|
||||
app_service_requester: The requester of an application service.
|
||||
|
||||
Returns:
|
||||
|
|
@ -438,7 +470,7 @@ class RoomBatchHandler:
|
|||
events_to_create=events_to_create,
|
||||
room_id=room_id,
|
||||
inherited_depth=inherited_depth,
|
||||
auth_event_ids=auth_event_ids,
|
||||
initial_state_event_ids=initial_state_event_ids,
|
||||
app_service_requester=app_service_requester,
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -271,7 +271,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
|
|||
membership: str,
|
||||
allow_no_prev_events: bool = False,
|
||||
prev_event_ids: Optional[List[str]] = None,
|
||||
auth_event_ids: Optional[List[str]] = None,
|
||||
state_event_ids: Optional[List[str]] = None,
|
||||
txn_id: Optional[str] = None,
|
||||
ratelimit: bool = True,
|
||||
content: Optional[dict] = None,
|
||||
|
|
@ -294,10 +294,14 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
|
|||
events should have a prev_event and we should only use this in special
|
||||
cases like MSC2716.
|
||||
prev_event_ids: The event IDs to use as the prev events
|
||||
auth_event_ids:
|
||||
The event ids to use as the auth_events for the new event.
|
||||
Should normally be left as None, which will cause them to be calculated
|
||||
based on the room state at the prev_events.
|
||||
state_event_ids:
|
||||
The full state at a given event. This is used particularly by the MSC2716
|
||||
/batch_send endpoint. One use case is the historical `state_events_at_start`;
|
||||
since each is marked as an `outlier`, the `EventContext.for_outlier()` won't
|
||||
have any `state_ids` set and therefore can't derive any state even though the
|
||||
prev_events are set so we need to set them ourself via this argument.
|
||||
This should normally be left as None, which will cause the auth_event_ids
|
||||
to be calculated based on the room state at the prev_events.
|
||||
|
||||
txn_id:
|
||||
ratelimit:
|
||||
|
|
@ -352,7 +356,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
|
|||
txn_id=txn_id,
|
||||
allow_no_prev_events=allow_no_prev_events,
|
||||
prev_event_ids=prev_event_ids,
|
||||
auth_event_ids=auth_event_ids,
|
||||
state_event_ids=state_event_ids,
|
||||
require_consent=require_consent,
|
||||
outlier=outlier,
|
||||
historical=historical,
|
||||
|
|
@ -455,7 +459,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
|
|||
historical: bool = False,
|
||||
allow_no_prev_events: bool = False,
|
||||
prev_event_ids: Optional[List[str]] = None,
|
||||
auth_event_ids: Optional[List[str]] = None,
|
||||
state_event_ids: Optional[List[str]] = None,
|
||||
) -> Tuple[str, int]:
|
||||
"""Update a user's membership in a room.
|
||||
|
||||
|
|
@ -483,10 +487,14 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
|
|||
events should have a prev_event and we should only use this in special
|
||||
cases like MSC2716.
|
||||
prev_event_ids: The event IDs to use as the prev events
|
||||
auth_event_ids:
|
||||
The event ids to use as the auth_events for the new event.
|
||||
Should normally be left as None, which will cause them to be calculated
|
||||
based on the room state at the prev_events.
|
||||
state_event_ids:
|
||||
The full state at a given event. This is used particularly by the MSC2716
|
||||
/batch_send endpoint. One use case is the historical `state_events_at_start`;
|
||||
since each is marked as an `outlier`, the `EventContext.for_outlier()` won't
|
||||
have any `state_ids` set and therefore can't derive any state even though the
|
||||
prev_events are set so we need to set them ourself via this argument.
|
||||
This should normally be left as None, which will cause the auth_event_ids
|
||||
to be calculated based on the room state at the prev_events.
|
||||
|
||||
Returns:
|
||||
A tuple of the new event ID and stream ID.
|
||||
|
|
@ -539,7 +547,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
|
|||
historical=historical,
|
||||
allow_no_prev_events=allow_no_prev_events,
|
||||
prev_event_ids=prev_event_ids,
|
||||
auth_event_ids=auth_event_ids,
|
||||
state_event_ids=state_event_ids,
|
||||
)
|
||||
|
||||
return result
|
||||
|
|
@ -561,7 +569,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
|
|||
historical: bool = False,
|
||||
allow_no_prev_events: bool = False,
|
||||
prev_event_ids: Optional[List[str]] = None,
|
||||
auth_event_ids: Optional[List[str]] = None,
|
||||
state_event_ids: Optional[List[str]] = None,
|
||||
) -> Tuple[str, int]:
|
||||
"""Helper for update_membership.
|
||||
|
||||
|
|
@ -591,10 +599,14 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
|
|||
events should have a prev_event and we should only use this in special
|
||||
cases like MSC2716.
|
||||
prev_event_ids: The event IDs to use as the prev events
|
||||
auth_event_ids:
|
||||
The event ids to use as the auth_events for the new event.
|
||||
Should normally be left as None, which will cause them to be calculated
|
||||
based on the room state at the prev_events.
|
||||
state_event_ids:
|
||||
The full state at a given event. This is used particularly by the MSC2716
|
||||
/batch_send endpoint. One use case is the historical `state_events_at_start`;
|
||||
since each is marked as an `outlier`, the `EventContext.for_outlier()` won't
|
||||
have any `state_ids` set and therefore can't derive any state even though the
|
||||
prev_events are set so we need to set them ourself via this argument.
|
||||
This should normally be left as None, which will cause the auth_event_ids
|
||||
to be calculated based on the room state at the prev_events.
|
||||
|
||||
Returns:
|
||||
A tuple of the new event ID and stream ID.
|
||||
|
|
@ -721,7 +733,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
|
|||
ratelimit=ratelimit,
|
||||
allow_no_prev_events=allow_no_prev_events,
|
||||
prev_event_ids=prev_event_ids,
|
||||
auth_event_ids=auth_event_ids,
|
||||
state_event_ids=state_event_ids,
|
||||
content=content,
|
||||
require_consent=require_consent,
|
||||
outlier=outlier,
|
||||
|
|
@ -945,7 +957,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
|
|||
txn_id=txn_id,
|
||||
ratelimit=ratelimit,
|
||||
prev_event_ids=latest_event_ids,
|
||||
auth_event_ids=auth_event_ids,
|
||||
state_event_ids=state_event_ids,
|
||||
content=content,
|
||||
require_consent=require_consent,
|
||||
outlier=outlier,
|
||||
|
|
|
|||
|
|
@ -54,6 +54,7 @@ class SearchHandler:
|
|||
self.clock = hs.get_clock()
|
||||
self.hs = hs
|
||||
self._event_serializer = hs.get_event_client_serializer()
|
||||
self._relations_handler = hs.get_relations_handler()
|
||||
self.storage = hs.get_storage()
|
||||
self.state_store = self.storage.state
|
||||
self.auth = hs.get_auth()
|
||||
|
|
@ -354,7 +355,7 @@ class SearchHandler:
|
|||
|
||||
aggregations = None
|
||||
if self._msc3666_enabled:
|
||||
aggregations = await self.store.get_bundled_aggregations(
|
||||
aggregations = await self._relations_handler.get_bundled_aggregations(
|
||||
# Generate an iterable of EventBase for all the events that will be
|
||||
# returned, including contextual events.
|
||||
itertools.chain(
|
||||
|
|
|
|||
|
|
@ -28,16 +28,16 @@ from typing import (
|
|||
import attr
|
||||
from prometheus_client import Counter
|
||||
|
||||
from synapse.api.constants import AccountDataTypes, EventTypes, Membership, ReceiptTypes
|
||||
from synapse.api.constants import EventTypes, Membership, ReceiptTypes
|
||||
from synapse.api.filtering import FilterCollection
|
||||
from synapse.api.presence import UserPresenceState
|
||||
from synapse.api.room_versions import KNOWN_ROOM_VERSIONS
|
||||
from synapse.events import EventBase
|
||||
from synapse.handlers.relations import BundledAggregations
|
||||
from synapse.logging.context import current_context
|
||||
from synapse.logging.opentracing import SynapseTags, log_kv, set_tag, start_active_span
|
||||
from synapse.push.clientformat import format_push_rules_for_user
|
||||
from synapse.storage.databases.main.event_push_actions import NotifCounts
|
||||
from synapse.storage.databases.main.relations import BundledAggregations
|
||||
from synapse.storage.roommember import MemberSummary
|
||||
from synapse.storage.state import StateFilter
|
||||
from synapse.types import (
|
||||
|
|
@ -269,6 +269,7 @@ class SyncHandler:
|
|||
self.store = hs.get_datastores().main
|
||||
self.notifier = hs.get_notifier()
|
||||
self.presence_handler = hs.get_presence_handler()
|
||||
self._relations_handler = hs.get_relations_handler()
|
||||
self.event_sources = hs.get_event_sources()
|
||||
self.clock = hs.get_clock()
|
||||
self.state = hs.get_state_handler()
|
||||
|
|
@ -638,8 +639,10 @@ class SyncHandler:
|
|||
# as clients will have all the necessary information.
|
||||
bundled_aggregations = None
|
||||
if limited or newly_joined_room:
|
||||
bundled_aggregations = await self.store.get_bundled_aggregations(
|
||||
recents, sync_config.user.to_string()
|
||||
bundled_aggregations = (
|
||||
await self._relations_handler.get_bundled_aggregations(
|
||||
recents, sync_config.user.to_string()
|
||||
)
|
||||
)
|
||||
|
||||
return TimelineBatch(
|
||||
|
|
@ -1601,7 +1604,7 @@ class SyncHandler:
|
|||
return set(), set(), set(), set()
|
||||
|
||||
# 3. Work out which rooms need reporting in the sync response.
|
||||
ignored_users = await self._get_ignored_users(user_id)
|
||||
ignored_users = await self.store.ignored_users(user_id)
|
||||
if since_token:
|
||||
room_changes = await self._get_rooms_changed(
|
||||
sync_result_builder, ignored_users
|
||||
|
|
@ -1627,7 +1630,6 @@ class SyncHandler:
|
|||
logger.debug("Generating room entry for %s", room_entry.room_id)
|
||||
await self._generate_room_entry(
|
||||
sync_result_builder,
|
||||
ignored_users,
|
||||
room_entry,
|
||||
ephemeral=ephemeral_by_room.get(room_entry.room_id, []),
|
||||
tags=tags_by_room.get(room_entry.room_id),
|
||||
|
|
@ -1657,29 +1659,6 @@ class SyncHandler:
|
|||
newly_left_users,
|
||||
)
|
||||
|
||||
async def _get_ignored_users(self, user_id: str) -> FrozenSet[str]:
|
||||
"""Retrieve the users ignored by the given user from their global account_data.
|
||||
|
||||
Returns an empty set if
|
||||
- there is no global account_data entry for ignored_users
|
||||
- there is such an entry, but it's not a JSON object.
|
||||
"""
|
||||
# TODO: Can we `SELECT ignored_user_id FROM ignored_users WHERE ignorer_user_id=?;` instead?
|
||||
ignored_account_data = (
|
||||
await self.store.get_global_account_data_by_type_for_user(
|
||||
user_id=user_id, data_type=AccountDataTypes.IGNORED_USER_LIST
|
||||
)
|
||||
)
|
||||
|
||||
# If there is ignored users account data and it matches the proper type,
|
||||
# then use it.
|
||||
ignored_users: FrozenSet[str] = frozenset()
|
||||
if ignored_account_data:
|
||||
ignored_users_data = ignored_account_data.get("ignored_users", {})
|
||||
if isinstance(ignored_users_data, dict):
|
||||
ignored_users = frozenset(ignored_users_data.keys())
|
||||
return ignored_users
|
||||
|
||||
async def _have_rooms_changed(
|
||||
self, sync_result_builder: "SyncResultBuilder"
|
||||
) -> bool:
|
||||
|
|
@ -2022,7 +2001,6 @@ class SyncHandler:
|
|||
async def _generate_room_entry(
|
||||
self,
|
||||
sync_result_builder: "SyncResultBuilder",
|
||||
ignored_users: FrozenSet[str],
|
||||
room_builder: "RoomSyncResultBuilder",
|
||||
ephemeral: List[JsonDict],
|
||||
tags: Optional[Dict[str, Dict[str, Any]]],
|
||||
|
|
@ -2051,7 +2029,6 @@ class SyncHandler:
|
|||
|
||||
Args:
|
||||
sync_result_builder
|
||||
ignored_users: Set of users ignored by user.
|
||||
room_builder
|
||||
ephemeral: List of new ephemeral events for room
|
||||
tags: List of *all* tags for room, or None if there has been
|
||||
|
|
|
|||
|
|
@ -19,8 +19,8 @@ import synapse.metrics
|
|||
from synapse.api.constants import EventTypes, HistoryVisibility, JoinRules, Membership
|
||||
from synapse.handlers.state_deltas import MatchChange, StateDeltasHandler
|
||||
from synapse.metrics.background_process_metrics import run_as_background_process
|
||||
from synapse.storage.databases.main.user_directory import SearchResult
|
||||
from synapse.storage.roommember import ProfileInfo
|
||||
from synapse.types import JsonDict
|
||||
from synapse.util.metrics import Measure
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
|
@ -78,7 +78,7 @@ class UserDirectoryHandler(StateDeltasHandler):
|
|||
|
||||
async def search_users(
|
||||
self, user_id: str, search_term: str, limit: int
|
||||
) -> JsonDict:
|
||||
) -> SearchResult:
|
||||
"""Searches for users in directory
|
||||
|
||||
Returns:
|
||||
|
|
|
|||
|
|
@ -111,6 +111,7 @@ from synapse.types import (
|
|||
StateMap,
|
||||
UserID,
|
||||
UserInfo,
|
||||
UserProfile,
|
||||
create_requester,
|
||||
)
|
||||
from synapse.util import Clock
|
||||
|
|
@ -150,6 +151,7 @@ __all__ = [
|
|||
"EventBase",
|
||||
"StateMap",
|
||||
"ProfileInfo",
|
||||
"UserProfile",
|
||||
]
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
|
@ -609,15 +611,18 @@ class ModuleApi:
|
|||
localpart: str,
|
||||
displayname: Optional[str] = None,
|
||||
emails: Optional[List[str]] = None,
|
||||
admin: bool = False,
|
||||
) -> "defer.Deferred[str]":
|
||||
"""Registers a new user with given localpart and optional displayname, emails.
|
||||
|
||||
Added in Synapse v1.2.0.
|
||||
Changed in Synapse v1.56.0: add 'admin' argument to register the user as admin.
|
||||
|
||||
Args:
|
||||
localpart: The localpart of the new user.
|
||||
displayname: The displayname of the new user.
|
||||
emails: Emails to bind to the new user.
|
||||
admin: True if the user should be registered as a server admin.
|
||||
|
||||
Raises:
|
||||
SynapseError if there is an error performing the registration. Check the
|
||||
|
|
@ -631,6 +636,7 @@ class ModuleApi:
|
|||
localpart=localpart,
|
||||
default_display_name=displayname,
|
||||
bind_emails=emails or [],
|
||||
admin=admin,
|
||||
)
|
||||
)
|
||||
|
||||
|
|
@ -665,7 +671,8 @@ class ModuleApi:
|
|||
def record_user_external_id(
|
||||
self, auth_provider_id: str, remote_user_id: str, registered_user_id: str
|
||||
) -> defer.Deferred:
|
||||
"""Record a mapping from an external user id to a mxid
|
||||
"""Record a mapping between an external user id from a single sign-on provider
|
||||
and a mxid.
|
||||
|
||||
Added in Synapse v1.9.0.
|
||||
|
||||
|
|
@ -1280,6 +1287,30 @@ class ModuleApi:
|
|||
"""
|
||||
await self._registration_handler.check_username(username)
|
||||
|
||||
async def store_remote_3pid_association(
|
||||
self, user_id: str, medium: str, address: str, id_server: str
|
||||
) -> None:
|
||||
"""Stores an existing association between a user ID and a third-party identifier.
|
||||
|
||||
The association must already exist on the remote identity server.
|
||||
|
||||
Added in Synapse v1.56.0.
|
||||
|
||||
Args:
|
||||
user_id: The user ID that's been associated with the 3PID.
|
||||
medium: The medium of the 3PID (current supported values are "msisdn" and
|
||||
"email").
|
||||
address: The address of the 3PID.
|
||||
id_server: The identity server the 3PID association has been registered on.
|
||||
This should only be the domain (or IP address, optionally with the port
|
||||
number) for the identity server. This will be used to reach out to the
|
||||
identity server using HTTPS (unless specified otherwise by Synapse's
|
||||
configuration) when attempting to unbind the third-party identifier.
|
||||
|
||||
|
||||
"""
|
||||
await self._store.add_user_bound_threepid(user_id, medium, address, id_server)
|
||||
|
||||
|
||||
class PublicRoomListManager:
|
||||
"""Contains methods for adding to, removing from and querying whether a room
|
||||
|
|
|
|||
|
|
@ -24,6 +24,7 @@ from synapse.event_auth import get_user_power_level
|
|||
from synapse.events import EventBase
|
||||
from synapse.events.snapshot import EventContext
|
||||
from synapse.state import POWER_KEY
|
||||
from synapse.storage.databases.main.roommember import EventIdMembership
|
||||
from synapse.util.async_helpers import Linearizer
|
||||
from synapse.util.caches import CacheMetric, register_cache
|
||||
from synapse.util.caches.descriptors import lru_cache
|
||||
|
|
@ -213,7 +214,7 @@ class BulkPushRuleEvaluator:
|
|||
if not event.is_state():
|
||||
ignorers = await self.store.ignored_by(event.sender)
|
||||
else:
|
||||
ignorers = set()
|
||||
ignorers = frozenset()
|
||||
|
||||
for uid, rules in rules_by_user.items():
|
||||
if event.sender == uid:
|
||||
|
|
@ -292,7 +293,7 @@ def _condition_checker(
|
|||
return True
|
||||
|
||||
|
||||
MemberMap = Dict[str, Tuple[str, str]]
|
||||
MemberMap = Dict[str, Optional[EventIdMembership]]
|
||||
Rule = Dict[str, dict]
|
||||
RulesByUser = Dict[str, List[Rule]]
|
||||
StateGroup = Union[object, int]
|
||||
|
|
@ -306,7 +307,7 @@ class RulesForRoomData:
|
|||
*only* include data, and not references to e.g. the data stores.
|
||||
"""
|
||||
|
||||
# event_id -> (user_id, state)
|
||||
# event_id -> EventIdMembership
|
||||
member_map: MemberMap = attr.Factory(dict)
|
||||
# user_id -> rules
|
||||
rules_by_user: RulesByUser = attr.Factory(dict)
|
||||
|
|
@ -447,11 +448,10 @@ class RulesForRoom:
|
|||
|
||||
res = self.data.member_map.get(event_id, None)
|
||||
if res:
|
||||
user_id, state = res
|
||||
if state == Membership.JOIN:
|
||||
rules = self.data.rules_by_user.get(user_id, None)
|
||||
if res.membership == Membership.JOIN:
|
||||
rules = self.data.rules_by_user.get(res.user_id, None)
|
||||
if rules:
|
||||
ret_rules_by_user[user_id] = rules
|
||||
ret_rules_by_user[res.user_id] = rules
|
||||
continue
|
||||
|
||||
# If a user has left a room we remove their push rule. If they
|
||||
|
|
@ -502,24 +502,26 @@ class RulesForRoom:
|
|||
"""
|
||||
sequence = self.data.sequence
|
||||
|
||||
rows = await self.store.get_membership_from_event_ids(member_event_ids.values())
|
||||
members = await self.store.get_membership_from_event_ids(
|
||||
member_event_ids.values()
|
||||
)
|
||||
|
||||
members = {row["event_id"]: (row["user_id"], row["membership"]) for row in rows}
|
||||
|
||||
# If the event is a join event then it will be in current state evnts
|
||||
# If the event is a join event then it will be in current state events
|
||||
# map but not in the DB, so we have to explicitly insert it.
|
||||
if event.type == EventTypes.Member:
|
||||
for event_id in member_event_ids.values():
|
||||
if event_id == event.event_id:
|
||||
members[event_id] = (event.state_key, event.membership)
|
||||
members[event_id] = EventIdMembership(
|
||||
user_id=event.state_key, membership=event.membership
|
||||
)
|
||||
|
||||
if logger.isEnabledFor(logging.DEBUG):
|
||||
logger.debug("Found members %r: %r", self.room_id, members.values())
|
||||
|
||||
joined_user_ids = {
|
||||
user_id
|
||||
for user_id, membership in members.values()
|
||||
if membership == Membership.JOIN
|
||||
entry.user_id
|
||||
for entry in members.values()
|
||||
if entry and entry.membership == Membership.JOIN
|
||||
}
|
||||
|
||||
logger.debug("Joined: %r", joined_user_ids)
|
||||
|
|
|
|||
|
|
@ -18,6 +18,7 @@ from typing import TYPE_CHECKING, Dict, Iterable, List, Optional, TypeVar
|
|||
|
||||
import bleach
|
||||
import jinja2
|
||||
from markupsafe import Markup
|
||||
|
||||
from synapse.api.constants import EventTypes, Membership, RoomTypes
|
||||
from synapse.api.errors import StoreError
|
||||
|
|
@ -867,7 +868,7 @@ class Mailer:
|
|||
)
|
||||
|
||||
|
||||
def safe_markup(raw_html: str) -> jinja2.Markup:
|
||||
def safe_markup(raw_html: str) -> Markup:
|
||||
"""
|
||||
Sanitise a raw HTML string to a set of allowed tags and attributes, and linkify any bare URLs.
|
||||
|
||||
|
|
@ -877,7 +878,7 @@ def safe_markup(raw_html: str) -> jinja2.Markup:
|
|||
Returns:
|
||||
A Markup object ready to safely use in a Jinja template.
|
||||
"""
|
||||
return jinja2.Markup(
|
||||
return Markup(
|
||||
bleach.linkify(
|
||||
bleach.clean(
|
||||
raw_html,
|
||||
|
|
@ -891,7 +892,7 @@ def safe_markup(raw_html: str) -> jinja2.Markup:
|
|||
)
|
||||
|
||||
|
||||
def safe_text(raw_text: str) -> jinja2.Markup:
|
||||
def safe_text(raw_text: str) -> Markup:
|
||||
"""
|
||||
Sanitise text (escape any HTML tags), and then linkify any bare URLs.
|
||||
|
||||
|
|
@ -901,7 +902,7 @@ def safe_text(raw_text: str) -> jinja2.Markup:
|
|||
Returns:
|
||||
A Markup object ready to safely use in a Jinja template.
|
||||
"""
|
||||
return jinja2.Markup(
|
||||
return Markup(
|
||||
bleach.linkify(bleach.clean(raw_text, tags=[], attributes=[], strip=False))
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -74,7 +74,10 @@ REQUIREMENTS = [
|
|||
# Note: 21.1.0 broke `/sync`, see #9936
|
||||
"attrs>=19.2.0,!=21.1.0",
|
||||
"netaddr>=0.7.18",
|
||||
"Jinja2>=2.9",
|
||||
# Jinja 2.x is incompatible with MarkupSafe>=2.1. To ensure that admins do not
|
||||
# end up with a broken installation, with recent MarkupSafe but old Jinja, we
|
||||
# add a lower bound to the Jinja2 dependency.
|
||||
"Jinja2>=3.0",
|
||||
"bleach>=1.4.3",
|
||||
# We use `ParamSpec`, which was added in `typing-extensions` 3.10.0.0.
|
||||
"typing-extensions>=3.10.0",
|
||||
|
|
|
|||
|
|
@ -32,6 +32,7 @@ from synapse.rest.client import (
|
|||
knock,
|
||||
login as v1_login,
|
||||
logout,
|
||||
mutual_rooms,
|
||||
notifications,
|
||||
openid,
|
||||
password_policy,
|
||||
|
|
@ -49,7 +50,6 @@ from synapse.rest.client import (
|
|||
room_keys,
|
||||
room_upgrade_rest_servlet,
|
||||
sendtodevice,
|
||||
shared_rooms,
|
||||
sync,
|
||||
tags,
|
||||
thirdparty,
|
||||
|
|
@ -132,4 +132,4 @@ class ClientRestResource(JsonResource):
|
|||
admin.register_servlets_for_client_rest_resource(hs, client_resource)
|
||||
|
||||
# unstable
|
||||
shared_rooms.register_servlets(hs, client_resource)
|
||||
mutual_rooms.register_servlets(hs, client_resource)
|
||||
|
|
|
|||
|
|
@ -28,13 +28,13 @@ if TYPE_CHECKING:
|
|||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class UserSharedRoomsServlet(RestServlet):
|
||||
class UserMutualRoomsServlet(RestServlet):
|
||||
"""
|
||||
GET /uk.half-shot.msc2666/user/shared_rooms/{user_id} HTTP/1.1
|
||||
GET /uk.half-shot.msc2666/user/mutual_rooms/{user_id} HTTP/1.1
|
||||
"""
|
||||
|
||||
PATTERNS = client_patterns(
|
||||
"/uk.half-shot.msc2666/user/shared_rooms/(?P<user_id>[^/]*)",
|
||||
"/uk.half-shot.msc2666/user/mutual_rooms/(?P<user_id>[^/]*)",
|
||||
releases=(), # This is an unstable feature
|
||||
)
|
||||
|
||||
|
|
@ -42,17 +42,19 @@ class UserSharedRoomsServlet(RestServlet):
|
|||
super().__init__()
|
||||
self.auth = hs.get_auth()
|
||||
self.store = hs.get_datastores().main
|
||||
self.user_directory_active = hs.config.server.update_user_directory
|
||||
self.user_directory_search_enabled = (
|
||||
hs.config.userdirectory.user_directory_search_enabled
|
||||
)
|
||||
|
||||
async def on_GET(
|
||||
self, request: SynapseRequest, user_id: str
|
||||
) -> Tuple[int, JsonDict]:
|
||||
|
||||
if not self.user_directory_active:
|
||||
if not self.user_directory_search_enabled:
|
||||
raise SynapseError(
|
||||
code=400,
|
||||
msg="The user directory is disabled on this server. Cannot determine shared rooms.",
|
||||
errcode=Codes.FORBIDDEN,
|
||||
msg="User directory searching is disabled. Cannot determine shared rooms.",
|
||||
errcode=Codes.UNKNOWN,
|
||||
)
|
||||
|
||||
UserID.from_string(user_id)
|
||||
|
|
@ -64,7 +66,8 @@ class UserSharedRoomsServlet(RestServlet):
|
|||
msg="You cannot request a list of shared rooms with yourself",
|
||||
errcode=Codes.FORBIDDEN,
|
||||
)
|
||||
rooms = await self.store.get_shared_rooms_for_users(
|
||||
|
||||
rooms = await self.store.get_mutual_rooms_for_users(
|
||||
requester.user.to_string(), user_id
|
||||
)
|
||||
|
||||
|
|
@ -72,4 +75,4 @@ class UserSharedRoomsServlet(RestServlet):
|
|||
|
||||
|
||||
def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
|
||||
UserSharedRoomsServlet(hs).register(http_server)
|
||||
UserMutualRoomsServlet(hs).register(http_server)
|
||||
|
|
@ -51,9 +51,7 @@ class RelationPaginationServlet(RestServlet):
|
|||
super().__init__()
|
||||
self.auth = hs.get_auth()
|
||||
self.store = hs.get_datastores().main
|
||||
self.clock = hs.get_clock()
|
||||
self._event_serializer = hs.get_event_client_serializer()
|
||||
self.event_handler = hs.get_event_handler()
|
||||
self._relations_handler = hs.get_relations_handler()
|
||||
|
||||
async def on_GET(
|
||||
self,
|
||||
|
|
@ -65,16 +63,6 @@ class RelationPaginationServlet(RestServlet):
|
|||
) -> Tuple[int, JsonDict]:
|
||||
requester = await self.auth.get_user_by_req(request, allow_guest=True)
|
||||
|
||||
await self.auth.check_user_in_room_or_world_readable(
|
||||
room_id, requester.user.to_string(), allow_departed_users=True
|
||||
)
|
||||
|
||||
# This gets the original event and checks that a) the event exists and
|
||||
# b) the user is allowed to view it.
|
||||
event = await self.event_handler.get_event(requester.user, room_id, parent_id)
|
||||
if event is None:
|
||||
raise SynapseError(404, "Unknown parent event.")
|
||||
|
||||
limit = parse_integer(request, "limit", default=5)
|
||||
direction = parse_string(
|
||||
request, "org.matrix.msc3715.dir", default="b", allowed_values=["f", "b"]
|
||||
|
|
@ -90,9 +78,9 @@ class RelationPaginationServlet(RestServlet):
|
|||
if to_token_str:
|
||||
to_token = await StreamToken.from_string(self.store, to_token_str)
|
||||
|
||||
pagination_chunk = await self.store.get_relations_for_event(
|
||||
result = await self._relations_handler.get_relations(
|
||||
requester=requester,
|
||||
event_id=parent_id,
|
||||
event=event,
|
||||
room_id=room_id,
|
||||
relation_type=relation_type,
|
||||
event_type=event_type,
|
||||
|
|
@ -102,30 +90,7 @@ class RelationPaginationServlet(RestServlet):
|
|||
to_token=to_token,
|
||||
)
|
||||
|
||||
events = await self.store.get_events_as_list(
|
||||
[c["event_id"] for c in pagination_chunk.chunk]
|
||||
)
|
||||
|
||||
now = self.clock.time_msec()
|
||||
# Do not bundle aggregations when retrieving the original event because
|
||||
# we want the content before relations are applied to it.
|
||||
original_event = self._event_serializer.serialize_event(
|
||||
event, now, bundle_aggregations=None
|
||||
)
|
||||
# The relations returned for the requested event do include their
|
||||
# bundled aggregations.
|
||||
aggregations = await self.store.get_bundled_aggregations(
|
||||
events, requester.user.to_string()
|
||||
)
|
||||
serialized_events = self._event_serializer.serialize_events(
|
||||
events, now, bundle_aggregations=aggregations
|
||||
)
|
||||
|
||||
return_value = await pagination_chunk.to_dict(self.store)
|
||||
return_value["chunk"] = serialized_events
|
||||
return_value["original_event"] = original_event
|
||||
|
||||
return 200, return_value
|
||||
return 200, result
|
||||
|
||||
|
||||
class RelationAggregationPaginationServlet(RestServlet):
|
||||
|
|
@ -245,9 +210,7 @@ class RelationAggregationGroupPaginationServlet(RestServlet):
|
|||
super().__init__()
|
||||
self.auth = hs.get_auth()
|
||||
self.store = hs.get_datastores().main
|
||||
self.clock = hs.get_clock()
|
||||
self._event_serializer = hs.get_event_client_serializer()
|
||||
self.event_handler = hs.get_event_handler()
|
||||
self._relations_handler = hs.get_relations_handler()
|
||||
|
||||
async def on_GET(
|
||||
self,
|
||||
|
|
@ -260,18 +223,6 @@ class RelationAggregationGroupPaginationServlet(RestServlet):
|
|||
) -> Tuple[int, JsonDict]:
|
||||
requester = await self.auth.get_user_by_req(request, allow_guest=True)
|
||||
|
||||
await self.auth.check_user_in_room_or_world_readable(
|
||||
room_id,
|
||||
requester.user.to_string(),
|
||||
allow_departed_users=True,
|
||||
)
|
||||
|
||||
# This checks that a) the event exists and b) the user is allowed to
|
||||
# view it.
|
||||
event = await self.event_handler.get_event(requester.user, room_id, parent_id)
|
||||
if event is None:
|
||||
raise SynapseError(404, "Unknown parent event.")
|
||||
|
||||
if relation_type != RelationTypes.ANNOTATION:
|
||||
raise SynapseError(400, "Relation type must be 'annotation'")
|
||||
|
||||
|
|
@ -286,9 +237,9 @@ class RelationAggregationGroupPaginationServlet(RestServlet):
|
|||
if to_token_str:
|
||||
to_token = await StreamToken.from_string(self.store, to_token_str)
|
||||
|
||||
result = await self.store.get_relations_for_event(
|
||||
result = await self._relations_handler.get_relations(
|
||||
requester=requester,
|
||||
event_id=parent_id,
|
||||
event=event,
|
||||
room_id=room_id,
|
||||
relation_type=relation_type,
|
||||
event_type=event_type,
|
||||
|
|
@ -298,17 +249,7 @@ class RelationAggregationGroupPaginationServlet(RestServlet):
|
|||
to_token=to_token,
|
||||
)
|
||||
|
||||
events = await self.store.get_events_as_list(
|
||||
[c["event_id"] for c in result.chunk]
|
||||
)
|
||||
|
||||
now = self.clock.time_msec()
|
||||
serialized_events = self._event_serializer.serialize_events(events, now)
|
||||
|
||||
return_value = await result.to_dict(self.store)
|
||||
return_value["chunk"] = serialized_events
|
||||
|
||||
return 200, return_value
|
||||
return 200, result
|
||||
|
||||
|
||||
def register_servlets(hs: "HomeServer", http_server: HttpServer) -> None:
|
||||
|
|
|
|||
|
|
@ -645,6 +645,7 @@ class RoomEventServlet(RestServlet):
|
|||
self._store = hs.get_datastores().main
|
||||
self.event_handler = hs.get_event_handler()
|
||||
self._event_serializer = hs.get_event_client_serializer()
|
||||
self._relations_handler = hs.get_relations_handler()
|
||||
self.auth = hs.get_auth()
|
||||
|
||||
async def on_GET(
|
||||
|
|
@ -663,7 +664,7 @@ class RoomEventServlet(RestServlet):
|
|||
|
||||
if event:
|
||||
# Ensure there are bundled aggregations available.
|
||||
aggregations = await self._store.get_bundled_aggregations(
|
||||
aggregations = await self._relations_handler.get_bundled_aggregations(
|
||||
[event], requester.user.to_string()
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -124,14 +124,14 @@ class RoomBatchSendEventRestServlet(RestServlet):
|
|||
)
|
||||
|
||||
# For the event we are inserting next to (`prev_event_ids_from_query`),
|
||||
# find the most recent auth events (derived from state events) that
|
||||
# allowed that message to be sent. We will use that as a base
|
||||
# to auth our historical messages against.
|
||||
auth_event_ids = await self.room_batch_handler.get_most_recent_auth_event_ids_from_event_id_list(
|
||||
# find the most recent state events that allowed that message to be
|
||||
# sent. We will use that as a base to auth our historical messages
|
||||
# against.
|
||||
state_event_ids = await self.room_batch_handler.get_most_recent_full_state_ids_from_event_id_list(
|
||||
prev_event_ids_from_query
|
||||
)
|
||||
|
||||
if not auth_event_ids:
|
||||
if not state_event_ids:
|
||||
raise SynapseError(
|
||||
HTTPStatus.BAD_REQUEST,
|
||||
"No auth events found for given prev_event query parameter. The prev_event=%s probably does not exist."
|
||||
|
|
@ -148,13 +148,13 @@ class RoomBatchSendEventRestServlet(RestServlet):
|
|||
await self.room_batch_handler.persist_state_events_at_start(
|
||||
state_events_at_start=body["state_events_at_start"],
|
||||
room_id=room_id,
|
||||
initial_auth_event_ids=auth_event_ids,
|
||||
initial_state_event_ids=state_event_ids,
|
||||
app_service_requester=requester,
|
||||
)
|
||||
)
|
||||
# Update our ongoing auth event ID list with all of the new state we
|
||||
# just created
|
||||
auth_event_ids.extend(state_event_ids_at_start)
|
||||
state_event_ids.extend(state_event_ids_at_start)
|
||||
|
||||
inherited_depth = await self.room_batch_handler.inherit_depth_from_prev_ids(
|
||||
prev_event_ids_from_query
|
||||
|
|
@ -196,7 +196,12 @@ class RoomBatchSendEventRestServlet(RestServlet):
|
|||
),
|
||||
base_insertion_event_dict,
|
||||
prev_event_ids=base_insertion_event_dict.get("prev_events"),
|
||||
auth_event_ids=auth_event_ids,
|
||||
# Also set the explicit state here because we want to resolve
|
||||
# any `state_events_at_start` here too. It's not strictly
|
||||
# necessary to accomplish anything but if someone asks for the
|
||||
# state at this point, we probably want to show them the
|
||||
# historical state that was part of this batch.
|
||||
state_event_ids=state_event_ids,
|
||||
historical=True,
|
||||
depth=inherited_depth,
|
||||
)
|
||||
|
|
@ -212,7 +217,7 @@ class RoomBatchSendEventRestServlet(RestServlet):
|
|||
room_id=room_id,
|
||||
batch_id_to_connect_to=batch_id_to_connect_to,
|
||||
inherited_depth=inherited_depth,
|
||||
auth_event_ids=auth_event_ids,
|
||||
initial_state_event_ids=state_event_ids,
|
||||
app_service_requester=requester,
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -19,7 +19,7 @@ from synapse.api.errors import SynapseError
|
|||
from synapse.http.server import HttpServer
|
||||
from synapse.http.servlet import RestServlet, parse_json_object_from_request
|
||||
from synapse.http.site import SynapseRequest
|
||||
from synapse.types import JsonDict
|
||||
from synapse.types import JsonMapping
|
||||
|
||||
from ._base import client_patterns
|
||||
|
||||
|
|
@ -38,7 +38,7 @@ class UserDirectorySearchRestServlet(RestServlet):
|
|||
self.auth = hs.get_auth()
|
||||
self.user_directory_handler = hs.get_user_directory_handler()
|
||||
|
||||
async def on_POST(self, request: SynapseRequest) -> Tuple[int, JsonDict]:
|
||||
async def on_POST(self, request: SynapseRequest) -> Tuple[int, JsonMapping]:
|
||||
"""Searches for users in directory
|
||||
|
||||
Returns:
|
||||
|
|
|
|||
|
|
@ -16,7 +16,6 @@ import itertools
|
|||
import logging
|
||||
import re
|
||||
from typing import TYPE_CHECKING, Dict, Generator, Iterable, Optional, Set, Union
|
||||
from urllib import parse as urlparse
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from lxml import etree
|
||||
|
|
@ -144,9 +143,7 @@ def decode_body(
|
|||
return etree.fromstring(body, parser)
|
||||
|
||||
|
||||
def parse_html_to_open_graph(
|
||||
tree: "etree.Element", media_uri: str
|
||||
) -> Dict[str, Optional[str]]:
|
||||
def parse_html_to_open_graph(tree: "etree.Element") -> Dict[str, Optional[str]]:
|
||||
"""
|
||||
Parse the HTML document into an Open Graph response.
|
||||
|
||||
|
|
@ -155,7 +152,6 @@ def parse_html_to_open_graph(
|
|||
|
||||
Args:
|
||||
tree: The parsed HTML document.
|
||||
media_url: The URI used to download the body.
|
||||
|
||||
Returns:
|
||||
The Open Graph response as a dictionary.
|
||||
|
|
@ -209,7 +205,7 @@ def parse_html_to_open_graph(
|
|||
"//*/meta[translate(@itemprop, 'IMAGE', 'image')='image']/@content"
|
||||
)
|
||||
if meta_image:
|
||||
og["og:image"] = rebase_url(meta_image[0], media_uri)
|
||||
og["og:image"] = meta_image[0]
|
||||
else:
|
||||
# TODO: consider inlined CSS styles as well as width & height attribs
|
||||
images = tree.xpath("//img[@src][number(@width)>10][number(@height)>10]")
|
||||
|
|
@ -320,37 +316,6 @@ def _iterate_over_text(
|
|||
)
|
||||
|
||||
|
||||
def rebase_url(url: str, base: str) -> str:
|
||||
"""
|
||||
Resolves a potentially relative `url` against an absolute `base` URL.
|
||||
|
||||
For example:
|
||||
|
||||
>>> rebase_url("subpage", "https://example.com/foo/")
|
||||
'https://example.com/foo/subpage'
|
||||
>>> rebase_url("sibling", "https://example.com/foo")
|
||||
'https://example.com/sibling'
|
||||
>>> rebase_url("/bar", "https://example.com/foo/")
|
||||
'https://example.com/bar'
|
||||
>>> rebase_url("https://alice.com/a/", "https://example.com/foo/")
|
||||
'https://alice.com/a'
|
||||
"""
|
||||
base_parts = urlparse.urlparse(base)
|
||||
# Convert the parsed URL to a list for (potential) modification.
|
||||
url_parts = list(urlparse.urlparse(url))
|
||||
# Add a scheme, if one does not exist.
|
||||
if not url_parts[0]:
|
||||
url_parts[0] = base_parts.scheme or "http"
|
||||
# Fix up the hostname, if this is not a data URL.
|
||||
if url_parts[0] != "data" and not url_parts[1]:
|
||||
url_parts[1] = base_parts.netloc
|
||||
# If the path does not start with a /, nest it under the base path's last
|
||||
# directory.
|
||||
if not url_parts[2].startswith("/"):
|
||||
url_parts[2] = re.sub(r"/[^/]+$", "/", base_parts.path) + url_parts[2]
|
||||
return urlparse.urlunparse(url_parts)
|
||||
|
||||
|
||||
def summarize_paragraphs(
|
||||
text_nodes: Iterable[str], min_size: int = 200, max_size: int = 500
|
||||
) -> Optional[str]:
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ import shutil
|
|||
import sys
|
||||
import traceback
|
||||
from typing import TYPE_CHECKING, BinaryIO, Iterable, Optional, Tuple
|
||||
from urllib import parse as urlparse
|
||||
from urllib.parse import urljoin, urlparse, urlsplit
|
||||
from urllib.request import urlopen
|
||||
|
||||
import attr
|
||||
|
|
@ -44,11 +44,7 @@ from synapse.metrics.background_process_metrics import run_as_background_process
|
|||
from synapse.rest.media.v1._base import get_filename_from_headers
|
||||
from synapse.rest.media.v1.media_storage import MediaStorage
|
||||
from synapse.rest.media.v1.oembed import OEmbedProvider
|
||||
from synapse.rest.media.v1.preview_html import (
|
||||
decode_body,
|
||||
parse_html_to_open_graph,
|
||||
rebase_url,
|
||||
)
|
||||
from synapse.rest.media.v1.preview_html import decode_body, parse_html_to_open_graph
|
||||
from synapse.types import JsonDict, UserID
|
||||
from synapse.util import json_encoder
|
||||
from synapse.util.async_helpers import ObservableDeferred
|
||||
|
|
@ -187,7 +183,7 @@ class PreviewUrlResource(DirectServeJsonResource):
|
|||
ts = self.clock.time_msec()
|
||||
|
||||
# XXX: we could move this into _do_preview if we wanted.
|
||||
url_tuple = urlparse.urlsplit(url)
|
||||
url_tuple = urlsplit(url)
|
||||
for entry in self.url_preview_url_blacklist:
|
||||
match = True
|
||||
for attrib in entry:
|
||||
|
|
@ -322,7 +318,7 @@ class PreviewUrlResource(DirectServeJsonResource):
|
|||
|
||||
# Parse Open Graph information from the HTML in case the oEmbed
|
||||
# response failed or is incomplete.
|
||||
og_from_html = parse_html_to_open_graph(tree, media_info.uri)
|
||||
og_from_html = parse_html_to_open_graph(tree)
|
||||
|
||||
# Compile the Open Graph response by using the scraped
|
||||
# information from the HTML and overlaying any information
|
||||
|
|
@ -588,12 +584,17 @@ class PreviewUrlResource(DirectServeJsonResource):
|
|||
if "og:image" not in og or not og["og:image"]:
|
||||
return
|
||||
|
||||
# The image URL from the HTML might be relative to the previewed page,
|
||||
# convert it to an URL which can be requested directly.
|
||||
image_url = og["og:image"]
|
||||
url_parts = urlparse(image_url)
|
||||
if url_parts.scheme != "data":
|
||||
image_url = urljoin(media_info.uri, image_url)
|
||||
|
||||
# FIXME: it might be cleaner to use the same flow as the main /preview_url
|
||||
# request itself and benefit from the same caching etc. But for now we
|
||||
# just rely on the caching on the master request to speed things up.
|
||||
image_info = await self._handle_url(
|
||||
rebase_url(og["og:image"], media_info.uri), user, allow_data_urls=True
|
||||
)
|
||||
image_info = await self._handle_url(image_url, user, allow_data_urls=True)
|
||||
|
||||
if _is_media(image_info.media_type):
|
||||
# TODO: make sure we don't choke on white-on-transparent images
|
||||
|
|
|
|||
|
|
@ -94,6 +94,7 @@ from synapse.handlers.profile import ProfileHandler
|
|||
from synapse.handlers.read_marker import ReadMarkerHandler
|
||||
from synapse.handlers.receipts import ReceiptsHandler
|
||||
from synapse.handlers.register import RegistrationHandler
|
||||
from synapse.handlers.relations import RelationsHandler
|
||||
from synapse.handlers.room import (
|
||||
RoomContextHandler,
|
||||
RoomCreationHandler,
|
||||
|
|
@ -719,6 +720,10 @@ class HomeServer(metaclass=abc.ABCMeta):
|
|||
def get_pagination_handler(self) -> PaginationHandler:
|
||||
return PaginationHandler(self)
|
||||
|
||||
@cache_in_self
|
||||
def get_relations_handler(self) -> RelationsHandler:
|
||||
return RelationsHandler(self)
|
||||
|
||||
@cache_in_self
|
||||
def get_room_context_handler(self) -> RoomContextHandler:
|
||||
return RoomContextHandler(self)
|
||||
|
|
|
|||
|
|
@ -41,6 +41,7 @@ from prometheus_client import Histogram
|
|||
from typing_extensions import Literal
|
||||
|
||||
from twisted.enterprise import adbapi
|
||||
from twisted.internet import defer
|
||||
|
||||
from synapse.api.errors import StoreError
|
||||
from synapse.config.database import DatabaseConnectionConfig
|
||||
|
|
@ -55,6 +56,7 @@ from synapse.metrics.background_process_metrics import run_as_background_process
|
|||
from synapse.storage.background_updates import BackgroundUpdater
|
||||
from synapse.storage.engines import BaseDatabaseEngine, PostgresEngine, Sqlite3Engine
|
||||
from synapse.storage.types import Connection, Cursor
|
||||
from synapse.util.async_helpers import delay_cancellation
|
||||
from synapse.util.iterutils import batch_iter
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
|
@ -286,13 +288,17 @@ class LoggingTransaction:
|
|||
"""
|
||||
|
||||
if isinstance(self.database_engine, PostgresEngine):
|
||||
from psycopg2.extras import execute_batch # type: ignore
|
||||
from psycopg2.extras import execute_batch
|
||||
|
||||
self._do_execute(lambda *x: execute_batch(self.txn, *x), sql, args)
|
||||
self._do_execute(
|
||||
lambda the_sql: execute_batch(self.txn, the_sql, args), sql
|
||||
)
|
||||
else:
|
||||
self.executemany(sql, args)
|
||||
|
||||
def execute_values(self, sql: str, *args: Any, fetch: bool = True) -> List[Tuple]:
|
||||
def execute_values(
|
||||
self, sql: str, values: Iterable[Iterable[Any]], fetch: bool = True
|
||||
) -> List[Tuple]:
|
||||
"""Corresponds to psycopg2.extras.execute_values. Only available when
|
||||
using postgres.
|
||||
|
||||
|
|
@ -300,10 +306,11 @@ class LoggingTransaction:
|
|||
rows (e.g. INSERTs).
|
||||
"""
|
||||
assert isinstance(self.database_engine, PostgresEngine)
|
||||
from psycopg2.extras import execute_values # type: ignore
|
||||
from psycopg2.extras import execute_values
|
||||
|
||||
return self._do_execute(
|
||||
lambda *x: execute_values(self.txn, *x, fetch=fetch), sql, *args
|
||||
lambda the_sql: execute_values(self.txn, the_sql, values, fetch=fetch),
|
||||
sql,
|
||||
)
|
||||
|
||||
def execute(self, sql: str, *args: Any) -> None:
|
||||
|
|
@ -732,34 +739,45 @@ class DatabasePool:
|
|||
Returns:
|
||||
The result of func
|
||||
"""
|
||||
after_callbacks: List[_CallbackListEntry] = []
|
||||
exception_callbacks: List[_CallbackListEntry] = []
|
||||
|
||||
if not current_context():
|
||||
logger.warning("Starting db txn '%s' from sentinel context", desc)
|
||||
async def _runInteraction() -> R:
|
||||
after_callbacks: List[_CallbackListEntry] = []
|
||||
exception_callbacks: List[_CallbackListEntry] = []
|
||||
|
||||
try:
|
||||
with opentracing.start_active_span(f"db.{desc}"):
|
||||
result = await self.runWithConnection(
|
||||
self.new_transaction,
|
||||
desc,
|
||||
after_callbacks,
|
||||
exception_callbacks,
|
||||
func,
|
||||
*args,
|
||||
db_autocommit=db_autocommit,
|
||||
isolation_level=isolation_level,
|
||||
**kwargs,
|
||||
)
|
||||
if not current_context():
|
||||
logger.warning("Starting db txn '%s' from sentinel context", desc)
|
||||
|
||||
for after_callback, after_args, after_kwargs in after_callbacks:
|
||||
after_callback(*after_args, **after_kwargs)
|
||||
except Exception:
|
||||
for after_callback, after_args, after_kwargs in exception_callbacks:
|
||||
after_callback(*after_args, **after_kwargs)
|
||||
raise
|
||||
try:
|
||||
with opentracing.start_active_span(f"db.{desc}"):
|
||||
result = await self.runWithConnection(
|
||||
self.new_transaction,
|
||||
desc,
|
||||
after_callbacks,
|
||||
exception_callbacks,
|
||||
func,
|
||||
*args,
|
||||
db_autocommit=db_autocommit,
|
||||
isolation_level=isolation_level,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
return cast(R, result)
|
||||
for after_callback, after_args, after_kwargs in after_callbacks:
|
||||
after_callback(*after_args, **after_kwargs)
|
||||
|
||||
return cast(R, result)
|
||||
except Exception:
|
||||
for after_callback, after_args, after_kwargs in exception_callbacks:
|
||||
after_callback(*after_args, **after_kwargs)
|
||||
raise
|
||||
|
||||
# To handle cancellation, we ensure that `after_callback`s and
|
||||
# `exception_callback`s are always run, since the transaction will complete
|
||||
# on another thread regardless of cancellation.
|
||||
#
|
||||
# We also wait until everything above is done before releasing the
|
||||
# `CancelledError`, so that logging contexts won't get used after they have been
|
||||
# finished.
|
||||
return await delay_cancellation(defer.ensureDeferred(_runInteraction()))
|
||||
|
||||
async def runWithConnection(
|
||||
self,
|
||||
|
|
|
|||
|
|
@ -14,7 +14,17 @@
|
|||
# limitations under the License.
|
||||
|
||||
import logging
|
||||
from typing import TYPE_CHECKING, Any, Dict, Iterable, List, Optional, Set, Tuple, cast
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
Any,
|
||||
Dict,
|
||||
FrozenSet,
|
||||
Iterable,
|
||||
List,
|
||||
Optional,
|
||||
Tuple,
|
||||
cast,
|
||||
)
|
||||
|
||||
from synapse.api.constants import AccountDataTypes
|
||||
from synapse.replication.slave.storage._slaved_id_tracker import SlavedIdTracker
|
||||
|
|
@ -365,7 +375,7 @@ class AccountDataWorkerStore(PushRulesWorkerStore, CacheInvalidationWorkerStore)
|
|||
)
|
||||
|
||||
@cached(max_entries=5000, iterable=True)
|
||||
async def ignored_by(self, user_id: str) -> Set[str]:
|
||||
async def ignored_by(self, user_id: str) -> FrozenSet[str]:
|
||||
"""
|
||||
Get users which ignore the given user.
|
||||
|
||||
|
|
@ -375,7 +385,7 @@ class AccountDataWorkerStore(PushRulesWorkerStore, CacheInvalidationWorkerStore)
|
|||
Return:
|
||||
The user IDs which ignore the given user.
|
||||
"""
|
||||
return set(
|
||||
return frozenset(
|
||||
await self.db_pool.simple_select_onecol(
|
||||
table="ignored_users",
|
||||
keyvalues={"ignored_user_id": user_id},
|
||||
|
|
@ -384,6 +394,26 @@ class AccountDataWorkerStore(PushRulesWorkerStore, CacheInvalidationWorkerStore)
|
|||
)
|
||||
)
|
||||
|
||||
@cached(max_entries=5000, iterable=True)
|
||||
async def ignored_users(self, user_id: str) -> FrozenSet[str]:
|
||||
"""
|
||||
Get users which the given user ignores.
|
||||
|
||||
Params:
|
||||
user_id: The user ID which is making the request.
|
||||
|
||||
Return:
|
||||
The user IDs which are ignored by the given user.
|
||||
"""
|
||||
return frozenset(
|
||||
await self.db_pool.simple_select_onecol(
|
||||
table="ignored_users",
|
||||
keyvalues={"ignorer_user_id": user_id},
|
||||
retcol="ignored_user_id",
|
||||
desc="ignored_users",
|
||||
)
|
||||
)
|
||||
|
||||
def process_replication_rows(
|
||||
self,
|
||||
stream_name: str,
|
||||
|
|
@ -529,6 +559,10 @@ class AccountDataWorkerStore(PushRulesWorkerStore, CacheInvalidationWorkerStore)
|
|||
else:
|
||||
currently_ignored_users = set()
|
||||
|
||||
# If the data has not changed, nothing to do.
|
||||
if previously_ignored_users == currently_ignored_users:
|
||||
return
|
||||
|
||||
# Delete entries which are no longer ignored.
|
||||
self.db_pool.simple_delete_many_txn(
|
||||
txn,
|
||||
|
|
@ -551,6 +585,7 @@ class AccountDataWorkerStore(PushRulesWorkerStore, CacheInvalidationWorkerStore)
|
|||
# Invalidate the cache for any ignored users which were added or removed.
|
||||
for ignored_user_id in previously_ignored_users ^ currently_ignored_users:
|
||||
self._invalidate_cache_and_stream(txn, self.ignored_by, (ignored_user_id,))
|
||||
self._invalidate_cache_and_stream(txn, self.ignored_users, (user_id,))
|
||||
|
||||
async def purge_account_data_for_user(self, user_id: str) -> None:
|
||||
"""
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ from synapse.replication.tcp.streams.events import (
|
|||
EventsStream,
|
||||
EventsStreamCurrentStateRow,
|
||||
EventsStreamEventRow,
|
||||
EventsStreamRow,
|
||||
)
|
||||
from synapse.storage._base import SQLBaseStore
|
||||
from synapse.storage.database import (
|
||||
|
|
@ -31,6 +32,7 @@ from synapse.storage.database import (
|
|||
LoggingTransaction,
|
||||
)
|
||||
from synapse.storage.engines import PostgresEngine
|
||||
from synapse.util.caches.descriptors import _CachedFunction
|
||||
from synapse.util.iterutils import batch_iter
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
|
@ -82,7 +84,9 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
|
|||
if last_id == current_id:
|
||||
return [], current_id, False
|
||||
|
||||
def get_all_updated_caches_txn(txn):
|
||||
def get_all_updated_caches_txn(
|
||||
txn: LoggingTransaction,
|
||||
) -> Tuple[List[Tuple[int, tuple]], int, bool]:
|
||||
# We purposefully don't bound by the current token, as we want to
|
||||
# send across cache invalidations as quickly as possible. Cache
|
||||
# invalidations are idempotent, so duplicates are fine.
|
||||
|
|
@ -107,7 +111,9 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
|
|||
"get_all_updated_caches", get_all_updated_caches_txn
|
||||
)
|
||||
|
||||
def process_replication_rows(self, stream_name, instance_name, token, rows):
|
||||
def process_replication_rows(
|
||||
self, stream_name: str, instance_name: str, token: int, rows: Iterable[Any]
|
||||
) -> None:
|
||||
if stream_name == EventsStream.NAME:
|
||||
for row in rows:
|
||||
self._process_event_stream_row(token, row)
|
||||
|
|
@ -142,10 +148,11 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
|
|||
|
||||
super().process_replication_rows(stream_name, instance_name, token, rows)
|
||||
|
||||
def _process_event_stream_row(self, token, row):
|
||||
def _process_event_stream_row(self, token: int, row: EventsStreamRow) -> None:
|
||||
data = row.data
|
||||
|
||||
if row.type == EventsStreamEventRow.TypeId:
|
||||
assert isinstance(data, EventsStreamEventRow)
|
||||
self._invalidate_caches_for_event(
|
||||
token,
|
||||
data.event_id,
|
||||
|
|
@ -157,9 +164,8 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
|
|||
backfilled=False,
|
||||
)
|
||||
elif row.type == EventsStreamCurrentStateRow.TypeId:
|
||||
self._curr_state_delta_stream_cache.entity_has_changed(
|
||||
row.data.room_id, token
|
||||
)
|
||||
assert isinstance(data, EventsStreamCurrentStateRow)
|
||||
self._curr_state_delta_stream_cache.entity_has_changed(data.room_id, token)
|
||||
|
||||
if data.type == EventTypes.Member:
|
||||
self.get_rooms_for_user_with_stream_ordering.invalidate(
|
||||
|
|
@ -170,15 +176,15 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
|
|||
|
||||
def _invalidate_caches_for_event(
|
||||
self,
|
||||
stream_ordering,
|
||||
event_id,
|
||||
room_id,
|
||||
etype,
|
||||
state_key,
|
||||
redacts,
|
||||
relates_to,
|
||||
backfilled,
|
||||
):
|
||||
stream_ordering: int,
|
||||
event_id: str,
|
||||
room_id: str,
|
||||
etype: str,
|
||||
state_key: Optional[str],
|
||||
redacts: Optional[str],
|
||||
relates_to: Optional[str],
|
||||
backfilled: bool,
|
||||
) -> None:
|
||||
self._invalidate_get_event_cache(event_id)
|
||||
self.have_seen_event.invalidate((room_id, event_id))
|
||||
|
||||
|
|
@ -186,6 +192,10 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
|
|||
|
||||
self.get_unread_event_push_actions_by_room_for_user.invalidate((room_id,))
|
||||
|
||||
# The `_get_membership_from_event_id` is immutable, except for the
|
||||
# case where we look up an event *before* persisting it.
|
||||
self._get_membership_from_event_id.invalidate((event_id,))
|
||||
|
||||
if not backfilled:
|
||||
self._events_stream_cache.entity_has_changed(room_id, stream_ordering)
|
||||
|
||||
|
|
@ -207,7 +217,9 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
|
|||
self.get_thread_summary.invalidate((relates_to,))
|
||||
self.get_thread_participated.invalidate((relates_to,))
|
||||
|
||||
async def invalidate_cache_and_stream(self, cache_name: str, keys: Tuple[Any, ...]):
|
||||
async def invalidate_cache_and_stream(
|
||||
self, cache_name: str, keys: Tuple[Any, ...]
|
||||
) -> None:
|
||||
"""Invalidates the cache and adds it to the cache stream so slaves
|
||||
will know to invalidate their caches.
|
||||
|
||||
|
|
@ -227,7 +239,12 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
|
|||
keys,
|
||||
)
|
||||
|
||||
def _invalidate_cache_and_stream(self, txn, cache_func, keys):
|
||||
def _invalidate_cache_and_stream(
|
||||
self,
|
||||
txn: LoggingTransaction,
|
||||
cache_func: _CachedFunction,
|
||||
keys: Tuple[Any, ...],
|
||||
) -> None:
|
||||
"""Invalidates the cache and adds it to the cache stream so slaves
|
||||
will know to invalidate their caches.
|
||||
|
||||
|
|
@ -238,7 +255,9 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
|
|||
txn.call_after(cache_func.invalidate, keys)
|
||||
self._send_invalidation_to_replication(txn, cache_func.__name__, keys)
|
||||
|
||||
def _invalidate_all_cache_and_stream(self, txn, cache_func):
|
||||
def _invalidate_all_cache_and_stream(
|
||||
self, txn: LoggingTransaction, cache_func: _CachedFunction
|
||||
) -> None:
|
||||
"""Invalidates the entire cache and adds it to the cache stream so slaves
|
||||
will know to invalidate their caches.
|
||||
"""
|
||||
|
|
@ -279,8 +298,8 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
|
|||
)
|
||||
|
||||
def _send_invalidation_to_replication(
|
||||
self, txn, cache_name: str, keys: Optional[Iterable[Any]]
|
||||
):
|
||||
self, txn: LoggingTransaction, cache_name: str, keys: Optional[Iterable[Any]]
|
||||
) -> None:
|
||||
"""Notifies replication that given cache has been invalidated.
|
||||
|
||||
Note that this does *not* invalidate the cache locally.
|
||||
|
|
@ -315,7 +334,7 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
|
|||
"instance_name": self._instance_name,
|
||||
"cache_func": cache_name,
|
||||
"keys": keys,
|
||||
"invalidation_ts": self.clock.time_msec(),
|
||||
"invalidation_ts": self._clock.time_msec(),
|
||||
},
|
||||
)
|
||||
|
||||
|
|
|
|||
|
|
@ -1073,9 +1073,15 @@ class EventFederationWorkerStore(SignatureWorkerStore, EventsWorkerStore, SQLBas
|
|||
/* Get the depth and stream_ordering of the prev_event_id from the events table */
|
||||
INNER JOIN events
|
||||
ON prev_event_id = events.event_id
|
||||
|
||||
/* exclude outliers from the results (we don't have the state, so cannot
|
||||
* verify if the requesting server can see them).
|
||||
*/
|
||||
WHERE NOT events.outlier
|
||||
|
||||
/* Look for an edge which matches the given event_id */
|
||||
WHERE event_edges.event_id = ?
|
||||
AND event_edges.is_state = ?
|
||||
AND event_edges.event_id = ? AND NOT event_edges.is_state
|
||||
|
||||
/* Because we can have many events at the same depth,
|
||||
* we want to also tie-break and sort on stream_ordering */
|
||||
ORDER BY depth DESC, stream_ordering DESC
|
||||
|
|
@ -1084,7 +1090,7 @@ class EventFederationWorkerStore(SignatureWorkerStore, EventsWorkerStore, SQLBas
|
|||
|
||||
txn.execute(
|
||||
connected_prev_event_query,
|
||||
(event_id, False, limit),
|
||||
(event_id, limit),
|
||||
)
|
||||
return [
|
||||
BackfillQueueNavigationItem(
|
||||
|
|
|
|||
|
|
@ -1745,6 +1745,13 @@ class PersistEventsStore:
|
|||
(event.state_key,),
|
||||
)
|
||||
|
||||
# The `_get_membership_from_event_id` is immutable, except for the
|
||||
# case where we look up an event *before* persisting it.
|
||||
txn.call_after(
|
||||
self.store._get_membership_from_event_id.invalidate,
|
||||
(event.event_id,),
|
||||
)
|
||||
|
||||
# We update the local_current_membership table only if the event is
|
||||
# "current", i.e., its something that has just happened.
|
||||
#
|
||||
|
|
|
|||
|
|
@ -13,13 +13,17 @@
|
|||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple
|
||||
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Tuple, cast
|
||||
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from synapse.api.errors import SynapseError
|
||||
from synapse.storage._base import SQLBaseStore, db_to_json
|
||||
from synapse.storage.database import DatabasePool, LoggingDatabaseConnection
|
||||
from synapse.storage.database import (
|
||||
DatabasePool,
|
||||
LoggingDatabaseConnection,
|
||||
LoggingTransaction,
|
||||
)
|
||||
from synapse.types import JsonDict
|
||||
from synapse.util import json_encoder
|
||||
|
||||
|
|
@ -75,7 +79,7 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
) -> List[Dict[str, Any]]:
|
||||
# TODO: Pagination
|
||||
|
||||
keyvalues = {"group_id": group_id}
|
||||
keyvalues: JsonDict = {"group_id": group_id}
|
||||
if not include_private:
|
||||
keyvalues["is_public"] = True
|
||||
|
||||
|
|
@ -117,7 +121,7 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
|
||||
# TODO: Pagination
|
||||
|
||||
def _get_rooms_in_group_txn(txn):
|
||||
def _get_rooms_in_group_txn(txn: LoggingTransaction) -> List[_RoomInGroup]:
|
||||
sql = """
|
||||
SELECT room_id, is_public FROM group_rooms
|
||||
WHERE group_id = ?
|
||||
|
|
@ -176,8 +180,10 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
* "order": int, the sort order of rooms in this category
|
||||
"""
|
||||
|
||||
def _get_rooms_for_summary_txn(txn):
|
||||
keyvalues = {"group_id": group_id}
|
||||
def _get_rooms_for_summary_txn(
|
||||
txn: LoggingTransaction,
|
||||
) -> Tuple[List[Dict[str, Any]], Dict[str, Any]]:
|
||||
keyvalues: JsonDict = {"group_id": group_id}
|
||||
if not include_private:
|
||||
keyvalues["is_public"] = True
|
||||
|
||||
|
|
@ -241,7 +247,7 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
"get_rooms_for_summary", _get_rooms_for_summary_txn
|
||||
)
|
||||
|
||||
async def get_group_categories(self, group_id):
|
||||
async def get_group_categories(self, group_id: str) -> JsonDict:
|
||||
rows = await self.db_pool.simple_select_list(
|
||||
table="group_room_categories",
|
||||
keyvalues={"group_id": group_id},
|
||||
|
|
@ -257,7 +263,7 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
for row in rows
|
||||
}
|
||||
|
||||
async def get_group_category(self, group_id, category_id):
|
||||
async def get_group_category(self, group_id: str, category_id: str) -> JsonDict:
|
||||
category = await self.db_pool.simple_select_one(
|
||||
table="group_room_categories",
|
||||
keyvalues={"group_id": group_id, "category_id": category_id},
|
||||
|
|
@ -269,7 +275,7 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
|
||||
return category
|
||||
|
||||
async def get_group_roles(self, group_id):
|
||||
async def get_group_roles(self, group_id: str) -> JsonDict:
|
||||
rows = await self.db_pool.simple_select_list(
|
||||
table="group_roles",
|
||||
keyvalues={"group_id": group_id},
|
||||
|
|
@ -285,7 +291,7 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
for row in rows
|
||||
}
|
||||
|
||||
async def get_group_role(self, group_id, role_id):
|
||||
async def get_group_role(self, group_id: str, role_id: str) -> JsonDict:
|
||||
role = await self.db_pool.simple_select_one(
|
||||
table="group_roles",
|
||||
keyvalues={"group_id": group_id, "role_id": role_id},
|
||||
|
|
@ -311,15 +317,19 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
desc="get_local_groups_for_room",
|
||||
)
|
||||
|
||||
async def get_users_for_summary_by_role(self, group_id, include_private=False):
|
||||
async def get_users_for_summary_by_role(
|
||||
self, group_id: str, include_private: bool = False
|
||||
) -> Tuple[List[JsonDict], JsonDict]:
|
||||
"""Get the users and roles that should be included in a summary request
|
||||
|
||||
Returns:
|
||||
([users], [roles])
|
||||
"""
|
||||
|
||||
def _get_users_for_summary_txn(txn):
|
||||
keyvalues = {"group_id": group_id}
|
||||
def _get_users_for_summary_txn(
|
||||
txn: LoggingTransaction,
|
||||
) -> Tuple[List[JsonDict], JsonDict]:
|
||||
keyvalues: JsonDict = {"group_id": group_id}
|
||||
if not include_private:
|
||||
keyvalues["is_public"] = True
|
||||
|
||||
|
|
@ -406,7 +416,9 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
allow_none=True,
|
||||
)
|
||||
|
||||
async def get_users_membership_info_in_group(self, group_id, user_id):
|
||||
async def get_users_membership_info_in_group(
|
||||
self, group_id: str, user_id: str
|
||||
) -> JsonDict:
|
||||
"""Get a dict describing the membership of a user in a group.
|
||||
|
||||
Example if joined:
|
||||
|
|
@ -421,7 +433,7 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
An empty dict if the user is not join/invite/etc
|
||||
"""
|
||||
|
||||
def _get_users_membership_in_group_txn(txn):
|
||||
def _get_users_membership_in_group_txn(txn: LoggingTransaction) -> JsonDict:
|
||||
row = self.db_pool.simple_select_one_txn(
|
||||
txn,
|
||||
table="group_users",
|
||||
|
|
@ -463,10 +475,14 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
desc="get_publicised_groups_for_user",
|
||||
)
|
||||
|
||||
async def get_attestations_need_renewals(self, valid_until_ms):
|
||||
async def get_attestations_need_renewals(
|
||||
self, valid_until_ms: int
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Get all attestations that need to be renewed until givent time"""
|
||||
|
||||
def _get_attestations_need_renewals_txn(txn):
|
||||
def _get_attestations_need_renewals_txn(
|
||||
txn: LoggingTransaction,
|
||||
) -> List[Dict[str, Any]]:
|
||||
sql = """
|
||||
SELECT group_id, user_id FROM group_attestations_renewals
|
||||
WHERE valid_until_ms <= ?
|
||||
|
|
@ -478,7 +494,9 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
"get_attestations_need_renewals", _get_attestations_need_renewals_txn
|
||||
)
|
||||
|
||||
async def get_remote_attestation(self, group_id, user_id):
|
||||
async def get_remote_attestation(
|
||||
self, group_id: str, user_id: str
|
||||
) -> Optional[JsonDict]:
|
||||
"""Get the attestation that proves the remote agrees that the user is
|
||||
in the group.
|
||||
"""
|
||||
|
|
@ -504,8 +522,8 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
desc="get_joined_groups",
|
||||
)
|
||||
|
||||
async def get_all_groups_for_user(self, user_id, now_token):
|
||||
def _get_all_groups_for_user_txn(txn):
|
||||
async def get_all_groups_for_user(self, user_id, now_token) -> List[JsonDict]:
|
||||
def _get_all_groups_for_user_txn(txn: LoggingTransaction) -> List[JsonDict]:
|
||||
sql = """
|
||||
SELECT group_id, type, membership, u.content
|
||||
FROM local_group_updates AS u
|
||||
|
|
@ -528,15 +546,16 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
"get_all_groups_for_user", _get_all_groups_for_user_txn
|
||||
)
|
||||
|
||||
async def get_groups_changes_for_user(self, user_id, from_token, to_token):
|
||||
from_token = int(from_token)
|
||||
has_changed = self._group_updates_stream_cache.has_entity_changed(
|
||||
async def get_groups_changes_for_user(
|
||||
self, user_id: str, from_token: int, to_token: int
|
||||
) -> List[JsonDict]:
|
||||
has_changed = self._group_updates_stream_cache.has_entity_changed( # type: ignore[attr-defined]
|
||||
user_id, from_token
|
||||
)
|
||||
if not has_changed:
|
||||
return []
|
||||
|
||||
def _get_groups_changes_for_user_txn(txn):
|
||||
def _get_groups_changes_for_user_txn(txn: LoggingTransaction) -> List[JsonDict]:
|
||||
sql = """
|
||||
SELECT group_id, membership, type, u.content
|
||||
FROM local_group_updates AS u
|
||||
|
|
@ -583,12 +602,14 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
"""
|
||||
|
||||
last_id = int(last_id)
|
||||
has_changed = self._group_updates_stream_cache.has_any_entity_changed(last_id)
|
||||
has_changed = self._group_updates_stream_cache.has_any_entity_changed(last_id) # type: ignore[attr-defined]
|
||||
|
||||
if not has_changed:
|
||||
return [], current_id, False
|
||||
|
||||
def _get_all_groups_changes_txn(txn):
|
||||
def _get_all_groups_changes_txn(
|
||||
txn: LoggingTransaction,
|
||||
) -> Tuple[List[Tuple[int, tuple]], int, bool]:
|
||||
sql = """
|
||||
SELECT stream_id, group_id, user_id, type, content
|
||||
FROM local_group_updates
|
||||
|
|
@ -596,10 +617,13 @@ class GroupServerWorkerStore(SQLBaseStore):
|
|||
LIMIT ?
|
||||
"""
|
||||
txn.execute(sql, (last_id, current_id, limit))
|
||||
updates = [
|
||||
(stream_id, (group_id, user_id, gtype, db_to_json(content_json)))
|
||||
for stream_id, group_id, user_id, gtype, content_json in txn
|
||||
]
|
||||
updates = cast(
|
||||
List[Tuple[int, tuple]],
|
||||
[
|
||||
(stream_id, (group_id, user_id, gtype, db_to_json(content_json)))
|
||||
for stream_id, group_id, user_id, gtype, content_json in txn
|
||||
],
|
||||
)
|
||||
|
||||
limited = False
|
||||
upto_token = current_id
|
||||
|
|
@ -633,8 +657,8 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
self,
|
||||
group_id: str,
|
||||
room_id: str,
|
||||
category_id: str,
|
||||
order: int,
|
||||
category_id: Optional[str],
|
||||
order: Optional[int],
|
||||
is_public: Optional[bool],
|
||||
) -> None:
|
||||
"""Add (or update) room's entry in summary.
|
||||
|
|
@ -661,11 +685,11 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
|
||||
def _add_room_to_summary_txn(
|
||||
self,
|
||||
txn,
|
||||
txn: LoggingTransaction,
|
||||
group_id: str,
|
||||
room_id: str,
|
||||
category_id: str,
|
||||
order: int,
|
||||
category_id: Optional[str],
|
||||
order: Optional[int],
|
||||
is_public: Optional[bool],
|
||||
) -> None:
|
||||
"""Add (or update) room's entry in summary.
|
||||
|
|
@ -750,7 +774,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
WHERE group_id = ? AND category_id = ?
|
||||
"""
|
||||
txn.execute(sql, (group_id, category_id))
|
||||
(order,) = txn.fetchone()
|
||||
(order,) = cast(Tuple[int], txn.fetchone())
|
||||
|
||||
if existing:
|
||||
to_update = {}
|
||||
|
|
@ -766,7 +790,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
"category_id": category_id,
|
||||
"room_id": room_id,
|
||||
},
|
||||
values=to_update,
|
||||
updatevalues=to_update,
|
||||
)
|
||||
else:
|
||||
if is_public is None:
|
||||
|
|
@ -785,7 +809,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
)
|
||||
|
||||
async def remove_room_from_summary(
|
||||
self, group_id: str, room_id: str, category_id: str
|
||||
self, group_id: str, room_id: str, category_id: Optional[str]
|
||||
) -> int:
|
||||
if category_id is None:
|
||||
category_id = _DEFAULT_CATEGORY_ID
|
||||
|
|
@ -808,8 +832,8 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
is_public: Optional[bool],
|
||||
) -> None:
|
||||
"""Add/update room category for group"""
|
||||
insertion_values = {}
|
||||
update_values = {"category_id": category_id} # This cannot be empty
|
||||
insertion_values: JsonDict = {}
|
||||
update_values: JsonDict = {"category_id": category_id} # This cannot be empty
|
||||
|
||||
if profile is None:
|
||||
insertion_values["profile"] = "{}"
|
||||
|
|
@ -844,8 +868,8 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
is_public: Optional[bool],
|
||||
) -> None:
|
||||
"""Add/remove user role"""
|
||||
insertion_values = {}
|
||||
update_values = {"role_id": role_id} # This cannot be empty
|
||||
insertion_values: JsonDict = {}
|
||||
update_values: JsonDict = {"role_id": role_id} # This cannot be empty
|
||||
|
||||
if profile is None:
|
||||
insertion_values["profile"] = "{}"
|
||||
|
|
@ -876,8 +900,8 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
self,
|
||||
group_id: str,
|
||||
user_id: str,
|
||||
role_id: str,
|
||||
order: int,
|
||||
role_id: Optional[str],
|
||||
order: Optional[int],
|
||||
is_public: Optional[bool],
|
||||
) -> None:
|
||||
"""Add (or update) user's entry in summary.
|
||||
|
|
@ -904,13 +928,13 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
|
||||
def _add_user_to_summary_txn(
|
||||
self,
|
||||
txn,
|
||||
txn: LoggingTransaction,
|
||||
group_id: str,
|
||||
user_id: str,
|
||||
role_id: str,
|
||||
order: int,
|
||||
role_id: Optional[str],
|
||||
order: Optional[int],
|
||||
is_public: Optional[bool],
|
||||
):
|
||||
) -> None:
|
||||
"""Add (or update) user's entry in summary.
|
||||
|
||||
Args:
|
||||
|
|
@ -989,7 +1013,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
WHERE group_id = ? AND role_id = ?
|
||||
"""
|
||||
txn.execute(sql, (group_id, role_id))
|
||||
(order,) = txn.fetchone()
|
||||
(order,) = cast(Tuple[int], txn.fetchone())
|
||||
|
||||
if existing:
|
||||
to_update = {}
|
||||
|
|
@ -1005,7 +1029,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
"role_id": role_id,
|
||||
"user_id": user_id,
|
||||
},
|
||||
values=to_update,
|
||||
updatevalues=to_update,
|
||||
)
|
||||
else:
|
||||
if is_public is None:
|
||||
|
|
@ -1024,7 +1048,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
)
|
||||
|
||||
async def remove_user_from_summary(
|
||||
self, group_id: str, user_id: str, role_id: str
|
||||
self, group_id: str, user_id: str, role_id: Optional[str]
|
||||
) -> int:
|
||||
if role_id is None:
|
||||
role_id = _DEFAULT_ROLE_ID
|
||||
|
|
@ -1065,7 +1089,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
Optional if the user and group are on the same server
|
||||
"""
|
||||
|
||||
def _add_user_to_group_txn(txn):
|
||||
def _add_user_to_group_txn(txn: LoggingTransaction) -> None:
|
||||
self.db_pool.simple_insert_txn(
|
||||
txn,
|
||||
table="group_users",
|
||||
|
|
@ -1108,7 +1132,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
await self.db_pool.runInteraction("add_user_to_group", _add_user_to_group_txn)
|
||||
|
||||
async def remove_user_from_group(self, group_id: str, user_id: str) -> None:
|
||||
def _remove_user_from_group_txn(txn):
|
||||
def _remove_user_from_group_txn(txn: LoggingTransaction) -> None:
|
||||
self.db_pool.simple_delete_txn(
|
||||
txn,
|
||||
table="group_users",
|
||||
|
|
@ -1159,7 +1183,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
)
|
||||
|
||||
async def remove_room_from_group(self, group_id: str, room_id: str) -> None:
|
||||
def _remove_room_from_group_txn(txn):
|
||||
def _remove_room_from_group_txn(txn: LoggingTransaction) -> None:
|
||||
self.db_pool.simple_delete_txn(
|
||||
txn,
|
||||
table="group_rooms",
|
||||
|
|
@ -1216,7 +1240,9 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
|
||||
content = content or {}
|
||||
|
||||
def _register_user_group_membership_txn(txn, next_id):
|
||||
def _register_user_group_membership_txn(
|
||||
txn: LoggingTransaction, next_id: int
|
||||
) -> int:
|
||||
# TODO: Upsert?
|
||||
self.db_pool.simple_delete_txn(
|
||||
txn,
|
||||
|
|
@ -1249,7 +1275,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
),
|
||||
},
|
||||
)
|
||||
self._group_updates_stream_cache.entity_has_changed(user_id, next_id)
|
||||
self._group_updates_stream_cache.entity_has_changed(user_id, next_id) # type: ignore[attr-defined]
|
||||
|
||||
# TODO: Insert profile to ensure it comes down stream if its a join.
|
||||
|
||||
|
|
@ -1289,7 +1315,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
|
||||
return next_id
|
||||
|
||||
async with self._group_updates_id_gen.get_next() as next_id:
|
||||
async with self._group_updates_id_gen.get_next() as next_id: # type: ignore[attr-defined]
|
||||
res = await self.db_pool.runInteraction(
|
||||
"register_user_group_membership",
|
||||
_register_user_group_membership_txn,
|
||||
|
|
@ -1298,7 +1324,13 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
return res
|
||||
|
||||
async def create_group(
|
||||
self, group_id, user_id, name, avatar_url, short_description, long_description
|
||||
self,
|
||||
group_id: str,
|
||||
user_id: str,
|
||||
name: str,
|
||||
avatar_url: str,
|
||||
short_description: str,
|
||||
long_description: str,
|
||||
) -> None:
|
||||
await self.db_pool.simple_insert(
|
||||
table="groups",
|
||||
|
|
@ -1313,7 +1345,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
desc="create_group",
|
||||
)
|
||||
|
||||
async def update_group_profile(self, group_id, profile):
|
||||
async def update_group_profile(self, group_id: str, profile: JsonDict) -> None:
|
||||
await self.db_pool.simple_update_one(
|
||||
table="groups",
|
||||
keyvalues={"group_id": group_id},
|
||||
|
|
@ -1361,8 +1393,8 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
desc="remove_attestation_renewal",
|
||||
)
|
||||
|
||||
def get_group_stream_token(self):
|
||||
return self._group_updates_id_gen.get_current_token()
|
||||
def get_group_stream_token(self) -> int:
|
||||
return self._group_updates_id_gen.get_current_token() # type: ignore[attr-defined]
|
||||
|
||||
async def delete_group(self, group_id: str) -> None:
|
||||
"""Deletes a group fully from the database.
|
||||
|
|
@ -1371,7 +1403,7 @@ class GroupServerStore(GroupServerWorkerStore):
|
|||
group_id: The group ID to delete.
|
||||
"""
|
||||
|
||||
def _delete_group_txn(txn):
|
||||
def _delete_group_txn(txn: LoggingTransaction) -> None:
|
||||
tables = [
|
||||
"groups",
|
||||
"group_users",
|
||||
|
|
|
|||
|
|
@ -156,7 +156,7 @@ class MediaRepositoryStore(MediaRepositoryBackgroundUpdateStore):
|
|||
hs: "HomeServer",
|
||||
):
|
||||
super().__init__(database, db_conn, hs)
|
||||
self.server_name = hs.hostname
|
||||
self.server_name: str = hs.hostname
|
||||
|
||||
async def get_local_media(self, media_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""Get the metadata for a local piece of media
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue