OpenCloud
/
MatrixSynapse
mirror of https://github.com/matrix-org/synapse
1
0
Fork 0
Code Issues Releases Wiki Activity

Merge branch 'release-v1.89' into develop

pull/16009/head
Mathieu Velten 2023-07-26 17:07:42 +02:00
parent 8d2a5586f7 76e392b0fa
commit 6dc019d9dd
39 changed files with 89 additions and 47 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

61
CHANGES.md
View File

@ -1,3 +1,64 @@
# Synapse 1.89.0rc1 (2023-07-25)
### Features
- Add Unix Socket support for HTTP Replication Listeners. [Document and provide usage instructions](https://matrix-org.github.io/synapse/v1.89/usage/configuration/config_documentation.html#listeners) for utilizing Unix sockets in Synapse. Contributed by Jason Little. ([\#15708](https://github.com/matrix-org/synapse/issues/15708), [\#15924](https://github.com/matrix-org/synapse/issues/15924))
- Allow `+` in Matrix IDs, per [MSC4009](https://github.com/matrix-org/matrix-spec-proposals/pull/4009). ([\#15911](https://github.com/matrix-org/synapse/issues/15911))
- Support room version 11 from [MSC3820](https://github.com/matrix-org/matrix-spec-proposals/pull/3820). ([\#15912](https://github.com/matrix-org/synapse/issues/15912))
- Allow configuring the set of workers to proxy outbound federation traffic through via `outbound_federation_restricted_to`. ([\#15913](https://github.com/matrix-org/synapse/issues/15913), [\#15969](https://github.com/matrix-org/synapse/issues/15969))
- Implement [MSC3814](https://github.com/matrix-org/matrix-spec-proposals/pull/3814), dehydrated devices v2/shrivelled sessions and move [MSC2697](https://github.com/matrix-org/matrix-spec-proposals/pull/2697) behind a config flag. Contributed by Nico from Famedly, H-Shay and poljar. ([\#15929](https://github.com/matrix-org/synapse/issues/15929))
### Bugfixes
- Fix a long-standing bug where remote invites weren't correctly pushed. ([\#15820](https://github.com/matrix-org/synapse/issues/15820))
- Fix background schema updates failing over a large upgrade gap. ([\#15887](https://github.com/matrix-org/synapse/issues/15887))
- Fix a bug introduced in 1.86.0 where Synapse starting with an empty `experimental_features` configuration setting. ([\#15925](https://github.com/matrix-org/synapse/issues/15925))
- Fixed deploy annotations in the provided Grafana dashboard config, so that it shows for any homeserver and not just matrix.org. Contributed by @wrjlewis. ([\#15957](https://github.com/matrix-org/synapse/issues/15957))
- Ensure a long state res does not starve CPU by occasionally yielding to the reactor. ([\#15960](https://github.com/matrix-org/synapse/issues/15960))
- Properly handle redactions of creation events. ([\#15973](https://github.com/matrix-org/synapse/issues/15973))
- Fix a bug where resyncing stale device lists could block responding to federation transactions, and thus delay receiving new data from the remote server. ([\#15975](https://github.com/matrix-org/synapse/issues/15975))
### Improved Documentation
- Better clarify how to run a worker instance (pass both configs). ([\#15921](https://github.com/matrix-org/synapse/issues/15921))
- Improve [the documentation](https://matrix-org.github.io/synapse/v1.89/admin_api/user_admin_api.html#login-as-a-user) for the login as a user admin API. ([\#15938](https://github.com/matrix-org/synapse/issues/15938))
- Fix broken Arch Linux package link. Contributed by @SnipeXandrej. ([\#15981](https://github.com/matrix-org/synapse/issues/15981))
### Deprecations and Removals
- Remove support for calling the `/register` endpoint with an unspecced `user` property for application services. ([\#15928](https://github.com/matrix-org/synapse/issues/15928))
### Internal Changes
- Mark `get_user_in_directory` private since it is only used in tests. Also remove the cache from it. ([\#15884](https://github.com/matrix-org/synapse/issues/15884))
- Document which Python version runs on a given Linux distribution so we can more easily clean up later. ([\#15909](https://github.com/matrix-org/synapse/issues/15909))
- Add details to warning in log when we fail to fetch an alias. ([\#15922](https://github.com/matrix-org/synapse/issues/15922))
- Remove unneeded `__init__`. ([\#15926](https://github.com/matrix-org/synapse/issues/15926))
- Fix bug with read/write lock implementation. This is currently unused so has no observable effects. ([\#15933](https://github.com/matrix-org/synapse/issues/15933), [\#15958](https://github.com/matrix-org/synapse/issues/15958))
- Unbreak the nix development environment by pinning the Rust version to 1.70.0. ([\#15940](https://github.com/matrix-org/synapse/issues/15940))
- Update presence metrics to differentiate remote vs local users. ([\#15952](https://github.com/matrix-org/synapse/issues/15952))
- Stop reading from column `user_id` of table `profiles`. ([\#15955](https://github.com/matrix-org/synapse/issues/15955))
- Build packages for Debian Trixie. ([\#15961](https://github.com/matrix-org/synapse/issues/15961))
- Reduce the amount of state we pull out. ([\#15968](https://github.com/matrix-org/synapse/issues/15968))
- Speed up updating state in large rooms. ([\#15971](https://github.com/matrix-org/synapse/issues/15971))
### Updates to locked dependencies
* Bump anyhow from 1.0.71 to 1.0.72. ([\#15949](https://github.com/matrix-org/synapse/issues/15949))
* Bump click from 8.1.3 to 8.1.6. ([\#15984](https://github.com/matrix-org/synapse/issues/15984))
* Bump cryptography from 41.0.1 to 41.0.2. ([\#15943](https://github.com/matrix-org/synapse/issues/15943))
* Bump jsonschema from 4.17.3 to 4.18.3. ([\#15948](https://github.com/matrix-org/synapse/issues/15948))
* Bump pillow from 9.4.0 to 10.0.0. ([\#15986](https://github.com/matrix-org/synapse/issues/15986))
* Bump prometheus-client from 0.17.0 to 0.17.1. ([\#15945](https://github.com/matrix-org/synapse/issues/15945))
* Bump pydantic from 1.10.10 to 1.10.11. ([\#15946](https://github.com/matrix-org/synapse/issues/15946))
* Bump pygithub from 1.58.2 to 1.59.0. ([\#15834](https://github.com/matrix-org/synapse/issues/15834))
* Bump pyo3-log from 0.8.2 to 0.8.3. ([\#15951](https://github.com/matrix-org/synapse/issues/15951))
* Bump sentry-sdk from 1.26.0 to 1.28.1. ([\#15985](https://github.com/matrix-org/synapse/issues/15985))
* Bump serde_json from 1.0.100 to 1.0.103. ([\#15950](https://github.com/matrix-org/synapse/issues/15950))
* Bump types-pillow from 9.5.0.4 to 10.0.0.1. ([\#15932](https://github.com/matrix-org/synapse/issues/15932))
* Bump types-requests from 2.31.0.1 to 2.31.0.2. ([\#15983](https://github.com/matrix-org/synapse/issues/15983))
* Bump typing-extensions from 4.5.0 to 4.7.1. ([\#15947](https://github.com/matrix-org/synapse/issues/15947))
# Synapse 1.88.0 (2023-07-18)
This release

1
changelog.d/15708.feature
View File

@ -1 +0,0 @@
Add Unix Socket support for HTTP Replication Listeners. Document and provide usage instructions for utilizing Unix sockets in Synapse. Contributed by Jason Little.

1
changelog.d/15820.bugfix
View File

@ -1 +0,0 @@
Fix long-standing bug where remote invites weren't correctly pushed.

1
changelog.d/15884.misc
View File

@ -1 +0,0 @@
Mark `get_user_in_directory` private since it is only used in tests. Also remove the cache from it.

1
changelog.d/15887.misc
View File

@ -1 +0,0 @@
Fix background schema updates failing over a large upgrade gap.

1
changelog.d/15909.misc
View File

@ -1 +0,0 @@
Document which Python version runs on a given Linux distribution so we can more easily clean up later.

1
changelog.d/15911.feature
View File

@ -1 +0,0 @@
Allow `+` in Matrix IDs, per [MSC4009](https://github.com/matrix-org/matrix-spec-proposals/pull/4009).

1
changelog.d/15912.feature
View File

@ -1 +0,0 @@
Support room version 11 from [MSC3820](https://github.com/matrix-org/matrix-spec-proposals/pull/3820).

1
changelog.d/15913.feature
View File

@ -1 +0,0 @@
Allow configuring the set of workers to proxy outbound federation traffic through via `outbound_federation_restricted_to`.

1
changelog.d/15921.doc
View File

@ -1 +0,0 @@
Better clarify how to run a worker instance (pass both configs).

1
changelog.d/15922.misc
View File

@ -1 +0,0 @@
Add details to warning in log when we fail to fetch an alias.

1
changelog.d/15924.feature
View File

@ -1 +0,0 @@
Add Unix Socket support for HTTP Replication Listeners. Document and provide usage instructions for utilizing Unix sockets in Synapse. Contributed by Jason Little.

1
changelog.d/15925.bugfix
View File

@ -1 +0,0 @@
Fix a bug introduced in 1.86.0 where Synapse starting with an empty `experimental_features` configuration setting.

1
changelog.d/15926.misc
View File

@ -1 +0,0 @@
Remove unneeded `__init__`.

1
changelog.d/15928.removal
View File

@ -1 +0,0 @@
Remove support for calling the `/register` endpoint with an unspecced `user` property for application services.

1
changelog.d/15929.feature
View File

@ -1 +0,0 @@
Implement [MSC3814](https://github.com/matrix-org/matrix-spec-proposals/pull/3814), dehydrated devices v2/shrivelled sessions and move [MSC2697](https://github.com/matrix-org/matrix-spec-proposals/pull/2697) behind a config flag. Contributed by Nico from Famedly and H-Shay.

1
changelog.d/15933.misc
View File

@ -1 +0,0 @@
Fix bug with read/write lock implementation. This is currently unused so has no observable effects.

1
changelog.d/15938.doc
View File

@ -1 +0,0 @@
Improve the documentation for the login as a user admin API.

1
changelog.d/15940.misc
View File

@ -1 +0,0 @@
Unbreak the nix development environment by pinning the Rust version to 1.70.0.

1
changelog.d/15952.misc
View File

@ -1 +0,0 @@
Update presence metrics to differentiate remote vs local users.

1
changelog.d/15955.misc
View File

@ -1 +0,0 @@
Stop reading from column `user_id` of table `profiles`.

1
changelog.d/15957.bugfix
View File

@ -1 +0,0 @@
Fixed deploy annotations in the provided Grafana dashboard config, so that it shows for any homeserver and not just matrix.org. Contributed by @wrjlewis.

1
changelog.d/15958.misc
View File

@ -1 +0,0 @@
Fix bug with read/write lock implementation. This is currently unused so has no observable effects.

1
changelog.d/15960.misc
View File

@ -1 +0,0 @@
Ensure a long state res does not starve CPU by occasionally yielding to the reactor.

1
changelog.d/15961.misc
View File

@ -1 +0,0 @@
Build packages for Debian Trixie.

1
changelog.d/15968.misc
View File

@ -1 +0,0 @@
Reduce the amount of state we pull out.

1
changelog.d/15969.feature
View File

@ -1 +0,0 @@
Allow configuring the set of workers to proxy outbound federation traffic through via `outbound_federation_restricted_to`.

1
changelog.d/15971.misc
View File

@ -1 +0,0 @@
Speed up updating state in large rooms.

1
changelog.d/15973.bugfix
View File

@ -1 +0,0 @@
Properly handle redactions of creation events.

1
changelog.d/15975.bugfix
View File

@ -1 +0,0 @@
Fix bug where resyncing stale device lists could block responding to federation transactions, and thus delay receiving new data from the remote server.

1
changelog.d/15981.doc
View File

@ -1 +0,0 @@
Fix broken Arch Linux package link. Contributed by @SnipeXandrej.

6
debian/changelog vendored
View File

@ -1,3 +1,9 @@
matrix-synapse-py3 (1.89.0~rc1) stable; urgency=medium
* New Synapse release 1.89.0rc1.
-- Synapse Packaging team <packages@matrix.org> Tue, 25 Jul 2023 14:31:07 +0200
matrix-synapse-py3 (1.88.0) stable; urgency=medium
* New Synapse release 1.88.0.

2
pyproject.toml
View File

@ -89,7 +89,7 @@ manifest-path = "rust/Cargo.toml"
[tool.poetry]
name = "matrix-synapse"
version = "1.88.0"
version = "1.89.0rc1"
description = "Homeserver for the Matrix decentralised comms protocol"
authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
license = "Apache-2.0"

6
synapse/storage/schema/__init__.py
View File

@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
SCHEMA_VERSION = 78 # remember to update the list below when updating
SCHEMA_VERSION = 79 # remember to update the list below when updating
"""Represents the expectations made by the codebase about the database schema
This should be incremented whenever the codebase changes its requirements on the
@ -106,6 +106,10 @@ Changes in SCHEMA_VERSION = 77
Changes in SCHEMA_VERSION = 78
- Validate check (full_user_id IS NOT NULL) on tables profiles and user_filters
Changes in SCHEMA_VERSION = 79
- Add tables to handle in DB read-write locks.
- Add some mitigations for a painful race between foreground and background updates, cf #15677.
"""

13
synapse/storage/schema/main/delta/78/04_read_write_locks_triggers.sql.postgres → synapse/storage/schema/main/delta/79/03_read_write_locks_triggers.sql.postgres
View File

@ -44,7 +44,7 @@
-- A table to track whether a lock is currently acquired, and if so whether its
-- in read or write mode.
CREATE TABLE worker_read_write_locks_mode (
CREATE TABLE IF NOT EXISTS worker_read_write_locks_mode (
lock_name TEXT NOT NULL,
lock_key TEXT NOT NULL,
-- Whether this lock is in read (false) or write (true) mode
@ -55,14 +55,14 @@ CREATE TABLE worker_read_write_locks_mode (
);
-- Ensure that we can only have one row per lock
CREATE UNIQUE INDEX worker_read_write_locks_mode_key ON worker_read_write_locks_mode (lock_name, lock_key);
CREATE UNIQUE INDEX IF NOT EXISTS worker_read_write_locks_mode_key ON worker_read_write_locks_mode (lock_name, lock_key);
-- We need this (redundant) constraint so that we can have a foreign key
-- constraint against this table.
CREATE UNIQUE INDEX worker_read_write_locks_mode_type ON worker_read_write_locks_mode (lock_name, lock_key, write_lock);
CREATE UNIQUE INDEX IF NOT EXISTS worker_read_write_locks_mode_type ON worker_read_write_locks_mode (lock_name, lock_key, write_lock);
-- A table to track who has currently acquired a given lock.
CREATE TABLE worker_read_write_locks (
CREATE TABLE IF NOT EXISTS worker_read_write_locks (
lock_name TEXT NOT NULL,
lock_key TEXT NOT NULL,
-- We write the instance name to ease manual debugging, we don't ever read
@ -84,9 +84,9 @@ CREATE TABLE worker_read_write_locks (
FOREIGN KEY (lock_name, lock_key, write_lock) REFERENCES worker_read_write_locks_mode (lock_name, lock_key, write_lock)
);
CREATE UNIQUE INDEX worker_read_write_locks_key ON worker_read_write_locks (lock_name, lock_key, token);
CREATE UNIQUE INDEX IF NOT EXISTS worker_read_write_locks_key ON worker_read_write_locks (lock_name, lock_key, token);
-- Ensures that only one instance can acquire a lock in write mode at a time.
CREATE UNIQUE INDEX worker_read_write_locks_write ON worker_read_write_locks (lock_name, lock_key) WHERE write_lock;
CREATE UNIQUE INDEX IF NOT EXISTS worker_read_write_locks_write ON worker_read_write_locks (lock_name, lock_key) WHERE write_lock;
-- Add a foreign key constraint to ensure that if a lock is in
@ -97,5 +97,6 @@ CREATE UNIQUE INDEX worker_read_write_locks_write ON worker_read_write_locks (lo
-- We only add to PostgreSQL as SQLite does not support adding constraints
-- after table creation, and so doesn't support "circular" foreign key
-- constraints.
ALTER TABLE worker_read_write_locks_mode DROP CONSTRAINT IF EXISTS worker_read_write_locks_mode_foreign;
ALTER TABLE worker_read_write_locks_mode ADD CONSTRAINT worker_read_write_locks_mode_foreign
FOREIGN KEY (lock_name, lock_key, token) REFERENCES worker_read_write_locks(lock_name, lock_key, token) DEFERRABLE INITIALLY DEFERRED;

12
synapse/storage/schema/main/delta/78/04_read_write_locks_triggers.sql.sqlite → synapse/storage/schema/main/delta/79/03_read_write_locks_triggers.sql.sqlite
View File

@ -22,7 +22,7 @@
-- A table to track whether a lock is currently acquired, and if so whether its
-- in read or write mode.
CREATE TABLE worker_read_write_locks_mode (
CREATE TABLE IF NOT EXISTS worker_read_write_locks_mode (
lock_name TEXT NOT NULL,
lock_key TEXT NOT NULL,
-- Whether this lock is in read (false) or write (true) mode
@ -38,14 +38,14 @@ CREATE TABLE worker_read_write_locks_mode (
);
-- Ensure that we can only have one row per lock
CREATE UNIQUE INDEX worker_read_write_locks_mode_key ON worker_read_write_locks_mode (lock_name, lock_key);
CREATE UNIQUE INDEX IF NOT EXISTS worker_read_write_locks_mode_key ON worker_read_write_locks_mode (lock_name, lock_key);
-- We need this (redundant) constraint so that we can have a foreign key
-- constraint against this table.
CREATE UNIQUE INDEX worker_read_write_locks_mode_type ON worker_read_write_locks_mode (lock_name, lock_key, write_lock);
CREATE UNIQUE INDEX IF NOT EXISTS worker_read_write_locks_mode_type ON worker_read_write_locks_mode (lock_name, lock_key, write_lock);
-- A table to track who has currently acquired a given lock.
CREATE TABLE worker_read_write_locks (
CREATE TABLE IF NOT EXISTS worker_read_write_locks (
lock_name TEXT NOT NULL,
lock_key TEXT NOT NULL,
-- We write the instance name to ease manual debugging, we don't ever read
@ -67,6 +67,6 @@ CREATE TABLE worker_read_write_locks (
FOREIGN KEY (lock_name, lock_key, write_lock) REFERENCES worker_read_write_locks_mode (lock_name, lock_key, write_lock)
);
CREATE UNIQUE INDEX worker_read_write_locks_key ON worker_read_write_locks (lock_name, lock_key, token);
CREATE UNIQUE INDEX IF NOT EXISTS worker_read_write_locks_key ON worker_read_write_locks (lock_name, lock_key, token);
-- Ensures that only one instance can acquire a lock in write mode at a time.
CREATE UNIQUE INDEX worker_read_write_locks_write ON worker_read_write_locks (lock_name, lock_key) WHERE write_lock;
CREATE UNIQUE INDEX IF NOT EXISTS worker_read_write_locks_write ON worker_read_write_locks (lock_name, lock_key) WHERE write_lock;

6
synapse/storage/schema/main/delta/78/05_mitigate_stream_ordering_update_race.py → synapse/storage/schema/main/delta/79/04_mitigate_stream_ordering_update_race.py
View File

@ -37,17 +37,17 @@ def run_create(
# after the background update has finished
if res:
drop_cse_sql = """
ALTER TABLE current_state_events DROP CONSTRAINT event_stream_ordering_fkey
ALTER TABLE current_state_events DROP CONSTRAINT IF EXISTS event_stream_ordering_fkey
"""
cur.execute(drop_cse_sql)
drop_lcm_sql = """
ALTER TABLE local_current_membership DROP CONSTRAINT event_stream_ordering_fkey
ALTER TABLE local_current_membership DROP CONSTRAINT IF EXISTS event_stream_ordering_fkey
"""
cur.execute(drop_lcm_sql)
drop_rm_sql = """
ALTER TABLE room_memberships DROP CONSTRAINT event_stream_ordering_fkey
ALTER TABLE room_memberships DROP CONSTRAINT IF EXISTS event_stream_ordering_fkey
"""
cur.execute(drop_rm_sql)

0
synapse/storage/schema/main/delta/78/06_read_write_locks_triggers.sql.postgres → synapse/storage/schema/main/delta/79/05_read_write_locks_triggers.sql.postgres
View File

0
synapse/storage/schema/main/delta/78/06_read_write_locks_triggers.sql.sqlite → synapse/storage/schema/main/delta/79/05_read_write_locks_triggers.sql.sqlite
View File