Merge branch 'release-v1.40' of github.com:matrix-org/synapse into matrix-org-hotfixes

CHANGES.md

@@ -1,3 +1,19 @@
+Synapse 1.40.0rc2 (2021-08-04)
+==============================
+
+Bugfixes
+--------
+
+- Fix the `PeriodicallyFlushingMemoryHandler` inhibiting application shutdown because of its background thread. ([\#10517](https://github.com/matrix-org/synapse/issues/10517))
+- Fix a bug introduced in Synapse v1.40.0rc1 that could cause Synapse to respond with an error when clients would update read receipts. ([\#10531](https://github.com/matrix-org/synapse/issues/10531))
+
+
+Internal Changes
+----------------
+
+- Fix release script to open the correct URL for the release. ([\#10516](https://github.com/matrix-org/synapse/issues/10516))
+
+
 Synapse 1.40.0rc1 (2021-08-03)
 ==============================
 

changelog.d/10449.bugfix

@@ -0,0 +1 @@
+Mark the experimental room version from [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) as unstable.

changelog.d/10449.feature

@@ -0,0 +1 @@
+Support [MSC3289: room version 8](https://github.com/matrix-org/matrix-doc/pull/3289).

changelog.d/10516.misc

@@ -1 +0,0 @@
-Fix release script to open correct URL for the release.

changelog.d/10517.bugfix

@@ -1 +0,0 @@
-Fix the `PeriodicallyFlushingMemoryHandler` inhibiting application shutdown because of its background thread.

changelog.d/10531.bugfix

@@ -1 +0,0 @@
-Fix a bug introduced in Synapse v1.40.0rc1 that would cause Synapse to respond with an error when clients would update their read receipts.

changelog.d/10543.doc

@@ -0,0 +1 @@
+Fix broken links in `upgrade.md`. Contributed by @dklimpel.

debian/changelog

@@ -1,3 +1,9 @@
+matrix-synapse-py3 (1.40.0~rc2) stable; urgency=medium
+
+  * New synapse release 1.40.0~rc2.
+
+ -- Synapse Packaging team <packages@matrix.org>  Wed, 04 Aug 2021 17:08:55 +0100
+
 matrix-synapse-py3 (1.40.0~rc1) stable; urgency=medium
 
   [ Richard van der Hoff ]

docs/upgrade.md

@@ -142,9 +142,9 @@ SQLite databases are unaffected by this change.
 
 The current spam checker interface is deprecated in favour of a new generic modules system.
 Authors of spam checker modules can refer to [this
-documentation](https://matrix-org.github.io/synapse/develop/modules.html#porting-an-existing-module-that-uses-the-old-interface)
+documentation](modules.md#porting-an-existing-module-that-uses-the-old-interface)
 to update their modules. Synapse administrators can refer to [this
-documentation](https://matrix-org.github.io/synapse/develop/modules.html#using-modules)
+documentation](modules.md#using-modules)
 to update their configuration once the modules they are using have been updated.
 
 We plan to remove support for the current spam checker interface in August 2021.
@@ -217,8 +217,7 @@ Instructions for doing so are provided
 
 ## Dropping support for old Python, Postgres and SQLite versions
 
-In line with our [deprecation
+In line with our [deprecation policy](deprecation_policy.md),
-policy](https://github.com/matrix-org/synapse/blob/release-v1.32.0/docs/deprecation_policy.md),
 we've dropped support for Python 3.5 and PostgreSQL 9.5, as they are no
 longer supported upstream.
 
@@ -231,8 +230,7 @@ The deprecated v1 "list accounts" admin API
 (`GET /_synapse/admin/v1/users/<user_id>`) has been removed in this
 version.
 
-The [v2 list accounts
+The [v2 list accounts API](admin_api/user_admin_api.md#list-accounts)
-API](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/user_admin_api.rst#list-accounts)
 has been available since Synapse 1.7.0 (2019-12-13), and is accessible
 under `GET /_synapse/admin/v2/users`.
 
@@ -267,7 +265,7 @@ by the client.
 
 Synapse also requires the [Host]{.title-ref} header to be preserved.
 
-See the [reverse proxy documentation](../reverse_proxy.md), where the
+See the [reverse proxy documentation](reverse_proxy.md), where the
 example configurations have been updated to show how to set these
 headers.
 
@@ -286,7 +284,7 @@ identity providers:
     `[synapse public baseurl]/_synapse/client/oidc/callback` to the list
     of permitted "redirect URIs" at the identity provider.
 
-    See the [OpenID docs](../openid.md) for more information on setting
+    See the [OpenID docs](openid.md) for more information on setting
     up OpenID Connect.
 
 -   If your server is configured for single sign-on via a SAML2 identity
@@ -486,8 +484,7 @@ lock down external access to the Admin API endpoints.
 This release deprecates use of the `structured: true` logging
 configuration for structured logging. If your logging configuration
 contains `structured: true` then it should be modified based on the
-[structured logging
+[structured logging documentation](structured_logging.md).
-documentation](../structured_logging.md).
 
 The `structured` and `drains` logging options are now deprecated and
 should be replaced by standard logging configuration of `handlers` and
@@ -517,14 +514,13 @@ acts the same as the `http_client` argument previously passed to
 
 ## Forwarding `/_synapse/client` through your reverse proxy
 
-The [reverse proxy
+The [reverse proxy documentation](reverse_proxy.md)
-documentation](https://github.com/matrix-org/synapse/blob/develop/docs/reverse_proxy.md)
 has been updated to include reverse proxy directives for
 `/_synapse/client/*` endpoints. As the user password reset flow now uses
 endpoints under this prefix, **you must update your reverse proxy
 configurations for user password reset to work**.
 
-Additionally, note that the [Synapse worker documentation](https://github.com/matrix-org/synapse/blob/develop/docs/workers.md) has been updated to
+Additionally, note that the [Synapse worker documentation](workers.md) has been updated to
 
 :   state that the `/_synapse/client/password_reset/email/submit_token`
     endpoint can be handled
@@ -588,7 +584,7 @@ updated.
 When setting up worker processes, we now recommend the use of a Redis
 server for replication. **The old direct TCP connection method is
 deprecated and will be removed in a future release.** See
-[workers](../workers.md) for more details.
+[workers](workers.md) for more details.
 
 # Upgrading to v1.14.0
 
@@ -720,8 +716,7 @@ participating in many rooms.
     omitting the `CONCURRENTLY` keyword. Note however that this
     operation may in itself cause Synapse to stop running for some time.
     Synapse admins are reminded that [SQLite is not recommended for use
-    outside a test
+    outside a test environment](postgres.md).
-    environment](https://github.com/matrix-org/synapse/blob/master/README.rst#using-postgresql).
 
 3.  Once the index has been created, the `SELECT` query in step 1 above
     should complete quickly. It is therefore safe to upgrade to Synapse
@@ -739,7 +734,7 @@ participating in many rooms.
 Synapse will now log a warning on start up if used with a PostgreSQL
 database that has a non-recommended locale set.
 
-See [Postgres](../postgres.md) for details.
+See [Postgres](postgres.md) for details.
 
 # Upgrading to v1.8.0
 
@@ -856,8 +851,8 @@ section headed `email`, and be sure to have at least the
 You may also need to set `smtp_user`, `smtp_pass`, and
 `require_transport_security`.
 
-See the [sample configuration file](docs/sample_config.yaml) for more
+See the [sample configuration file](usage/configuration/homeserver_sample_config.md)
-details on these settings.
+for more details on these settings.
 
 #### Delegate email to an identity server
 
@@ -959,7 +954,7 @@ back to v1.3.1, subject to the following:
 
 Some counter metrics have been renamed, with the old names deprecated.
 See [the metrics
-documentation](../metrics-howto.md#renaming-of-metrics--deprecation-of-old-names-in-12)
+documentation](metrics-howto.md#renaming-of-metrics--deprecation-of-old-names-in-12)
 for details.
 
 # Upgrading to v1.1.0
@@ -995,7 +990,7 @@ more details on upgrading your database.
 Synapse v1.0 is the first release to enforce validation of TLS
 certificates for the federation API. It is therefore essential that your
 certificates are correctly configured. See the
-[FAQ](../MSC1711_certificates_FAQ.md) for more information.
+[FAQ](MSC1711_certificates_FAQ.md) for more information.
 
 Note, v1.0 installations will also no longer be able to federate with
 servers that have not correctly configured their certificates.
@@ -1010,8 +1005,8 @@ ways:-
 -   Configure a whitelist of server domains to trust via
     `federation_certificate_verification_whitelist`.
 
-See the [sample configuration file](docs/sample_config.yaml) for more
+See the [sample configuration file](usage/configuration/homeserver_sample_config.md)
-details on these settings.
+for more details on these settings.
 
 ## Email
 
@@ -1036,8 +1031,8 @@ If you are absolutely certain that you wish to continue using an
 identity server for password resets, set
 `trust_identity_server_for_password_resets` to `true`.
 
-See the [sample configuration file](docs/sample_config.yaml) for more
+See the [sample configuration file](usage/configuration/homeserver_sample_config.md)
-details on these settings.
+for more details on these settings.
 
 ## New email templates
 
@@ -1057,11 +1052,11 @@ sent to them.
 
 Please be aware that, before Synapse v1.0 is released around March 2019,
 you will need to replace any self-signed certificates with those
-verified by a root CA. Information on how to do so can be found at [the
+verified by a root CA. Information on how to do so can be found at the
-ACME docs](../ACME.md).
+ACME docs.
 
 For more information on configuring TLS certificates see the
-[FAQ](../MSC1711_certificates_FAQ.md).
+[FAQ](MSC1711_certificates_FAQ.md).
 
 # Upgrading to v0.34.0
 

synapse/__init__.py

@@ -47,7 +47,7 @@ try:
 except ImportError:
     pass
 
-__version__ = "1.40.0rc1"
+__version__ = "1.40.0rc2"
 
 if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
     # We import here so that we don't have to install a bunch of deps when

synapse/api/constants.py

@@ -62,7 +62,7 @@ class JoinRules:
     INVITE = "invite"
     PRIVATE = "private"
     # As defined for MSC3083.
-    MSC3083_RESTRICTED = "restricted"
+    RESTRICTED = "restricted"
 
 
 class RestrictedJoinRuleTypes:

synapse/api/room_versions.py

@@ -177,20 +177,6 @@ class RoomVersions:
         msc2403_knocking=False,
         msc2716_historical=False,
     )
-    MSC3083 = RoomVersion(
-        "org.matrix.msc3083.v2",
-        RoomDisposition.UNSTABLE,
-        EventFormatVersions.V3,
-        StateResolutionVersions.V2,
-        enforce_key_validity=True,
-        special_case_aliases_auth=False,
-        strict_canonicaljson=True,
-        limit_notifications_power_levels=True,
-        msc2176_redaction_rules=False,
-        msc3083_join_rules=True,
-        msc2403_knocking=False,
-        msc2716_historical=False,
-    )
     V7 = RoomVersion(
         "7",
         RoomDisposition.STABLE,
@@ -207,7 +193,7 @@ class RoomVersions:
     )
     MSC2716 = RoomVersion(
         "org.matrix.msc2716",
-        RoomDisposition.STABLE,
+        RoomDisposition.UNSTABLE,
         EventFormatVersions.V3,
         StateResolutionVersions.V2,
         enforce_key_validity=True,
@@ -219,6 +205,20 @@ class RoomVersions:
         msc2403_knocking=True,
         msc2716_historical=True,
     )
+    V8 = RoomVersion(
+        "8",
+        RoomDisposition.STABLE,
+        EventFormatVersions.V3,
+        StateResolutionVersions.V2,
+        enforce_key_validity=True,
+        special_case_aliases_auth=False,
+        strict_canonicaljson=True,
+        limit_notifications_power_levels=True,
+        msc2176_redaction_rules=False,
+        msc3083_join_rules=True,
+        msc2403_knocking=True,
+        msc2716_historical=False,
+    )
 
 
 KNOWN_ROOM_VERSIONS: Dict[str, RoomVersion] = {
@@ -231,9 +231,9 @@ KNOWN_ROOM_VERSIONS: Dict[str, RoomVersion] = {
         RoomVersions.V5,
         RoomVersions.V6,
         RoomVersions.MSC2176,
-        RoomVersions.MSC3083,
         RoomVersions.V7,
         RoomVersions.MSC2716,
+        RoomVersions.V8,
     )
 }
 

synapse/event_auth.py

@@ -370,10 +370,7 @@ def _is_membership_change_allowed(
             raise AuthError(403, "You are banned from this room")
         elif join_rule == JoinRules.PUBLIC:
             pass
-        elif (
+        elif room_version.msc3083_join_rules and join_rule == JoinRules.RESTRICTED:
-            room_version.msc3083_join_rules
-            and join_rule == JoinRules.MSC3083_RESTRICTED
-        ):
             # This is the same as public, but the event must contain a reference
             # to the server who authorised the join. If the event does not contain
             # the proper content it is rejected.

synapse/handlers/event_auth.py

@@ -240,7 +240,7 @@ class EventAuthHandler:
 
         # If the join rule is not restricted, this doesn't apply.
         join_rules_event = await self._store.get_event(join_rules_event_id)
-        return join_rules_event.content.get("join_rule") == JoinRules.MSC3083_RESTRICTED
+        return join_rules_event.content.get("join_rule") == JoinRules.RESTRICTED
 
     async def get_rooms_that_allow_join(
         self, state_ids: StateMap[str]

tests/events/test_utils.py

@@ -341,7 +341,7 @@ class PruneEventTestCase(unittest.TestCase):
                 "signatures": {},
                 "unsigned": {},
             },
-            room_version=RoomVersions.MSC3083,
+            room_version=RoomVersions.V8,
         )
 
 

tests/handlers/test_space_summary.py

@@ -231,13 +231,13 @@ class SpaceSummaryTestCase(unittest.HomeserverTestCase):
         invited_room = self._create_room_with_join_rule(JoinRules.INVITE)
         self.helper.invite(invited_room, targ=user2, tok=self.token)
         restricted_room = self._create_room_with_join_rule(
-            JoinRules.MSC3083_RESTRICTED,
+            JoinRules.RESTRICTED,
-            room_version=RoomVersions.MSC3083.identifier,
+            room_version=RoomVersions.V8.identifier,
             allow=[],
         )
         restricted_accessible_room = self._create_room_with_join_rule(
-            JoinRules.MSC3083_RESTRICTED,
+            JoinRules.RESTRICTED,
-            room_version=RoomVersions.MSC3083.identifier,
+            room_version=RoomVersions.V8.identifier,
             allow=[
                 {
                     "type": RestrictedJoinRuleTypes.ROOM_MEMBERSHIP,
@@ -459,13 +459,13 @@ class SpaceSummaryTestCase(unittest.HomeserverTestCase):
                 {
                     "room_id": restricted_room,
                     "world_readable": False,
-                    "join_rules": JoinRules.MSC3083_RESTRICTED,
+                    "join_rules": JoinRules.RESTRICTED,
                     "allowed_spaces": [],
                 },
                 {
                     "room_id": restricted_accessible_room,
                     "world_readable": False,
-                    "join_rules": JoinRules.MSC3083_RESTRICTED,
+                    "join_rules": JoinRules.RESTRICTED,
                     "allowed_spaces": [self.room],
                 },
                 {

tests/test_event_auth.py

@@ -384,7 +384,7 @@ class EventAuthTestCase(unittest.TestCase):
             },
         )
         event_auth.check(
-            RoomVersions.MSC3083,
+            RoomVersions.V8,
             authorised_join_event,
             auth_events,
             do_sig_check=False,
@@ -400,7 +400,7 @@ class EventAuthTestCase(unittest.TestCase):
             "@inviter:foo.test"
         )
         event_auth.check(
-            RoomVersions.MSC3083,
+            RoomVersions.V8,
             _join_event(
                 pleb,
                 additional_content={
@@ -414,7 +414,7 @@ class EventAuthTestCase(unittest.TestCase):
         # A join which is missing an authorised server is rejected.
         with self.assertRaises(AuthError):
             event_auth.check(
-                RoomVersions.MSC3083,
+                RoomVersions.V8,
                 _join_event(pleb),
                 auth_events,
                 do_sig_check=False,
@@ -427,7 +427,7 @@ class EventAuthTestCase(unittest.TestCase):
         )
         with self.assertRaises(AuthError):
             event_auth.check(
-                RoomVersions.MSC3083,
+                RoomVersions.V8,
                 _join_event(
                     pleb,
                     additional_content={
@@ -442,7 +442,7 @@ class EventAuthTestCase(unittest.TestCase):
         # *would* be valid, but is sent be a different user.)
         with self.assertRaises(AuthError):
             event_auth.check(
-                RoomVersions.MSC3083,
+                RoomVersions.V8,
                 _member_event(
                     pleb,
                     "join",
@@ -459,7 +459,7 @@ class EventAuthTestCase(unittest.TestCase):
         auth_events[("m.room.member", pleb)] = _member_event(pleb, "ban")
         with self.assertRaises(AuthError):
             event_auth.check(
-                RoomVersions.MSC3083,
+                RoomVersions.V8,
                 authorised_join_event,
                 auth_events,
                 do_sig_check=False,
@@ -468,7 +468,7 @@ class EventAuthTestCase(unittest.TestCase):
         # A user who left can re-join.
         auth_events[("m.room.member", pleb)] = _member_event(pleb, "leave")
         event_auth.check(
-            RoomVersions.MSC3083,
+            RoomVersions.V8,
             authorised_join_event,
             auth_events,
             do_sig_check=False,
@@ -478,7 +478,7 @@ class EventAuthTestCase(unittest.TestCase):
         # be authorised since the user is already joined.)
         auth_events[("m.room.member", pleb)] = _member_event(pleb, "join")
         event_auth.check(
-            RoomVersions.MSC3083,
+            RoomVersions.V8,
             _join_event(pleb),
             auth_events,
             do_sig_check=False,
@@ -490,7 +490,7 @@ class EventAuthTestCase(unittest.TestCase):
             pleb, "invite", sender=creator
         )
         event_auth.check(
-            RoomVersions.MSC3083,
+            RoomVersions.V8,
             _join_event(pleb),
             auth_events,
             do_sig_check=False,