Merge pull request #8858 from matrix-org/rav/sso_uia

UIA: offer only available auth flows
remove unused FakeResponse (#8864 )
2020-12-02 20:06:53 +00:00 · 2020-12-02 18:58:25 +00:00 · 2020-12-02 18:54:15 +00:00 · 2020-12-02 18:54:15 +00:00 · 2020-12-02 18:30:29 +00:00 · 2020-12-02 16:30:01 +00:00
119 changed files with 854 additions and 272 deletions
--- a/CHANGES.md
+++ b/CHANGES.md
@ -1,3 +1,75 @@
+Synapse 1.24.0rc1 (2020-12-02)
+==============================
+
+Features
+--------
+
+- Add admin API for logging in as a user. ([\#8617](https://github.com/matrix-org/synapse/issues/8617))
+- Allow specification of the SAML IdP if the metadata returns multiple IdPs. ([\#8630](https://github.com/matrix-org/synapse/issues/8630))
+- Add support for re-trying generation of a localpart for OpenID Connect mapping providers. ([\#8801](https://github.com/matrix-org/synapse/issues/8801), [\#8855](https://github.com/matrix-org/synapse/issues/8855))
+- Allow the `Date` header through CORS. Contributed by Nicolas Chamo. ([\#8804](https://github.com/matrix-org/synapse/issues/8804))
+- Add a config option, `push.group_by_unread_count`, which controls whether unread message counts in push notifications are defined as "the number of rooms with unread messages" or "total unread messages". ([\#8820](https://github.com/matrix-org/synapse/issues/8820))
+- Add `force_purge` option to delete-room admin api. ([\#8843](https://github.com/matrix-org/synapse/issues/8843))
+
+
+Bugfixes
+--------
+
+- Fix a bug where appservices may be sent an excessive amount of read receipts and presence. Broke in v1.22.0. ([\#8744](https://github.com/matrix-org/synapse/issues/8744))
+- Fix a bug in some federation APIs which could lead to unexpected behaviour if different parameters were set in the URI and the request body. ([\#8776](https://github.com/matrix-org/synapse/issues/8776))
+- Fix a bug where synctl could spawn duplicate copies of a worker. Contributed by Waylon Cude. ([\#8798](https://github.com/matrix-org/synapse/issues/8798))
+- Allow per-room profiles to be used for the server notice user. ([\#8799](https://github.com/matrix-org/synapse/issues/8799))
+- Fix a bug where logging could break after a call to SIGHUP. ([\#8817](https://github.com/matrix-org/synapse/issues/8817))
+- Fix `register_new_matrix_user` failing with "Bad Request" when trailing slash is included in server URL. Contributed by @angdraug. ([\#8823](https://github.com/matrix-org/synapse/issues/8823))
+- Fix a minor long-standing bug in login, where we would offer the `password` login type if a custom auth provider supported it, even if password login was disabled. ([\#8835](https://github.com/matrix-org/synapse/issues/8835))
+- Fix a long-standing bug which caused Synapse to require unspecified parameters during user-interactive authentication. ([\#8848](https://github.com/matrix-org/synapse/issues/8848))
+- Fix a bug introduced in v1.20.0 where the user-agent and IP address reported during user registration for CAS, OpenID Connect, and SAML were of the wrong form. ([\#8784](https://github.com/matrix-org/synapse/issues/8784))
+
+
+Improved Documentation
+----------------------
+
+- Clarify the usecase for a msisdn delegate. Contributed by Adrian Wannenmacher. ([\#8734](https://github.com/matrix-org/synapse/issues/8734))
+- Remove extraneous comma from JSON example in User Admin API docs. ([\#8771](https://github.com/matrix-org/synapse/issues/8771))
+- Update `turn-howto.md` with troubleshooting notes. ([\#8779](https://github.com/matrix-org/synapse/issues/8779))
+- Fix the example on how to set the `Content-Type` header in nginx for the Client Well-Known URI. ([\#8793](https://github.com/matrix-org/synapse/issues/8793))
+- Improve the documentation for the admin API to list all media in a room with respect to encrypted events. ([\#8795](https://github.com/matrix-org/synapse/issues/8795))
+- Update the formatting of the `push` section of the homeserver config file to better align with the [code style guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format). ([\#8818](https://github.com/matrix-org/synapse/issues/8818))
+- Improve documentation how to configure prometheus for workers. ([\#8822](https://github.com/matrix-org/synapse/issues/8822))
+- Update example prometheus console. ([\#8824](https://github.com/matrix-org/synapse/issues/8824))
+
+
+Deprecations and Removals
+-------------------------
+
+- Remove old `/_matrix/client/*/admin` endpoints which were deprecated since Synapse 1.20.0. ([\#8785](https://github.com/matrix-org/synapse/issues/8785))
+- Disable pretty printing JSON responses for curl. Users who want pretty-printed output should use [jq](https://stedolan.github.io/jq/) in combination with curl. Contributed by @tulir. ([\#8833](https://github.com/matrix-org/synapse/issues/8833))
+
+
+Internal Changes
+----------------
+
+- Simplify the way the `HomeServer` object caches its internal attributes. ([\#8565](https://github.com/matrix-org/synapse/issues/8565), [\#8851](https://github.com/matrix-org/synapse/issues/8851))
+- Add an example and documentation for clock skew to the SAML2 sample configuration to allow for clock/time difference between the homserver and IdP. Contributed by @localguru. ([\#8731](https://github.com/matrix-org/synapse/issues/8731))
+- Generalise `RoomMemberHandler._locally_reject_invite` to apply to more flows than just invite. ([\#8751](https://github.com/matrix-org/synapse/issues/8751))
+- Generalise `RoomStore.maybe_store_room_on_invite` to handle other, non-invite membership events. ([\#8754](https://github.com/matrix-org/synapse/issues/8754))
+- Refactor test utilities for injecting HTTP requests. ([\#8757](https://github.com/matrix-org/synapse/issues/8757), [\#8758](https://github.com/matrix-org/synapse/issues/8758), [\#8759](https://github.com/matrix-org/synapse/issues/8759), [\#8760](https://github.com/matrix-org/synapse/issues/8760), [\#8761](https://github.com/matrix-org/synapse/issues/8761), [\#8777](https://github.com/matrix-org/synapse/issues/8777))
+- Consolidate logic between the OpenID Connect and SAML code. ([\#8765](https://github.com/matrix-org/synapse/issues/8765))
+- Use `TYPE_CHECKING` instead of magic `MYPY` variable. ([\#8770](https://github.com/matrix-org/synapse/issues/8770))
+- Add a commandline script to sign arbitrary json objects. ([\#8772](https://github.com/matrix-org/synapse/issues/8772))
+- Minor log line improvements for the SSO mapping code used to generate Matrix IDs from SSO IDs. ([\#8773](https://github.com/matrix-org/synapse/issues/8773))
+- Add additional error checking for OpenID Connect and SAML mapping providers. ([\#8774](https://github.com/matrix-org/synapse/issues/8774), [\#8800](https://github.com/matrix-org/synapse/issues/8800))
+- Add type hints to HTTP abstractions. ([\#8806](https://github.com/matrix-org/synapse/issues/8806), [\#8812](https://github.com/matrix-org/synapse/issues/8812))
+- Remove unnecessary function arguments and add typing to several membership replication classes. ([\#8809](https://github.com/matrix-org/synapse/issues/8809))
+- Optimise the lookup for an invite from another homeserver when trying to reject it. ([\#8815](https://github.com/matrix-org/synapse/issues/8815))
+- Add tests for `password_auth_provider`s. ([\#8819](https://github.com/matrix-org/synapse/issues/8819))
+- Drop redundant database index on `event_json`. ([\#8845](https://github.com/matrix-org/synapse/issues/8845))
+- Simplify `uk.half-shot.msc2778.login.application_service` login handler. ([\#8847](https://github.com/matrix-org/synapse/issues/8847))
+- Refactor `password_auth_provider` support code. ([\#8849](https://github.com/matrix-org/synapse/issues/8849))
+- Add missing `ordering` to background database updates. ([\#8850](https://github.com/matrix-org/synapse/issues/8850))
+- Allow for specifying a room version when creating a room in unit tests via `RestHelper.create_room_as`. ([\#8854](https://github.com/matrix-org/synapse/issues/8854))
+
+
 Synapse 1.23.0 (2020-11-18)
 ===========================

--- a/changelog.d/8565.misc
+++ b/changelog.d/8565.misc
@ -1 +0,0 @@
-Simplify the way the `HomeServer` object caches its internal attributes.
--- a/changelog.d/8617.feature
+++ b/changelog.d/8617.feature
@ -1 +0,0 @@
-Add admin API for logging in as a user.
--- a/changelog.d/8630.feature
+++ b/changelog.d/8630.feature
@ -1 +0,0 @@
-Allow specification of the SAML IdP if the metadata returns multiple IdPs.
--- a/changelog.d/8731.misc
+++ b/changelog.d/8731.misc
@ -1 +0,0 @@
-Add an example and documentation for clock skew to the SAML2 sample configuration to allow for clock/time difference between the homserver and IdP. Contributed by @localguru.
--- a/changelog.d/8734.doc
+++ b/changelog.d/8734.doc
@ -1 +0,0 @@
-Clarify the usecase for an msisdn delegate. Contributed by Adrian Wannenmacher.
--- a/changelog.d/8744.bugfix
+++ b/changelog.d/8744.bugfix
@ -1 +0,0 @@
-Fix a bug where appservices may be sent an excessive amount of read receipts and presence. Broke in v1.22.0. 
--- a/changelog.d/8751.misc
+++ b/changelog.d/8751.misc
@ -1 +0,0 @@
-Generalise `RoomMemberHandler._locally_reject_invite` to apply to more flows than just invite.
--- a/changelog.d/8754.misc
+++ b/changelog.d/8754.misc
@ -1 +0,0 @@
-Generalise `RoomStore.maybe_store_room_on_invite` to handle other, non-invite membership events.
--- a/changelog.d/8757.misc
+++ b/changelog.d/8757.misc
@ -1 +0,0 @@
-Refactor test utilities for injecting HTTP requests.
--- a/changelog.d/8758.misc
+++ b/changelog.d/8758.misc
@ -1 +0,0 @@
-Refactor test utilities for injecting HTTP requests.
--- a/changelog.d/8759.misc
+++ b/changelog.d/8759.misc
@ -1 +0,0 @@
-Refactor test utilities for injecting HTTP requests.
--- a/changelog.d/8760.misc
+++ b/changelog.d/8760.misc
@ -1 +0,0 @@
-Refactor test utilities for injecting HTTP requests.
--- a/changelog.d/8761.misc
+++ b/changelog.d/8761.misc
@ -1 +0,0 @@
- Refactor test utilities for injecting HTTP requests.
--- a/changelog.d/8765.misc
+++ b/changelog.d/8765.misc
@ -1 +0,0 @@
-Consolidate logic between the OpenID Connect and SAML code.
--- a/changelog.d/8770.misc
+++ b/changelog.d/8770.misc
@ -1 +0,0 @@
-Use `TYPE_CHECKING` instead of magic `MYPY` variable.
--- a/changelog.d/8771.doc
+++ b/changelog.d/8771.doc
@ -1 +0,0 @@
-Remove extraneous comma from JSON example in User Admin API docs.
--- a/changelog.d/8772.misc
+++ b/changelog.d/8772.misc
@ -1 +0,0 @@
-Add a commandline script to sign arbitrary json objects.
--- a/changelog.d/8773.misc
+++ b/changelog.d/8773.misc
@ -1 +0,0 @@
-Minor log line improvements for the SSO mapping code used to generate Matrix IDs from SSO IDs.
--- a/changelog.d/8774.misc
+++ b/changelog.d/8774.misc
@ -1 +0,0 @@
-Add additional error checking for OpenID Connect and SAML mapping providers.
--- a/changelog.d/8776.bugfix
+++ b/changelog.d/8776.bugfix
@ -1 +0,0 @@
-Fix a bug in some federation APIs which could lead to unexpected behaviour if different parameters were set in the URI and the request body.
--- a/changelog.d/8777.misc
+++ b/changelog.d/8777.misc
@ -1 +0,0 @@
- Refactor test utilities for injecting HTTP requests.
--- a/changelog.d/8779.doc
+++ b/changelog.d/8779.doc
@ -1 +0,0 @@
-Update `turn-howto.md` with troubleshooting notes.
--- a/changelog.d/8784.misc
+++ b/changelog.d/8784.misc
@ -1 +0,0 @@
-Fix a bug introduced in v1.20.0 where the user-agent and IP address reported during user registration for CAS, OpenID Connect, and SAML were of the wrong form.
--- a/changelog.d/8785.removal
+++ b/changelog.d/8785.removal
@ -1 +0,0 @@
-Remove old `/_matrix/client/*/admin` endpoints which was deprecated since Synapse 1.20.0.
--- a/changelog.d/8793.doc
+++ b/changelog.d/8793.doc
@ -1 +0,0 @@
-Fix the example on how to set the `Content-Type` header in nginx for the Client Well-Known URI.
--- a/changelog.d/8795.doc
+++ b/changelog.d/8795.doc
@ -1 +0,0 @@
-Improve the documentation for the admin API to list all media in a room with respect to encrypted events.
--- a/changelog.d/8798.bugfix
+++ b/changelog.d/8798.bugfix
@ -1 +0,0 @@
-Fix a bug where synctl could spawn duplicate copies of a worker. Contributed by Waylon Cude.
--- a/changelog.d/8799.bugfix
+++ b/changelog.d/8799.bugfix
@ -1 +0,0 @@
-Allow per-room profiles to be used for the server notice user.
--- a/changelog.d/8800.misc
+++ b/changelog.d/8800.misc
@ -1 +0,0 @@
-Add additional error checking for OpenID Connect and SAML mapping providers.
--- a/changelog.d/8801.feature
+++ b/changelog.d/8801.feature
@ -1 +0,0 @@
-Add support for re-trying generation of a localpart for OpenID Connect mapping providers.
--- a/changelog.d/8802.doc
+++ b/changelog.d/8802.doc
@ -0,0 +1 @@
+Fix the "Event persist rate" section of the included grafana dashboard by adding missing prometheus rules.
--- a/changelog.d/8804.feature
+++ b/changelog.d/8804.feature
@ -1 +0,0 @@
-Allow Date header through CORS. Contributed by Nicolas Chamo.
--- a/changelog.d/8806.misc
+++ b/changelog.d/8806.misc
@ -1 +0,0 @@
-Add type hints to HTTP abstractions.
--- a/changelog.d/8809.misc
+++ b/changelog.d/8809.misc
@ -1 +0,0 @@
-Remove unnecessary function arguments and add typing to several membership replication classes.
--- a/changelog.d/8812.misc
+++ b/changelog.d/8812.misc
@ -1 +0,0 @@
-Add type hints to HTTP abstractions.
--- a/changelog.d/8815.misc
+++ b/changelog.d/8815.misc
@ -1 +0,0 @@
-Optimise the lookup for an invite from another homeserver when trying to reject it.
--- a/changelog.d/8817.bugfix
+++ b/changelog.d/8817.bugfix
@ -1 +0,0 @@
-Fix bug where logging could break after a call to SIGHUP.
--- a/changelog.d/8818.doc
+++ b/changelog.d/8818.doc
@ -1 +0,0 @@
-Update the formatting of the `push` section of the homeserver config file to better align with the [code style guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format).
--- a/changelog.d/8819.misc
+++ b/changelog.d/8819.misc
@ -1 +0,0 @@
-Add tests for `password_auth_provider`s.
--- a/changelog.d/8820.feature
+++ b/changelog.d/8820.feature
@ -1 +0,0 @@
-Add a config option, `push.group_by_unread_count`, which controls whether unread message counts in push notifications are defined as "the number of rooms with unread messages" or "total unread messages".
--- a/changelog.d/8821.bugfix
+++ b/changelog.d/8821.bugfix
@ -0,0 +1 @@
+Apply the `federation_ip_range_blacklist` to push and key revocation requests.
--- a/changelog.d/8822.doc
+++ b/changelog.d/8822.doc
@ -1 +0,0 @@
-Improve documentation how to configure prometheus for workers.
--- a/changelog.d/8823.bugfix
+++ b/changelog.d/8823.bugfix
@ -1 +0,0 @@
-Fix `register_new_matrix_user` failing with "Bad Request" when trailing slash is included in server URL. Contributed by @angdraug.
--- a/changelog.d/8824.doc
+++ b/changelog.d/8824.doc
@ -1 +0,0 @@
-Update example prometheus console.
--- a/changelog.d/8827.bugfix
+++ b/changelog.d/8827.bugfix
@ -0,0 +1 @@
+Fix bug where we might not correctly calculate the current state for rooms with multiple extremities.
--- a/changelog.d/8833.removal
+++ b/changelog.d/8833.removal
@ -1 +0,0 @@
-Disable pretty printing JSON responses for curl. Users who want pretty-printed output should use [jq](https://stedolan.github.io/jq/) in combination with curl. Contributed by @tulir.
--- a/changelog.d/8835.bugfix
+++ b/changelog.d/8835.bugfix
@ -1 +0,0 @@
-Fix minor long-standing bug in login, where we would offer the `password` login type if a custom auth provider supported it, even if password login was disabled.
--- a/changelog.d/8837.bugfix
+++ b/changelog.d/8837.bugfix
@ -0,0 +1 @@
+Fix a long standing bug in the register admin endpoint (`/_synapse/admin/v1/register`) when the `mac` field was not provided. The endpoint now properly returns a 400 error. Contributed by @edwargix.
--- a/changelog.d/8843.feature
+++ b/changelog.d/8843.feature
@ -1 +0,0 @@
-Add `force_purge` option to delete-room admin api.
--- a/changelog.d/8845.misc
+++ b/changelog.d/8845.misc
@ -1 +0,0 @@
-Drop redundant database index on `event_json`.
--- a/changelog.d/8847.misc
+++ b/changelog.d/8847.misc
@ -1 +0,0 @@
-Simplify `uk.half-shot.msc2778.login.application_service` login handler.
--- a/changelog.d/8848.bugfix
+++ b/changelog.d/8848.bugfix
@ -1 +0,0 @@
-Fix a long-standing bug which caused Synapse to require unspecified parameters during user-interactive authentication.
--- a/changelog.d/8849.misc
+++ b/changelog.d/8849.misc
@ -1 +0,0 @@
-Refactor `password_auth_provider` support code.
--- a/changelog.d/8850.misc
+++ b/changelog.d/8850.misc
@ -1 +0,0 @@
-Add missing `ordering` to background database updates.
--- a/changelog.d/8851.misc
+++ b/changelog.d/8851.misc
@ -1 +0,0 @@
-Simplify the way the `HomeServer` object caches its internal attributes.
--- a/changelog.d/8854.misc
+++ b/changelog.d/8854.misc
@ -1 +0,0 @@
-Allow for specifying a room version when creating a room in unit tests via `RestHelper.create_room_as`.
--- a/changelog.d/8855.feature
+++ b/changelog.d/8855.feature
@ -1 +0,0 @@
-Add support for re-trying generation of a localpart for OpenID Connect mapping providers.
--- a/changelog.d/8858.bugfix
+++ b/changelog.d/8858.bugfix
@ -0,0 +1 @@
+Fix a long-standing bug on Synapse instances supporting Single-Sign-On, where users would be prompted to enter their password to confirm certain actions, even though they have not set a password.
--- a/changelog.d/8864.misc
+++ b/changelog.d/8864.misc
@ -0,0 +1 @@
+Remove unused `FakeResponse` class from unit tests.
--- a/contrib/prometheus/synapse-v2.rules
+++ b/contrib/prometheus/synapse-v2.rules
@ -58,3 +58,21 @@ groups:
    labels:
      type: "PDU"
    expr: 'synapse_federation_transaction_queue_pending_pdus + 0'
+
+  - record: synapse_storage_events_persisted_by_source_type
+    expr: sum without(type, origin_type, origin_entity) (synapse_storage_events_persisted_events_sep{origin_type="remote"})
+    labels:
+      type: remote
+  - record: synapse_storage_events_persisted_by_source_type
+    expr: sum without(type, origin_type, origin_entity) (synapse_storage_events_persisted_events_sep{origin_entity="*client*",origin_type="local"})
+    labels:
+      type: local
+  - record: synapse_storage_events_persisted_by_source_type
+    expr: sum without(type, origin_type, origin_entity) (synapse_storage_events_persisted_events_sep{origin_entity!="*client*",origin_type="local"})
+    labels:
+      type: bridges
+  - record: synapse_storage_events_persisted_by_event_type
+    expr: sum without(origin_entity, origin_type) (synapse_storage_events_persisted_events_sep)
+  - record: synapse_storage_events_persisted_by_origin
+    expr: sum without(type) (synapse_storage_events_persisted_events_sep)
+
--- a/docs/sample_config.yaml
+++ b/docs/sample_config.yaml
@ -642,17 +642,19 @@ acme:
 #  - nyc.example.com
 #  - syd.example.com

-# Prevent federation requests from being sent to the following
-# blacklist IP address CIDR ranges. If this option is not specified, or
-# specified with an empty list, no ip range blacklist will be enforced.
+# Prevent outgoing requests from being sent to the following blacklisted IP address
+# CIDR ranges. If this option is not specified, or specified with an empty list,
+# no IP range blacklist will be enforced.
 #
-# As of Synapse v1.4.0 this option also affects any outbound requests to identity
-# servers provided by user input.
+# The blacklist applies to the outbound requests for federation, identity servers,
+# push servers, and for checking key validitity for third-party invite events.
 #
 # (0.0.0.0 and :: are always blacklisted, whether or not they are explicitly
 # listed here, since they correspond to unroutable addresses.)
 #
-federation_ip_range_blacklist:
+# This option replaces federation_ip_range_blacklist in Synapse v1.24.0.
+#
+ip_range_blacklist:
  - '127.0.0.0/8'
  - '10.0.0.0/8'
  - '172.16.0.0/12'
--- a/synapse/init.py
+++ b/synapse/init.py
@ -48,7 +48,7 @@ try:
 except ImportError:
    pass

-__version__ = "1.23.0"
+__version__ = "1.24.0rc1"

 if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
    # We import here so that we don't have to install a bunch of deps when
--- a/synapse/app/generic_worker.py
+++ b/synapse/app/generic_worker.py
@ -266,7 +266,6 @@ class GenericWorkerPresence(BasePresenceHandler):
        super().__init__(hs)
        self.hs = hs
        self.is_mine_id = hs.is_mine_id
-        self.http_client = hs.get_simple_http_client()

        self._presence_enabled = hs.config.use_presence

--- a/synapse/config/federation.py
+++ b/synapse/config/federation.py
@ -36,22 +36,30 @@ class FederationConfig(Config):
            for domain in federation_domain_whitelist:
                self.federation_domain_whitelist[domain] = True

-        self.federation_ip_range_blacklist = config.get(
-            "federation_ip_range_blacklist", []
-        )
+        ip_range_blacklist = config.get("ip_range_blacklist", [])

        # Attempt to create an IPSet from the given ranges
        try:
-            self.federation_ip_range_blacklist = IPSet(
-                self.federation_ip_range_blacklist
-            )
+            self.ip_range_blacklist = IPSet(ip_range_blacklist)
+        except Exception as e:
+            raise ConfigError("Invalid range(s) provided in ip_range_blacklist: %s" % e)
+        # Always blacklist 0.0.0.0, ::
+        self.ip_range_blacklist.update(["0.0.0.0", "::"])

-            # Always blacklist 0.0.0.0, ::
-            self.federation_ip_range_blacklist.update(["0.0.0.0", "::"])
+        # The federation_ip_range_blacklist is used for backwards-compatibility
+        # and only applies to federation and identity servers. If it is not given,
+        # default to ip_range_blacklist.
+        federation_ip_range_blacklist = config.get(
+            "federation_ip_range_blacklist", ip_range_blacklist
+        )
+        try:
+            self.federation_ip_range_blacklist = IPSet(federation_ip_range_blacklist)
        except Exception as e:
            raise ConfigError(
                "Invalid range(s) provided in federation_ip_range_blacklist: %s" % e
            )
+        # Always blacklist 0.0.0.0, ::
+        self.federation_ip_range_blacklist.update(["0.0.0.0", "::"])

        federation_metrics_domains = config.get("federation_metrics_domains") or []
        validate_config(
@ -76,17 +84,19 @@ class FederationConfig(Config):
        #  - nyc.example.com
        #  - syd.example.com

-        # Prevent federation requests from being sent to the following
-        # blacklist IP address CIDR ranges. If this option is not specified, or
-        # specified with an empty list, no ip range blacklist will be enforced.
+        # Prevent outgoing requests from being sent to the following blacklisted IP address
+        # CIDR ranges. If this option is not specified, or specified with an empty list,
+        # no IP range blacklist will be enforced.
        #
-        # As of Synapse v1.4.0 this option also affects any outbound requests to identity
-        # servers provided by user input.
+        # The blacklist applies to the outbound requests for federation, identity servers,
+        # push servers, and for checking key validitity for third-party invite events.
        #
        # (0.0.0.0 and :: are always blacklisted, whether or not they are explicitly
        # listed here, since they correspond to unroutable addresses.)
        #
-        federation_ip_range_blacklist:
+        # This option replaces federation_ip_range_blacklist in Synapse v1.24.0.
+        #
+        ip_range_blacklist:
          - '127.0.0.0/8'
          - '10.0.0.0/8'
          - '172.16.0.0/12'
--- a/synapse/crypto/keyring.py
+++ b/synapse/crypto/keyring.py
@ -578,7 +578,7 @@ class PerspectivesKeyFetcher(BaseV2KeyFetcher):
    def __init__(self, hs):
        super().__init__(hs)
        self.clock = hs.get_clock()
-        self.client = hs.get_http_client()
+        self.client = hs.get_federation_http_client()
        self.key_servers = self.config.key_servers

    async def get_keys(self, keys_to_fetch):
@ -748,7 +748,7 @@ class ServerKeyFetcher(BaseV2KeyFetcher):
    def __init__(self, hs):
        super().__init__(hs)
        self.clock = hs.get_clock()
-        self.client = hs.get_http_client()
+        self.client = hs.get_federation_http_client()

    async def get_keys(self, keys_to_fetch):
        """
--- a/synapse/federation/federation_server.py
+++ b/synapse/federation/federation_server.py
@ -845,7 +845,6 @@ class FederationHandlerRegistry:

    def __init__(self, hs: "HomeServer"):
        self.config = hs.config
-        self.http_client = hs.get_simple_http_client()
        self.clock = hs.get_clock()
        self._instance_name = hs.get_instance_name()

--- a/synapse/federation/transport/client.py
+++ b/synapse/federation/transport/client.py
@ -35,7 +35,7 @@ class TransportLayerClient:

    def __init__(self, hs):
        self.server_name = hs.hostname
-        self.client = hs.get_http_client()
+        self.client = hs.get_federation_http_client()

    @log_function
    def get_room_state_ids(self, destination, room_id, event_id):
--- a/synapse/federation/transport/server.py
+++ b/synapse/federation/transport/server.py
Author	SHA1	Message	Date
Richard van der Hoff	ed5172852a	Merge pull request #8858 from matrix-org/rav/sso_uia UIA: offer only available auth flows	2020-12-02 20:06:53 +00:00
Richard van der Hoff	f347f0cd58	remove unused FakeResponse (#8864 )	2020-12-02 18:58:25 +00:00
Richard van der Hoff	935732768c	newsfile	2020-12-02 18:54:15 +00:00
Richard van der Hoff	0bac276890	UIA: offer only available auth flows During user-interactive auth, do not offer password auth to users with no password, nor SSO auth to users with no SSO. Fixes #7559.	2020-12-02 18:54:15 +00:00
Richard van der Hoff	76469898ee	Factor out FakeResponse from test_oidc	2020-12-02 18:30:29 +00:00
Richard van der Hoff	7ea85302f3	fix up various test cases A few test cases were relying on being able to mount non-client servlets on the test resource. it's better to give them their own Resources.	2020-12-02 16:30:01 +00:00
Patrick Cloke	30fba62108	Apply an IP range blacklist to push and key revocation requests. (#8821 ) Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).	2020-12-02 11:09:24 -05:00
Erik Johnston	c5b6abd53d	Correctly handle unpersisted events when calculating auth chain difference. (#8827 ) We do state res with unpersisted events when calculating the new current state of the room, so that should be the only thing impacted. I don't think this is tooooo big of a deal as: 1. the next time a state event happens in the room the current state should correct itself; 2. in the common case all the unpersisted events' auth events will be pulled in by other state, so will still return the correct result (or one which is sufficiently close to not affect the result); and 3. we mostly use the state at an event to do important operations, which isn't affected by this.	2020-12-02 15:22:37 +00:00
Richard van der Hoff	693516e756	Add `create_resource_dict` method to HomeserverTestCase Rather than using a single JsonResource, construct a resource tree, as we do in the prod code, and allow testcases to add extra resources by overriding `create_resource_dict`.	2020-12-02 15:21:00 +00:00
Johanna Dorothea Reichmann	0fed46ebe5	Add missing prometheus rules for persisted events (#8802 ) The official dashboard uses data from these rules, but they were never added to the synapse-v2.rules. They are mentioned in this issue: https://github.com/matrix-org/synapse/issues/7917#issuecomment-661330409, but never got added to the rules. Adding them results in all graphs in the "Event persist rate" section to function as intended. Signed-off-by: Johanna Dorothea Reichmann <transcaffeine@finallycoffee.eu>	2020-12-02 15:18:41 +00:00
David Florness	c4675e1b24	Add additional validation for the admin register endpoint. (#8837 ) Raise a proper 400 error if the `mac` field is missing.	2020-12-02 10:01:15 -05:00
Patrick Cloke	e41720d85f	Minor changes to the CHANGES doc.	2020-12-02 09:17:42 -05:00
Patrick Cloke	c67af840aa	Minor fixes to changelog.	2020-12-02 09:03:12 -05:00
Patrick Cloke	53b12688dd	1.24.0rc1	2020-12-02 08:57:51 -05:00
				`@ -1 +0,0 @@`
				Simplify the way the `HomeServer` object caches its internal attributes.
				`@ -1 +0,0 @@`
				`Allow specification of the SAML IdP if the metadata returns multiple IdPs.`
				`@ -1 +0,0 @@`
				`Add an example and documentation for clock skew to the SAML2 sample configuration to allow for clock/time difference between the homserver and IdP. Contributed by @localguru.`
				`@ -1 +0,0 @@`
				`Clarify the usecase for an msisdn delegate. Contributed by Adrian Wannenmacher.`
				`@ -1 +0,0 @@`
				`Fix a bug where appservices may be sent an excessive amount of read receipts and presence. Broke in v1.22.0.`
				`@ -1 +0,0 @@`
				Generalise `RoomMemberHandler._locally_reject_invite` to apply to more flows than just invite.
				`@ -1 +0,0 @@`
				Generalise `RoomStore.maybe_store_room_on_invite` to handle other, non-invite membership events.
				`@ -1 +0,0 @@`
				`Refactor test utilities for injecting HTTP requests.`
				`@ -1 +0,0 @@`
				`Consolidate logic between the OpenID Connect and SAML code.`