Merge branch 'develop' into matrix-org-hotfixes

CHANGES.md

@@ -1,3 +1,104 @@
+Synapse 1.21.0rc1 (2020-10-01)
+==============================
+
+Features
+--------
+
+- Require the user to confirm that their password should be reset after clicking the email confirmation link. ([\#8004](https://github.com/matrix-org/synapse/issues/8004))
+- Add an admin API `GET /_synapse/admin/v1/event_reports` to read entries of table `event_reports`. Contributed by @dklimpel. ([\#8217](https://github.com/matrix-org/synapse/issues/8217))
+- Consolidate the SSO error template across all configuration. ([\#8248](https://github.com/matrix-org/synapse/issues/8248), [\#8405](https://github.com/matrix-org/synapse/issues/8405))
+- Add a configuration option to specify a whitelist of domains that a user can be redirected to after validating their email or phone number. ([\#8275](https://github.com/matrix-org/synapse/issues/8275), [\#8417](https://github.com/matrix-org/synapse/issues/8417))
+- Add experimental support for sharding event persister. ([\#8294](https://github.com/matrix-org/synapse/issues/8294), [\#8387](https://github.com/matrix-org/synapse/issues/8387), [\#8396](https://github.com/matrix-org/synapse/issues/8396), [\#8419](https://github.com/matrix-org/synapse/issues/8419))
+- Add the room topic and avatar to the room details admin API. ([\#8305](https://github.com/matrix-org/synapse/issues/8305))
+- Add an admin API for querying rooms where a user is a member. Contributed by @dklimpel. ([\#8306](https://github.com/matrix-org/synapse/issues/8306))
+- Add `uk.half-shot.msc2778.login.application_service` login type to allow appservices to login. ([\#8320](https://github.com/matrix-org/synapse/issues/8320))
+- Add a configuration option that allows existing users to log in with OpenID Connect. Contributed by @BBBSnowball and @OmmyZhang. ([\#8345](https://github.com/matrix-org/synapse/issues/8345))
+- Add prometheus metrics for replication requests. ([\#8406](https://github.com/matrix-org/synapse/issues/8406))
+- Support passing additional single sign-on parameters to the client. ([\#8413](https://github.com/matrix-org/synapse/issues/8413))
+- Add experimental reporting of metrics on expensive rooms for state-resolution. ([\#8420](https://github.com/matrix-org/synapse/issues/8420))
+- Add experimental prometheus metric to track numbers of "large" rooms for state resolutiom. ([\#8425](https://github.com/matrix-org/synapse/issues/8425))
+- Add prometheus metrics to track federation delays. ([\#8430](https://github.com/matrix-org/synapse/issues/8430))
+
+
+Bugfixes
+--------
+
+- Fix a bug in the media repository where remote thumbnails with the same size but different crop methods would overwrite each other. Contributed by @deepbluev7. ([\#7124](https://github.com/matrix-org/synapse/issues/7124))
+- Fix inconsistent handling of non-existent push rules, and stop tracking the `enabled` state of removed push rules. ([\#7796](https://github.com/matrix-org/synapse/issues/7796))
+- Fix a longstanding bug when storing a media file with an empty `upload_name`. ([\#7905](https://github.com/matrix-org/synapse/issues/7905))
+- Fix messages not being sent over federation until an event is sent into the same room. ([\#8230](https://github.com/matrix-org/synapse/issues/8230), [\#8247](https://github.com/matrix-org/synapse/issues/8247), [\#8258](https://github.com/matrix-org/synapse/issues/8258), [\#8272](https://github.com/matrix-org/synapse/issues/8272), [\#8322](https://github.com/matrix-org/synapse/issues/8322))
+- Fix a longstanding bug where files that could not be thumbnailed would result in an Internal Server Error. ([\#8236](https://github.com/matrix-org/synapse/issues/8236), [\#8435](https://github.com/matrix-org/synapse/issues/8435))
+- Upgrade minimum version of `canonicaljson` to version 1.4.0, to fix an unicode encoding issue. ([\#8262](https://github.com/matrix-org/synapse/issues/8262))
+- Fix longstanding bug which could lead to incomplete database upgrades on SQLite. ([\#8265](https://github.com/matrix-org/synapse/issues/8265))
+- Fix stack overflow when stderr is redirected to the logging system, and the logging system encounters an error. ([\#8268](https://github.com/matrix-org/synapse/issues/8268))
+- Fix a bug which cause the logging system to report errors, if `DEBUG` was enabled and no `context` filter was applied. ([\#8278](https://github.com/matrix-org/synapse/issues/8278))
+- Fix edge case where push could get delayed for a user until a later event was pushed. ([\#8287](https://github.com/matrix-org/synapse/issues/8287))
+- Fix fetching malformed events from remote servers. ([\#8324](https://github.com/matrix-org/synapse/issues/8324))
+- Fix `UnboundLocalError` from occuring when appservices send a malformed register request. ([\#8329](https://github.com/matrix-org/synapse/issues/8329))
+- Don't send push notifications to expired user accounts. ([\#8353](https://github.com/matrix-org/synapse/issues/8353))
+- Fix a regression in v1.19.0 with reactivating users through the admin API. ([\#8362](https://github.com/matrix-org/synapse/issues/8362))
+- Fix a bug where during device registration the length of the device name wasn't limited. ([\#8364](https://github.com/matrix-org/synapse/issues/8364))
+- Include `guest_access` in the fields that are checked for null bytes when updating `room_stats_state`. Broke in v1.7.2. ([\#8373](https://github.com/matrix-org/synapse/issues/8373))
+- Fix theoretical race condition where events are not sent down `/sync` if the synchrotron worker is restarted without restarting other workers. ([\#8374](https://github.com/matrix-org/synapse/issues/8374))
+- Fix a bug which could cause errors in rooms with malformed membership events, on servers using sqlite. ([\#8385](https://github.com/matrix-org/synapse/issues/8385))
+- Fix "Re-starting finished log context" warning when receiving an event we already had over federation. ([\#8398](https://github.com/matrix-org/synapse/issues/8398))
+- Fix incorrect handling of timeouts on outgoing HTTP requests. ([\#8400](https://github.com/matrix-org/synapse/issues/8400))
+- Fix a regression in v1.20.0 in the `synapse_port_db` script regarding the `ui_auth_sessions_ips` table. ([\#8410](https://github.com/matrix-org/synapse/issues/8410))
+- Remove unnecessary 3PID registration check when resetting password via an email address. Bug introduced in v0.34.0rc2. ([\#8414](https://github.com/matrix-org/synapse/issues/8414))
+
+
+Improved Documentation
+----------------------
+
+- Add `/_synapse/client` to the reverse proxy documentation. ([\#8227](https://github.com/matrix-org/synapse/issues/8227))
+- Add note to the reverse proxy settings documentation about disabling Apache's mod_security2. Contributed by Julian Fietkau (@jfietkau). ([\#8375](https://github.com/matrix-org/synapse/issues/8375))
+- Improve description of `server_name` config option in `homserver.yaml`. ([\#8415](https://github.com/matrix-org/synapse/issues/8415))
+
+
+Deprecations and Removals
+-------------------------
+
+- Drop support for `prometheus_client` older than 0.4.0. ([\#8426](https://github.com/matrix-org/synapse/issues/8426))
+
+
+Internal Changes
+----------------
+
+- Fix tests on distros which disable TLSv1.0. Contributed by @danc86. ([\#8208](https://github.com/matrix-org/synapse/issues/8208))
+- Simplify the distributor code to avoid unnecessary work. ([\#8216](https://github.com/matrix-org/synapse/issues/8216))
+- Remove the `populate_stats_process_rooms_2` background job and restore functionality to `populate_stats_process_rooms`. ([\#8243](https://github.com/matrix-org/synapse/issues/8243))
+- Clean up type hints for `PaginationConfig`. ([\#8250](https://github.com/matrix-org/synapse/issues/8250), [\#8282](https://github.com/matrix-org/synapse/issues/8282))
+- Track the latest event for every destination and room for catch-up after federation outage. ([\#8256](https://github.com/matrix-org/synapse/issues/8256))
+- Fix non-user visible bug in implementation of `MultiWriterIdGenerator.get_current_token_for_writer`. ([\#8257](https://github.com/matrix-org/synapse/issues/8257))
+- Switch to the JSON implementation from the standard library. ([\#8259](https://github.com/matrix-org/synapse/issues/8259))
+- Add type hints to `synapse.util.async_helpers`. ([\#8260](https://github.com/matrix-org/synapse/issues/8260))
+- Simplify tests that mock asynchronous functions. ([\#8261](https://github.com/matrix-org/synapse/issues/8261))
+- Add type hints to `StreamToken` and `RoomStreamToken` classes. ([\#8279](https://github.com/matrix-org/synapse/issues/8279))
+- Change `StreamToken.room_key` to be a `RoomStreamToken` instance. ([\#8281](https://github.com/matrix-org/synapse/issues/8281))
+- Refactor notifier code to correctly use the max event stream position. ([\#8288](https://github.com/matrix-org/synapse/issues/8288))
+- Use slotted classes where possible. ([\#8296](https://github.com/matrix-org/synapse/issues/8296))
+- Support testing the local Synapse checkout against the [Complement homeserver test suite](https://github.com/matrix-org/complement/). ([\#8317](https://github.com/matrix-org/synapse/issues/8317))
+- Update outdated usages of `metaclass` to python 3 syntax. ([\#8326](https://github.com/matrix-org/synapse/issues/8326))
+- Move lint-related dependencies to package-extra field, update CONTRIBUTING.md to utilise this. ([\#8330](https://github.com/matrix-org/synapse/issues/8330), [\#8377](https://github.com/matrix-org/synapse/issues/8377))
+- Use the `admin_patterns` helper in additional locations. ([\#8331](https://github.com/matrix-org/synapse/issues/8331))
+- Fix test logging to allow braces in log output. ([\#8335](https://github.com/matrix-org/synapse/issues/8335))
+- Remove `__future__` imports related to Python 2 compatibility. ([\#8337](https://github.com/matrix-org/synapse/issues/8337))
+- Simplify `super()` calls to Python 3 syntax. ([\#8344](https://github.com/matrix-org/synapse/issues/8344))
+- Fix bad merge from `release-v1.20.0` branch to `develop`. ([\#8354](https://github.com/matrix-org/synapse/issues/8354))
+- Factor out a `_send_dummy_event_for_room` method. ([\#8370](https://github.com/matrix-org/synapse/issues/8370))
+- Improve logging of state resolution. ([\#8371](https://github.com/matrix-org/synapse/issues/8371))
+- Add type annotations to `SimpleHttpClient`. ([\#8372](https://github.com/matrix-org/synapse/issues/8372))
+- Refactor ID generators to use `async with` syntax. ([\#8383](https://github.com/matrix-org/synapse/issues/8383))
+- Add `EventStreamPosition` type. ([\#8388](https://github.com/matrix-org/synapse/issues/8388))
+- Create a mechanism for marking tests "logcontext clean". ([\#8399](https://github.com/matrix-org/synapse/issues/8399))
+- A pair of tiny cleanups in the federation request code. ([\#8401](https://github.com/matrix-org/synapse/issues/8401))
+- Add checks on startup that PostgreSQL sequences are consistent with their associated tables. ([\#8402](https://github.com/matrix-org/synapse/issues/8402))
+- Do not include appservice users when calculating the total MAU for a server. ([\#8404](https://github.com/matrix-org/synapse/issues/8404))
+- Typing fixes for `synapse.handlers.federation`. ([\#8422](https://github.com/matrix-org/synapse/issues/8422))
+- Various refactors to simplify stream token handling. ([\#8423](https://github.com/matrix-org/synapse/issues/8423))
+- Make stream token serializing/deserializing async. ([\#8427](https://github.com/matrix-org/synapse/issues/8427))
+
+
 Synapse 1.20.1 (2020-09-24)
 ===========================
 

changelog.d/7124.bugfix

@@ -1 +0,0 @@
-Fix a bug in the media repository where remote thumbnails with the same size but different crop methods would overwrite each other. Contributed by @deepbluev7.

changelog.d/7796.bugfix

@@ -1 +0,0 @@
-Fix inconsistent handling of non-existent push rules, and stop tracking the `enabled` state of removed push rules.

changelog.d/7905.bugfix

@@ -1 +0,0 @@
-Fix a longstanding bug when storing a media file with an empty `upload_name`.

changelog.d/8004.feature

@@ -1 +0,0 @@
-Require the user to confirm that their password should be reset after clicking the email confirmation link.

changelog.d/8208.misc

@@ -1 +0,0 @@
-Fix tests on distros which disable TLSv1.0. Contributed by @danc86.

changelog.d/8216.misc

@@ -1 +0,0 @@
-Simplify the distributor code to avoid unnecessary work.

changelog.d/8217.feature

@@ -1 +0,0 @@
-Add an admin API `GET /_synapse/admin/v1/event_reports` to read entries of table `event_reports`. Contributed by @dklimpel.

changelog.d/8227.doc

@@ -1 +0,0 @@
-Add `/_synapse/client` to the reverse proxy documentation.

changelog.d/8230.bugfix

@@ -1 +0,0 @@
-Fix messages over federation being lost until an event is sent into the same room.

changelog.d/8236.bugfix

@@ -1 +0,0 @@
-Fix a longstanding bug where files that could not be thumbnailed would result in an Internal Server Error.

changelog.d/8243.misc

@@ -1 +0,0 @@
-Remove the 'populate_stats_process_rooms_2' background job and restore functionality to 'populate_stats_process_rooms'.

changelog.d/8247.bugfix

@@ -1 +0,0 @@
-Fix messages over federation being lost until an event is sent into the same room.

changelog.d/8248.feature

@@ -1 +0,0 @@
-Consolidate the SSO error template across all configuration.

changelog.d/8250.misc

@@ -1 +0,0 @@
-Clean up type hints for `PaginationConfig`.

changelog.d/8256.misc

@@ -1 +0,0 @@
-Track the latest event for every destination and room for catch-up after federation outage.

changelog.d/8257.misc

@@ -1 +0,0 @@
-Fix non-user visible bug in implementation of `MultiWriterIdGenerator.get_current_token_for_writer`.

changelog.d/8258.bugfix

@@ -1 +0,0 @@
-Fix messages over federation being lost until an event is sent into the same room.

changelog.d/8259.misc

@@ -1 +0,0 @@
-Switch to the JSON implementation from the standard library.

changelog.d/8260.misc

@@ -1 +0,0 @@
-Add type hints to `synapse.util.async_helpers`.

changelog.d/8261.misc

@@ -1 +0,0 @@
-Simplify tests that mock asynchronous functions.

changelog.d/8262.bugfix

@@ -1 +0,0 @@
-Upgrade canonicaljson to version 1.4.0 to fix an unicode encoding issue.

changelog.d/8265.bugfix

@@ -1 +0,0 @@
-Fix logstanding bug which could lead to incomplete database upgrades on SQLite.

changelog.d/8268.bugfix

@@ -1 +0,0 @@
-Fix stack overflow when stderr is redirected to the logging system, and the logging system encounters an error.

changelog.d/8272.bugfix

@@ -1 +0,0 @@
-Fix messages over federation being lost until an event is sent into the same room.

changelog.d/8275.feature

@@ -1 +0,0 @@
-Add a config option to specify a whitelist of domains that a user can be redirected to after validating their email or phone number.

changelog.d/8278.bugfix

@@ -1 +0,0 @@
-Fix a bug which cause the logging system to report errors, if `DEBUG` was enabled and no `context` filter was applied.

changelog.d/8279.misc

@@ -1 +0,0 @@
-Add type hints to `StreamToken` and `RoomStreamToken` classes.

changelog.d/8281.misc

@@ -1 +0,0 @@
-Change `StreamToken.room_key` to be a `RoomStreamToken` instance.

changelog.d/8282.misc

@@ -1 +0,0 @@
-Clean up type hints for `PaginationConfig`.

changelog.d/8287.bugfix

@@ -1 +0,0 @@
-Fix edge case where push could get delayed for a user until a later event was pushed.

changelog.d/8288.misc

@@ -1 +0,0 @@
-Refactor notifier code to correctly use the max event stream position.

changelog.d/8294.feature

@@ -1 +0,0 @@
-Add experimental support for sharding event persister.

changelog.d/8296.misc

@@ -1 +0,0 @@
-Use slotted classes where possible.

changelog.d/8305.feature

@@ -1 +0,0 @@
-Add the room topic and avatar to the room details admin API.

changelog.d/8306.feature

@@ -1 +0,0 @@
-Add an admin API for querying rooms where a user is a member. Contributed by @dklimpel.

changelog.d/8317.feature

@@ -1 +0,0 @@
-Support testing the local Synapse checkout against the [Complement homeserver test suite](https://github.com/matrix-org/complement/).

changelog.d/8320.feature

@@ -1 +0,0 @@
-Add `uk.half-shot.msc2778.login.application_service` login type to allow appservices to login.

changelog.d/8322.bugfix

@@ -1 +0,0 @@
-Fix messages over federation being lost until an event is sent into the same room.

changelog.d/8324.bugfix

@@ -1 +0,0 @@
-Fix fetching events from remote servers that are malformed.

changelog.d/8326.misc

@@ -1 +0,0 @@
-Update outdated usages of `metaclass` to python 3 syntax.

changelog.d/8329.bugfix

@@ -1 +0,0 @@
-Fix UnboundLocalError from occuring when appservices send malformed register request.

changelog.d/8330.misc

@@ -1 +0,0 @@
-Move lint-related dependencies to package-extra field, update CONTRIBUTING.md to utilise this.

changelog.d/8331.misc

@@ -1 +0,0 @@
-Use the `admin_patterns` helper in additional locations.

changelog.d/8335.misc

@@ -1 +0,0 @@
-Fix test logging to allow braces in log output.

changelog.d/8337.misc

@@ -1 +0,0 @@
-Remove `__future__` imports related to Python 2 compatibility.

changelog.d/8344.misc

@@ -1 +0,0 @@
-Simplify `super()` calls to Python 3 syntax.

changelog.d/8345.feature

@@ -1 +0,0 @@
-Add a configuration option that allows existing users to log in with OpenID Connect. Contributed by @BBBSnowball and @OmmyZhang.

changelog.d/8353.bugfix

@@ -1 +0,0 @@
-Don't send push notifications to expired user accounts.

changelog.d/8354.misc

@@ -1 +0,0 @@
-Fix bad merge from `release-v1.20.0` branch to `develop`.

changelog.d/8362.bugfix

@@ -1 +0,0 @@
-Fixed a regression in v1.19.0 with reactivating users through the admin API.

changelog.d/8364.bugfix

@@ -1,2 +0,0 @@
-Fix a bug where during device registration the length of the device name wasn't
-limited.

changelog.d/8370.misc

@@ -1 +0,0 @@
-Factor out a `_send_dummy_event_for_room` method.

changelog.d/8371.misc

@@ -1 +0,0 @@
-Improve logging of state resolution.

changelog.d/8372.misc

@@ -1 +0,0 @@
-Add type annotations to `SimpleHttpClient`.

changelog.d/8373.bugfix

@@ -1 +0,0 @@
-Include `guest_access` in the fields that are checked for null bytes when updating `room_stats_state`. Broke in v1.7.2.

changelog.d/8374.bugfix

@@ -1 +0,0 @@
-Fix theoretical race condition where events are not sent down `/sync` if the synchrotron worker is restarted without restarting other workers.

changelog.d/8375.doc

@@ -1 +0,0 @@
-Add note to the reverse proxy settings documentation about disabling Apache's mod_security2. Contributed by Julian Fietkau (@jfietkau).

changelog.d/8377.misc

@@ -1 +0,0 @@
-Move lint-related dependencies to package-extra field, update CONTRIBUTING.md to utilise this.

changelog.d/8383.misc

@@ -1 +0,0 @@
-Refactor ID generators to use `async with` syntax.

changelog.d/8385.bugfix

@@ -1 +0,0 @@
-Fix a bug which could cause errors in rooms with malformed membership events, on servers using sqlite.

changelog.d/8386.bugfix

@@ -1 +0,0 @@
-Fix a bug introduced in v1.20.0 which caused the `synapse_port_db` script to fail.

changelog.d/8387.feature

@@ -1 +0,0 @@
-Add experimental support for sharding event persister.

changelog.d/8388.misc

@@ -1 +0,0 @@
-Add `EventStreamPosition` type.

changelog.d/8396.feature

@@ -1 +0,0 @@
-Add experimental support for sharding event persister.

changelog.d/8398.bugfix

@@ -1 +0,0 @@
-Fix "Re-starting finished log context" warning when receiving an event we already had over federation.

changelog.d/8399.misc

@@ -1 +0,0 @@
-Create a mechanism for marking tests "logcontext clean".

changelog.d/8400.bugfix

@@ -1 +0,0 @@
-Fix incorrect handling of timeouts on outgoing HTTP requests.

changelog.d/8401.misc

@@ -1 +0,0 @@
-A pair of tiny cleanups in the federation request code.

changelog.d/8402.misc

@@ -1 +0,0 @@
-Add checks on startup that PostgreSQL sequences are consistent with their associated tables.

changelog.d/8404.misc

@@ -1 +0,0 @@
-Do not include appservice users when calculating the total MAU for a server.

changelog.d/8405.feature

@@ -1 +0,0 @@
-Consolidate the SSO error template across all configuration.

changelog.d/8406.feature

@@ -1 +0,0 @@
-Add prometheus metrics for replication requests.

changelog.d/8410.bugfix

@@ -1 +0,0 @@
-Fix a v1.20.0 regression in the `synapse_port_db` script regarding the `ui_auth_sessions_ips` table.

changelog.d/8413.feature

@@ -1 +0,0 @@
-Support passing additional single sign-on parameters to the client.

changelog.d/8414.bugfix

@@ -1 +0,0 @@
-Remove unnecessary 3PID registration check when resetting password via an email address. Bug introduced in v0.34.0rc2.

changelog.d/8415.doc

@@ -1 +0,0 @@
-Improve description of `server_name` config option in `homserver.yaml`.

changelog.d/8417.feature

@@ -1 +0,0 @@
-Add a config option to specify a whitelist of domains that a user can be redirected to after validating their email or phone number.

changelog.d/8419.feature

@@ -1 +0,0 @@
-Add experimental support for sharding event persister.

changelog.d/8420.feature

@@ -1 +0,0 @@
-Add experimental reporting of metrics on expensive rooms for state-resolution.

changelog.d/8422.misc

@@ -1 +0,0 @@
-Typing fixes for `synapse.handlers.federation`.

changelog.d/8423.misc

@@ -1 +0,0 @@
-Various refactors to simplify stream token handling.

changelog.d/8425.feature

@@ -1 +0,0 @@
-Add experimental prometheus metric to track numbers of "large" rooms for state resolutiom.

changelog.d/8426.removal

@@ -1 +0,0 @@
-Drop support for `prometheus_client` older than 0.4.0.

changelog.d/8427.misc

@@ -1 +0,0 @@
-Make stream token serializing/deserializing async.

changelog.d/8430.feature

@@ -1 +0,0 @@
-Add prometheus metrics to track federation delays.

changelog.d/8432.misc

@@ -0,0 +1 @@
+Check for unreachable code with mypy.

mypy.ini

@@ -6,6 +6,7 @@ check_untyped_defs = True
 show_error_codes = True
 show_traceback = True
 mypy_path = stubs
+warn_unreachable = True
 files =
   synapse/api,
   synapse/appservice,

synapse/__init__.py

@@ -48,7 +48,7 @@ try:
 except ImportError:
     pass
 
-__version__ = "1.20.1"
+__version__ = "1.21.0rc1"
 
 if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
     # We import here so that we don't have to install a bunch of deps when

synapse/config/tls.py

@@ -18,7 +18,7 @@ import os
 import warnings
 from datetime import datetime
 from hashlib import sha256
-from typing import List
+from typing import List, Optional
 
 from unpaddedbase64 import encode_base64
 
@@ -177,8 +177,8 @@ class TlsConfig(Config):
             "use_insecure_ssl_client_just_for_testing_do_not_use"
         )
 
-        self.tls_certificate = None
+        self.tls_certificate = None  # type: Optional[crypto.X509]
-        self.tls_private_key = None
+        self.tls_private_key = None  # type: Optional[crypto.PKey]
 
     def is_disk_cert_valid(self, allow_self_signed=True):
         """
@@ -226,12 +226,12 @@ class TlsConfig(Config):
         days_remaining = (expires_on - now).days
         return days_remaining
 
-    def read_certificate_from_disk(self, require_cert_and_key):
+    def read_certificate_from_disk(self, require_cert_and_key: bool):
         """
         Read the certificates and private key from disk.
 
         Args:
-            require_cert_and_key (bool): set to True to throw an error if the certificate
+            require_cert_and_key: set to True to throw an error if the certificate
                 and key file are not given
         """
         if require_cert_and_key:
@@ -479,13 +479,13 @@ class TlsConfig(Config):
             }
         )
 
-    def read_tls_certificate(self):
+    def read_tls_certificate(self) -> crypto.X509:
         """Reads the TLS certificate from the configured file, and returns it
 
         Also checks if it is self-signed, and warns if so
 
         Returns:
-            OpenSSL.crypto.X509: the certificate
+            The certificate
         """
         cert_path = self.tls_certificate_file
         logger.info("Loading TLS certificate from %s", cert_path)
@@ -504,11 +504,11 @@ class TlsConfig(Config):
 
         return cert
 
-    def read_tls_private_key(self):
+    def read_tls_private_key(self) -> crypto.PKey:
         """Reads the TLS private key from the configured file, and returns it
 
         Returns:
-            OpenSSL.crypto.PKey: the private key
+            The private key
         """
         private_key_path = self.tls_private_key_file
         logger.info("Loading TLS key from %s", private_key_path)

synapse/federation/federation_server.py

@@ -22,7 +22,6 @@ from typing import (
     Callable,
     Dict,
     List,
-    Match,
     Optional,
     Tuple,
     Union,
@@ -825,14 +824,14 @@ def server_matches_acl_event(server_name: str, acl_event: EventBase) -> bool:
     return False
 
 
-def _acl_entry_matches(server_name: str, acl_entry: str) -> Match:
+def _acl_entry_matches(server_name: str, acl_entry: Any) -> bool:
     if not isinstance(acl_entry, str):
         logger.warning(
             "Ignoring non-str ACL entry '%s' (is %s)", acl_entry, type(acl_entry)
         )
         return False
     regex = glob_to_regex(acl_entry)
-    return regex.match(server_name)
+    return bool(regex.match(server_name))
 
 
 class FederationHandlerRegistry:

synapse/handlers/directory.py

@@ -383,7 +383,7 @@ class DirectoryHandler(BaseHandler):
         """
         creator = await self.store.get_room_alias_creator(alias.to_string())
 
-        if creator is not None and creator == user_id:
+        if creator == user_id:
             return True
 
         # Resolve the alias to the corresponding room.

synapse/handlers/room.py

@@ -962,8 +962,6 @@ class RoomCreationHandler(BaseHandler):
             try:
                 random_string = stringutils.random_string(18)
                 gen_room_id = RoomID(random_string, self.hs.hostname).to_string()
-                if isinstance(gen_room_id, bytes):
-                    gen_room_id = gen_room_id.decode("utf-8")
                 await self.store.store_room(
                     room_id=gen_room_id,
                     room_creator_user_id=creator_id,

synapse/handlers/room_member.py

@@ -662,7 +662,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
 
     async def send_membership_event(
         self,
-        requester: Requester,
+        requester: Optional[Requester],
         event: EventBase,
         context: EventContext,
         ratelimit: bool = True,

synapse/handlers/sync.py

@@ -88,7 +88,7 @@ class SyncConfig:
 class TimelineBatch:
     prev_batch = attr.ib(type=StreamToken)
     events = attr.ib(type=List[EventBase])
-    limited = attr.ib(bool)
+    limited = attr.ib(type=bool)
 
     def __bool__(self) -> bool:
         """Make the result appear empty if there are no updates. This is used

synapse/http/server.py

@@ -257,7 +257,7 @@ class _AsyncResource(resource.Resource, metaclass=abc.ABCMeta):
             if isinstance(raw_callback_return, (defer.Deferred, types.CoroutineType)):
                 callback_return = await raw_callback_return
             else:
-                callback_return = raw_callback_return
+                callback_return = raw_callback_return  # type: ignore
 
             return callback_return
 
@@ -406,7 +406,7 @@ class JsonResource(DirectServeJsonResource):
         if isinstance(raw_callback_return, (defer.Deferred, types.CoroutineType)):
             callback_return = await raw_callback_return
         else:
-            callback_return = raw_callback_return
+            callback_return = raw_callback_return  # type: ignore
 
         return callback_return
 

synapse/logging/_structured.py

@@ -12,7 +12,6 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-
 import logging
 import os.path
 import sys
@@ -89,14 +88,7 @@ class LogContextObserver:
         context = current_context()
 
         # Copy the context information to the log event.
-        if context is not None:
         context.copy_to_twisted_log_entry(event)
-        else:
-            # If there's no logging context, not even the root one, we might be
-            # starting up or it might be from non-Synapse code. Log it as if it
-            # came from the root logger.
-            event["request"] = None
-            event["scope"] = None
 
         self.observer(event)
 

synapse/push/push_rule_evaluator.py

@@ -16,7 +16,7 @@
 
 import logging
 import re
-from typing import Any, Dict, List, Pattern, Union
+from typing import Any, Dict, List, Optional, Pattern, Union
 
 from synapse.events import EventBase
 from synapse.types import UserID
@@ -181,7 +181,7 @@ class PushRuleEvaluatorForEvent:
 
         return r.search(body)
 
-    def _get_value(self, dotted_key: str) -> str:
+    def _get_value(self, dotted_key: str) -> Optional[str]:
         return self._value_cache.get(dotted_key, None)
 
 

synapse/replication/tcp/protocol.py

@@ -51,10 +51,11 @@ import fcntl
 import logging
 import struct
 from inspect import isawaitable
-from typing import TYPE_CHECKING, List
+from typing import TYPE_CHECKING, List, Optional
 
 from prometheus_client import Counter
 
+from twisted.internet import task
 from twisted.protocols.basic import LineOnlyReceiver
 from twisted.python.failure import Failure
 
@@ -152,9 +153,10 @@ class BaseReplicationStreamProtocol(LineOnlyReceiver):
 
         self.last_received_command = self.clock.time_msec()
         self.last_sent_command = 0
-        self.time_we_closed = None  # When we requested the connection be closed
+        # When we requested the connection be closed
+        self.time_we_closed = None  # type: Optional[int]
 
-        self.received_ping = False  # Have we reecived a ping from the other side
+        self.received_ping = False  # Have we received a ping from the other side
 
         self.state = ConnectionStates.CONNECTING
 
@@ -165,7 +167,7 @@ class BaseReplicationStreamProtocol(LineOnlyReceiver):
         self.pending_commands = []  # type: List[Command]
 
         # The LoopingCall for sending pings.
-        self._send_ping_loop = None
+        self._send_ping_loop = None  # type: Optional[task.LoopingCall]
 
         # a logcontext which we use for processing incoming commands. We declare it as a
         # background process so that the CPU stats get reported to prometheus.

synapse/rest/media/v1/media_repository.py

@@ -637,7 +637,7 @@ class MediaRepository:
             thumbnailer = Thumbnailer(input_path)
         except ThumbnailError as e:
             logger.warning(
-                "Unable to generate thumbnails for remote media %s from %s using a method of %s and type of %s: %s",
+                "Unable to generate thumbnails for remote media %s from %s of type %s: %s",
                 media_id,
                 server_name,
                 media_type,