Merge remote-tracking branch 'origin/release-v1.62' into matrix-org-hotfixes
Andrew Morgan
commit
f825db3d6c
116
CHANGES.md
116
CHANGES.md
|
|
@ -1,3 +1,119 @@
|
|||
Synapse 1.62.0rc1 (2022-06-28)
|
||||
==============================
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
- Port the spam-checker API callbacks to a new, richer API. This is part of an ongoing change to let spam-checker modules inform users of the reason their event or operation is rejected. ([\#12857](https://github.com/matrix-org/synapse/issues/12857), [\#13047](https://github.com/matrix-org/synapse/issues/13047))
|
||||
- Allow server admins to customise the response of the `/.well-known/matrix/client` endpoint. ([\#13035](https://github.com/matrix-org/synapse/issues/13035))
|
||||
- Add metrics measuring the CPU and DB time spent in state resolution. ([\#13036](https://github.com/matrix-org/synapse/issues/13036))
|
||||
- Speed up fetching of device list changes in `/sync` and `/keys/changes`. ([\#13045](https://github.com/matrix-org/synapse/issues/13045), [\#13098](https://github.com/matrix-org/synapse/issues/13098))
|
||||
- Improve URL previews for sites which only provide Twitter Card metadata, e.g. LWN.net. ([\#13056](https://github.com/matrix-org/synapse/issues/13056))
|
||||
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Update [MSC3786](https://github.com/matrix-org/matrix-spec-proposals/pull/3786) implementation to check `state_key`. ([\#12939](https://github.com/matrix-org/synapse/issues/12939))
|
||||
- Fix a bug introduced in Synapse 1.58 where Synapse would not report full version information when installed from a git checkout. This is a best-effort affair and not guaranteed to be stable. ([\#12973](https://github.com/matrix-org/synapse/issues/12973))
|
||||
- Fix a bug introduced in Synapse 1.60 where Synapse would fail to start if the `sqlite3` module was not available. ([\#12979](https://github.com/matrix-org/synapse/issues/12979))
|
||||
- Fix a bug where non-standard information was required when requesting the `/hierarchy` API over federation. Introduced
|
||||
in Synapse v1.41.0. ([\#12991](https://github.com/matrix-org/synapse/issues/12991))
|
||||
- Fix a long-standing bug which meant that rate limiting was not restrictive enough in some cases. ([\#13018](https://github.com/matrix-org/synapse/issues/13018))
|
||||
- Fix a bug introduced in Synapse 1.58 where profile requests for a malformed user ID would ccause an internal error. Synapse now returns 400 Bad Request in this situation. ([\#13041](https://github.com/matrix-org/synapse/issues/13041))
|
||||
- Fix some inconsistencies in the event authentication code. ([\#13087](https://github.com/matrix-org/synapse/issues/13087), [\#13088](https://github.com/matrix-org/synapse/issues/13088))
|
||||
- Fix a long-standing bug where room directory requests would cause an internal server error if given a malformed room alias. ([\#13106](https://github.com/matrix-org/synapse/issues/13106))
|
||||
|
||||
|
||||
Improved Documentation
|
||||
----------------------
|
||||
|
||||
- Add documentation for how to configure Synapse with Workers using Docker Compose. Includes example worker config and docker-compose.yaml. Contributed by @Thumbscrew. ([\#12737](https://github.com/matrix-org/synapse/issues/12737))
|
||||
- Ensure the [Poetry cheat sheet](https://matrix-org.github.io/synapse/develop/development/dependencies.html) is available in the online documentation. ([\#13022](https://github.com/matrix-org/synapse/issues/13022))
|
||||
- Mention removed community/group worker endpoints in upgrade.md. Contributed by @olmari. ([\#13023](https://github.com/matrix-org/synapse/issues/13023))
|
||||
- Add instructions for running Complement with `gotestfmt`-formatted output locally. ([\#13073](https://github.com/matrix-org/synapse/issues/13073))
|
||||
- Update OpenTracing docs to reference the configuration manual rather than the configuration file. ([\#13076](https://github.com/matrix-org/synapse/issues/13076))
|
||||
- Update information on downstream Debian packages. ([\#13095](https://github.com/matrix-org/synapse/issues/13095))
|
||||
- Remove documentation for the Delete Group Admin API which no longer exists. ([\#13112](https://github.com/matrix-org/synapse/issues/13112))
|
||||
|
||||
|
||||
Deprecations and Removals
|
||||
-------------------------
|
||||
|
||||
- Remove the unspecced `DELETE /directory/list/room/{roomId}` endpoint, which hid rooms from the [public room directory](https://spec.matrix.org/v1.3/client-server-api/#listing-rooms). Instead, `PUT` to the same URL with a visibility of `"private"`. ([\#13123](https://github.com/matrix-org/synapse/issues/13123))
|
||||
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Add tests for cancellation of `GET /rooms/$room_id/members` and `GET /rooms/$room_id/state` requests. ([\#12674](https://github.com/matrix-org/synapse/issues/12674))
|
||||
- Report login failures due to unknown third party identifiers in the same way as failures due to invalid passwords. This prevents an attacker from using the error response to determine if the identifier exists. Contributed by Daniel Aloni. ([\#12738](https://github.com/matrix-org/synapse/issues/12738))
|
||||
- Merge the Complement testing Docker images into a single, multi-purpose image. ([\#12881](https://github.com/matrix-org/synapse/issues/12881), [\#13075](https://github.com/matrix-org/synapse/issues/13075))
|
||||
- Simplify the database schema for `event_edges`. ([\#12893](https://github.com/matrix-org/synapse/issues/12893))
|
||||
- Clean up the test code for client disconnection. ([\#12929](https://github.com/matrix-org/synapse/issues/12929))
|
||||
- Remove code generating comments in configuration. ([\#12941](https://github.com/matrix-org/synapse/issues/12941))
|
||||
- Add `Cross-Origin-Resource-Policy: cross-origin` header to content repository's thumbnail and download endpoints. ([\#12944](https://github.com/matrix-org/synapse/issues/12944))
|
||||
- Replace noop background updates with `DELETE` delta. ([\#12954](https://github.com/matrix-org/synapse/issues/12954), [\#13050](https://github.com/matrix-org/synapse/issues/13050))
|
||||
- Use lower isolation level when inserting read receipts to avoid serialization errors. Contributed by Nick @ Beeper. ([\#12957](https://github.com/matrix-org/synapse/issues/12957))
|
||||
- Reduce the amount of state we pull from the DB. ([\#12963](https://github.com/matrix-org/synapse/issues/12963))
|
||||
- Enable testing against PostgreSQL databases in Complement CI. ([\#12965](https://github.com/matrix-org/synapse/issues/12965), [\#13034](https://github.com/matrix-org/synapse/issues/13034))
|
||||
- Fix an inaccurate comment. ([\#12969](https://github.com/matrix-org/synapse/issues/12969))
|
||||
- Remove the `delete_device` method and always call `delete_devices`. ([\#12970](https://github.com/matrix-org/synapse/issues/12970))
|
||||
- Use a GitHub form for issues rather than a hard-to-read, easy-to-ignore template. ([\#12982](https://github.com/matrix-org/synapse/issues/12982))
|
||||
- Move [MSC3715](https://github.com/matrix-org/matrix-spec-proposals/pull/3715) behind an experimental config flag. ([\#12984](https://github.com/matrix-org/synapse/issues/12984))
|
||||
- Add type hints to tests. ([\#12985](https://github.com/matrix-org/synapse/issues/12985), [\#13099](https://github.com/matrix-org/synapse/issues/13099))
|
||||
- Refactor macaroon tokens generation and move the unsubscribe link in notification emails to `/_synapse/client/unsubscribe`. ([\#12986](https://github.com/matrix-org/synapse/issues/12986))
|
||||
- Fix documentation for running complement tests. ([\#12990](https://github.com/matrix-org/synapse/issues/12990))
|
||||
- Faster joins: add issue links to the TODO comments in the code. ([\#13004](https://github.com/matrix-org/synapse/issues/13004))
|
||||
- Reduce DB usage of `/sync` when a large number of unread messages have recently been sent in a room. ([\#13005](https://github.com/matrix-org/synapse/issues/13005), [\#13096](https://github.com/matrix-org/synapse/issues/13096), [\#13118](https://github.com/matrix-org/synapse/issues/13118))
|
||||
- Replaced usage of PyJWT with methods from Authlib in `org.matrix.login.jwt`. Contributed by Hannes Lerchl. ([\#13011](https://github.com/matrix-org/synapse/issues/13011))
|
||||
- Modernize the `contrib/graph/` scripts. ([\#13013](https://github.com/matrix-org/synapse/issues/13013))
|
||||
- Remove redundant `room_version` parameters from event auth functions. ([\#13017](https://github.com/matrix-org/synapse/issues/13017))
|
||||
- Decouple `synapse.api.auth_blocking.AuthBlocking` from `synapse.api.auth.Auth`. ([\#13021](https://github.com/matrix-org/synapse/issues/13021))
|
||||
- Add type annotations to `synapse.storage.databases.main.devices`. ([\#13025](https://github.com/matrix-org/synapse/issues/13025))
|
||||
- Set default `sync_response_cache_duration` to two minutes. ([\#13042](https://github.com/matrix-org/synapse/issues/13042))
|
||||
- Rename CI test runs. ([\#13046](https://github.com/matrix-org/synapse/issues/13046))
|
||||
- Increase timeout of complement CI test runs. ([\#13048](https://github.com/matrix-org/synapse/issues/13048))
|
||||
- Refactor entry points so that they all have a `main` function. ([\#13052](https://github.com/matrix-org/synapse/issues/13052))
|
||||
- Refactor the Dockerfile-workers configuration script to use Jinja2 templates in Synapse workers' Supervisord blocks. ([\#13054](https://github.com/matrix-org/synapse/issues/13054))
|
||||
- Add headers to individual options in config documentation to allow for linking. ([\#13055](https://github.com/matrix-org/synapse/issues/13055))
|
||||
- Make Complement CI logs easier to read. ([\#13057](https://github.com/matrix-org/synapse/issues/13057), [\#13058](https://github.com/matrix-org/synapse/issues/13058), [\#13069](https://github.com/matrix-org/synapse/issues/13069))
|
||||
- Don't instantiate modules with keyword arguments. ([\#13060](https://github.com/matrix-org/synapse/issues/13060))
|
||||
- Fix type checking errors against Twisted trunk. ([\#13061](https://github.com/matrix-org/synapse/issues/13061))
|
||||
- Allow MSC3030 `timestamp_to_event` calls from anyone on world-readable rooms. ([\#13062](https://github.com/matrix-org/synapse/issues/13062))
|
||||
- Add a CI job to check that schema deltas are in the correct folder. ([\#13063](https://github.com/matrix-org/synapse/issues/13063))
|
||||
- Avoid rechecking event auth rules which are independent of room state. ([\#13065](https://github.com/matrix-org/synapse/issues/13065))
|
||||
- Reduce the duplication of code that invokes the rate limiter. ([\#13070](https://github.com/matrix-org/synapse/issues/13070))
|
||||
- Add a Subject Alternative Name to the certificate generated for Complement tests. ([\#13071](https://github.com/matrix-org/synapse/issues/13071))
|
||||
- Add more tests for room upgrades. ([\#13074](https://github.com/matrix-org/synapse/issues/13074))
|
||||
- Pin dependencies maintained by matrix.org to [semantic version](https://semver.org/) bounds. ([\#13082](https://github.com/matrix-org/synapse/issues/13082))
|
||||
- Correctly report prometheus DB stats for `get_earliest_token_for_stats`. ([\#13085](https://github.com/matrix-org/synapse/issues/13085))
|
||||
- Fix a long-standing bug where a finished logging context would be re-started when Synapse failed to persist an event from federation. ([\#13089](https://github.com/matrix-org/synapse/issues/13089))
|
||||
- Simplify the alias deletion logic as an application service. ([\#13093](https://github.com/matrix-org/synapse/issues/13093))
|
||||
- Add type annotations to `tests.test_server`. ([\#13124](https://github.com/matrix-org/synapse/issues/13124))
|
||||
|
||||
|
||||
Synapse 1.61.1 (2022-06-28)
|
||||
===========================
|
||||
|
||||
This patch release fixes a security issue regarding URL previews, affecting all prior versions of Synapse. Server administrators are encouraged to update Synapse as soon as possible. We are not aware of these vulnerabilities being exploited in the wild.
|
||||
|
||||
Server administrators who are unable to update Synapse may use the workarounds described in the linked GitHub Security Advisory below.
|
||||
|
||||
## Security advisory
|
||||
|
||||
The following issue is fixed in 1.61.1.
|
||||
|
||||
* [GHSA-22p3-qrh9-cx32](https://github.com/matrix-org/synapse/security/advisories/GHSA-22p3-qrh9-cx32) / [CVE-2022-31052](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-31052)
|
||||
|
||||
Synapse instances with the [`url_preview_enabled`](https://matrix-org.github.io/synapse/v1.61/usage/configuration/config_documentation.html#media-store) homeserver config option set to `true` are affected. URL previews of some web pages can lead to unbounded recursion, causing the request to either fail, or in some cases crash the running Synapse process.
|
||||
|
||||
Requesting URL previews requires authentication. Nevertheless, it is possible to exploit this maliciously, either by malicious users on the homeserver, or by remote users sending URLs that a local user's client may automatically request a URL preview for.
|
||||
|
||||
Homeservers with the `url_preview_enabled` configuration option set to `false` (the default) are unaffected. Instances with the `enable_media_repo` configuration option set to `false` are also unaffected, as this also disables URL preview functionality.
|
||||
|
||||
Fixed by [fa1308061802ac7b7d20e954ba7372c5ac292333](https://github.com/matrix-org/synapse/commit/fa1308061802ac7b7d20e954ba7372c5ac292333).
|
||||
|
||||
Synapse 1.61.0 (2022-06-14)
|
||||
===========================
|
||||
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
Add tests for cancellation of `GET /rooms/$room_id/members` and `GET /rooms/$room_id/state` requests.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add documentation for how to configure Synapse with Workers using Docker Compose. Includes example worker config and docker-compose.yaml. Contributed by @Thumbscrew.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Report login failures due to unknown third party identifiers in the same way as failures due to invalid passwords. This prevents an attacker from using the error response to determine if the identifier exists. Contributed by Daniel Aloni.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Port spam-checker API callbacks to a new, richer API. This is part of an ongoing change to let spam-checker modules inform users of the reason their event or operation is rejected.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Merge the Complement testing Docker images into a single, multi-purpose image.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Simplify the database schema for `event_edges`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Clean up the test code for client disconnection.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update [MSC3786](https://github.com/matrix-org/matrix-spec-proposals/pull/3786) implementation to check `state_key`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove code generating comments in configuration.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add `Cross-Origin-Resource-Policy: cross-origin` header to content repository's thumbnail and download endpoints.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Replace noop background updates with `DELETE` delta.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Use lower isolation level when inserting read receipts to avoid serialization errors. Contributed by Nick @ Beeper.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Reduce the amount of state we pull from the DB.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Enable testing against PostgreSQL databases in Complement CI.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix an inaccurate comment.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove the `delete_device` method and always call `delete_devices`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a bug introduced in Synapse 1.58 where Synapse would not report full version information when installed from a git checkout. This is a best-effort affair and not guaranteed to be stable.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a bug introduced in Synapse 1.60 where Synapse would fail to start if the `sqlite3` module was not available.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Use a GitHub form for issues rather than a hard-to-read, easy-to-ignore template.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Move [MSC3715](https://github.com/matrix-org/matrix-spec-proposals/pull/3715) behind an experimental config flag.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add type hints to tests.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Refactor macaroon tokens generation and move the unsubscribe link in notification emails to `/_synapse/client/unsubscribe`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix documentation for running complement tests.
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
Fix a bug where non-standard information was required when requesting the `/hierarchy` API over federation. Introduced
|
||||
in Synapse v1.41.0.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Faster joins: add issue links to the TODO comments in the code.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Reduce DB usage of `/sync` when a large number of unread messages have recently been sent in a room.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Replaced usage of PyJWT with methods from Authlib in `org.matrix.login.jwt`. Contributed by Hannes Lerchl.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Modernize the `contrib/graph/` scripts.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove redundant `room_version` parameters from event auth functions.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a long-standing bug which meant that rate limiting was not restrictive enough in some cases.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Decouple `synapse.api.auth_blocking.AuthBlocking` from `synapse.api.auth.Auth`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Ensure the [Poetry cheat sheet](https://matrix-org.github.io/synapse/develop/development/dependencies.html) is available in the online documentation.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Mention removed community/group worker endpoints in upgrade.md. Contributed by @olmari.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add type annotations to `synapse.storage.databases.main.devices`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Enable testing against PostgreSQL databases in Complement CI.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Allow server admins to customise the response of the `/.well-known/matrix/client` endpoint.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add metrics measuring the CPU and DB time spent in state resolution.
|
||||
|
|
@ -1,2 +0,0 @@
|
|||
Fix a bug introduced in Synapse 1.58 where profile requests for a malformed user ID would ccause an internal error. Synapse now returns 400 Bad Request in this situation.
|
||||
|
||||
|
|
@ -1 +0,0 @@
|
|||
Set default `sync_response_cache_duration` to two minutes.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Speed up fetching of device list changes in `/sync` and `/keys/changes`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Rename CI test runs.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Port spam-checker API callbacks to a new, richer API. This is part of an ongoing change to let spam-checker modules inform users of the reason their event or operation is rejected.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Increase timeout of complement CI test runs.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Replace noop background updates with `DELETE` delta.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Refactor entry points so that they all have a `main` function.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Refactor the Dockerfile-workers configuration script to use Jinja2 templates in Synapse workers' Supervisord blocks.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add headers to individual options in config documentation to allow for linking.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Improve URL previews for sites which only provide Twitter Card metadata, e.g. LWN.net.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Make Complement CI logs easier to read.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Make Complement CI logs easier to read.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Don't instantiate modules with keyword arguments.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix type checking errors against Twisted trunk.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Allow MSC3030 'timestamp_to_event' calls from anyone on world-readable rooms.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add a CI job to check that schema deltas are in the correct folder.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Avoid rechecking event auth rules which are independent of room state.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Make Complement CI logs easier to read.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Reduce the duplication of code that invokes the rate limiter.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add a Subject Alternative Name to the certificate generated for Complement tests.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add instructions for running Complement with `gotestfmt`-formatted output locally.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add more tests for room upgrades.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Merge the Complement testing Docker images into a single, multi-purpose image.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update OpenTracing docs to reference the configuration manual rather than the configuration file.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Pin dependencies maintained by matrix.org to [semantic version](https://semver.org/) bounds.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Correctly report prometheus DB stats for `get_earliest_token_for_stats`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix some inconsistencies in the event authentication code.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix some inconsistencies in the event authentication code.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a long-standing bug where a finished logging context would be re-started when Synapse failed to persist an event from federation.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Simplify the alias deletion logic as an application service.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update information on downstream Debian packages.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Reduce DB usage of `/sync` when a large number of unread messages have recently been sent in a room.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Speed up fetching of device list changes in `/sync` and `/keys/changes`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add type hints to tests.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a long-standing bug where room directory requests would cause an internal server error if given a malformed room alias.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove documentation for the Delete Group Admin API which no longer exists.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Reduce DB usage of `/sync` when a large number of unread messages have recently been sent in a room.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove the unspecced `DELETE /directory/list/room/{roomId}` endpoint, which hid rooms from the [public room directory](https://spec.matrix.org/v1.3/client-server-api/#listing-rooms). Instead, `PUT` to the same URL with a visibility of `"private"`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add type annotations to `tests.test_server`.
|
||||
|
|
@ -1,3 +1,15 @@
|
|||
matrix-synapse-py3 (1.62.0~rc1) stable; urgency=medium
|
||||
|
||||
* New Synapse release 1.62.0rc1.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Tue, 28 Jun 2022 16:34:57 +0100
|
||||
|
||||
matrix-synapse-py3 (1.61.1) stable; urgency=medium
|
||||
|
||||
* New Synapse release 1.61.1.
|
||||
|
||||
-- Synapse Packaging team <packages@matrix.org> Tue, 28 Jun 2022 14:33:46 +0100
|
||||
|
||||
matrix-synapse-py3 (1.61.0) stable; urgency=medium
|
||||
|
||||
* New Synapse release 1.61.0.
|
||||
|
|
|
|||
|
|
@ -54,7 +54,7 @@ skip_gitignore = true
|
|||
|
||||
[tool.poetry]
|
||||
name = "matrix-synapse"
|
||||
version = "1.61.0"
|
||||
version = "1.62.0rc1"
|
||||
description = "Homeserver for the Matrix decentralised comms protocol"
|
||||
authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
|
||||
license = "Apache-2.0"
|
||||
|
|
|
|||
|
|
@ -12,7 +12,6 @@
|
|||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
import codecs
|
||||
import itertools
|
||||
import logging
|
||||
import re
|
||||
from typing import (
|
||||
|
|
@ -21,6 +20,7 @@ from typing import (
|
|||
Dict,
|
||||
Generator,
|
||||
Iterable,
|
||||
List,
|
||||
Optional,
|
||||
Set,
|
||||
Union,
|
||||
|
|
@ -354,7 +354,7 @@ def parse_html_description(tree: "etree.Element") -> Optional[str]:
|
|||
|
||||
from lxml import etree
|
||||
|
||||
TAGS_TO_REMOVE = (
|
||||
TAGS_TO_REMOVE = {
|
||||
"header",
|
||||
"nav",
|
||||
"aside",
|
||||
|
|
@ -369,31 +369,42 @@ def parse_html_description(tree: "etree.Element") -> Optional[str]:
|
|||
"img",
|
||||
"picture",
|
||||
etree.Comment,
|
||||
)
|
||||
}
|
||||
|
||||
# Split all the text nodes into paragraphs (by splitting on new
|
||||
# lines)
|
||||
text_nodes = (
|
||||
re.sub(r"\s+", "\n", el).strip()
|
||||
for el in _iterate_over_text(tree.find("body"), *TAGS_TO_REMOVE)
|
||||
for el in _iterate_over_text(tree.find("body"), TAGS_TO_REMOVE)
|
||||
)
|
||||
return summarize_paragraphs(text_nodes)
|
||||
|
||||
|
||||
def _iterate_over_text(
|
||||
tree: "etree.Element", *tags_to_ignore: Union[str, "etree.Comment"]
|
||||
tree: Optional["etree.Element"],
|
||||
tags_to_ignore: Set[Union[str, "etree.Comment"]],
|
||||
stack_limit: int = 1024,
|
||||
) -> Generator[str, None, None]:
|
||||
"""Iterate over the tree returning text nodes in a depth first fashion,
|
||||
skipping text nodes inside certain tags.
|
||||
|
||||
Args:
|
||||
tree: The parent element to iterate. Can be None if there isn't one.
|
||||
tags_to_ignore: Set of tags to ignore
|
||||
stack_limit: Maximum stack size limit for depth-first traversal.
|
||||
Nodes will be dropped if this limit is hit, which may truncate the
|
||||
textual result.
|
||||
Intended to limit the maximum working memory when generating a preview.
|
||||
"""
|
||||
# This is basically a stack that we extend using itertools.chain.
|
||||
# This will either consist of an element to iterate over *or* a string
|
||||
|
||||
if tree is None:
|
||||
return
|
||||
|
||||
# This is a stack whose items are elements to iterate over *or* strings
|
||||
# to be returned.
|
||||
elements = iter([tree])
|
||||
while True:
|
||||
el = next(elements, None)
|
||||
if el is None:
|
||||
return
|
||||
elements: List[Union[str, "etree.Element"]] = [tree]
|
||||
while elements:
|
||||
el = elements.pop()
|
||||
|
||||
if isinstance(el, str):
|
||||
yield el
|
||||
|
|
@ -407,17 +418,22 @@ def _iterate_over_text(
|
|||
if el.text:
|
||||
yield el.text
|
||||
|
||||
# We add to the stack all the elements children, interspersed with
|
||||
# each child's tail text (if it exists). The tail text of a node
|
||||
# is text that comes *after* the node, so we always include it even
|
||||
# if we ignore the child node.
|
||||
elements = itertools.chain(
|
||||
itertools.chain.from_iterable( # Basically a flatmap
|
||||
[child, child.tail] if child.tail else [child]
|
||||
for child in el.iterchildren()
|
||||
),
|
||||
elements,
|
||||
)
|
||||
# We add to the stack all the element's children, interspersed with
|
||||
# each child's tail text (if it exists).
|
||||
#
|
||||
# We iterate in reverse order so that earlier pieces of text appear
|
||||
# closer to the top of the stack.
|
||||
for child in el.iterchildren(reversed=True):
|
||||
if len(elements) > stack_limit:
|
||||
# We've hit our limit for working memory
|
||||
break
|
||||
|
||||
if child.tail:
|
||||
# The tail text of a node is text that comes *after* the node,
|
||||
# so we always include it even if we ignore the child node.
|
||||
elements.append(child.tail)
|
||||
|
||||
elements.append(child)
|
||||
|
||||
|
||||
def summarize_paragraphs(
|
||||
|
|
|
|||
|
|
@ -411,6 +411,23 @@ class OpenGraphFromHtmlTestCase(unittest.TestCase):
|
|||
},
|
||||
)
|
||||
|
||||
def test_nested_nodes(self) -> None:
|
||||
"""A body with some nested nodes. Tests that we iterate over children
|
||||
in the right order (and don't reverse the order of the text)."""
|
||||
html = b"""
|
||||
<a href="somewhere">Welcome <b>the bold <u>and underlined text <svg>
|
||||
with a cheeky SVG</svg></u> and <strong>some</strong> tail text</b></a>
|
||||
"""
|
||||
tree = decode_body(html, "http://example.com/test.html")
|
||||
og = parse_html_to_open_graph(tree)
|
||||
self.assertEqual(
|
||||
og,
|
||||
{
|
||||
"og:title": None,
|
||||
"og:description": "Welcome\n\nthe bold\n\nand underlined text\n\nand\n\nsome\n\ntail text",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
class MediaEncodingTestCase(unittest.TestCase):
|
||||
def test_meta_charset(self) -> None:
|
||||
|
|
|
|||
Loading…
Reference in New Issue