60fa4935b5
Improve opentracing for incoming HTTP requests ( #11618 )
...
* remove `start_active_span_from_request`
Instead, pull out a separate function, `span_context_from_request`, to extract
the parent span, which we can then pass into `start_active_span` as
normal. This seems to be clearer all round.
* Remove redundant tags from `incoming-federation-request`
These are all wrapped up inside a parent span generated in AsyncResource, so
there's no point duplicating all the tags that are set there.
* Leave request spans open until the request completes
It may take some time for the response to be encoded into JSON, and that JSON
to be streamed back to the client, and really we want that inside the top-level
span, so let's hand responsibility for closure to the SynapseRequest.
* opentracing logs for HTTP request events
* changelog
2021-12-20 17:45:03 +00:00
8e4083e2f6
Merge remote-tracking branch 'origin/release-v1.49' into develop
2021-12-20 16:34:01 +00:00
76aa5537ad
Disable aggregation bundling on `/sync` responses ( #11583 )
...
* Disable aggregation bundling on `/sync` responses
A partial revert of #11478 . This turns out to have had a significant CPU impact
on initial-sync handling. For now, let's disable it, until we find a more
efficient way of achieving this.
* Fix tests.
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
2021-12-20 16:33:35 +00:00
3e0cfd447e
Return JSON errors for unknown resources under /matrix/client. ( #11602 )
...
Instead of returning 404 errors with HTML bodies when an unknown
prefix was requested (e.g. /matrix/client/v1 before Synapse v1.49.0).
2021-12-20 16:00:13 +00:00
7a7ca8f226
Use mock from standard library ( #11588 )
...
Instead of the backported version.
2021-12-20 10:34:46 -05:00
8ad39438fa
Add opentracing types ( #11603 )
2021-12-20 12:18:09 +00:00
8428ef66c7
Add type hints to `synapse/tests/rest/admin` ( #11590 )
2021-12-16 14:59:56 -05:00
1847d027e6
Add type hints to `synapse/storage/databases/main/transactions.py` ( #11589 )
2021-12-16 14:59:35 -05:00
43f5cc7adc
Add MSC2716 and MSC3030 to `/versions` -> `unstable_features` ( #11582 )
...
As suggested in https://github.com/matrix-org/matrix-react-sdk/pull/7372#discussion_r769523369
2021-12-16 11:25:37 -06:00
c7fe32edb4
Add type hints to `synapse/storage/databases/main/room.py` ( #11575 )
2021-12-15 18:00:48 +00:00
f901f8b70e
Require Collections as the parameters for simple_* methods. ( #11580 )
...
Instead of Iterable since the generators are not allowed due
to the potential for their re-use.
2021-12-15 17:00:50 +00:00
323151b787
Convert EventStreamResult to attrs. ( #11574 )
2021-12-15 11:10:02 -05:00
17886d2603
Add experimental support for MSC3202: allowing application services to masquerade as specific devices. ( #11538 )
2021-12-15 10:40:52 +00:00
ecfcd9bbbe
Add type hints to `synapse/storage/databases/main/e2e_room_keys.py` ( #11549 )
2021-12-14 17:46:47 +00:00
0147b3de20
Add missing type hints to `synapse.logging.context` ( #11556 )
2021-12-14 17:35:28 +00:00
2519beaad2
Add missing type hints to `synapse.appservice` ( #11360 )
2021-12-14 17:02:46 +00:00
70ca05373b
Merge branch 'master' into develop
2021-12-14 14:22:48 +00:00
a91698df90
Synapse 1.49.0 (2021-12-14)
...
===========================
No significant changes since version 1.49.0rc1.
Support for Ubuntu 21.04 ends next month on the 20th of January
---------------------------------------------------------------
For users of Ubuntu 21.04 (Hirsute Hippo), please be aware that [upstream support for this version of Ubuntu will end next month][Ubuntu2104EOL].
We will stop producing packages for Ubuntu 21.04 after upstream support ends.
[Ubuntu2104EOL]: https://lists.ubuntu.com/archives/ubuntu-announce/2021-December/000275.html
The wiki has been migrated to the documentation website
-------------------------------------------------------
We've decided to move the existing, somewhat stagnant pages from the GitHub wiki
to the [documentation website](https://matrix-org.github.io/synapse/latest/ ).
This was done for two reasons. The first was to ensure that changes are checked by
multiple authors before being committed (everyone makes mistakes!) and the second
was visibility of the documentation. Not everyone knows that Synapse has some very
useful information hidden away in its GitHub wiki pages. Bringing them to the
documentation website should help with visibility, as well as keep all Synapse documentation
in one, easily-searchable location.
Note that contributions to the documentation website happen through [GitHub pull
requests](https://github.com/matrix-org/synapse/pulls ). Please visit [#synapse-dev:matrix.org](https://matrix.to/#/#synapse-dev:matrix.org )
if you need help with the process!
Synapse 1.49.0rc1 (2021-12-07)
==============================
Features
--------
- Add [MSC3030](https://github.com/matrix-org/matrix-doc/pull/3030 ) experimental client and federation API endpoints to get the closest event to a given timestamp. ([\#9445](https://github.com/matrix-org/synapse/issues/9445 ))
- Include bundled relation aggregations during a limited `/sync` request and `/relations` request, per [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675 ). ([\#11284](https://github.com/matrix-org/synapse/issues/11284 ), [\#11478](https://github.com/matrix-org/synapse/issues/11478 ))
- Add plugin support for controlling database background updates. ([\#11306](https://github.com/matrix-org/synapse/issues/11306 ), [\#11475](https://github.com/matrix-org/synapse/issues/11475 ), [\#11479](https://github.com/matrix-org/synapse/issues/11479 ))
- Support the stable API endpoints for [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946 ): the room `/hierarchy` endpoint. ([\#11329](https://github.com/matrix-org/synapse/issues/11329 ))
- Add admin API to get some information about federation status with remote servers. ([\#11407](https://github.com/matrix-org/synapse/issues/11407 ))
- Support expiry of refresh tokens and expiry of the overall session when refresh tokens are in use. ([\#11425](https://github.com/matrix-org/synapse/issues/11425 ))
- Stabilise support for [MSC2918](https://github.com/matrix-org/matrix-doc/blob/main/proposals/2918-refreshtokens.md#msc2918-refresh-tokens ) refresh tokens as they have now been merged into the Matrix specification. ([\#11435](https://github.com/matrix-org/synapse/issues/11435 ), [\#11522](https://github.com/matrix-org/synapse/issues/11522 ))
- Update [MSC2918 refresh token](https://github.com/matrix-org/matrix-doc/blob/main/proposals/2918-refreshtokens.md#msc2918-refresh-tokens ) support to confirm with the latest revision: accept the `refresh_tokens` parameter in the request body rather than in the URL parameters. ([\#11430](https://github.com/matrix-org/synapse/issues/11430 ))
- Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. ([\#11445](https://github.com/matrix-org/synapse/issues/11445 ))
- Expose `synapse_homeserver` and `synapse_worker` commands as entry points to run Synapse's main process and worker processes, respectively. Contributed by @Ma27. ([\#11449](https://github.com/matrix-org/synapse/issues/11449 ))
- `synctl stop` will now wait for Synapse to exit before returning. ([\#11459](https://github.com/matrix-org/synapse/issues/11459 ), [\#11490](https://github.com/matrix-org/synapse/issues/11490 ))
- Extend the "delete room" admin api to work correctly on rooms which have previously been partially deleted. ([\#11523](https://github.com/matrix-org/synapse/issues/11523 ))
- Add support for the `/_matrix/client/v3/login/sso/redirect/{idpId}` API from Matrix v1.1. This endpoint was overlooked when support for v3 endpoints was added in Synapse 1.48.0rc1. ([\#11451](https://github.com/matrix-org/synapse/issues/11451 ))
Bugfixes
--------
- Fix using [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) batch sending in combination with event persistence workers. Contributed by @tulir at Beeper. ([\#11220](https://github.com/matrix-org/synapse/issues/11220 ))
- Fix a long-standing bug where all requests that read events from the database could get stuck as a result of losing the database connection, properly this time. Also fix a race condition introduced in the previous insufficient fix in Synapse 1.47.0. ([\#11376](https://github.com/matrix-org/synapse/issues/11376 ))
- The `/send_join` response now includes the stable `event` field instead of the unstable field from [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083 ). ([\#11413](https://github.com/matrix-org/synapse/issues/11413 ))
- Fix a bug introduced in Synapse 1.47.0 where `send_join` could fail due to an outdated `ijson` version. ([\#11439](https://github.com/matrix-org/synapse/issues/11439 ), [\#11441](https://github.com/matrix-org/synapse/issues/11441 ), [\#11460](https://github.com/matrix-org/synapse/issues/11460 ))
- Fix a bug introduced in Synapse 1.36.0 which could cause problems fetching event-signing keys from trusted key servers. ([\#11440](https://github.com/matrix-org/synapse/issues/11440 ))
- Fix a bug introduced in Synapse 1.47.1 where the media repository would fail to work if the media store path contained any symbolic links. ([\#11446](https://github.com/matrix-org/synapse/issues/11446 ))
- Fix an `LruCache` corruption bug, introduced in Synapse 1.38.0, that would cause certain requests to fail until the next Synapse restart. ([\#11454](https://github.com/matrix-org/synapse/issues/11454 ))
- Fix a long-standing bug where invites from ignored users were included in incremental syncs. ([\#11511](https://github.com/matrix-org/synapse/issues/11511 ))
- Fix a regression in Synapse 1.48.0 where presence workers would not clear their presence updates over replication on shutdown. ([\#11518](https://github.com/matrix-org/synapse/issues/11518 ))
- Fix a regression in Synapse 1.48.0 where the module API's `looping_background_call` method would spam errors to the logs when given a non-async function. ([\#11524](https://github.com/matrix-org/synapse/issues/11524 ))
Updates to the Docker image
---------------------------
- Update `Dockerfile-workers` to healthcheck all workers in the container. ([\#11429](https://github.com/matrix-org/synapse/issues/11429 ))
Improved Documentation
----------------------
- Update the media repository documentation. ([\#11415](https://github.com/matrix-org/synapse/issues/11415 ))
- Update section about backward extremities in the room DAG concepts doc to correct the misconception about backward extremities indicating whether we have fetched an events' `prev_events`. ([\#11469](https://github.com/matrix-org/synapse/issues/11469 ))
Internal Changes
----------------
- Add `Final` annotation to string constants in `synapse.api.constants` so that they get typed as `Literal`s. ([\#11356](https://github.com/matrix-org/synapse/issues/11356 ))
- Add a check to ensure that users cannot start the Synapse master process when `worker_app` is set. ([\#11416](https://github.com/matrix-org/synapse/issues/11416 ))
- Add a note about postgres memory management and hugepages to postgres doc. ([\#11467](https://github.com/matrix-org/synapse/issues/11467 ))
- Add missing type hints to `synapse.config` module. ([\#11465](https://github.com/matrix-org/synapse/issues/11465 ))
- Add missing type hints to `synapse.federation`. ([\#11483](https://github.com/matrix-org/synapse/issues/11483 ))
- Add type annotations to `tests.storage.test_appservice`. ([\#11488](https://github.com/matrix-org/synapse/issues/11488 ), [\#11492](https://github.com/matrix-org/synapse/issues/11492 ))
- Add type annotations to some of the configuration surrounding refresh tokens. ([\#11428](https://github.com/matrix-org/synapse/issues/11428 ))
- Add type hints to `synapse/tests/rest/admin`. ([\#11501](https://github.com/matrix-org/synapse/issues/11501 ))
- Add type hints to storage classes. ([\#11411](https://github.com/matrix-org/synapse/issues/11411 ))
- Add wiki pages to documentation website. ([\#11402](https://github.com/matrix-org/synapse/issues/11402 ))
- Clean up `tests.storage.test_main` to remove use of legacy code. ([\#11493](https://github.com/matrix-org/synapse/issues/11493 ))
- Clean up `tests.test_visibility` to remove legacy code. ([\#11495](https://github.com/matrix-org/synapse/issues/11495 ))
- Convert status codes to `HTTPStatus` in `synapse.rest.admin`. ([\#11452](https://github.com/matrix-org/synapse/issues/11452 ), [\#11455](https://github.com/matrix-org/synapse/issues/11455 ))
- Extend the `scripts-dev/sign_json` script to support signing events. ([\#11486](https://github.com/matrix-org/synapse/issues/11486 ))
- Improve internal types in push code. ([\#11409](https://github.com/matrix-org/synapse/issues/11409 ))
- Improve type annotations in `synapse.module_api`. ([\#11029](https://github.com/matrix-org/synapse/issues/11029 ))
- Improve type hints for `LruCache`. ([\#11453](https://github.com/matrix-org/synapse/issues/11453 ))
- Preparation for database schema simplifications: disambiguate queries on `state_key`. ([\#11497](https://github.com/matrix-org/synapse/issues/11497 ))
- Refactor `backfilled` into specific behavior function arguments (`_persist_events_and_state_updates` and downstream calls). ([\#11417](https://github.com/matrix-org/synapse/issues/11417 ))
- Refactor `get_version_string` to fix-up types and duplicated code. ([\#11468](https://github.com/matrix-org/synapse/issues/11468 ))
- Refactor various parts of the `/sync` handler. ([\#11494](https://github.com/matrix-org/synapse/issues/11494 ), [\#11515](https://github.com/matrix-org/synapse/issues/11515 ))
- Remove unnecessary `json.dumps` from `tests.rest.admin`. ([\#11461](https://github.com/matrix-org/synapse/issues/11461 ))
- Save the OpenID Connect session ID on login. ([\#11482](https://github.com/matrix-org/synapse/issues/11482 ))
- Update and clean up recently ported documentation pages. ([\#11466](https://github.com/matrix-org/synapse/issues/11466 ))
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEE8SRSDO7gYkSP4chELS76LzL74EcFAmG4l40ACgkQLS76LzL7
4EcAag//XXFGXWsGA3TMF6aDE9Sr6Ei6QucgP6bEMZ3TxBoLDnrwueWHUku7CgLa
vndmmimRtZBWkpdmjGEMs7rmY6U/f6DoGoiSm6L4Ef3nZ9kr9KVvDo5WB2LmnzCD
jOlmhsI/wZJPU14I2+y4yhEbwHCOzSgZmvWg8K6L8Tpd8cBEw+Ee8nESj8R20T5x
wguzmeI/oXvORaQ+G48E0K9wZQEBka0CDTP1TgFfaeSYkBNNtLQBUcbmQou7v7fs
hieLX9Jq8PvEQ5COvXxoxwTtdCho6+23DPxAfUTdXLbXctgEBmoPw2xte9aRSMyB
M3vdt4C9UF54QRDygcy94WgCdMfMHvHKmjlXVd6l7NUtyRjNmHnNsoKU/MZmvvgy
1VIvSxijeVfR5QYpswdEIn4cTVT2xu8oJgw5do1klVC7BL90QCTFDTi5CSRIkqae
C/YUed7Ch/HVW8/s+1B9PHvP6NSlombLnC+cxardrCgABOs25KSwBoi0vGPFYRCQ
KsgdOwxLna3OL3hYEyYUg43S0eEvDkOf1CXWTcQeg5WNMYZiS+uPwxznOHj4YCUL
3Ysawhh0+d0d/6JKCf4ZdR9VC3EaZXv5zd2JtKtaXCR72QG8aaoiNS4YKX9+8wNX
3pKuzX4rvdUdsvHuXDpaipWgmzJu3bLS4uRmuThuzh/XlNmiuAw=
=uMvf
-----END PGP SIGNATURE-----
Merge tag 'v1.49.0'
Synapse 1.49.0 (2021-12-14)
===========================
No significant changes since version 1.49.0rc1.
Support for Ubuntu 21.04 ends next month on the 20th of January
---------------------------------------------------------------
For users of Ubuntu 21.04 (Hirsute Hippo), please be aware that [upstream support for this version of Ubuntu will end next month][Ubuntu2104EOL].
We will stop producing packages for Ubuntu 21.04 after upstream support ends.
[Ubuntu2104EOL]: https://lists.ubuntu.com/archives/ubuntu-announce/2021-December/000275.html
The wiki has been migrated to the documentation website
-------------------------------------------------------
We've decided to move the existing, somewhat stagnant pages from the GitHub wiki
to the [documentation website](https://matrix-org.github.io/synapse/latest/ ).
This was done for two reasons. The first was to ensure that changes are checked by
multiple authors before being committed (everyone makes mistakes!) and the second
was visibility of the documentation. Not everyone knows that Synapse has some very
useful information hidden away in its GitHub wiki pages. Bringing them to the
documentation website should help with visibility, as well as keep all Synapse documentation
in one, easily-searchable location.
Note that contributions to the documentation website happen through [GitHub pull
requests](https://github.com/matrix-org/synapse/pulls ). Please visit [#synapse-dev:matrix.org](https://matrix.to/#/#synapse-dev:matrix.org )
if you need help with the process!
Synapse 1.49.0rc1 (2021-12-07)
==============================
Features
--------
- Add [MSC3030](https://github.com/matrix-org/matrix-doc/pull/3030 ) experimental client and federation API endpoints to get the closest event to a given timestamp. ([\#9445](https://github.com/matrix-org/synapse/issues/9445 ))
- Include bundled relation aggregations during a limited `/sync` request and `/relations` request, per [MSC2675](https://github.com/matrix-org/matrix-doc/pull/2675 ). ([\#11284](https://github.com/matrix-org/synapse/issues/11284 ), [\#11478](https://github.com/matrix-org/synapse/issues/11478 ))
- Add plugin support for controlling database background updates. ([\#11306](https://github.com/matrix-org/synapse/issues/11306 ), [\#11475](https://github.com/matrix-org/synapse/issues/11475 ), [\#11479](https://github.com/matrix-org/synapse/issues/11479 ))
- Support the stable API endpoints for [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946 ): the room `/hierarchy` endpoint. ([\#11329](https://github.com/matrix-org/synapse/issues/11329 ))
- Add admin API to get some information about federation status with remote servers. ([\#11407](https://github.com/matrix-org/synapse/issues/11407 ))
- Support expiry of refresh tokens and expiry of the overall session when refresh tokens are in use. ([\#11425](https://github.com/matrix-org/synapse/issues/11425 ))
- Stabilise support for [MSC2918](https://github.com/matrix-org/matrix-doc/blob/main/proposals/2918-refreshtokens.md#msc2918-refresh-tokens ) refresh tokens as they have now been merged into the Matrix specification. ([\#11435](https://github.com/matrix-org/synapse/issues/11435 ), [\#11522](https://github.com/matrix-org/synapse/issues/11522 ))
- Update [MSC2918 refresh token](https://github.com/matrix-org/matrix-doc/blob/main/proposals/2918-refreshtokens.md#msc2918-refresh-tokens ) support to confirm with the latest revision: accept the `refresh_tokens` parameter in the request body rather than in the URL parameters. ([\#11430](https://github.com/matrix-org/synapse/issues/11430 ))
- Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. ([\#11445](https://github.com/matrix-org/synapse/issues/11445 ))
- Expose `synapse_homeserver` and `synapse_worker` commands as entry points to run Synapse's main process and worker processes, respectively. Contributed by @Ma27. ([\#11449](https://github.com/matrix-org/synapse/issues/11449 ))
- `synctl stop` will now wait for Synapse to exit before returning. ([\#11459](https://github.com/matrix-org/synapse/issues/11459 ), [\#11490](https://github.com/matrix-org/synapse/issues/11490 ))
- Extend the "delete room" admin api to work correctly on rooms which have previously been partially deleted. ([\#11523](https://github.com/matrix-org/synapse/issues/11523 ))
- Add support for the `/_matrix/client/v3/login/sso/redirect/{idpId}` API from Matrix v1.1. This endpoint was overlooked when support for v3 endpoints was added in Synapse 1.48.0rc1. ([\#11451](https://github.com/matrix-org/synapse/issues/11451 ))
Bugfixes
--------
- Fix using [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) batch sending in combination with event persistence workers. Contributed by @tulir at Beeper. ([\#11220](https://github.com/matrix-org/synapse/issues/11220 ))
- Fix a long-standing bug where all requests that read events from the database could get stuck as a result of losing the database connection, properly this time. Also fix a race condition introduced in the previous insufficient fix in Synapse 1.47.0. ([\#11376](https://github.com/matrix-org/synapse/issues/11376 ))
- The `/send_join` response now includes the stable `event` field instead of the unstable field from [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083 ). ([\#11413](https://github.com/matrix-org/synapse/issues/11413 ))
- Fix a bug introduced in Synapse 1.47.0 where `send_join` could fail due to an outdated `ijson` version. ([\#11439](https://github.com/matrix-org/synapse/issues/11439 ), [\#11441](https://github.com/matrix-org/synapse/issues/11441 ), [\#11460](https://github.com/matrix-org/synapse/issues/11460 ))
- Fix a bug introduced in Synapse 1.36.0 which could cause problems fetching event-signing keys from trusted key servers. ([\#11440](https://github.com/matrix-org/synapse/issues/11440 ))
- Fix a bug introduced in Synapse 1.47.1 where the media repository would fail to work if the media store path contained any symbolic links. ([\#11446](https://github.com/matrix-org/synapse/issues/11446 ))
- Fix an `LruCache` corruption bug, introduced in Synapse 1.38.0, that would cause certain requests to fail until the next Synapse restart. ([\#11454](https://github.com/matrix-org/synapse/issues/11454 ))
- Fix a long-standing bug where invites from ignored users were included in incremental syncs. ([\#11511](https://github.com/matrix-org/synapse/issues/11511 ))
- Fix a regression in Synapse 1.48.0 where presence workers would not clear their presence updates over replication on shutdown. ([\#11518](https://github.com/matrix-org/synapse/issues/11518 ))
- Fix a regression in Synapse 1.48.0 where the module API's `looping_background_call` method would spam errors to the logs when given a non-async function. ([\#11524](https://github.com/matrix-org/synapse/issues/11524 ))
Updates to the Docker image
---------------------------
- Update `Dockerfile-workers` to healthcheck all workers in the container. ([\#11429](https://github.com/matrix-org/synapse/issues/11429 ))
Improved Documentation
----------------------
- Update the media repository documentation. ([\#11415](https://github.com/matrix-org/synapse/issues/11415 ))
- Update section about backward extremities in the room DAG concepts doc to correct the misconception about backward extremities indicating whether we have fetched an events' `prev_events`. ([\#11469](https://github.com/matrix-org/synapse/issues/11469 ))
Internal Changes
----------------
- Add `Final` annotation to string constants in `synapse.api.constants` so that they get typed as `Literal`s. ([\#11356](https://github.com/matrix-org/synapse/issues/11356 ))
- Add a check to ensure that users cannot start the Synapse master process when `worker_app` is set. ([\#11416](https://github.com/matrix-org/synapse/issues/11416 ))
- Add a note about postgres memory management and hugepages to postgres doc. ([\#11467](https://github.com/matrix-org/synapse/issues/11467 ))
- Add missing type hints to `synapse.config` module. ([\#11465](https://github.com/matrix-org/synapse/issues/11465 ))
- Add missing type hints to `synapse.federation`. ([\#11483](https://github.com/matrix-org/synapse/issues/11483 ))
- Add type annotations to `tests.storage.test_appservice`. ([\#11488](https://github.com/matrix-org/synapse/issues/11488 ), [\#11492](https://github.com/matrix-org/synapse/issues/11492 ))
- Add type annotations to some of the configuration surrounding refresh tokens. ([\#11428](https://github.com/matrix-org/synapse/issues/11428 ))
- Add type hints to `synapse/tests/rest/admin`. ([\#11501](https://github.com/matrix-org/synapse/issues/11501 ))
- Add type hints to storage classes. ([\#11411](https://github.com/matrix-org/synapse/issues/11411 ))
- Add wiki pages to documentation website. ([\#11402](https://github.com/matrix-org/synapse/issues/11402 ))
- Clean up `tests.storage.test_main` to remove use of legacy code. ([\#11493](https://github.com/matrix-org/synapse/issues/11493 ))
- Clean up `tests.test_visibility` to remove legacy code. ([\#11495](https://github.com/matrix-org/synapse/issues/11495 ))
- Convert status codes to `HTTPStatus` in `synapse.rest.admin`. ([\#11452](https://github.com/matrix-org/synapse/issues/11452 ), [\#11455](https://github.com/matrix-org/synapse/issues/11455 ))
- Extend the `scripts-dev/sign_json` script to support signing events. ([\#11486](https://github.com/matrix-org/synapse/issues/11486 ))
- Improve internal types in push code. ([\#11409](https://github.com/matrix-org/synapse/issues/11409 ))
- Improve type annotations in `synapse.module_api`. ([\#11029](https://github.com/matrix-org/synapse/issues/11029 ))
- Improve type hints for `LruCache`. ([\#11453](https://github.com/matrix-org/synapse/issues/11453 ))
- Preparation for database schema simplifications: disambiguate queries on `state_key`. ([\#11497](https://github.com/matrix-org/synapse/issues/11497 ))
- Refactor `backfilled` into specific behavior function arguments (`_persist_events_and_state_updates` and downstream calls). ([\#11417](https://github.com/matrix-org/synapse/issues/11417 ))
- Refactor `get_version_string` to fix-up types and duplicated code. ([\#11468](https://github.com/matrix-org/synapse/issues/11468 ))
- Refactor various parts of the `/sync` handler. ([\#11494](https://github.com/matrix-org/synapse/issues/11494 ), [\#11515](https://github.com/matrix-org/synapse/issues/11515 ))
- Remove unnecessary `json.dumps` from `tests.rest.admin`. ([\#11461](https://github.com/matrix-org/synapse/issues/11461 ))
- Save the OpenID Connect session ID on login. ([\#11482](https://github.com/matrix-org/synapse/issues/11482 ))
- Update and clean up recently ported documentation pages. ([\#11466](https://github.com/matrix-org/synapse/issues/11466 ))
2021-12-14 14:22:16 +00:00
4dd9ea8f4f
Revert "Revert accidental fast-forward merge from v1.49.0rc1"
...
This reverts commit 158d73ebdd
2021-12-14 14:22:01 +00:00
92906e1b60
Restructure changelog
2021-12-14 13:00:46 +00:00
9f3c7e85a4
1.49.0
2021-12-14 12:56:14 +00:00
a4dce5b53d
Remove redundant `COALESCE()`s around `COUNT()`s in database queries ( #11570 )
...
`COUNT()` never returns `NULL`. A `COUNT(*)` over 0 rows is 0 and a
`COUNT(NULL)` is also 0.
2021-12-14 12:34:30 +00:00
33abbc3278
Add missing type hints to synapse.http. ( #11571 )
2021-12-14 07:00:47 -05:00
ff6fd52160
checks for generators in database functions ( #11564 )
...
A couple of safety-checks to hopefully stop people doing what I just did, and create a storage
function which only works the first time it is called (and not when it is re-run due to a database
concurrency error or similar).
2021-12-13 19:01:27 +00:00
eb39da6782
Move HTML parsing to a separate file for URL previews. ( #11566 )
...
* Splits the logic for parsing HTML from the resource handling code.
* Fix a circular import in the oEmbed code (which uses the HTML parsing code).
* Renames some of the HTML parsing methods to:
* Make it clear which methods are "internal" to the module.
* Clarify what the methods do.
2021-12-13 17:55:07 +00:00
5305a5e881
Type hint the constructors of the data store classes ( #11555 )
2021-12-13 17:05:00 +00:00
1abfb15f07
Add type hints to `synapse/storage/databases/main/end_to_end_keys.py` ( #11551 )
2021-12-13 16:28:26 +00:00
6da8591f2e
Add type hints to `synapse/storage/databases/main/account_data.py` ( #11546 )
2021-12-13 16:28:10 +00:00
e5cdb9e233
Make `get_device` return None if the device doesn't exist rather than raising an exception. ( #11565 )
...
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-12-13 15:39:43 +00:00
aa8708ebed
Allow events to be created with no `prev_events` (MSC2716) ( #11243 )
...
The event still needs to have `auth_events` defined to be valid.
Split out from https://github.com/matrix-org/synapse/pull/11114
2021-12-10 23:08:51 -06:00
8391bd6ab5
Test to ensure we share the same `state_group` across the whole historical batch (MSC2716) ( #11487 )
...
Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716
We did some work on making sure the `state_groups` were shared in
https://github.com/matrix-org/synapse/pull/10975
2021-12-10 20:59:20 -06:00
fd2dadb815
Adjust _get_rooms_changed comments ( #11550 )
...
C.f. https://github.com/matrix-org/synapse/pull/11494#pullrequestreview-827780886
2021-12-10 19:19:48 +00:00
f0562183e7
skip some dict munging in event persistence ( #11560 )
...
Create a new dict helper method `simple_insert_many_values_txn`, which takes
raw row values, rather than {key=>value} dicts. This saves us a bunch of dict
munging, and makes it easier to use generators rather than creating
intermediate lists and dicts.
2021-12-10 15:02:33 +00:00
86e7a6d16e
Stop populating `state_events.prev_state` ( #11558 )
...
this field is never read, so we may as well stop populating it.
2021-12-10 14:13:23 +00:00
9562f0c2f1
Ensure emails are canonicalized before fetching associated user. ( #11547 )
...
This should fix pushers with an email in non-canonical form is used as
the pushkey.
2021-12-10 07:17:28 -05:00
3b8872299a
Do not allow cross-room relations, per MSC2674. ( #11516 )
2021-12-09 13:16:01 -05:00
0cc3bf97b4
Additional type hints for the config module, part 2. ( #11480 )
2021-12-09 16:15:46 +00:00
941ebe49ff
Use HTTPStatus constants in place of literals in `synapse.http` ( #11543 )
2021-12-09 11:58:25 +00:00
b47d10dc46
Support unprefixed versions of fallback key property names. ( #11541 )
2021-12-09 11:41:27 +00:00
b3bcacf3c1
Add missing `errcode` to `parse_string` and `parse_boolean` ( #11542 )
2021-12-09 11:23:34 +00:00
afa0a5e4fc
Allow guests to send state events ( #11378 )
2021-12-09 11:02:05 +00:00
d93362d87f
Add a constant for receipt types (m.read). ( #11531 )
...
And expand some type hints in the receipts storage module.
2021-12-08 12:26:29 -05:00
7ecaa3b976
Clean up `synapse.rest.admin` ( #11535 )
2021-12-08 16:59:40 +00:00
83a74d9350
Document the usage of refresh tokens. ( #11427 )
...
Co-authored-by: David Robertson <davidr@element.io>
2021-12-08 15:31:17 +00:00
365e9482fe
Use HTTPStatus constants in place of literals in `tests.rest.client.test_auth`. ( #11520 )
2021-12-08 14:54:47 +00:00
ff7cc17b57
Improve log messages for stream ids ( #11536 )
...
Somehow I'd managed to get my database in a pickle with stream ids. These
changes were useful to debug.
2021-12-08 14:15:14 +00:00
8541809cb9
Send and handle cross-signing messages using the stable prefix. ( #10520 )
2021-12-08 10:01:38 +00:00
d6fb96e056
Fix case in `wait_for_background_updates` where `self.store` does not exist ( #11331 )
...
Pull the DataStore from the HomeServer instance, which
always exists.
2021-12-07 11:51:53 -05:00
158d73ebdd
Revert accidental fast-forward merge from v1.49.0rc1
...
Revert "Sort internal changes in changelog"
Revert "Update CHANGES.md"
Revert "1.49.0rc1"
Revert "Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505 ) (#11527 )"
Revert "Refactors in `_generate_sync_entry_for_rooms` (#11515 )"
Revert "Correctly register shutdown handler for presence workers (#11518 )"
Revert "Fix `ModuleApi.looping_background_call` for non-async functions (#11524 )"
Revert "Fix 'delete room' admin api to work on incomplete rooms (#11523 )"
Revert "Correctly ignore invites from ignored users (#11511 )"
Revert "Fix the test breakage introduced by #11435 as a result of concurrent PRs (#11522 )"
Revert "Stabilise support for MSC2918 refresh tokens as they have now been merged into the Matrix specification. (#11435 )"
Revert "Save the OIDC session ID (sid) with the device on login (#11482 )"
Revert "Add admin API to get some information about federation status (#11407 )"
Revert "Include bundled aggregations in /sync and related fixes (#11478 )"
Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505 )"
Revert "Update backward extremity docs to make it clear that it does not indicate whether we have fetched an events' `prev_events` (#11469 )"
Revert "Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. (#11445 )"
Revert "Add type hints to `synapse/tests/rest/admin` (#11501 )"
Revert "Revert accidental commits to develop."
Revert "Newsfile"
Revert "Give `tests.server.setup_test_homeserver` (nominally!) the same behaviour"
Revert "Move `tests.utils.setup_test_homeserver` to `tests.server`"
Revert "Convert one of the `setup_test_homeserver`s to `make_test_homeserver_synchronous`"
Revert "Disambiguate queries on `state_key` (#11497 )"
Revert "Comments on the /sync tentacles (#11494 )"
Revert "Clean up tests.storage.test_appservice (#11492 )"
Revert "Clean up `tests.storage.test_main` to remove use of legacy code. (#11493 )"
Revert "Clean up `tests.test_visibility` to remove legacy code. (#11495 )"
Revert "Minor cleanup on recently ported doc pages (#11466 )"
Revert "Add most of the missing type hints to `synapse.federation`. (#11483 )"
Revert "Avoid waiting for zombie processes in `synctl stop` (#11490 )"
Revert "Fix media repository failing when media store path contains symlinks (#11446 )"
Revert "Add type annotations to `tests.storage.test_appservice`. (#11488 )"
Revert "`scripts-dev/sign_json`: support for signing events (#11486 )"
Revert "Add MSC3030 experimental client and federation API endpoints to get the closest event to a given timestamp (#9445 )"
Revert "Port wiki pages to documentation website (#11402 )"
Revert "Add a license header and comment. (#11479 )"
Revert "Clean-up get_version_string (#11468 )"
Revert "Link background update controller docs to summary (#11475 )"
Revert "Additional type hints for config module. (#11465 )"
Revert "Register the login redirect endpoint for v3. (#11451 )"
Revert "Update openid.md"
Revert "Remove mention of OIDC certification from Dex (#11470 )"
Revert "Add a note about huge pages to our Postgres doc (#11467 )"
Revert "Don't start Synapse master process if `worker_app` is set (#11416 )"
Revert "Expose worker & homeserver as entrypoints in `setup.py` (#11449 )"
Revert "Bundle relations of relations into the `/relations` result. (#11284 )"
Revert "Fix `LruCache` corruption bug with a `size_callback` that can return 0 (#11454 )"
Revert "Eliminate a few `Any`s in `LruCache` type hints (#11453 )"
Revert "Remove unnecessary `json.dumps` from `tests.rest.admin` (#11461 )"
Revert "Merge branch 'master' into develop"
This reverts commit 26b5d2320fbce4220f38966b5d0fa0088d748f2c14d593f72d2a3ec6facfeccc49d755b1ecd19c5d9c55dedc8c2d42e586a82f053f3f82a15a893df88b4b153c9e494ebd7347a77c3698974eb77965cd637df95de6e5f426cd548cd68b81026cae125e207be88fbf48b3fd99b74af7ec6e7d9e5640992d17d26808dd85f91624a59516d39a54908a4c29698749e1356ee3d2279f471bb50e39df57858d80bf0f435f044807f61462e1bea6f1a3abec84dc50e160ed635d32857b62791e00153194c771f44d729d4ca265fbd397b9fef1a7cdb0eb64ff7bf1795463bf70cbb1a5e342bf020463379f2650cf7ff22d6da45a0b652d36432a174bc1b14f8a1bafe713855dca
2021-12-07 16:47:31 +00:00
26b5d2320f
Sort internal changes in changelog
2021-12-07 14:46:37 +00:00
Richard van der Hoff