941ebe49ff
Use HTTPStatus constants in place of literals in `synapse.http` ( #11543 )
2021-12-09 11:58:25 +00:00
b3bcacf3c1
Add missing `errcode` to `parse_string` and `parse_boolean` ( #11542 )
2021-12-09 11:23:34 +00:00
a6f1a3abec
Add MSC3030 experimental client and federation API endpoints to get the closest event to a given timestamp ( #9445 )
...
MSC3030: https://github.com/matrix-org/matrix-doc/pull/3030
Client API endpoint. This will also go and fetch from the federation API endpoint if unable to find an event locally or we found an extremity with possibly a closer event we don't know about.
```
GET /_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>
{
"event_id": ...
"origin_server_ts": ...
}
```
Federation API endpoint:
```
GET /_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>
{
"event_id": ...
"origin_server_ts": ...
}
```
Co-authored-by: Erik Johnston <erik@matrix.org>
2021-12-02 01:02:20 -06:00
b64b6d12d4
Add more type hints to synapse.util. ( #11321 )
2021-11-12 13:43:06 +00:00
1bfd141205
Type hints for the remaining two files in `synapse.http`. ( #11164 )
...
* Teach MyPy that the sentinel context is False
This means that if `ctx: LoggingContextOrSentinel`
then `bool(ctx)` narrows us to `ctx:LoggingContext`, which is a really
neat find!
* Annotate RequestMetrics
- Raise errors for sentry if we use the sentinel context
- Ensure we don't raise an error and carry on, but not recording stats
- Include stack trace in the error case to lower Sean's blood pressure
* Make mypy pass for synapse.http.request_metrics
* Make synapse.http.connectproxyclient pass mypy
Co-authored-by: reivilibre <oliverw@matrix.org>
2021-10-28 14:14:42 +01:00
2b82ec425f
Add type hints for most `HomeServer` parameters ( #11095 )
2021-10-22 18:15:41 +01:00
797ee7812d
Relax `ignore-missing-imports` for modules that have stubs now and update mypy ( #11006 )
...
Updating mypy past version 0.9 means that third-party stubs are no-longer distributed with typeshed. See http://mypy-lang.blogspot.com/2021/06/mypy-0900-released.html for details.
We therefore pull in stub packages in setup.py
Additionally, some modules that we were previously ignoring import failures for now have stubs. So let's use them.
The rest of this change consists of fixups to make the newer mypy + stubs pass CI.
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-10-08 14:49:41 +01:00
52aefd5086
Catch AttributeErrors when calling registerProducer ( #10995 )
...
Looks like the wrong exception type was caught in #10932 .
2021-10-07 10:37:10 +00:00
a19aa8b162
Synapse 1.44.0rc1 (2021-09-29)
...
==============================
Features
--------
- Only allow the [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) `/batch_send?chunk_id=xxx` endpoint to connect to an already existing insertion event. ([\#10776](https://github.com/matrix-org/synapse/issues/10776 ))
- Improve oEmbed URL previews by processing the author name, photo, and video information. ([\#10814](https://github.com/matrix-org/synapse/issues/10814 ), [\#10819](https://github.com/matrix-org/synapse/issues/10819 ))
- Speed up responding with large JSON objects to requests. ([\#10868](https://github.com/matrix-org/synapse/issues/10868 ), [\#10905](https://github.com/matrix-org/synapse/issues/10905 ))
- Add a `user_may_create_room_with_invites` spam checker callback to allow modules to allow or deny a room creation request based on the invites and/or 3PID invites it includes. ([\#10898](https://github.com/matrix-org/synapse/issues/10898 ))
Bugfixes
--------
- Fix a long-standing bug that caused an `AssertionError` when purging history in certain rooms. Contributed by @Kokokokoka. ([\#10690](https://github.com/matrix-org/synapse/issues/10690 ))
- Fix a long-standing bug which caused deactivated users that were later reactivated to be missing from the user directory. ([\#10782](https://github.com/matrix-org/synapse/issues/10782 ))
- Fix a long-standing bug that caused unbanning a user by sending a membership event to fail. Contributed by @aaronraimist. ([\#10807](https://github.com/matrix-org/synapse/issues/10807 ))
- Fix a long-standing bug where logging contexts would go missing when federation requests time out. ([\#10810](https://github.com/matrix-org/synapse/issues/10810 ))
- Fix a long-standing bug causing an error in the deprecated `/initialSync` endpoint when using the undocumented `from` and `to` parameters. ([\#10827](https://github.com/matrix-org/synapse/issues/10827 ))
- Fix a bug causing the `remove_stale_pushers` background job to repeatedly fail and log errors. This bug affected Synapse servers that had been upgraded from version 1.28 or older and are using SQLite. ([\#10843](https://github.com/matrix-org/synapse/issues/10843 ))
- Fix a long-standing bug in Unicode support of the room search admin API breaking search for rooms with non-ASCII characters. ([\#10859](https://github.com/matrix-org/synapse/issues/10859 ))
- Fix a bug introduced in Synapse 1.37.0 which caused `knock` membership events which we sent to remote servers to be incorrectly stored in the local database. ([\#10873](https://github.com/matrix-org/synapse/issues/10873 ))
- Fix invalidating one-time key count cache after claiming keys. The bug was introduced in Synapse v1.41.0. Contributed by Tulir at Beeper. ([\#10875](https://github.com/matrix-org/synapse/issues/10875 ))
- Fix a long-standing bug causing application service users to be subject to MAU blocking if the MAU limit had been reached, even if configured not to be blocked. ([\#10881](https://github.com/matrix-org/synapse/issues/10881 ))
- Fix a long-standing bug which could cause events pulled over federation to be incorrectly rejected. ([\#10907](https://github.com/matrix-org/synapse/issues/10907 ))
- Fix a long-standing bug causing URL cache files to be stored in storage providers. Server admins may safely delete the `url_cache/` and `url_cache_thumbnails/` directories from any configured storage providers to reclaim space. ([\#10911](https://github.com/matrix-org/synapse/issues/10911 ))
- Fix a long-standing bug leading to race conditions when creating media store and config directories. ([\#10913](https://github.com/matrix-org/synapse/issues/10913 ))
Improved Documentation
----------------------
- Fix some crashes in the Module API example code, by adding JSON encoding/decoding. ([\#10845](https://github.com/matrix-org/synapse/issues/10845 ))
- Add developer documentation about experimental configuration flags. ([\#10865](https://github.com/matrix-org/synapse/issues/10865 ))
- Properly remove deleted files from GitHub pages when generating the documentation. ([\#10869](https://github.com/matrix-org/synapse/issues/10869 ))
Internal Changes
----------------
- Fix GitHub Actions config so we can run sytest on synapse from parallel branches. ([\#10659](https://github.com/matrix-org/synapse/issues/10659 ))
- Split out [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) meta events to their own fields in the `/batch_send` response. ([\#10777](https://github.com/matrix-org/synapse/issues/10777 ))
- Add missing type hints to REST servlets. ([\#10785](https://github.com/matrix-org/synapse/issues/10785 ), [\#10817](https://github.com/matrix-org/synapse/issues/10817 ))
- Simplify the internal logic which maintains the user directory database tables. ([\#10796](https://github.com/matrix-org/synapse/issues/10796 ))
- Use direct references to config flags. ([\#10812](https://github.com/matrix-org/synapse/issues/10812 ), [\#10885](https://github.com/matrix-org/synapse/issues/10885 ), [\#10893](https://github.com/matrix-org/synapse/issues/10893 ), [\#10897](https://github.com/matrix-org/synapse/issues/10897 ))
- Specify the type of token in generic "Invalid token" error messages. ([\#10815](https://github.com/matrix-org/synapse/issues/10815 ))
- Make `StateFilter` frozen so it is hashable. ([\#10816](https://github.com/matrix-org/synapse/issues/10816 ))
- Fix a long-standing bug where an `m.room.message` event containing a null byte would cause an internal server error. ([\#10820](https://github.com/matrix-org/synapse/issues/10820 ))
- Add type hints to the state database. ([\#10823](https://github.com/matrix-org/synapse/issues/10823 ))
- Opt out of cache expiry for `get_users_who_share_room_with_user`, to hopefully improve `/sync` performance when you
haven't synced recently. ([\#10826](https://github.com/matrix-org/synapse/issues/10826 ))
- Track cache eviction rates more finely in Prometheus's monitoring. ([\#10829](https://github.com/matrix-org/synapse/issues/10829 ))
- Add missing type hints to `synapse.handlers`. ([\#10831](https://github.com/matrix-org/synapse/issues/10831 ), [\#10856](https://github.com/matrix-org/synapse/issues/10856 ))
- Extend the Module API to let plug-ins check whether an ID is local and to access IP + User Agent data. ([\#10833](https://github.com/matrix-org/synapse/issues/10833 ))
- Factor out PNG image data to a constant to be used in several tests. ([\#10834](https://github.com/matrix-org/synapse/issues/10834 ))
- Add a test to ensure state events sent by modules get persisted correctly. ([\#10835](https://github.com/matrix-org/synapse/issues/10835 ))
- Rename [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) fields and event types from `chunk` to `batch` to match the `/batch_send` endpoint. ([\#10838](https://github.com/matrix-org/synapse/issues/10838 ))
- Rename [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) `/batch_send` query parameter from `?prev_event` to more obvious usage with `?prev_event_id`. ([\#10839](https://github.com/matrix-org/synapse/issues/10839 ))
- Add type hints to `synapse.http.site`. ([\#10867](https://github.com/matrix-org/synapse/issues/10867 ))
- Include outlier status when we log V2 or V3 events. ([\#10879](https://github.com/matrix-org/synapse/issues/10879 ))
- Break down Grafana's cache expiry time series based on reason for eviction, c.f. [\#10829](https://github.com/matrix-org/synapse/issues/10829 ). ([\#10880](https://github.com/matrix-org/synapse/issues/10880 ))
- Clean up some of the federation event authentication code for clarity. ([\#10883](https://github.com/matrix-org/synapse/issues/10883 ), [\#10884](https://github.com/matrix-org/synapse/issues/10884 ), [\#10896](https://github.com/matrix-org/synapse/issues/10896 ), [\#10901](https://github.com/matrix-org/synapse/issues/10901 ))
- Allow the `.` and `~` characters when creating registration tokens as per the change to [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231 ). ([\#10887](https://github.com/matrix-org/synapse/issues/10887 ))
- Clean up some unnecessary parentheses in places around the codebase. ([\#10889](https://github.com/matrix-org/synapse/issues/10889 ))
- Improve type hinting in the user directory code. ([\#10891](https://github.com/matrix-org/synapse/issues/10891 ))
- Update development testing script `test_postgresql.sh` to use a supported Python version and make re-runs quicker. ([\#10906](https://github.com/matrix-org/synapse/issues/10906 ))
- Document and summarize changes in schema version `61` – `64`. ([\#10917](https://github.com/matrix-org/synapse/issues/10917 ))
- Update release script to sign the newly created git tags. ([\#10925](https://github.com/matrix-org/synapse/issues/10925 ))
- Fix Debian builds due to `dh-virtualenv` no longer being able to build their docs. ([\#10931](https://github.com/matrix-org/synapse/issues/10931 ))
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEE8SRSDO7gYkSP4chELS76LzL74EcFAmFUY3UACgkQLS76LzL7
4EdDdg//egZvwUizHJAEWi5Vz029S2D3LN8Hl57TG+/KR+GgIJvOfgX9sgWDNsLA
4wTlsw6xhH9VcDzFhiAPrUTwhi4Gi80BZmnGXuR97yAE1z3oyS4IQg/g5LUXq9el
a/2JsK1Sau354TBMvLQgZsplqC+RPxE0k0MSTvEC+d3ft929jLXUftnI/nhGcDpX
7pP9RscVThkLLNC5Xcuu2ev4yPSrNNWhBeqz5J8CfbEW6Roej7a1d3WdehhoWJ5V
hwPOxOhL89lWwJ8uL/H5sDFMkxk+ua+IVQx8gJkQ1eRa3RIdFFSh1+exFtYwjIhI
Pd4wwUH7hGe1VIvmlVOoDZ1hka0RTjRLeJg39vq+yvWCBQW7syVE6sRUxTjeUvRe
7Cyz4sVG0g8uvWsF3YYva7S4SlYPPSj2Z+5/yL4RHlMZsPk/apmCRZBzaWlVEGur
XqTExi2rKzKsi2HZctHDWOeYF+G3c13FDNeXG+N2g/DfGKuKCTuhbdMm7LeIJ2L2
B8ki/nq3wc1FCJ0mkVlM+/UX3i3wXG+2UbOx29hU9yuEUIl4p4jyE1OjuH5l45TX
OWfW25BOGh5gYKvX8p9Kj4A9MIJ2YFRDFfcfF7g+S0ZA8KdUAjZkYBzfJtEzn2iK
uxVsSVdYOVqbbSwE5wto8WuOWu1op6RP3l1Wp92SoM5vqWjRmyA=
=SzDp
-----END PGP SIGNATURE-----
Merge tag 'v1.44.0rc1' into develop
Synapse 1.44.0rc1 (2021-09-29)
==============================
Features
--------
- Only allow the
[MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 )
`/batch_send?chunk_id=xxx` endpoint to connect to an already existing
insertion event.
([\#10776](https://github.com/matrix-org/synapse/issues/10776 ))
- Improve oEmbed URL previews by processing the author name, photo, and
video information.
([\#10814](https://github.com/matrix-org/synapse/issues/10814 ),
[\#10819](https://github.com/matrix-org/synapse/issues/10819 ))
- Speed up responding with large JSON objects to requests.
([\#10868](https://github.com/matrix-org/synapse/issues/10868 ),
[\#10905](https://github.com/matrix-org/synapse/issues/10905 ))
- Add a `user_may_create_room_with_invites` spam checker callback to
allow modules to allow or deny a room creation request based on the
invites and/or 3PID invites it includes.
([\#10898](https://github.com/matrix-org/synapse/issues/10898 ))
Bugfixes
--------
- Fix a long-standing bug that caused an `AssertionError` when purging
history in certain rooms. Contributed by @Kokokokoka.
([\#10690](https://github.com/matrix-org/synapse/issues/10690 ))
- Fix a long-standing bug which caused deactivated users that were later
reactivated to be missing from the user directory.
([\#10782](https://github.com/matrix-org/synapse/issues/10782 ))
- Fix a long-standing bug that caused unbanning a user by sending a
membership event to fail. Contributed by @aaronraimist.
([\#10807](https://github.com/matrix-org/synapse/issues/10807 ))
- Fix a long-standing bug where logging contexts would go missing when
federation requests time out.
([\#10810](https://github.com/matrix-org/synapse/issues/10810 ))
- Fix a long-standing bug causing an error in the deprecated
`/initialSync` endpoint when using the undocumented `from` and `to`
parameters.
([\#10827](https://github.com/matrix-org/synapse/issues/10827 ))
- Fix a bug causing the `remove_stale_pushers` background job to
repeatedly fail and log errors. This bug affected Synapse servers that
had been upgraded from version 1.28 or older and are using SQLite.
([\#10843](https://github.com/matrix-org/synapse/issues/10843 ))
- Fix a long-standing bug in Unicode support of the room search admin
API breaking search for rooms with non-ASCII characters.
([\#10859](https://github.com/matrix-org/synapse/issues/10859 ))
- Fix a bug introduced in Synapse 1.37.0 which caused `knock` membership
events which we sent to remote servers to be incorrectly stored in the
local database.
([\#10873](https://github.com/matrix-org/synapse/issues/10873 ))
- Fix invalidating one-time key count cache after claiming keys. The bug
was introduced in Synapse v1.41.0. Contributed by Tulir at Beeper.
([\#10875](https://github.com/matrix-org/synapse/issues/10875 ))
- Fix a long-standing bug causing application service users to be
subject to MAU blocking if the MAU limit had been reached, even if
configured not to be blocked.
([\#10881](https://github.com/matrix-org/synapse/issues/10881 ))
- Fix a long-standing bug which could cause events pulled over
federation to be incorrectly rejected.
([\#10907](https://github.com/matrix-org/synapse/issues/10907 ))
- Fix a long-standing bug causing URL cache files to be stored in
storage providers. Server admins may safely delete the `url_cache/` and
`url_cache_thumbnails/` directories from any configured storage
providers to reclaim space.
([\#10911](https://github.com/matrix-org/synapse/issues/10911 ))
- Fix a long-standing bug leading to race conditions when creating media
store and config directories.
([\#10913](https://github.com/matrix-org/synapse/issues/10913 ))
Improved Documentation
----------------------
- Fix some crashes in the Module API example code, by adding JSON
encoding/decoding.
([\#10845](https://github.com/matrix-org/synapse/issues/10845 ))
- Add developer documentation about experimental configuration flags.
([\#10865](https://github.com/matrix-org/synapse/issues/10865 ))
- Properly remove deleted files from GitHub pages when generating the
documentation.
([\#10869](https://github.com/matrix-org/synapse/issues/10869 ))
Internal Changes
----------------
- Fix GitHub Actions config so we can run sytest on synapse from
parallel branches.
([\#10659](https://github.com/matrix-org/synapse/issues/10659 ))
- Split out
[MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) meta
events to their own fields in the `/batch_send` response.
([\#10777](https://github.com/matrix-org/synapse/issues/10777 ))
- Add missing type hints to REST servlets.
([\#10785](https://github.com/matrix-org/synapse/issues/10785 ),
[\#10817](https://github.com/matrix-org/synapse/issues/10817 ))
- Simplify the internal logic which maintains the user directory
database tables.
([\#10796](https://github.com/matrix-org/synapse/issues/10796 ))
- Use direct references to config flags.
([\#10812](https://github.com/matrix-org/synapse/issues/10812 ),
[\#10885](https://github.com/matrix-org/synapse/issues/10885 ),
[\#10893](https://github.com/matrix-org/synapse/issues/10893 ),
[\#10897](https://github.com/matrix-org/synapse/issues/10897 ))
- Specify the type of token in generic "Invalid token" error messages.
([\#10815](https://github.com/matrix-org/synapse/issues/10815 ))
- Make `StateFilter` frozen so it is hashable.
([\#10816](https://github.com/matrix-org/synapse/issues/10816 ))
- Fix a long-standing bug where an `m.room.message` event containing a
null byte would cause an internal server error.
([\#10820](https://github.com/matrix-org/synapse/issues/10820 ))
- Add type hints to the state database.
([\#10823](https://github.com/matrix-org/synapse/issues/10823 ))
- Opt out of cache expiry for `get_users_who_share_room_with_user`, to
hopefully improve `/sync` performance when you
haven't synced recently.
([\#10826](https://github.com/matrix-org/synapse/issues/10826 ))
- Track cache eviction rates more finely in Prometheus's monitoring.
([\#10829](https://github.com/matrix-org/synapse/issues/10829 ))
- Add missing type hints to `synapse.handlers`.
([\#10831](https://github.com/matrix-org/synapse/issues/10831 ),
[\#10856](https://github.com/matrix-org/synapse/issues/10856 ))
- Extend the Module API to let plug-ins check whether an ID is local and
to access IP + User Agent data.
([\#10833](https://github.com/matrix-org/synapse/issues/10833 ))
- Factor out PNG image data to a constant to be used in several tests.
([\#10834](https://github.com/matrix-org/synapse/issues/10834 ))
- Add a test to ensure state events sent by modules get persisted
correctly.
([\#10835](https://github.com/matrix-org/synapse/issues/10835 ))
- Rename [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 )
fields and event types from `chunk` to `batch` to match the
`/batch_send` endpoint.
([\#10838](https://github.com/matrix-org/synapse/issues/10838 ))
- Rename [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 )
`/batch_send` query parameter from `?prev_event` to more obvious usage
with `?prev_event_id`.
([\#10839](https://github.com/matrix-org/synapse/issues/10839 ))
- Add type hints to `synapse.http.site`.
([\#10867](https://github.com/matrix-org/synapse/issues/10867 ))
- Include outlier status when we log V2 or V3 events.
([\#10879](https://github.com/matrix-org/synapse/issues/10879 ))
- Break down Grafana's cache expiry time series based on reason for
eviction, c.f.
[\#10829](https://github.com/matrix-org/synapse/issues/10829 ).
([\#10880](https://github.com/matrix-org/synapse/issues/10880 ))
- Clean up some of the federation event authentication code for clarity.
([\#10883](https://github.com/matrix-org/synapse/issues/10883 ),
[\#10884](https://github.com/matrix-org/synapse/issues/10884 ),
[\#10896](https://github.com/matrix-org/synapse/issues/10896 ),
[\#10901](https://github.com/matrix-org/synapse/issues/10901 ))
- Allow the `.` and `~` characters when creating registration tokens as
per the change to
[MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231 ).
([\#10887](https://github.com/matrix-org/synapse/issues/10887 ))
- Clean up some unnecessary parentheses in places around the codebase.
([\#10889](https://github.com/matrix-org/synapse/issues/10889 ))
- Improve type hinting in the user directory code.
([\#10891](https://github.com/matrix-org/synapse/issues/10891 ))
- Update development testing script `test_postgresql.sh` to use a
supported Python version and make re-runs quicker.
([\#10906](https://github.com/matrix-org/synapse/issues/10906 ))
- Document and summarize changes in schema version `61` – `64`.
([\#10917](https://github.com/matrix-org/synapse/issues/10917 ))
- Update release script to sign the newly created git tags.
([\#10925](https://github.com/matrix-org/synapse/issues/10925 ))
- Fix Debian builds due to `dh-virtualenv` no longer being able to build
their docs.
([\#10931](https://github.com/matrix-org/synapse/issues/10931 ))
F124520CEEE062448FE1C8442D2EFA2F32FBE047
<olivier@librepush.net>" [ultimate]
2021-09-29 14:23:39 +01:00
94b620a5ed
Use direct references for configuration variables (part 6). ( #10916 )
2021-09-29 06:44:15 -04:00
37bb93d181
Fix exception responding to request that has been closed ( #10932 )
...
Introduced in #10905
2021-09-28 14:36:19 +01:00
707d5e4e48
Encode JSON responses on a thread in C, mk2 ( #10905 )
...
Currently we use `JsonEncoder.iterencode` to write JSON responses, which ensures that we don't block the main reactor thread when encoding huge objects. The downside to this is that `iterencode` falls back to using a pure Python encoder that is *much* less efficient and can easily burn a lot of CPU for huge responses. To fix this, while still ensuring we don't block the reactor loop, we encode the JSON on a threadpool using the standard `JsonEncoder.encode` functions, which is backed by a C library.
Doing so, however, requires `respond_with_json` to have access to the reactor, which it previously didn't. There are two ways of doing this:
1. threading through the reactor object, which is a bit fiddly as e.g. `DirectServeJsonResource` doesn't currently take a reactor, but is exposed to modules and so is a PITA to change; or
2. expose the reactor in `SynapseRequest`, which requires updating a bunch of servlet types.
I went with the latter as that is just a mechanical change, and I think makes sense as a request already has a reactor associated with it (via its http channel).
2021-09-28 09:37:58 +00:00
50022cff96
Add reactor to `SynapseRequest` and fix up types. ( #10868 )
2021-09-24 11:01:25 +01:00
47854c71e9
Use direct references for configuration variables (part 4). ( #10893 )
2021-09-23 12:03:01 -04:00
aa2c027792
Remove unnecessary parentheses around tuples returned from methods ( #10889 )
2021-09-23 11:59:07 +01:00
b25a494779
Add types to http.site ( #10867 )
2021-09-21 16:41:27 +00:00
8c7a531e27
Use direct references for some configuration variables (part 2) ( #10812 )
2021-09-15 08:34:52 -04:00
14b8c0476f
Prevent logging context going missing on federation request timeout ( #10810 )
...
In `MatrixFederationHttpClient._send_request()`, we make a HTTP request
using an `Agent`, wrap that request in a timeout and await the resulting
`Deferred`. On its own, the `Agent` performing the HTTP request
correctly stashes and restores the logging context while waiting.
The addition of the timeout introduces a path where the logging context
is not restored when execution resumes.
To address this, we wrap the timeout `Deferred` in a
`make_deferred_yieldable()` to stash the logging context and restore it
on completion of the `await`. However this is not sufficient, since by
the time we construct the timeout `Deferred`, the `Agent` has already
stashed and cleared the logging context when using
`make_deferred_yieldable()` to produce its `Deferred` for the request.
Hence, we wrap the `Agent` request in a `run_in_background()` to "fork"
and preserve the logging context so that we can stash and restore it
when `await`ing the timeout `Deferred`.
This approach is similar to the one used with `defer.gatherResults`.
Note that the code is still not fully correct. When a timeout occurs,
the request remains running in the background (existing behavior which
is nothing to do with the new call to `run_in_background`) and may
re-start the logging context after it has finished.
2021-09-14 13:01:30 +01:00
66d72b7e17
Change logging of puppeted requests to better differentiate users ( #10779 )
...
This used to be a comma and got accidentally changed to a period in #9654 , but a pipe character is more easier to parse visually.
2021-09-08 12:59:15 +00:00
ecbfa4fe4f
Additional type hints for client REST servlets (part 5) ( #10736 )
...
Additionally this enforce type hints on all function signatures inside
of the synapse.rest.client package.
2021-09-03 09:22:22 -04:00
0c3565da4c
Additional type hints for the proxy agent and SRV resolver modules. ( #10608 )
2021-08-18 13:53:20 -04:00
0ace38b7b3
Experimental support for MSC3266 Room Summary API. ( #10394 )
2021-08-16 14:49:12 +00:00
339c3918e1
support federation queries through http connect proxy ( #10475 )
...
Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-08-11 15:34:59 +01:00
d9cb658c78
Fix up type hints for Twisted 21.7 ( #10490 )
...
Mostly this involves decorating a few Deferred declarations with extra type hints. We wrap the types in quotes to avoid runtime errors when running against older versions of Twisted that don't have generics on Deferred.
2021-07-28 12:04:11 +00:00
076deade02
allow specifying https:// proxy ( #10411 )
2021-07-27 17:31:06 +01:00
d8324b8238
Fix a handful of type annotations. ( #10446 )
...
* switch from `types.CoroutineType` to `typing.Coroutine`
these should be identical semantically, and since `defer.ensureDeferred` is
defined to take a `typing.Coroutine`, will keep mypy happy
* Fix some annotations on inlineCallbacks functions
* changelog
2021-07-22 12:00:16 +01:00
590cc4e888
Add type hints to additional servlet functions ( #10437 )
...
Improves type hints for:
* parse_{boolean,integer}
* parse_{boolean,integer}_from_args
* parse_json_{value,object}_from_request
And fixes any incorrect calls that resulted from unknown types.
2021-07-21 18:12:22 +00:00
5db118626b
Add a return type to parse_string. ( #10438 )
...
And set the required attribute in a few places which will error if
a parameter is not provided.
2021-07-21 09:47:56 -04:00
95e47b2e78
[pyupgrade] `synapse/` ( #10348 )
...
This PR is tantamount to running
```
pyupgrade --py36-plus --keep-percent-format `find synapse/ -type f -name "*.py"`
```
Part of #9744
2021-07-19 15:28:05 +01:00
bdfde6dca1
Use inline type hints in `http/federation/`, `storage/` and `util/` ( #10381 )
2021-07-15 12:46:54 -04:00
bf72d10dbf
Use inline type hints in various other places (in `synapse/`) ( #10380 )
2021-07-15 11:02:43 +01:00
c7603af1d0
Allow providing credentials to `http_proxy` ( #10360 )
2021-07-15 10:37:08 +01:00
7695ca0618
Fix a number of logged errors caused by remote servers being down. ( #10400 )
2021-07-15 10:35:46 +01:00
0555d7b0dc
Add additional types to the federation transport server. ( #10213 )
2021-06-28 07:36:41 -04:00
27c06a6e06
Drop Origin & Accept from Access-Control-Allow-Headers value ( #10114 )
...
* Drop Origin & Accept from Access-Control-Allow-Headers value
This change drops the Origin and Accept header names from the value of the
Access-Control-Allow-Headers response header sent by Synapse. Per the CORS
protocol, it’s not necessary or useful to include those header names.
Details:
Per-spec at https://fetch.spec.whatwg.org/#forbidden-header-name , Origin
is a “forbidden header name” set by the browser and that frontend
JavaScript code is never allowed to set.
So the value of Access-Control-Allow-Headers isn’t relevant to Origin or
in general to other headers set by the browser itself — the browser
never ever consults the Access-Control-Allow-Headers value to confirm
that it’s OK for the request to include an Origin header.
And per-spec at https://fetch.spec.whatwg.org/#cors-safelisted-request-header ,
Accept is a “CORS-safelisted request-header”, which means that browsers
allow requests to contain the Accept header regardless of whether the
Access-Control-Allow-Headers value contains "Accept".
So it’s unnecessary for the Access-Control-Allow-Headers to explicitly
include Accept. Browsers will not perform a CORS preflight for requests
containing an Accept request header.
Related: https://github.com/matrix-org/matrix-doc/pull/3225
Signed-off-by: Michael[tm] Smith <mike@w3.org>
2021-06-23 11:25:03 +01:00
8070b893db
update black to 21.6b0 ( #10197 )
...
Reformat all files with the new version.
Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
2021-06-17 15:20:06 +01:00
aac2c49b9b
Fix 'ip_range_whitelist' not working for federation servers ( #10115 )
...
Add 'federation_ip_range_whitelist'. This allows backwards-compatibility, If 'federation_ip_range_blacklist' is set. Otherwise 'ip_range_whitelist' will be used for federation servers.
Signed-off-by: Michael Kutzner 1mikure@gmail.com
2021-06-15 08:53:55 +01:00
d936371b69
Implement knock feature ( #6739 )
...
This PR aims to implement the knock feature as proposed in https://github.com/matrix-org/matrix-doc/pull/2403
Signed-off-by: Sorunome mail@sorunome.de
Signed-off-by: Andrew Morgan andrewm@element.io
2021-06-09 19:39:51 +01:00
1bf83a191b
Clean up the interface for injecting opentracing over HTTP ( #10143 )
...
* Remove unused helper functions
* Clean up the interface for injecting opentracing over HTTP
* changelog
2021-06-09 11:33:00 +01:00
c7f3fb2745
Add type hints to the federation server transport. ( #10080 )
2021-06-08 11:19:25 -04:00
9e4610cc27
Correct type hints for parse_string(s)_from_args. ( #10137 )
2021-06-08 08:30:48 -04:00
3ff6fe2851
Merge branch 'master' into develop
2021-06-01 13:47:27 +01:00
1641c5c707
Log method and path when dropping request due to size limit ( #10091 )
2021-05-28 16:29:09 +01:00
84cf3e47a0
Allow response of `/send_join` to be larger. ( #10093 )
...
Fixes #10087 .
2021-05-28 16:28:01 +01:00
3f96dbbda7
Log method and path when dropping request due to size limit ( #10091 )
2021-05-28 15:57:53 +01:00
ac3e02d089
Add `parse_strings_from_args` to get `prev_events` array ( #10048 )
...
Split out from https://github.com/matrix-org/synapse/pull/9247
Strings:
- `parse_string`
- `parse_string_from_args`
- `parse_strings_from_args`
For comparison with ints:
- `parse_integer`
- `parse_integer_from_args`
Previous discussions:
- https://github.com/matrix-org/synapse/pull/9247#discussion_r573195687
- https://github.com/matrix-org/synapse/pull/9247#discussion_r574214156
- https://github.com/matrix-org/synapse/pull/9247#discussion_r573264791
Signed-off-by: Eric Eastwood <erice@element.io>
2021-05-28 14:19:06 +01:00
64887f06fc
Use ijson to parse the response to `/send_join`, reducing memory usage. ( #9958 )
...
Instead of parsing the full response to `/send_join` into Python objects (which can be huge for large rooms) and *then* parsing that into events, we instead use ijson to stream parse the response directly into `EventBase` objects.
2021-05-20 16:11:48 +01:00
3ff2251754
Improved validation for received requests ( #9817 )
...
* Simplify `start_listening` callpath
* Correctly check the size of uploaded files
2021-04-23 19:20:44 +01:00
59d24c5bef
pass a reactor into SynapseSite ( #9874 )
2021-04-23 17:06:47 +01:00
51a20914a8
Limit the size of HTTP responses read over federation. ( #9833 )
2021-04-23 11:08:41 +01:00
Dirk Klimpel