Commit Graph

15597 Commits (0a5f4f766514b84aff84ff17dffd5301a437c797)

Author SHA1 Message Date
Shay 0a5f4f7665
Move support for application service query parameter authorization behind a configuration option (#16017) 2023-08-03 11:43:51 -07:00
Mathieu Velten f0a860908b
Allow config of the backoff algorithm for the federation client. (#15754)
Adds three new configuration variables:

* destination_min_retry_interval is identical to before (10mn).
* destination_retry_multiplier is now 2 instead of 5, the maximum value will
  be reached slower.
* destination_max_retry_interval is one day instead of (essentially) infinity.

Capping this will cause destinations to continue to be retried sometimes instead
of being lost forever. The previous value was 2 ^ 62 milliseconds.
2023-08-03 14:36:55 -04:00
reivilibre 9c462f18a4
Allow modules to check whether the current worker is configured to run background tasks. (#15991) 2023-08-03 08:42:19 -04:00
Patrick Cloke 4f5bccbbba
Add forward-compatibility for the redacts property (MSC2174). (#16013)
The location of the redacts field changes in room version 11. Ensure
it is copied to the *new* location for *old* room versions for
forwards-compatibility with clients.

Note that copying it to the *old* location for the *new* room version
was previously handled.
2023-08-02 15:35:54 +00:00
Patrick Cloke 90ad836ed8
Properly setup the additional sequences in the portdb script. (#16043)
The un_partial_stated_event_stream_sequence and
application_services_txn_id_seq were never properly configured
in the portdb script, resulting in an error on start-up.
2023-08-01 10:36:33 -04:00
Mohit Rathee 5eb3fd785b
Trim whitespace when setting display names (#16031) 2023-08-01 09:14:02 -04:00
Jason Little 7cbb2a00d1
Add metrics tracking for eviction to ResponseCache (#16028)
Track whether the ResponseCache is evicting due to invalidation
or due to time.
2023-08-01 08:10:49 -04:00
Patrick Cloke b7695ac388
Combine duplicated code for calculating an event ID from a txn ID (#16023)
Refactoring related to stabilization of MSC3970, refactor to combine
code which has the same logic.
2023-07-31 08:44:45 -04:00
Erik Johnston ae55cc1e6b
Add ability to wait for locks and add locks to purge history / room deletion (#15791)
c.f. #13476
2023-07-31 10:58:03 +01:00
Shay 68b2611783
Clarify comment on key uploads over replication (#16016) 2023-07-27 15:08:46 -07:00
Mathieu Velten a719b703d9
Fix 404 on /profile when the display name is empty but not the avatar (#16012) 2023-07-27 15:45:05 +02:00
Shay f98f4f2e16
Remove support for legacy application service paths (#15964) 2023-07-26 12:59:47 -07:00
Anshul Madnawat 58f8305114
Inline SQL queries using boolean parameters (#15525)
SQLite now supports TRUE and FALSE constants, simplify some
queries by inlining those instead of passing them as arguments.
2023-07-26 18:45:47 +00:00
Mathieu Velten 8ebfd577e2
Bump DB version to 79 since synapse v1.88 was already there (#15998) 2023-07-26 14:51:44 +02:00
Shay 641ff9ef7e
Support MSC3814: Dehydrated Devices (#15929)
Signed-off-by: Nicolas Werner <n.werner@famedly.com>
Co-authored-by: Nicolas Werner <n.werner@famedly.com>
Co-authored-by: Nicolas Werner <89468146+nico-famedly@users.noreply.github.com>
Co-authored-by: Hubert Chathi <hubert@uhoreg.ca>
2023-07-24 08:23:19 -07:00
Erik Johnston 654902a758
Resync stale devices in background (#15975)
This is so we don't block responding to federation transaction while we
try and fetch the device lists.
2023-07-24 13:43:43 +01:00
Shay 5c7364fea5
Properly handle redactions of creation events (#15973) 2023-07-23 16:32:01 -07:00
Shay f08d05dd2c
Actually stop reading from column `user_id` of tables `profiles` (#15955) 2023-07-23 16:30:54 -07:00
Erik Johnston fc1e534e41
Speed up updating state in large rooms (#15971)
This should speed up updating state in rooms with lots of state.
2023-07-20 15:51:28 +01:00
Erik Johnston fd44053b84
Don't log exceptions for every non-200 response (#15969)
Introduced in #15913
2023-07-20 11:07:58 +01:00
Erik Johnston ad52db3b5c
Reduce the amount of state we pull out (#15968) 2023-07-20 10:46:37 +01:00
Erik Johnston 67f9e5293e
Ensure a long state res does not starve CPU (#15960)
We do this by yielding the reactor in hot loops.
2023-07-19 17:00:33 +00:00
Erik Johnston 19796e20aa
Fix bad merge of #15933 (#15958)
This was because we reverted the bump of the schema version, so we were not applying the new deltas.
2023-07-19 12:17:08 +00:00
Erik Johnston 40a3583ba1
Fix race in triggers for read/write locks. (#15933) 2023-07-19 12:06:38 +01:00
Shay cb6e2c6cc7
Fix background schema updates failing over a large upgrade gap (#15887) 2023-07-18 16:59:27 -07:00
Olivier Wilkinson (reivilibre) 8e8431bc6e Merge branch 'master' into develop 2023-07-18 16:45:39 +01:00
Patrick Cloke 6d81aec09f
Support room version 11 (#15912)
And fix a bug in the implementation of the updated redaction
format (MSC2174) where the top-level redacts field was not
properly added for backwards-compatibility.
2023-07-18 08:44:59 -04:00
Shay e625c3dca0
Revert "Stop writing to column `user_id` of tables `profiles` and `user_filters`. (#15953)
* Revert "Stop writing to column `user_id` of tables `profiles` and `user_filters` (#15787)"

This reverts commit f25b0f8808.

* newsfragement
2023-07-18 11:44:09 +01:00
Jason Little 199c270947
Add a locality to a few presence metrics (#15952) 2023-07-18 10:36:40 +01:00
Eric Eastwood 1c802de626
Re-introduce the outbound federation proxy (#15913)
Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).

This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.
2023-07-18 09:49:21 +01:00
Patrick Cloke 20ae617d14
Stop accepting 'user' parameter for application service registration. (#15928)
This is unspecced, but has existed for a very long time.
2023-07-13 07:23:56 -04:00
Patrick Cloke 204b66c203
Remove unneeded __init__. (#15926)
Remove an __init__ which only calls super() without changing the
input arguments.
2023-07-12 14:30:05 +00:00
Patrick Cloke 5bdf01fccd
Fix running with an empty experimental features section. (#15925) 2023-07-12 12:39:25 +00:00
Erik Johnston 36c6b92bfc
Fix push for invites received over federation (#15820) 2023-07-12 11:02:11 +00:00
Mathieu Velten 8eb7bb975e
Mark get_user_in_directory private since only used in tests (#15884) 2023-07-12 11:09:13 +02:00
Eric Eastwood ae391db777
Better warning in logs when we fail to fetch an alias (#15922)
**Before:**
```
Error retrieving alias
```

**After:**
```
Error retrieving alias #foo:bar -> 401 Unauthorized
```

*Spawning from creating the [manual testing strategy for the outbound federation proxy](https://github.com/matrix-org/synapse/pull/15773).*
2023-07-11 17:12:41 -05:00
Jason Little 224ef0b669
Unix Sockets for HTTP Replication (#15708)
Unix socket support for `federation` and `client` Listeners has existed now for a little while(since [1.81.0](https://github.com/matrix-org/synapse/pull/15353)), but there was one last hold out before it could be complete: HTTP Replication communication. This should finish it up. The Listeners would have always worked, but would have had no way to be talked to/at.

---------

Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Co-authored-by: Eric Eastwood <erice@element.io>
2023-07-11 13:08:06 -05:00
Patrick Cloke a4243183f0
Add + as an allowed character for Matrix IDs (MSC4009) (#15911) 2023-07-11 12:21:00 -04:00
Michael Telatynski b516d91999
Add `Server` to Access-Control-Expose-Headers header (#15908) 2023-07-11 09:18:50 +01:00
Eric Eastwood 2328e90fbb
Make the media `/upload` tracing less ambiguous (#15888)
A lot of the functions have the same name in this space like `store_file`,
and we also do it multiple times for different reasons (main media repo,
other storage providers, thumbnails, etc) so it's good to differentiate
them so your head doesn't explode.

Follow-up to https://github.com/matrix-org/synapse/pull/15850

Tracing instrumentation to media `/upload` code paths to investigate https://github.com/matrix-org/synapse/issues/15841
2023-07-10 17:23:11 -05:00
Eric Eastwood c9bf644fa0
Revert "Federation outbound proxy" (#15910)
Revert "Federation outbound proxy (#15773)"

This reverts commit b07b14b494.
2023-07-10 11:10:20 -05:00
Erik Johnston e55a9b3e41
Fix downgrading to previous version of Synapse (#15907)
We do this by marking the constraint as deferrable.
2023-07-10 16:24:42 +01:00
Shay f25b0f8808
Stop writing to column `user_id` of tables `profiles` and `user_filters` (#15787) 2023-07-07 09:23:27 -07:00
Jason Little 2481b7dfa4
Remove `worker_replication_*` deprecated settings, with helpful errors on startup (#15860)
Co-authored-by: reivilibre <oliverw@matrix.org>
2023-07-07 07:45:25 +00:00
sarthak shah f19dd39dfc
Update link to the clients webpage, fix #15825 (#15874) 2023-07-06 17:28:09 +02:00
Eric Eastwood b07b14b494
Federation outbound proxy (#15773)
Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).

This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.

The original code is from @erikjohnston's branches which I've gotten in-shape to merge.
2023-07-05 18:53:55 -05:00
Eric Eastwood 561d06b481
Remove support for Python 3.7 (#15851)
Fix https://github.com/matrix-org/synapse/issues/15836
2023-07-05 18:45:42 -05:00
Erik Johnston 39d131b016
Add basic read/write lock (#15782) 2023-07-05 17:25:00 +01:00
Eric Eastwood ce857c05d5
Add tracing to media `/upload` endpoint (#15850)
Add tracing instrumentation to media `/upload` code paths to investigate https://github.com/matrix-org/synapse/issues/15841
2023-07-05 10:22:21 -05:00
Jason Little 4cf9f92f39
Fix could not serialize access due to concurrent `DELETE` from presence_stream (#15826)
* Change update_presence to have a isolation level of READ_COMMITTED

* changelog
2023-07-05 11:44:02 +01:00