Commit Graph

3431 Commits (7cf88d743e9dd9f9b2d67d13e666d0258cd8e541)

Author SHA1 Message Date
Shay 0a5f4f7665
Move support for application service query parameter authorization behind a configuration option (#16017) 2023-08-03 11:43:51 -07:00
Mathieu Velten f0a860908b
Allow config of the backoff algorithm for the federation client. (#15754)
Adds three new configuration variables:

* destination_min_retry_interval is identical to before (10mn).
* destination_retry_multiplier is now 2 instead of 5, the maximum value will
  be reached slower.
* destination_max_retry_interval is one day instead of (essentially) infinity.

Capping this will cause destinations to continue to be retried sometimes instead
of being lost forever. The previous value was 2 ^ 62 milliseconds.
2023-08-03 14:36:55 -04:00
Patrick Cloke 4f5bccbbba
Add forward-compatibility for the redacts property (MSC2174). (#16013)
The location of the redacts field changes in room version 11. Ensure
it is copied to the *new* location for *old* room versions for
forwards-compatibility with clients.

Note that copying it to the *old* location for the *new* room version
was previously handled.
2023-08-02 15:35:54 +00:00
Patrick Cloke 01a45869f0
Update MSC3958 support to interact with intentional mentions. (#15992)
* Updates the rule ID.
* Use `event_property_is` instead of `event_match`.

This updates the implementation of MSC3958 to match the latest
text from the MSC.
2023-08-02 08:41:32 -04:00
Mohit Rathee 5eb3fd785b
Trim whitespace when setting display names (#16031) 2023-08-01 09:14:02 -04:00
Erik Johnston ae55cc1e6b
Add ability to wait for locks and add locks to purge history / room deletion (#15791)
c.f. #13476
2023-07-31 10:58:03 +01:00
Mathieu Velten a719b703d9
Fix 404 on /profile when the display name is empty but not the avatar (#16012) 2023-07-27 15:45:05 +02:00
Shay f98f4f2e16
Remove support for legacy application service paths (#15964) 2023-07-26 12:59:47 -07:00
Shay 641ff9ef7e
Support MSC3814: Dehydrated Devices (#15929)
Signed-off-by: Nicolas Werner <n.werner@famedly.com>
Co-authored-by: Nicolas Werner <n.werner@famedly.com>
Co-authored-by: Nicolas Werner <89468146+nico-famedly@users.noreply.github.com>
Co-authored-by: Hubert Chathi <hubert@uhoreg.ca>
2023-07-24 08:23:19 -07:00
Shay 5c7364fea5
Properly handle redactions of creation events (#15973) 2023-07-23 16:32:01 -07:00
Shay f08d05dd2c
Actually stop reading from column `user_id` of tables `profiles` (#15955) 2023-07-23 16:30:54 -07:00
Olivier Wilkinson (reivilibre) 8e8431bc6e Merge branch 'master' into develop 2023-07-18 16:45:39 +01:00
Patrick Cloke 6d81aec09f
Support room version 11 (#15912)
And fix a bug in the implementation of the updated redaction
format (MSC2174) where the top-level redacts field was not
properly added for backwards-compatibility.
2023-07-18 08:44:59 -04:00
Shay e625c3dca0
Revert "Stop writing to column `user_id` of tables `profiles` and `user_filters`. (#15953)
* Revert "Stop writing to column `user_id` of tables `profiles` and `user_filters` (#15787)"

This reverts commit f25b0f8808.

* newsfragement
2023-07-18 11:44:09 +01:00
Eric Eastwood 1c802de626
Re-introduce the outbound federation proxy (#15913)
Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).

This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.
2023-07-18 09:49:21 +01:00
Mathieu Velten 8eb7bb975e
Mark get_user_in_directory private since only used in tests (#15884) 2023-07-12 11:09:13 +02:00
Jason Little 224ef0b669
Unix Sockets for HTTP Replication (#15708)
Unix socket support for `federation` and `client` Listeners has existed now for a little while(since [1.81.0](https://github.com/matrix-org/synapse/pull/15353)), but there was one last hold out before it could be complete: HTTP Replication communication. This should finish it up. The Listeners would have always worked, but would have had no way to be talked to/at.

---------

Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Co-authored-by: Eric Eastwood <erice@element.io>
2023-07-11 13:08:06 -05:00
Patrick Cloke a4243183f0
Add + as an allowed character for Matrix IDs (MSC4009) (#15911) 2023-07-11 12:21:00 -04:00
Michael Telatynski b516d91999
Add `Server` to Access-Control-Expose-Headers header (#15908) 2023-07-11 09:18:50 +01:00
Eric Eastwood c9bf644fa0
Revert "Federation outbound proxy" (#15910)
Revert "Federation outbound proxy (#15773)"

This reverts commit b07b14b494.
2023-07-10 11:10:20 -05:00
Erik Johnston e55a9b3e41
Fix downgrading to previous version of Synapse (#15907)
We do this by marking the constraint as deferrable.
2023-07-10 16:24:42 +01:00
Shay f25b0f8808
Stop writing to column `user_id` of tables `profiles` and `user_filters` (#15787) 2023-07-07 09:23:27 -07:00
Jason Little 2481b7dfa4
Remove `worker_replication_*` deprecated settings, with helpful errors on startup (#15860)
Co-authored-by: reivilibre <oliverw@matrix.org>
2023-07-07 07:45:25 +00:00
Eric Eastwood b07b14b494
Federation outbound proxy (#15773)
Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).

This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.

The original code is from @erikjohnston's branches which I've gotten in-shape to merge.
2023-07-05 18:53:55 -05:00
Eric Eastwood 561d06b481
Remove support for Python 3.7 (#15851)
Fix https://github.com/matrix-org/synapse/issues/15836
2023-07-05 18:45:42 -05:00
Erik Johnston 39d131b016
Add basic read/write lock (#15782) 2023-07-05 17:25:00 +01:00
Erik Johnston 95a96b21eb
Add foreign key constraint to `event_forward_extremities`. (#15751) 2023-07-05 09:43:19 +00:00
Michael Weimann c8e81898b6
Add not_user_type param to the list accounts admin API (#15844)
Signed-off-by: Michael Weimann <michaelw@element.io>
2023-07-04 15:03:20 -07:00
Erik Johnston 25c55a9d22
Add login spam checker API (#15838) 2023-06-26 14:12:20 +00:00
Nicolas Werner e0c39d6bb5
Fix forgotten rooms missing in initial sync (#15815)
If you leave a room and forget it, then rejoin it, the room would be
missing from the next initial sync.

fixes #13262

Signed-off-by: Nicolas Werner <n.werner@famedly.com>
2023-06-21 14:56:31 +01:00
Mathieu Velten 496f73103d
Allow for the configuration of max request retries and min/max retry delays in the matrix federation client (#15783) 2023-06-21 10:41:11 +02:00
Eric Eastwood 887fa4b66b
Switch from `matrix://` to `matrix-federation://` scheme for internal Synapse routing of outbound federation traffic (#15806)
`matrix://` is a registered specced scheme nowadays and doesn't make sense for
our internal to Synapse use case anymore. ([discussion]
(https://github.com/matrix-org/synapse/pull/15773#discussion_r1227598679))
2023-06-20 10:05:31 +01:00
Eric Eastwood 0f02f0b4da
Remove experimental MSC2716 implementation to incrementally import history into existing rooms (#15748)
Context for why we're removing the implementation:

 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010
 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734

Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737

Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.
2023-06-16 14:12:24 -05:00
Mathieu Velten 0618bf94cd
push rules: fix internal conversion from _type to value (#15781)
Also fix wrong rule names for `is_user_mention` and `is_room_mention`.
2023-06-16 14:17:02 +02:00
Mathieu Velten f63d4a3a65
Regularly try to wake up dests instead of waiting for next PDU/EDU (#15743) 2023-06-16 10:15:12 +00:00
Josh Qou d939120421
Fix unsafe hotserving behaviour for non-multimedia uploads. (#15680)
* Fix unsafe hotserving behaviour for non-multimedia uploads.

* invert disposition assert

* test_media_storage.py: run lint

* test_base.py: /inline/attachment/s

* Only return attachment for disposition type, update tests

* Update synapse/media/_base.py

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Update changelog.d/15680.bugfix

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* add attribution

* Update changelog.

---------

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2023-06-15 14:23:27 +01:00
Mathieu Velten 87e5df9a6e Merge branch 'release-v1.86' into develop 2023-06-14 14:54:19 +02:00
Mathieu Velten ef0d3d7bd9 Revert "Allow for the configuration of max request retries and min/max retry delays in the matrix federation client (#12504)"
This reverts commit d84e66144d.
2023-06-14 11:55:57 +02:00
Jason Little 21fea6b749
Prefill events after invalidate not before when persisting events (#15758)
Fixes #15757
2023-06-14 09:42:18 +01:00
Shay 553f2f53e7
Replace `EventContext` fields `prev_group` and `delta_ids` with field `state_group_deltas` (#15233) 2023-06-13 13:22:06 -07:00
Shay d84e66144d
Allow for the configuration of max request retries and min/max retry delays in the matrix federation client (#12504)
Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Erik Johnston <erik@matrix.org>
2023-06-09 09:00:46 +02:00
Erik Johnston c485ed1c5a
Clear event caches when we purge history (#15609)
This should help a little with #13476

---------

Co-authored-by: Patrick Cloke <patrickc@matrix.org>
2023-06-08 13:14:40 +01:00
Grant McLean 5c24d7b9eb
Check required power levels earlier in createRoom handler. (#15695)
* Check required power levels earlier in createRoom handler.

- If a server was configured to reject the creation of rooms with E2EE
  enabled (by specifying an unattainably high power level for
  "m.room.encryption" in default_power_level_content_override), the 403
  error was not being triggered until after the room was created and
  before the "m.room.power_levels" was sent.  This allowed a user to
  access the partially-configured room and complete the setup of E2EE
  and power levels manually.

- This change causes the power level overrides to be checked earlier and
  the request to be rejected before the user gains access to the room.

- A new `_validate_room_config` method is added to contain checks that
  should be run before a room is created.

- The new test case confirms that a user request is rejected by the new
  validation method.

Signed-off-by: Grant McLean <grant@catalyst.net.nz>

* Add a changelog file.

* Formatting fix for black.

* Remove unneeded line from test.

---------

Signed-off-by: Grant McLean <grant@catalyst.net.nz>
2023-06-07 16:21:25 +01:00
Eric Eastwood 8bfded81f3
Trace functions which return `Awaitable` (#15650) 2023-06-06 17:39:22 -05:00
Patrick Cloke f880e64b11
Stabilize support for MSC3952: Intentional mentions. (#15520) 2023-06-06 09:11:07 +01:00
Shay d0c4257f14
`N + 3`: Read from column `full_user_id` rather than `user_id` of tables `profiles` and `user_filters` (#15649) 2023-06-02 17:24:13 -07:00
Mathieu Velten e0f2429d13
Add a catch-all * to the supported relation types when redacting (#15705)
This is an update to MSC3912 implementation
2023-06-02 13:13:50 +00:00
Eric Eastwood 30a5076da8
Log when events are (unexpectedly) filtered out of responses in tests (#14213)
See https://github.com/matrix-org/synapse/pull/14095#discussion_r990335492

This is useful because when see that a relevant event is an `outlier` or `soft-failed`, then that's a good unexpected indicator explaining why it's not showing up. `filter_events_for_client` is used in `/sync`, `/messages`, `/context` which are all common end-to-end assertion touch points (also notifications, relations).
2023-06-01 21:27:18 -05:00
Hugh Nimmo-Smith d1693f0362
Implement stable support for MSC3882 to allow an existing device/session to generate a login token for use on a new device/session (#15388)
Implements stable support for MSC3882; this involves updating Synapse's support to
match the MSC / the spec says.

Continue to support the unstable version to allow clients to transition.
2023-06-01 08:52:51 -04:00
Patrick Cloke 6f18812bb0
Add stubs package for lxml. (#15697)
The stubs have some issues so this has some generous cast
and ignores in it, but it is better than not having stubs.

Note that confusing that Element is a function which creates
_Element instances (and similarly for Comment).
2023-05-31 17:06:57 +00:00