Commit Graph

829 Commits (ea75346f6af8c182a42d1ca29119a10361693a7b)

Author SHA1 Message Date
Patrick Cloke ea75346f6a
Track presence state per-device and combine to a user state. (#16066)
Tracks presence on an individual per-device basis and combine
the per-device state into a per-user state. This should help in
situations where a user has multiple devices with conflicting status
(e.g. one is syncing with unavailable and one is syncing with online).

The tie-breaking is done by priority:

    BUSY > ONLINE > UNAVAILABLE > OFFLINE
2023-09-05 09:58:51 -04:00
Erik Johnston d35bed8369
Don't wake up destination transaction queue if they're not due for retry. (#16223) 2023-09-04 17:14:09 +01:00
Erik Johnston f84baecb6f
Don't reset retry timers on "valid" error codes (#16221) 2023-09-04 14:04:43 +01:00
Patrick Cloke ebd8374fb5
Stop writing to the event_txn_id table (#16175) 2023-08-30 11:10:56 +01:00
Patrick Cloke 9ec3da06da
Bump mypy-zope & mypy. (#16188) 2023-08-29 10:38:56 -04:00
Evilham 63b51ef3fb
Support IPv6-only SMTP servers (#16155)
Use Twisted HostnameEndpoint to connect to SMTP servers (instead
of connectTCP/connectSSL) which properly supports IPv6-only servers.
2023-08-29 09:33:58 -04:00
Patrick Cloke 40901af5e0
Pass the device ID around in the presence handler (#16171)
Refactoring to pass the device ID (in addition to the user ID) through
the presence handler (specifically the `user_syncing`, `set_state`,
and `bump_presence_active_time` methods and their replication
versions).
2023-08-28 13:08:49 -04:00
Patrick Cloke 1bf143699c
Combine logic about not overriding BUSY presence. (#16170)
Simplify some of the presence code by reducing duplicated code between
worker & non-worker modes.

The main change is to push some of the logic from `user_syncing` into
`set_state`. This is done by passing whether the user is setting the presence
via a `/sync` with a new `is_sync` flag to `set_state`. If this is `true` some
additional logic is performed:

* Don't override `busy` presence.
* Update the `last_user_sync_ts`.
* Never update the status message.
2023-08-28 11:03:23 -04:00
V02460 84f441f88f
Prepare unit tests for Python 3.12 (#16099) 2023-08-25 15:05:10 -04:00
Patrick Cloke a8a46b1336
Replace simple_async_mock with AsyncMock (#16180)
Python 3.8 has a native AsyncMock, use it instead of a custom
implementation.
2023-08-25 09:27:21 -04:00
Patrick Cloke daf11e26ef
Replace make_awaitable with AsyncMock (#16179)
Python 3.8 provides a native AsyncMock, we can replace the
homegrown version we have.
2023-08-24 19:38:46 -04:00
Patrick Cloke da162cbe4e
Add tests for restoring the presence state after a restart. (#16151) 2023-08-23 07:31:00 -04:00
Patrick Cloke 3f17178728
Clean-up presence tests (#16158)
Reduce duplicated code & remove unused variables.
2023-08-22 11:43:44 -04:00
Shay 69048f7b48
Add an admin endpoint to allow authorizing server to signal token revocations (#16125) 2023-08-22 14:15:34 +00:00
Patrick Cloke b657e89005
Fix user directory test for deactivated support user. (#16157)
Support users should not be added to the user directory after
being deactivated.
2023-08-22 09:08:24 -04:00
Mathieu Velten 2d15e39684
MSC3861: allow impersonation by an admin using a query param (#16132) 2023-08-18 15:46:46 +02:00
Shay 54a51ff6c1
Cache token introspection response from OIDC provider (#16117) 2023-08-17 10:53:10 -07:00
Shay 0328b56468
Support MSC3814: Dehydrated Devices Part 2 (#16010) 2023-08-08 12:04:46 -07:00
Erik Johnston ae55cc1e6b
Add ability to wait for locks and add locks to purge history / room deletion (#15791)
c.f. #13476
2023-07-31 10:58:03 +01:00
Mathieu Velten a719b703d9
Fix 404 on /profile when the display name is empty but not the avatar (#16012) 2023-07-27 15:45:05 +02:00
Shay 641ff9ef7e
Support MSC3814: Dehydrated Devices (#15929)
Signed-off-by: Nicolas Werner <n.werner@famedly.com>
Co-authored-by: Nicolas Werner <n.werner@famedly.com>
Co-authored-by: Nicolas Werner <89468146+nico-famedly@users.noreply.github.com>
Co-authored-by: Hubert Chathi <hubert@uhoreg.ca>
2023-07-24 08:23:19 -07:00
Eric Eastwood 1c802de626
Re-introduce the outbound federation proxy (#15913)
Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).

This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.
2023-07-18 09:49:21 +01:00
Mathieu Velten 8eb7bb975e
Mark get_user_in_directory private since only used in tests (#15884) 2023-07-12 11:09:13 +02:00
Patrick Cloke a4243183f0
Add + as an allowed character for Matrix IDs (MSC4009) (#15911) 2023-07-11 12:21:00 -04:00
Eric Eastwood c9bf644fa0
Revert "Federation outbound proxy" (#15910)
Revert "Federation outbound proxy (#15773)"

This reverts commit b07b14b494.
2023-07-10 11:10:20 -05:00
Eric Eastwood b07b14b494
Federation outbound proxy (#15773)
Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).

This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.

The original code is from @erikjohnston's branches which I've gotten in-shape to merge.
2023-07-05 18:53:55 -05:00
Nicolas Werner e0c39d6bb5
Fix forgotten rooms missing in initial sync (#15815)
If you leave a room and forget it, then rejoin it, the room would be
missing from the next initial sync.

fixes #13262

Signed-off-by: Nicolas Werner <n.werner@famedly.com>
2023-06-21 14:56:31 +01:00
Erik Johnston c485ed1c5a
Clear event caches when we purge history (#15609)
This should help a little with #13476

---------

Co-authored-by: Patrick Cloke <patrickc@matrix.org>
2023-06-08 13:14:40 +01:00
Shay d0c4257f14
`N + 3`: Read from column `full_user_id` rather than `user_id` of tables `profiles` and `user_filters` (#15649) 2023-06-02 17:24:13 -07:00
Quentin Gliech ceb3dd77db Enforce that an admin token also has the basic Matrix API scope 2023-05-30 09:43:06 -04:00
Quentin Gliech f739bde962 Reject tokens with multiple device scopes 2023-05-30 09:43:06 -04:00
Quentin Gliech 14a5be9c4d Handle errors when introspecting tokens
This returns a proper 503 when the introspection endpoint is not working
for some reason, which should avoid logging out clients in those cases.
2023-05-30 09:43:06 -04:00
Quentin Gliech e343125b38 Disable incompatible Admin API endpoints 2023-05-30 09:43:06 -04:00
Quentin Gliech 4d0231b364 Make AS tokens work & allow ASes to /register 2023-05-30 09:43:06 -04:00
Hugh Nimmo-Smith 249f4a338d Refactor config to be an experimental feature
Also enforce you can't combine it with incompatible config options
2023-05-30 09:43:06 -04:00
Quentin Gliech 31691d6151 Disable account related endpoints when using OAuth delegation 2023-05-30 09:43:06 -04:00
Hugh Nimmo-Smith 5fe96082d0 Actually enforce guest + return www-authenticate header 2023-05-30 09:43:06 -04:00
Hugh Nimmo-Smith 28a9663bdf Initial tests for OAuth delegation 2023-05-30 09:43:06 -04:00
Quentin Gliech e2c8458bba Make the api.auth.Auth a Protocol 2023-05-30 09:43:06 -04:00
Eric Eastwood 77156a4bc1
Process previously failed backfill events in the background (#15585)
Process previously failed backfill events in the background because they are bound to fail again and we don't need to waste time holding up the request for something that is bound to fail again.

Fix https://github.com/matrix-org/synapse/issues/13623

Follow-up to https://github.com/matrix-org/synapse/issues/13621 and https://github.com/matrix-org/synapse/issues/13622

Part of making `/messages` faster: https://github.com/matrix-org/synapse/issues/13356
2023-05-24 23:22:24 -05:00
Shay 8839b6c2f8
Add requesting user id parameter to key claim methods in `TransportLayerClient` (#15663) 2023-05-24 13:23:26 -07:00
Patrick Cloke 7c9b91790c
Consolidate logic to check for deactivated users. (#15634)
This moves the deactivated user check to the method which
all login types call.

Additionally updates the application service tests to be more
realistic by removing invalid tests and fixing server names.
2023-05-23 10:35:43 -04:00
Patrick Cloke 1e89976b26
Rename blacklist/whitelist internally. (#15620)
Avoid renaming configuration settings for now and rename internal code
to use blocklist and allowlist instead.
2023-05-19 12:25:25 +00:00
Patrick Cloke 28bceef84e
Check appservices for devices during a /user/devices query. (#15539)
MSC3984 proxies /keys/query requests to appservices, but servers will
can also requests devices / keys from the /user/devices endpoint.

The formats are close enough that we can "proxy" that /user/devices to
appservices (by calling /keys/query) and then change the format of the
returned data before returning it over federation.
2023-05-05 15:18:47 -04:00
Patrick Cloke 36df9c5e36
Implement MSC4009 to widen the allowed Matrix ID grammar (#15536)
Behind a configuration flag this adds + to the list of allowed
characters in Matrix IDs. The main feature this enables is
using full E.164 phone numbers as Matrix IDs.
2023-05-05 12:13:50 -04:00
Sean Quah 04e79e6a18
Add config option to forget rooms automatically when users leave them (#15224)
This is largely based off the stats and user directory updater code.

Signed-off-by: Sean Quah <seanq@matrix.org>
2023-05-03 12:27:33 +01:00
Patrick Cloke 57aeeb308b
Add support for claiming multiple OTKs at once. (#15468)
MSC3983 provides a way to request multiple OTKs at once from appservices,
this extends this concept to the Client-Server API.

Note that this will likely be spit out into a separate MSC, but is currently part of
MSC3983.
2023-04-27 12:57:46 -04:00
Shay 301b4156d5
Add column `full_user_id` to tables `profiles` and `user_filters`. (#15458) 2023-04-26 16:03:26 -07:00
Patrick Cloke 8e9739449d
Add unstable /keys/claim endpoint which always returns fallback keys. (#15462)
It can be useful to always return the fallback key when attempting to
claim keys. This adds an unstable endpoint for `/keys/claim` which
always returns fallback keys in addition to one-time-keys.

The fallback key(s) are not marked as "used" unless there are no
corresponding OTKs.

This is currently defined in MSC3983 (although likely to be split out
to a separate MSC). The endpoint shape may change or be requested
differently (i.e. a keyword parameter on the current endpoint), but the
core logic should be reasonable.
2023-04-25 13:30:41 -04:00
Andrew Morgan aec639e3e3
Move Spam Checker callbacks to a dedicated file (#15453) 2023-04-18 00:57:40 +00:00