Commit Graph

14587 Commits (9ad2197fa7b1f9f0fd308b37cefc78061e95c04b)

Author SHA1 Message Date
David Robertson 941dc3db13
Track a histogram of state res durations (#13036) 2022-06-15 15:19:49 +01:00
reivilibre 0dbdc39940
Fix a long-standing bug which meant that rate limiting was not restrictive enough in some cases. (#13018) 2022-06-15 14:11:55 +00:00
Brendan Abolivier 417f4cf40b
Don't use keyword arguments when initialising modules (#13060) 2022-06-15 15:36:16 +02:00
Richard van der Hoff 75fb10ee45
Clean up schema for `event_edges` (#12893)
* Remove redundant references to `event_edges.room_id`

We don't need to care about the room_id here, because we are already checking
the event id.

* Clean up the event_edges table

We make a number of changes to `event_edges`:

 * We give the `room_id` and `is_state` columns defaults (null and false
   respectively) so that we can stop populating them.
 * We drop any rows that have `is_state` set true - they should no longer
   exist.
 * We drop any rows that do not exist in `events` - these should not exist
   either.
 * We drop the old unique constraint on all the colums, which wasn't much use.
 * We create a new unique index on `(event_id, prev_event_id)`.
 * We add a foreign key constraint to `events`.

These happen rather differently depending on whether we are on Postgres or
SQLite. For SQLite, we just rebuild the whole table, copying only the rows we
want to keep. For Postgres, we try to do things in the background as much as
possible.

* Stop populating `event_edges.room_id` and `is_state`

We can just rely on the defaults.
2022-06-15 12:29:42 +01:00
reivilibre 5b645ae2ad
Refactor entry points so that they all have a `main` function. (#13052) 2022-06-14 17:41:06 +00:00
David Robertson c99b511db9
Fix `destination_is` errors seen in sentry. (#13041)
* Rename test_fedclient to match its source file
* Require at least one destination to be truthy
* Explicitly validate user ID in profile endpoint GETs
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-06-14 18:28:26 +01:00
Shay 493c2fc44a
Remove code generating comments in configuration file (#12941) 2022-06-14 07:53:42 -07:00
Patrick Cloke 5f4ecf759d
Rename delta to apply in the proper schema version. (#13050) 2022-06-14 14:34:04 +00:00
Quentin Gliech fe1daad672
Move the "email unsubscribe" resource, refactor the macaroon generator & simplify the access token verification logic. (#12986)
This simplifies the access token verification logic by removing the `rights`
parameter which was only ever used for the unsubscribe link in email
notifications. The latter has been moved under the `/_synapse` namespace,
since it is not a standard API.

This also makes the email verification link more secure, by embedding the
app_id and pushkey in the macaroon and verifying it. This prevents the user
from tampering the query parameters of that unsubscribe link.

Macaroon generation is refactored:

- Centralised all macaroon generation and verification logic to the
  `MacaroonGenerator`
- Moved to `synapse.utils`
- Changed the constructor to require only a `Clock`, hostname, and a secret key
  (instead of a full `Homeserver`).
- Added tests for all methods.
2022-06-14 09:12:08 -04:00
Quentin Gliech 92103cb2c8
Decouple `synapse.api.auth_blocking.AuthBlocking` from `synapse.api.auth.Auth`. (#13021) 2022-06-14 09:51:15 +01:00
David Teller a164a46038
Uniformize spam-checker API, part 4: port other spam-checker callbacks to return `Union[Allow, Codes]`. (#12857)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-06-13 18:16:16 +00:00
Patrick Cloke 53b77b203a
Replace noop background updates with DELETE. (#12954)
Removes the `register_noop_background_update` and deletes the background
updates directly in a delta file.
2022-06-13 14:06:27 -04:00
Richard van der Hoff f68b5e5773 Merge branch 'rav/simplify_event_auth_interface' into develop 2022-06-13 11:34:59 +01:00
Richard van der Hoff c1b28b8842 Remove redundant `room_version` param from `check_auth_rules_from_context`
It's now implied by the room_version property on the event.
2022-06-12 23:13:10 +01:00
Richard van der Hoff 0d9d36b15c Remove `room_version` param from `check_auth_rules_for_event`
Instead, use the `room_version` property of the event we're checking.

The `room_version` was originally added as a parameter somewhere around #4482,
but really it's been redundant since #6875 added a `room_version` field to `EventBase`.
2022-06-12 23:13:10 +01:00
Richard van der Hoff 68be42f6b6 Remove `room_version` param from `validate_event_for_room_version`
Instead, use the `room_version` property of the event we're validating.

The `room_version` was originally added as a parameter somewhere around #4482,
but really it's been redundant since #6875 added a `room_version` field to `EventBase`.
2022-06-12 23:13:09 +01:00
Patrick Cloke 81608490e3
Stop depending on `room_id` to be returned for children state in the hierarchy response. (#12991)
The `room_id` field was removed from MSC2946 before
it was accepted. It was initially kept for backwards compatibility
and should be removed now that the stable form of the API
is used.

This change only stops Synapse from validating that it is returned,
a future PR will remove returning it as part of the response.
2022-06-10 07:15:51 -04:00
Richard van der Hoff 7c6b2204d1
Faster joins: add issue links to the TODOs (#13004)
... to help us keep track of these things
2022-06-09 10:13:03 +00:00
David Robertson 97053c9406
Type annotations for `test_v2` (#12985) 2022-06-09 09:48:04 +01:00
Nick Mills-Barrett 04ca3a52f6
Use READ COMMITTED isolation level when inserting read receipts (#12957) 2022-06-09 09:44:16 +01:00
Patrick Cloke dd2d66b0c9
Move the (unstable) `dir` parameter for /relations behind an experimental flag. (#12984)
MSC3715 defines this parameter, but the unstable version of it should be
behind an experimental flag.
2022-06-08 09:00:35 -04:00
David Robertson 586bfc6dc0
Use dummy fallback engines if imports fail (#12979) 2022-06-07 17:33:55 +01:00
Daniel Aloni b5a3aecf18
Return the same error message from `/login` when password is incorrect and when account doesn't exist. (#12738) 2022-06-07 14:58:48 +00:00
David Robertson f30bcbd84a
Fix Synapse git info missing in version strings (#12973) 2022-06-07 15:24:11 +01:00
Erik Johnston a7e506ddee
Reduce amount of state we pull out when attempting to send catchup PDUs. (#12963)
* Don't pull out state for catchup

* Newsfile

* Merge newsfile
2022-06-07 14:35:56 +01:00
Patrick Cloke d2fd7f7b5c
Fix a stale comment in get_room_version_id_txn. (#12969) 2022-06-07 07:44:31 -04:00
Patrick Cloke 9dc3293e0b
Consolidate the logic of delete_device/delete_devices. (#12970)
By always using delete_devices and sometimes passing a list
with a single device ID.

Previously these methods had gotten out of sync with each
other and it seems there's little benefit to the single-device
variant.
2022-06-07 07:43:35 -04:00
Andrew Morgan c51f5b9592
Prevent breaking old sqlite's when media retention is enabled (#12977) 2022-06-07 11:28:08 +00:00
Andrew Morgan a47636c570
Prevent local quarantined media from being claimed by media retention (#12972) 2022-06-07 10:53:47 +00:00
Patrick Cloke f7baffd8ec
Remove remaining pieces of groups code. (#12966)
* Remove an unused stream ID generator.
* Remove the now unused remote profile cache.
2022-06-06 13:20:05 -04:00
Erik Johnston 44de53bb79
Reduce state pulled from DB due to sending typing and receipts over federation (#12964)
Reducing the amount of state we pull from the DB is useful as fetching state is expensive in terms of DB, CPU and memory.
2022-06-06 16:46:11 +01:00
Patrick Cloke 148fe58a24
Do not break URL previews if an image is unreachable. (#12950)
Avoid breaking a URL preview completely if the chosen image 404s
or is unreachable for some other reason (e.g. DNS).
2022-06-06 07:46:04 -04:00
Patrick Cloke 1acc897c31
Implement MSC3816, consider the root event for thread participation. (#12766)
As opposed to only considering a user to have "participated" if they
replied to the thread.
2022-06-06 07:18:04 -04:00
Jan Christian Grünhage fcd8703508
Allow updating passwords using the admin api without logging out devices (#12952) 2022-06-06 12:10:13 +01:00
Erik Johnston e3163e2e11
Reduce the amount of state we pull from the DB (#12811) 2022-06-06 09:24:12 +01:00
Patrick Cloke 6b46c3eb3d
Remove groups code from synapse_port_db. (#12899) 2022-06-03 12:13:35 -04:00
Patrick Cloke 01df5bacac
Improve URL previews for some pages (#12951)
* Skip `og` and `meta` tags where the value is empty.
* Fallback to the favicon if there are no other images.
* Ignore tags meant for navigation.
2022-06-03 12:09:12 -04:00
Erik Johnston 888a29f412
Wait for lazy join to complete when getting current state (#12872) 2022-06-01 16:02:53 +01:00
Patrick Cloke 7bc08f3201
Remove remaining bits of groups code. (#12936)
* Update worker docs to remove group endpoints.
* Removes an unused parameter to `ApplicationService`.
* Break dependency between media repo and groups.
* Avoid copying `m.room.related_groups` state events during room upgrades.
2022-06-01 09:41:25 -04:00
Richard van der Hoff 79dadf7216
Fix 404 on `/sync` when the last event is a redaction of an unknown/purged event (#12905)
Currently, we try to pull the event corresponding to a sync token from the database. However, when
we fetch redaction events, we check the target of that redaction (because we aren't allowed to send
redactions to clients without validating them). So, if the sync token points to a redaction of an event
that we don't have, we have a problem.

It turns out we don't really need that event, and can just work with its ID and metadata, which
sidesteps the whole problem.
2022-06-01 11:29:51 +00:00
Erik Johnston 5949ab86f8
Fix potential thumbnail memory leaks. (#12932) 2022-06-01 10:57:49 +00:00
Patrick Cloke 2e8763ec96
Remove most groups datastore code. (#12895)
The remaining piece is a background update that is needed
for backwards compatibility.
2022-05-31 20:28:17 -04:00
Richard van der Hoff f0aec0abef
Improve logging when signature checks fail (#12925)
* Raise a dedicated `InvalidEventSignatureError` from `_check_sigs_on_pdu`

* Downgrade logging about redactions to DEBUG

this can be very spammy during a room join, and it's not very useful.

* Raise `InvalidEventSignatureError` from `_check_sigs_and_hash`

... and, more importantly, move the logging out to the callers.

* changelog
2022-05-31 23:32:56 +01:00
Patrick Cloke cf05258f76
Remove groups replication code. (#12900)
The replication logic for groups is no longer used, so the message
passing infrastructure can be removed.
2022-05-31 13:04:08 -04:00
Andrew Morgan 2fc787c341
Add config options for media retention (#12732) 2022-05-31 16:35:29 +00:00
Sean Quah 641908f72f
Faster room joins: Resume state re-syncing after a Synapse restart (#12813)
Signed-off-by: Sean Quah <seanq@matrix.org>
2022-05-31 15:15:08 +00:00
Sean Quah 2fba1076c5
Faster room joins: Try other destinations when resyncing the state of a partial-state room (#12812)
Signed-off-by: Sean Quah <seanq@matrix.org>
2022-05-31 15:50:29 +01:00
Erik Johnston 3594f6c1f3 Merge branch 'master' into develop 2022-05-31 14:48:22 +01:00
Erik Johnston c8684e6792
Reduce DB load of /sync when using presence (#12885)
While the query was fast, we were calling it *a lot*.
2022-05-31 13:01:05 +00:00
Richard van der Hoff 5e17922ef7
Stop reading from `event_edges.room_id`. (#12914)
event_edges.room_id is implied by the event id, so there is no need to join on the room id.
2022-05-31 13:51:49 +01:00