Commit Graph

4322 Commits (af13a3be29dd2d84d9255f8e613ca70c16819436)

Author SHA1 Message Date
Richard van der Hoff 5572e6cc4b
Comments and typing for `_update_outliers_txn` (#11776)
A couple of surprises for me here, so thought I'd document them
2022-01-19 19:45:36 +00:00
Patrick Cloke 68acb0a29d
Include whether the requesting user has participated in a thread. (#11577)
Per updates to MSC3440.

This is implement as a separate method since it needs to be cached
on a per-user basis, instead of a per-thread basis.
2022-01-18 11:38:57 -05:00
Richard van der Hoff 251b5567ec
Remove `log_function` and its uses (#11761)
I've never found this terribly useful. I think it was added in the early days
of Synapse, without much thought as to what would actually be useful to log,
and has just been cargo-culted ever since.

Rather, it tends to clutter up debug logs with useless information.
2022-01-18 13:06:04 +00:00
Daniel Sonck 6b241f5286
Make pagination of rooms in admin api stable (#11737)
Always add state.room_id after the configurable ORDER BY. Otherwise,
for any sort, certain pages can contain results from
other pages. (Especially when sorting by creator, since there may
be many rooms by the same creator)

* Document different order direction of numerical fields

"joined_members", "joined_local_members", "version" and "state_events"
are ordered in descending direction by default (dir=f). Added a note
in tests to explain the differences in ordering.

Signed-off-by: Daniël Sonck <daniel@sonck.nl>
2022-01-17 11:42:51 +00:00
Olivier Wilkinson (reivilibre) e7da1ced24 Merge branch 'release-v1.50' into develop 2022-01-14 15:25:16 +00:00
Patrick Cloke 3e0536cd2a
Replace uses of simple_insert_many with simple_insert_many_values. (#11742)
This should be (slightly) more efficient and it is simpler
to have a single method for inserting multiple values.
2022-01-13 19:44:18 -05:00
reivilibre b602ba194b
Fix a bug introduced in Synapse v1.50.0rc1 whereby outbound federation could fail because too many EDUs were produced for device updates. (#11730)
Co-authored-by: David Robertson <davidr@element.io>
2022-01-13 18:12:18 +00:00
Patrick Cloke 10a88ba91c
Use auto_attribs/native type hints for attrs classes. (#11692) 2022-01-13 13:49:28 +00:00
reivilibre 22abfca8d9
Fix a bug introduced in Synapse v1.0.0 whereby device list updates would not be sent to remote homeservers if there were too many to send at once. (#11729)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-01-12 15:21:13 +00:00
reivilibre ffd227c382
Fix docstring on `add_account_data_for_user`. (#11716) 2022-01-10 15:38:22 +00:00
Patrick Cloke 6bf81a7a61
Bundle aggregations outside of the serialization method. (#11612)
This makes the serialization of events synchronous (and it no
longer access the database), but we must manually calculate and
provide the bundled aggregations.

Overall this should cause no change in behavior, but is prep work
for other improvements.
2022-01-07 09:10:46 -05:00
Dirk Klimpel 3b51c763ba
Fix get federation status of destination if no error occured (#11593) 2022-01-05 14:46:50 -05:00
Shay d8f94eeec2
Run `pyupgrade --py37-plus --keep-percent-format` on Synapse (#11685)
* newsfragment

* fix newsfragment number

* update changelog

* remove extra space
2022-01-05 09:53:05 -08:00
David Robertson 88a78c6577
Cache empty responses from `/user/devices` (#11587)
If we've never made a request to a remote homeserver, we should cache the response---even if the response is "this user has no devices".
2022-01-05 13:33:28 +00:00
Richard van der Hoff 2359ee3864
Remove redundant `get_current_events_token` (#11643)
* Push `get_room_{min,max_stream_ordering}` into StreamStore

Both implementations of this are identical, so we may as well push it down and
get rid of the abstract base class nonsense.

* Remove redundant `StreamStore` class

This is empty now

* Remove redundant `get_current_events_token`

This was an exact duplicate of `get_room_max_stream_ordering`, so let's get rid
of it.

* newsfile
2022-01-04 16:10:27 +00:00
Richard van der Hoff bd9821f7f1
Better error messages from `get_create_event_for_room` (#11638)
"Unknown room" can mean a multitude of things here. To help with debugging, add
some more words to the exception text.
2022-01-04 16:10:05 +00:00
Patrick Cloke cbd82d0b2d
Convert all namedtuples to attrs. (#11665)
To improve type hints throughout the code.
2021-12-30 18:47:12 +00:00
Dirk Klimpel 07a3b5daba
Add type hints to `synapse/storage/databases/main/events_bg_updates.py` (#11654) 2021-12-30 07:22:31 -05:00
Dirk Klimpel f82d38ed2e
Improve type hints in storage classes. (#11652)
By using cast and making ignores more specific.
2021-12-29 13:04:28 +00:00
Dirk Klimpel 15bb1c8511
Add type hints to `synapse/storage/databases/main/stats.py` (#11653) 2021-12-29 08:01:13 -05:00
Patrick Cloke b6102230a7
Add type hints to event_push_actions. (#11594) 2021-12-21 13:25:34 +00:00
Dirk Klimpel 1847d027e6
Add type hints to `synapse/storage/databases/main/transactions.py` (#11589) 2021-12-16 14:59:35 -05:00
Sean Quah c7fe32edb4
Add type hints to `synapse/storage/databases/main/room.py` (#11575) 2021-12-15 18:00:48 +00:00
Patrick Cloke f901f8b70e
Require Collections as the parameters for simple_* methods. (#11580)
Instead of Iterable since the generators are not allowed due
to the potential for their re-use.
2021-12-15 17:00:50 +00:00
reivilibre 17886d2603
Add experimental support for MSC3202: allowing application services to masquerade as specific devices. (#11538) 2021-12-15 10:40:52 +00:00
Sean Quah ecfcd9bbbe
Add type hints to `synapse/storage/databases/main/e2e_room_keys.py` (#11549) 2021-12-14 17:46:47 +00:00
Sean Quah a4dce5b53d
Remove redundant `COALESCE()`s around `COUNT()`s in database queries (#11570)
`COUNT()` never returns `NULL`. A `COUNT(*)` over 0 rows is 0 and a
`COUNT(NULL)` is also 0.
2021-12-14 12:34:30 +00:00
Richard van der Hoff ff6fd52160
checks for generators in database functions (#11564)
A couple of safety-checks to hopefully stop people doing what I just did, and create a storage
function which only works the first time it is called (and not when it is re-run due to a database
concurrency error or similar).
2021-12-13 19:01:27 +00:00
Sean Quah 5305a5e881
Type hint the constructors of the data store classes (#11555) 2021-12-13 17:05:00 +00:00
Sean Quah 1abfb15f07
Add type hints to `synapse/storage/databases/main/end_to_end_keys.py` (#11551) 2021-12-13 16:28:26 +00:00
Sean Quah 6da8591f2e
Add type hints to `synapse/storage/databases/main/account_data.py` (#11546) 2021-12-13 16:28:10 +00:00
reivilibre e5cdb9e233
Make `get_device` return None if the device doesn't exist rather than raising an exception. (#11565)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-12-13 15:39:43 +00:00
Richard van der Hoff f0562183e7
skip some dict munging in event persistence (#11560)
Create a new dict helper method `simple_insert_many_values_txn`, which takes
raw row values, rather than {key=>value} dicts. This saves us a bunch of dict
munging, and makes it easier to use generators rather than creating
intermediate lists and dicts.
2021-12-10 15:02:33 +00:00
Richard van der Hoff 86e7a6d16e
Stop populating `state_events.prev_state` (#11558)
this field is never read, so we may as well stop populating it.
2021-12-10 14:13:23 +00:00
Patrick Cloke 9562f0c2f1
Ensure emails are canonicalized before fetching associated user. (#11547)
This should fix pushers with an email in non-canonical form is used as
the pushkey.
2021-12-10 07:17:28 -05:00
Patrick Cloke 3b8872299a
Do not allow cross-room relations, per MSC2674. (#11516) 2021-12-09 13:16:01 -05:00
Patrick Cloke d93362d87f
Add a constant for receipt types (m.read). (#11531)
And expand some type hints in the receipts storage module.
2021-12-08 12:26:29 -05:00
Richard van der Hoff ff7cc17b57
Improve log messages for stream ids (#11536)
Somehow I'd managed to get my database in a pickle with stream ids. These
changes were useful to debug.
2021-12-08 14:15:14 +00:00
Hubert Chathi 8541809cb9
Send and handle cross-signing messages using the stable prefix. (#10520) 2021-12-08 10:01:38 +00:00
Quentin Gliech a15a893df8
Save the OIDC session ID (sid) with the device on login (#11482)
As a step towards allowing back-channel logout for OIDC.
2021-12-06 12:43:06 -05:00
Dirk Klimpel 8b4b153c9e
Add admin API to get some information about federation status (#11407) 2021-12-06 16:59:50 +00:00
Richard van der Hoff 5640992d17
Disambiguate queries on `state_key` (#11497)
We're going to add a `state_key` column to the `events` table, so we need to
add some disambiguation to queries which use it.
2021-12-02 22:42:58 +00:00
David Robertson d26808dd85
Comments on the /sync tentacles (#11494)
This mainly consists of docstrings and inline comments. There are one or two type annotations and variable renames thrown in while I was here.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-12-02 20:58:32 +00:00
reivilibre 435f044807
Add type annotations to `tests.storage.test_appservice`. (#11488) 2021-12-02 15:30:05 +00:00
Eric Eastwood a6f1a3abec
Add MSC3030 experimental client and federation API endpoints to get the closest event to a given timestamp (#9445)
MSC3030: https://github.com/matrix-org/matrix-doc/pull/3030

Client API endpoint. This will also go and fetch from the federation API endpoint if unable to find an event locally or we found an extremity with possibly a closer event we don't know about.
```
GET /_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>
{
    "event_id": ...
    "origin_server_ts": ...
}
```

Federation API endpoint:
```
GET /_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>
{
    "event_id": ...
    "origin_server_ts": ...
}
```

Co-authored-by: Erik Johnston <erik@matrix.org>
2021-12-02 01:02:20 -06:00
Marcus a9481223d1
Improved push typing (#11409)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-11-30 11:49:20 +00:00
Eric Eastwood fb58611d21
Refactor `backfilled` into specific behavior function arguments (`_persist_events_and_state_updates`) (#11417)
Part of https://github.com/matrix-org/synapse/issues/11300

Call stack:

 - `_persist_events_and_state_updates` (added `use_negative_stream_ordering`)
    - `_persist_events_txn`
       - `_update_room_depths_txn` (added `update_room_forward_stream_ordering`)
       - `_update_metadata_tables_txn`
          - `_store_room_members_txn` (added `inhibit_local_membership_updates`)

Using keyword-only arguments (`*`) to reduce the mistakes from `backfilled` being left as a positional argument somewhere and being interpreted wrong by our new arguments.
2021-11-29 16:01:54 -06:00
Erik Johnston d08ef6f155
Make background updates controllable via a plugin (#11306)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-11-29 17:57:06 +01:00
Sean Quah ffd858aa68
Add type hints to `synapse/storage/databases/main/events_worker.py` (#11411)
Also refactor the stream ID trackers/generators a bit and try to
document them better.
2021-11-26 18:41:31 +00:00
reivilibre 1d8b80b334
Support expiry of refresh tokens and expiry of the overall session when refresh tokens are in use. (#11425) 2021-11-26 14:27:14 +00:00