Commit Graph

3554 Commits (3f1cd147919582c101f8ce1c00f83c4a5549f0be)

Author SHA1 Message Date
Erik Johnston 8a380d0fe2
Increase perf of `get_auth_chain_ids` used in state res v2. (#6937)
We do this by moving the recursive query to be fully in the DB.
2020-02-18 15:39:09 +00:00
Richard van der Hoff 46fa66bbfd
wait for current_state_events_membership before delete_old_current_state_events (#6924) 2020-02-17 11:30:50 +00:00
Richard van der Hoff e1d858984d
Remove unused `get_room_stats_state` method. (#6869) 2020-02-07 15:30:26 +00:00
Erik Johnston de2d267375
Allow moving group read APIs to workers (#6866) 2020-02-07 11:14:19 +00:00
Erik Johnston ed630ea17c
Reduce amount of logging at INFO level. (#6862)
A lot of the things we log at INFO are now a bit superfluous, so lets
make them DEBUG logs to reduce the amount we log by default.

Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
2020-02-06 13:31:05 +00:00
Erik Johnston 2201ef8556 Synapse 1.10.0rc2 (2020-02-06)
==============================
 
 Bugfixes
 --------
 
 - Fix an issue with cross-signing where device signatures were not sent to remote servers. ([\#6844](https://github.com/matrix-org/synapse/issues/6844))
 - Fix to the unknown remote device detection which was introduced in 1.10.rc1. ([\#6848](https://github.com/matrix-org/synapse/issues/6848))
 
 Internal Changes
 ----------------
 
 - Detect unexpected sender keys on remote encrypted events and resync device lists. ([\#6850](https://github.com/matrix-org/synapse/issues/6850))
 -----BEGIN PGP SIGNATURE-----
 
 iQJEBAABCgAuFiEEumuwyPtYLL2OMhYdOtoG7cdT0R4FAl478l8QHGVyaWtAbWF0
 cml4Lm9yZwAKCRA62gbtx1PRHmKfD/0ZpBV95gMk0iecfvO6IY8+0KigzDUypgXf
 zp1k+k5DBUmH5/83h7+cXWHIJyofv1At1YdIq9J/nNeyqr3V3dcQwPabOKYbI96d
 kVD0EZX+2NjuYejAuF9eJNdiwRq5yMLluCfnOXr5jCS0NNdpO1xVb4bWYm50RmcD
 WJvBvmfwXwCC3AFyNb4IAipaVUXuVXXx1YhjFs6DsbtpPUG88e4uIhpfvrS4v0t8
 GFJZNuVJStOvyKHHTfNRIMkhy4xidj3HRUMSmjNpx07WnPBnbv4LnUOD1boYxgaP
 EoODYnoAPshdiKB0AywNNjue3TmFD/Z7vVEzlFP/lNZ4GDU4kN9C/EwFYw0C2Jqq
 f9O/E2ZuP9Qqume5O9UwMQQAmhV5lMBaIsYRbixU+9bSB893zRHRffA11HypzD00
 ZXj8QDOXpiXp2jpAwN1Rk9e/aZEX3qd+zUAgRmk+kYb0KTC2/gcY956rodNSSO/U
 RFYg4DFvoCgCrRSxZ4LQMlFu3YY2E7qWH+p/OHk3WG79jW64VpEFaytvv8fR+GKq
 g3EbtWy6mUzlKYqbjZ+ZTUDe5AWdv6ZX8xJqRD/S6cpiwyh6Gp89HHNvBThXoCmK
 fxkgw8if7eIITuTlNuDrqunxyWqwd3oVlzd2mi2bg0yRfcqJ9C6OuBV1VTLFZeky
 3sjCiU0IRw==
 =kcSy
 -----END PGP SIGNATURE-----

Merge tag 'v1.10.0rc2' into develop

Synapse 1.10.0rc2 (2020-02-06)
==============================

Bugfixes
--------

- Fix an issue with cross-signing where device signatures were not sent to remote servers. ([\#6844](https://github.com/matrix-org/synapse/issues/6844))
- Fix to the unknown remote device detection which was introduced in 1.10.rc1. ([\#6848](https://github.com/matrix-org/synapse/issues/6848))

Internal Changes
----------------

- Detect unexpected sender keys on remote encrypted events and resync device lists. ([\#6850](https://github.com/matrix-org/synapse/issues/6850))
2020-02-06 11:04:03 +00:00
Hubert Chathi 60d0672426 Merge pull request #6844 from matrix-org/uhoreg/cross_signing_fix_device_fed
add device signatures to device key query results
2020-02-05 10:54:49 +00:00
Michael Kaye a831d2e4e3
Reduce performance logging to DEBUG (#6833)
* Reduce tnx performance logging to DEBUG
* Changelog.d
2020-02-05 08:57:37 +00:00
Richard van der Hoff d88e0ec080
Database updates to populate rooms.room_version (#6847)
We're going to need this so that we can figure out how to handle redactions when fetching events from the database.
2020-02-04 21:31:08 +00:00
Erik Johnston 6475382d80
Fix detecting unknown devices from remote encrypted events. (#6848)
We were looking at the wrong event type (`m.room.encryption` vs
`m.room.encrypted`).

Also fixup the duplicate `EvenTypes` entries.

Introduced in #6776.
2020-02-04 17:25:54 +00:00
Hubert Chathi 23d8a55c7a add device signatures to device key query results 2020-02-04 00:13:12 -05:00
Erik Johnston 83b0ea047b
Fix deleting of stale marker for device lists (#6819)
We were in fact only deleting stale marker when we got an incremental
update, rather than when we did a full resync.
2020-01-31 14:04:15 +00:00
Richard van der Hoff 08f41a6f05 Add `get_room_version` method
So that we can start factoring out some of this boilerplatey boilerplate.
2020-01-31 10:28:15 +00:00
Richard van der Hoff d7bf793cc1 s/get_room_version/get_room_version_id/
... to make way for a forthcoming get_room_version which returns a RoomVersion
object.
2020-01-31 10:06:21 +00:00
Erik Johnston e0992fcc5b
Log when we delete room in bg update (#6816) 2020-01-30 17:55:34 +00:00
Erik Johnston 57ad702af0
Backgroud update to clean out rooms from current state (#6802) 2020-01-30 17:17:44 +00:00
Erik Johnston a5bab2d058
When server leaves room check for stale device lists. (#6801)
When a server leaves a room it may stop sharing a room with remote
users, and thus not get any updates to their device lists. So we need to
check for this case and delete those device lists from the cache.

We don't need to do this if we stop sharing a room because the remote
user leaves the room, because we track that case via looking at
membership changes.
2020-01-30 16:10:30 +00:00
Erik Johnston c80a9fe13d
When a client asks for remote keys check if should resync. (#6797)
If we detect that the remote users' keys may have changed then we should
attempt to resync against the remote server rather than using the
(potentially) stale local cache.
2020-01-30 15:06:58 +00:00
Erik Johnston 611215a49c
Delete current state when server leaves a room (#6792)
Otherwise its just stale data, which may get deleted later anyway so
can't be relied on. It's also a bit of a shotgun if we're trying to get
the current state of a room we're not in.
2020-01-29 11:01:32 +00:00
Erik Johnston e17a110661
Detect unknown remote devices and mark cache as stale (#6776)
We just mark the fact that the cache may be stale in the database for
now.
2020-01-28 14:43:21 +00:00
Erik Johnston 02b44db922
Warn if postgres database has non-C locale. (#6734)
As using non-C locale can cause issues on upgrading OS.
2020-01-28 13:44:21 +00:00
Erik Johnston 33f904835a Merge branch 'master' into develop 2020-01-28 13:39:39 +00:00
Erik Johnston bdbeeb94ec
Fix setting `mau_limit_reserved_threepids` config (#6793)
Calling the invalidation function during initialisation of the data
stores introduces a circular dependency, causing Synapse to fail to
start.
2020-01-28 13:05:24 +00:00
Erik Johnston 8df862e45d
Add `rooms.room_version` column (#6729)
This is so that we don't have to rely on pulling it out from `current_state_events` table.
2020-01-27 14:30:57 +00:00
Erik Johnston d5275fc55f
Propagate cache invalidates from workers to other workers. (#6748)
Currently if a worker invalidates a cache it will be streamed to master, which then didn't forward those to other workers.
2020-01-27 13:47:50 +00:00
Erik Johnston fa4d609e20
Make 'event.redacts' never raise. (#6771)
There are quite a few places that we assume that a redaction event has a
corresponding `redacts` key, which is not always the case. So lets
cheekily make it so that event.redacts just returns None instead.
2020-01-23 15:19:03 +00:00
Andrew Morgan ce84dd9e20
Remove unnecessary abstractions in admin handler (#6751) 2020-01-22 15:09:57 +00:00
Andrew Morgan 90a28fb475
Admin API to list, filter and sort rooms (#6720) 2020-01-22 13:36:43 +00:00
Neil Johnson 5e52d8563b Allow monthly active user limiting support for worker mode, fixes #4639. (#6742) 2020-01-22 11:05:14 +00:00
Erik Johnston 5d7a6ad223
Allow streaming cache invalidate all to workers. (#6749) 2020-01-22 10:37:00 +00:00
Erik Johnston 0e68760078
Add a DeltaState to track changes to be made to current state (#6716) 2020-01-20 18:07:20 +00:00
Satsuki Yanagi 722b4f302d Fix syntax error in run_upgrade for schema 57 (#6728)
Fix #6727
Related #6655

Co-authored-by: Erik Johnston <erikj@jki.re>
2020-01-17 14:30:35 +00:00
Brendan Abolivier 3b72bb780a
Merge pull request #6714 from matrix-org/babolivier/retention_select_event
Fix instantiation of message retention purge jobs
2020-01-17 14:23:51 +00:00
Richard van der Hoff 14d8f342d5 move batch_iter to a separate module 2020-01-16 22:25:32 +00:00
Brendan Abolivier dac148341b
Fixup diff 2020-01-16 20:25:09 +00:00
Brendan Abolivier 842c2cfbf1
Remove get_room_event_after_stream_ordering entirely 2020-01-16 20:24:17 +00:00
Erik Johnston d386f2f339
Add StateMap type alias (#6715) 2020-01-16 13:31:22 +00:00
Brendan Abolivier e601f35d3b
Lint 2020-01-16 09:55:11 +00:00
Brendan Abolivier 066b9f52b8
Correctly order when selecting before stream ordering 2020-01-15 19:32:47 +00:00
Brendan Abolivier 8363588237
Fix typo 2020-01-15 19:13:22 +00:00
Brendan Abolivier 855af069a4
Fix instantiation of message retention purge jobs
When figuring out which topological token to start a purge job at, we
need to do the following:

1. Figure out a timestamp before which events will be purged
2. Select the first stream ordering after that timestamp
3. Select info about the first event after that stream ordering
4. Build a topological token from that info

In some situations (e.g. quiet rooms with a short max_lifetime), there
might not be an event after the stream ordering at step 3, therefore we
abort the purge with the error `No event found`. To mitigate that, this
patch fetches the first event _before_ the stream ordering, instead of
after.
2020-01-15 18:56:18 +00:00
Erik Johnston 19a1aac48c
Fix purge_room admin API (#6711) 2020-01-15 18:13:47 +00:00
Erik Johnston 28c98e51ff
Add `local_current_membership` table (#6655)
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
2020-01-15 14:59:33 +00:00
Andrew Morgan 1177d3f3a3
Quarantine media by ID or user ID (#6681) 2020-01-13 18:10:43 +00:00
Richard van der Hoff bf46821180 Refuse to start if sqlite is older than 3.11.0 2020-01-09 18:11:04 +00:00
Richard van der Hoff e48ba84e0b Check postgres version in check_database
this saves doing it on each connection, and will allow us to pass extra options
in.
2020-01-09 18:05:59 +00:00
Richard van der Hoff e97d1cf001 Modify check_database to take a connection rather than a cursor
We might not need the cursor at all.
2020-01-09 18:05:50 +00:00
Manuel Stahl d2906fe666 Allow admin users to create or modify users without a shared secret (#6495)
Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
2020-01-09 13:31:00 +00:00
Erik Johnston 7c232bd98b
Merge pull request #6664 from matrix-org/erikj/media_admin_apis
Fix media repo admin APIs when using a media worker.
2020-01-08 15:50:06 +00:00
Erik Johnston d74054afda Shuffle the code 2020-01-08 14:57:45 +00:00