Commit Graph

11604 Commits (bdf3cdaec800966a493998c65665ee951b3d39f5)

Author SHA1 Message Date
Erik Johnston b08b0a22d5
Add typing to synapse.federation.sender (#6871) 2020-02-07 13:56:38 +00:00
Erik Johnston de2d267375
Allow moving group read APIs to workers (#6866) 2020-02-07 11:14:19 +00:00
Dirk Klimpel 56ca93ef59
Admin api to add an email address (#6789) 2020-02-07 10:29:36 +00:00
Richard van der Hoff f4884444c3
remove unused room_version_to_event_format (#6857) 2020-02-07 09:26:57 +00:00
Richard van der Hoff e1b240329e
Merge pull request #6856 from matrix-org/rav/redact_changes/6
Pass room_version into `event_from_pdu_json`
2020-02-07 09:22:15 +00:00
Patrick Cloke 7765bf3989
Limit the number of events that can be requested when backfilling events (#6864)
Limit the maximum number of events requested when backfilling events.
2020-02-06 13:25:24 -05:00
Richard van der Hoff 928edef979 Pass room_version into `event_from_pdu_json`
It's called from all over the shop, so this one's a bit messy.
2020-02-06 16:08:27 +00:00
Richard van der Hoff b0c8bdd49d
pass room version into FederationClient.send_join (#6854)
... which allows us to sanity-check the create event.
2020-02-06 15:50:39 +00:00
timfi bce557175b
Allow empty federation_certificate_verification_whitelist (#6849) 2020-02-06 14:45:01 +00:00
PeerD 99fcc96289
Third party event rules Update (#6781) 2020-02-06 14:15:29 +00:00
Erik Johnston ed630ea17c
Reduce amount of logging at INFO level. (#6862)
A lot of the things we log at INFO are now a bit superfluous, so lets
make them DEBUG logs to reduce the amount we log by default.

Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
2020-02-06 13:31:05 +00:00
Richard van der Hoff 9bcd37146e
Merge pull request #6823 from matrix-org/rav/redact_changes/5
pass room versions around
2020-02-06 11:32:33 +00:00
Erik Johnston 2201ef8556 Synapse 1.10.0rc2 (2020-02-06)
==============================
 
 Bugfixes
 --------
 
 - Fix an issue with cross-signing where device signatures were not sent to remote servers. ([\#6844](https://github.com/matrix-org/synapse/issues/6844))
 - Fix to the unknown remote device detection which was introduced in 1.10.rc1. ([\#6848](https://github.com/matrix-org/synapse/issues/6848))
 
 Internal Changes
 ----------------
 
 - Detect unexpected sender keys on remote encrypted events and resync device lists. ([\#6850](https://github.com/matrix-org/synapse/issues/6850))
 -----BEGIN PGP SIGNATURE-----
 
 iQJEBAABCgAuFiEEumuwyPtYLL2OMhYdOtoG7cdT0R4FAl478l8QHGVyaWtAbWF0
 cml4Lm9yZwAKCRA62gbtx1PRHmKfD/0ZpBV95gMk0iecfvO6IY8+0KigzDUypgXf
 zp1k+k5DBUmH5/83h7+cXWHIJyofv1At1YdIq9J/nNeyqr3V3dcQwPabOKYbI96d
 kVD0EZX+2NjuYejAuF9eJNdiwRq5yMLluCfnOXr5jCS0NNdpO1xVb4bWYm50RmcD
 WJvBvmfwXwCC3AFyNb4IAipaVUXuVXXx1YhjFs6DsbtpPUG88e4uIhpfvrS4v0t8
 GFJZNuVJStOvyKHHTfNRIMkhy4xidj3HRUMSmjNpx07WnPBnbv4LnUOD1boYxgaP
 EoODYnoAPshdiKB0AywNNjue3TmFD/Z7vVEzlFP/lNZ4GDU4kN9C/EwFYw0C2Jqq
 f9O/E2ZuP9Qqume5O9UwMQQAmhV5lMBaIsYRbixU+9bSB893zRHRffA11HypzD00
 ZXj8QDOXpiXp2jpAwN1Rk9e/aZEX3qd+zUAgRmk+kYb0KTC2/gcY956rodNSSO/U
 RFYg4DFvoCgCrRSxZ4LQMlFu3YY2E7qWH+p/OHk3WG79jW64VpEFaytvv8fR+GKq
 g3EbtWy6mUzlKYqbjZ+ZTUDe5AWdv6ZX8xJqRD/S6cpiwyh6Gp89HHNvBThXoCmK
 fxkgw8if7eIITuTlNuDrqunxyWqwd3oVlzd2mi2bg0yRfcqJ9C6OuBV1VTLFZeky
 3sjCiU0IRw==
 =kcSy
 -----END PGP SIGNATURE-----

Merge tag 'v1.10.0rc2' into develop

Synapse 1.10.0rc2 (2020-02-06)
==============================

Bugfixes
--------

- Fix an issue with cross-signing where device signatures were not sent to remote servers. ([\#6844](https://github.com/matrix-org/synapse/issues/6844))
- Fix to the unknown remote device detection which was introduced in 1.10.rc1. ([\#6848](https://github.com/matrix-org/synapse/issues/6848))

Internal Changes
----------------

- Detect unexpected sender keys on remote encrypted events and resync device lists. ([\#6850](https://github.com/matrix-org/synapse/issues/6850))
2020-02-06 11:04:03 +00:00
Erik Johnston 6a7e90ad78 1.10.0rc2 2020-02-06 10:40:08 +00:00
Richard van der Hoff ee0525b2b2 Simplify `room_version` handling in `FederationClient.send_invite` 2020-02-05 17:43:57 +00:00
Richard van der Hoff f84700fba8 Pass room version object into `FederationClient.get_pdu` 2020-02-05 17:25:46 +00:00
Richard van der Hoff 577f460369
Merge pull request #6840 from matrix-org/rav/federation_client_async
Port much of `synapse.federation.federation_client` to async/await
2020-02-05 16:56:39 +00:00
Richard van der Hoff 6bbd890f05 make FederationClient._do_send_invite async 2020-02-05 15:50:31 +00:00
Richard van der Hoff 146fec0820
Apply suggestions from code review
Co-Authored-By: Erik Johnston <erik@matrix.org>
2020-02-05 15:47:00 +00:00
Erik Johnston a58860e480
Check sender_key matches on inbound encrypted events. (#6850)
If they don't then the device lists are probably out of sync.
2020-02-05 14:02:39 +00:00
Hubert Chathi 60d0672426 Merge pull request #6844 from matrix-org/uhoreg/cross_signing_fix_device_fed
add device signatures to device key query results
2020-02-05 10:54:49 +00:00
Michael Kaye a831d2e4e3
Reduce performance logging to DEBUG (#6833)
* Reduce tnx performance logging to DEBUG
* Changelog.d
2020-02-05 08:57:37 +00:00
Richard van der Hoff d88e0ec080
Database updates to populate rooms.room_version (#6847)
We're going to need this so that we can figure out how to handle redactions when fetching events from the database.
2020-02-04 21:31:08 +00:00
Erik Johnston 6475382d80
Fix detecting unknown devices from remote encrypted events. (#6848)
We were looking at the wrong event type (`m.room.encryption` vs
`m.room.encrypted`).

Also fixup the duplicate `EvenTypes` entries.

Introduced in #6776.
2020-02-04 17:25:54 +00:00
Hubert Chathi 74bf3fdbb9
Merge pull request #6844 from matrix-org/uhoreg/cross_signing_fix_device_fed
add device signatures to device key query results
2020-02-04 12:03:54 -05:00
Richard van der Hoff 5ef91b96f1 Merge remote-tracking branch 'origin/develop' into rav/federation_client_async 2020-02-04 12:07:05 +00:00
Richard van der Hoff c7d6d5c69e
Merge pull request #6837 from matrix-org/rav/federation_async
Port much of `synapse.handlers.federation` to async/await.
2020-02-04 12:06:18 +00:00
Hubert Chathi 23d8a55c7a add device signatures to device key query results 2020-02-04 00:13:12 -05:00
Richard van der Hoff ea23210b2d make FederationClient.send_invite async 2020-02-03 22:29:49 +00:00
Richard van der Hoff 6deeefb68c make FederationClient.get_missing_events async 2020-02-03 22:28:45 +00:00
Richard van der Hoff abadf44eb2 make FederationClient._do_send_leave async 2020-02-03 22:28:45 +00:00
Richard van der Hoff e88b90aaeb make FederationClient.send_leave.send_request async 2020-02-03 22:28:45 +00:00
Richard van der Hoff 638001116d make FederationClient._do_send_join async 2020-02-03 22:28:45 +00:00
Richard van der Hoff 3960527c2e make FederationClient.send_join.send_request async 2020-02-03 22:28:45 +00:00
Richard van der Hoff ad09ee9262 make FederationClient.make_membership_event.send_request async 2020-02-03 22:28:45 +00:00
Richard van der Hoff 1330c311b7 make FederationClient._try_destination_list async 2020-02-03 22:28:45 +00:00
Richard van der Hoff a46fabf17b make FederationClient.send_leave async 2020-02-03 22:28:45 +00:00
Richard van der Hoff 8af9f11bea make FederationClient.send_join async 2020-02-03 22:28:45 +00:00
Richard van der Hoff 3f11cbb404 make FederationClient.make_membership_event async 2020-02-03 22:28:45 +00:00
Richard van der Hoff 24d814ca23 make FederationClient.get_event_auth async 2020-02-03 22:28:45 +00:00
Richard van der Hoff d73683c363 make FederationClient.get_room_state_ids async 2020-02-03 22:28:45 +00:00
Richard van der Hoff 0cb0c7bcd5 make FederationClient.get_pdu async 2020-02-03 22:28:45 +00:00
Richard van der Hoff 0536d0c9be make FederationClient.backfill async 2020-02-03 22:28:45 +00:00
Richard van der Hoff 5d17c31596 make FederationHandler.send_invite async 2020-02-03 22:28:11 +00:00
Richard van der Hoff e81c093974 make FederationHandler.on_get_missing_events async 2020-02-03 19:15:08 +00:00
Erik Johnston b9391c9575
Add typing to SyncHandler (#6821)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-02-03 18:05:44 +00:00
Erik Johnston ae5b3104f0
Fix stacktraces when using ObservableDeferred and async/await (#6836) 2020-02-03 17:10:54 +00:00
Richard van der Hoff f64c96662e make FederationHandler.user_joined_room async 2020-02-03 16:29:30 +00:00
Richard van der Hoff 52642860da make FederationHandler._clean_room_for_join async 2020-02-03 16:29:30 +00:00
Richard van der Hoff 814cc00cb9 make FederationHandler._notify_persisted_event async 2020-02-03 16:29:30 +00:00
Richard van der Hoff 05299599b6 make FederationHandler.persist_events_and_notify async 2020-02-03 16:29:30 +00:00
Richard van der Hoff 3b7e0e002b make FederationHandler._make_and_verify_event async 2020-02-03 16:22:30 +00:00
Richard van der Hoff 4286e429a7 make FederationHandler.do_remotely_reject_invite async 2020-02-03 16:19:18 +00:00
Richard van der Hoff c3f296af32 make FederationHandler._check_for_soft_fail async 2020-02-03 16:16:31 +00:00
Richard van der Hoff dbdf843012 make FederationHandler._persist_auth_tree async 2020-02-03 16:14:58 +00:00
Richard van der Hoff ebd6a15af3 make FederationHandler.do_invite_join async 2020-02-03 16:13:13 +00:00
Richard van der Hoff 94f7b4cd54 make FederationHandler.on_event_auth async 2020-02-03 16:06:46 +00:00
Richard van der Hoff 863087d186 make FederationHandler.on_exchange_third_party_invite_request async 2020-02-03 16:02:50 +00:00
Richard van der Hoff 957129f4a7 make FederationHandler.construct_auth_difference async 2020-02-03 16:00:46 +00:00
Richard van der Hoff 0d5f2f4bb0 make FederationHandler._update_context_for_auth_events async 2020-02-03 15:55:35 +00:00
Richard van der Hoff a25ddf26a3 make FederationHandler._update_auth_events_and_context_for_auth async 2020-02-03 15:53:54 +00:00
Richard van der Hoff bc9b75c6f0 make FederationHandler.do_auth async 2020-02-03 15:51:24 +00:00
Richard van der Hoff 8033b257a7 make FederationHandler._prep_event async 2020-02-03 15:49:32 +00:00
Richard van der Hoff 1cdc253e0a make FederationHandler._handle_new_event async 2020-02-03 15:48:33 +00:00
Richard van der Hoff c556ed9e15 make FederationHandler._handle_new_events async 2020-02-03 15:43:51 +00:00
Richard van der Hoff 6e89ec5e32 make FederationHandler.on_make_leave_request async 2020-02-03 15:40:41 +00:00
Richard van der Hoff d184cbc031 make FederationHandler.on_send_leave_request async 2020-02-03 15:39:24 +00:00
Richard van der Hoff 98681f90cb make FederationHandler.on_make_join_request async 2020-02-03 15:38:02 +00:00
Richard van der Hoff af8ba6b525 make FederationHandler.on_invite_request async 2020-02-03 15:33:42 +00:00
Richard van der Hoff 7571bf86f0 make FederationHandler.on_send_join_request async 2020-02-03 15:32:48 +00:00
Richard van der Hoff b3e44f0bdf make FederationHandler.on_query_auth async 2020-02-03 15:30:23 +00:00
Andrew Morgan 370080531e
Allow URL-encoded user IDs on user admin api paths (#6825) 2020-02-03 13:18:42 +00:00
Richard van der Hoff b0d112e78b
Fix `room_version` in `on_invite_request` flow (#6827)
I messed this up a bit in #6805, but fortunately we weren't actually doing
anything with the room_version so it didn't matter that it was a str not a RoomVersion.
2020-02-03 13:15:23 +00:00
Erik Johnston ac0d45b78b 1.10.0rc1 2020-01-31 15:35:37 +00:00
Erik Johnston 83b0ea047b
Fix deleting of stale marker for device lists (#6819)
We were in fact only deleting stale marker when we got an incremental
update, rather than when we did a full resync.
2020-01-31 14:04:15 +00:00
Richard van der Hoff 7f93eb1903
pass room_version into compute_event_signature (#6807) 2020-01-31 13:47:43 +00:00
Richard van der Hoff a5afdd15e5
Merge pull request #6806 from matrix-org/rav/redact_changes/3
Pass room_version into add_hashes_and_signatures
2020-01-31 10:57:03 +00:00
Richard van der Hoff 160522e32c
Merge pull request #6820 from matrix-org/rav/get_room_version_id
Make `get_room_version` return a RoomVersion object
2020-01-31 10:56:42 +00:00
Richard van der Hoff 08f41a6f05 Add `get_room_version` method
So that we can start factoring out some of this boilerplatey boilerplate.
2020-01-31 10:28:15 +00:00
Richard van der Hoff d7bf793cc1 s/get_room_version/get_room_version_id/
... to make way for a forthcoming get_room_version which returns a RoomVersion
object.
2020-01-31 10:06:21 +00:00
Erik Johnston 7d846e8704
Fix bug with getting missing auth event during join 500'ed (#6810) 2020-01-31 09:49:13 +00:00
Richard van der Hoff 2a81393a4b Pass room_version into add_hashes_and_signatures 2020-01-30 22:15:50 +00:00
Richard van der Hoff 54f3f369bd Pass room_version into create_local_event_from_event_dict 2020-01-30 22:15:50 +00:00
Richard van der Hoff ef6bdafb29 Store the room version in EventBuilder 2020-01-30 22:15:50 +00:00
Richard van der Hoff 46a446828d
pass room version into FederationHandler.on_invite_request (#6805) 2020-01-30 22:13:02 +00:00
Erik Johnston e0992fcc5b
Log when we delete room in bg update (#6816) 2020-01-30 17:55:34 +00:00
Richard van der Hoff 184303b865
MSC2260: Block direct sends of m.room.aliases events (#6794)
as per MSC2260
2020-01-30 17:20:55 +00:00
Erik Johnston 57ad702af0
Backgroud update to clean out rooms from current state (#6802) 2020-01-30 17:17:44 +00:00
Erik Johnston b660327056
Resync remote device list when detected as stale. (#6786) 2020-01-30 17:06:38 +00:00
Erik Johnston c3d4ad8afd
Fix sending server up commands from workers (#6811)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2020-01-30 16:42:11 +00:00
Erik Johnston a5bab2d058
When server leaves room check for stale device lists. (#6801)
When a server leaves a room it may stop sharing a room with remote
users, and thus not get any updates to their device lists. So we need to
check for this case and delete those device lists from the cache.

We don't need to do this if we stop sharing a room because the remote
user leaves the room, because we track that case via looking at
membership changes.
2020-01-30 16:10:30 +00:00
Erik Johnston c80a9fe13d
When a client asks for remote keys check if should resync. (#6797)
If we detect that the remote users' keys may have changed then we should
attempt to resync against the remote server rather than using the
(potentially) stale local cache.
2020-01-30 15:06:58 +00:00
Richard van der Hoff 5a246611e3
Type defintions for use in refactoring for redaction changes (#6803)
* Bump signedjson to 1.1

... so that we can use the type definitions

* Fix breakage caused by upgrade to signedjson 1.1

Thanks, @illicitonion...
2020-01-30 11:25:59 +00:00
Erik Johnston a855b7c3a8
Remove unused DeviceRow class (#6800) 2020-01-29 12:06:31 +00:00
Richard van der Hoff dcd85b976d Make /directory/room/<alias> handle restrictive power levels
Fixes a bug where the alias would be added, but `PUT /directory/room/<alias>`
would return a 403.
2020-01-29 11:52:52 +00:00
Richard van der Hoff b36095ae5c Set the PL for aliases events to 0. 2020-01-29 11:52:52 +00:00
Richard van der Hoff ee42a5513e Factor out a `copy_power_levels_contents` method
I'm going to need another copy (hah!) of this.
2020-01-29 11:52:52 +00:00
Erik Johnston 6b9e1014cf
Fix race in federation sender that delayed device updates. (#6799)
We were sending device updates down both the federation stream and
device streams. This mean there was a race if the federation sender
worker processed the federation stream first, as when the sender checked
if there were new device updates the slaved ID generator hadn't been
updated with the new stream IDs and so returned nothing.

This situation is correctly handled by events/receipts/etc by not
sending updates down the federation stream and instead having the
federation sender worker listen on the other streams and poke the
transaction queues as appropriate.
2020-01-29 11:23:01 +00:00
Erik Johnston 611215a49c
Delete current state when server leaves a room (#6792)
Otherwise its just stale data, which may get deleted later anyway so
can't be relied on. It's also a bit of a shotgun if we're trying to get
the current state of a room we're not in.
2020-01-29 11:01:32 +00:00
Erik Johnston 2cad8baa70
Fix bug when querying remote user keys that require a resync. (#6796)
We ended up only returning a single device, rather than all of them.
2020-01-29 09:56:41 +00:00
Erik Johnston fcfb591b31
Fix outbound federation request metrics (#6795) 2020-01-28 18:59:48 +00:00
Richard van der Hoff cc109b79dd
Merge pull request #6787 from matrix-org/rav/msc2260
Implement updated auth rules from MSC2260
2020-01-28 15:11:22 +00:00
Richard van der Hoff a1f307f7d1 fix bad variable ref 2020-01-28 14:55:22 +00:00
Erik Johnston e17a110661
Detect unknown remote devices and mark cache as stale (#6776)
We just mark the fact that the cache may be stale in the database for
now.
2020-01-28 14:43:21 +00:00
Richard van der Hoff 49d3bca37b Implement updated auth rules from MSC2260 2020-01-28 14:20:10 +00:00
Richard van der Hoff a8ce7aeb43
Pass room version object into event_auth.check and check_redaction (#6788)
These are easier to work with than the strings and we normally have one around.

This fixes `FederationHander._persist_auth_tree` which was passing a
RoomVersion object into event_auth.check instead of a string.
2020-01-28 14:18:29 +00:00
Erik Johnston 02b44db922
Warn if postgres database has non-C locale. (#6734)
As using non-C locale can cause issues on upgrading OS.
2020-01-28 13:44:21 +00:00
Erik Johnston 33f904835a Merge branch 'master' into develop 2020-01-28 13:39:39 +00:00
Erik Johnston 77d9357226 1.9.1 2020-01-28 13:09:36 +00:00
Erik Johnston bdbeeb94ec
Fix setting `mau_limit_reserved_threepids` config (#6793)
Calling the invalidation function during initialisation of the data
stores introduces a circular dependency, causing Synapse to fail to
start.
2020-01-28 13:05:24 +00:00
Erik Johnston 8df862e45d
Add `rooms.room_version` column (#6729)
This is so that we don't have to rely on pulling it out from `current_state_events` table.
2020-01-27 14:30:57 +00:00
Erik Johnston d5275fc55f
Propagate cache invalidates from workers to other workers. (#6748)
Currently if a worker invalidates a cache it will be streamed to master, which then didn't forward those to other workers.
2020-01-27 13:47:50 +00:00
Andrew Morgan 9f7aaf90b5
Validate client_secret parameter (#6767) 2020-01-24 14:28:40 +00:00
Erik Johnston fa4d609e20
Make 'event.redacts' never raise. (#6771)
There are quite a few places that we assume that a redaction event has a
corresponding `redacts` key, which is not always the case. So lets
cheekily make it so that event.redacts just returns None instead.
2020-01-23 15:19:03 +00:00
Brendan Abolivier 51fc3f693e Merge branch 'master' into develop 2020-01-23 13:45:23 +00:00
Brendan Abolivier f3eac2b3e9 1.9.0 2020-01-23 12:57:55 +00:00
Richard van der Hoff 6b7462a13f
a bit of debugging for media storage providers (#6757)
* a bit of debugging for media storage providers

* changelog
2020-01-23 12:11:44 +00:00
Richard van der Hoff 5bd3cb7260
Minor fixes to user admin api (#6761)
* don't insist on a password (this is valid if you have an SSO login)
* fix reference to undefined `requester`
2020-01-23 12:03:58 +00:00
Andrew Morgan ce84dd9e20
Remove unnecessary abstractions in admin handler (#6751) 2020-01-22 15:09:57 +00:00
Brendan Abolivier 0d0f32bc53 1.9.0rc1 2020-01-22 14:03:46 +00:00
Andrew Morgan 90a28fb475
Admin API to list, filter and sort rooms (#6720) 2020-01-22 13:36:43 +00:00
Brendan Abolivier ed83c3a018
Fix typo in _select_thumbnail 2020-01-22 12:27:42 +00:00
Andrew Morgan aa9b00fb2f
Fix and add test to deprecated quarantine media admin api (#6756) 2020-01-22 11:05:50 +00:00
Neil Johnson 5e52d8563b Allow monthly active user limiting support for worker mode, fixes #4639. (#6742) 2020-01-22 11:05:14 +00:00
Erik Johnston 5d7a6ad223
Allow streaming cache invalidate all to workers. (#6749) 2020-01-22 10:37:00 +00:00
Erik Johnston 0e68760078
Add a DeltaState to track changes to be made to current state (#6716) 2020-01-20 18:07:20 +00:00
Erik Johnston b0a66ab83c
Fixup synapse.rest to pass mypy (#6732) 2020-01-20 17:38:21 +00:00
Erik Johnston 74b74462f1
Fix `/events/:event_id` deprecated API. (#6731) 2020-01-20 17:38:09 +00:00
Erik Johnston 0f6e525be3
Fixup synapse.api to pass mypy (#6733) 2020-01-20 17:34:13 +00:00
Erik Johnston ceecedc68b
Fix changing password via user admin API. (#6730) 2020-01-20 17:23:59 +00:00
Andrew Morgan 198d52da3a Fix empty account_validity config block 2020-01-20 14:05:29 +00:00
Brendan Abolivier a17f64361c
Add more logging around message retention policies support (#6717)
So we can debug issues like #6683 more easily
2020-01-17 20:51:44 +00:00
Richard van der Hoff 0b885d62ef bump version to v1.9.0.dev2 2020-01-17 14:58:58 +00:00
Satsuki Yanagi 722b4f302d Fix syntax error in run_upgrade for schema 57 (#6728)
Fix #6727
Related #6655

Co-authored-by: Erik Johnston <erikj@jki.re>
2020-01-17 14:30:35 +00:00
Brendan Abolivier 3b72bb780a
Merge pull request #6714 from matrix-org/babolivier/retention_select_event
Fix instantiation of message retention purge jobs
2020-01-17 14:23:51 +00:00
Richard van der Hoff 1dee1e900b bump version to v1.9.0.dev1 2020-01-17 10:44:12 +00:00
Richard van der Hoff 59dc87c618
Merge pull request #6724 from matrix-org/rav/log_saml_attributes
Log saml assertions rather than the whole response
2020-01-17 10:33:24 +00:00
Richard van der Hoff 2b6a77fcde
Delegate remote_user_id mapping to the saml mapping provider (#6723)
Turns out that figuring out a remote user id for the SAML user isn't quite as obvious as it seems. Factor it out to the SamlMappingProvider so that it's easy to control.
2020-01-17 10:32:47 +00:00
Erik Johnston a8a50f5b57
Wake up transaction queue when remote server comes back online (#6706)
This will be used to retry outbound transactions to a remote server if
we think it might have come back up.
2020-01-17 10:27:19 +00:00
Richard van der Hoff 5ce0b17e38
Clarify the `account_validity` and `email` sections of the sample configuration. (#6685)
Generally try to make this more comprehensible, and make it match the
conventions.

I've removed the documentation for all the settings which allow you to change
the names of the template files, because I can't really see why they are
useful.
2020-01-17 10:04:15 +00:00
Richard van der Hoff acc7820574 Log saml assertions rather than the whole response
... since the whole response is huge.

We even need to break up the assertions, since kibana otherwise truncates them.
2020-01-16 22:26:34 +00:00
Richard van der Hoff 14d8f342d5 move batch_iter to a separate module 2020-01-16 22:25:32 +00:00
Brendan Abolivier dac148341b
Fixup diff 2020-01-16 20:25:09 +00:00
Brendan Abolivier 842c2cfbf1
Remove get_room_event_after_stream_ordering entirely 2020-01-16 20:24:17 +00:00
Erik Johnston d386f2f339
Add StateMap type alias (#6715) 2020-01-16 13:31:22 +00:00
Brendan Abolivier e601f35d3b
Lint 2020-01-16 09:55:11 +00:00
Neil Johnson 38e0e59f42
Add org.matrix.e2e_cross_signing to unstable_features in /versions as per MSC1756 (#6712) 2020-01-16 09:46:14 +00:00
Erik Johnston 48c3a96886
Port synapse.replication.tcp to async/await (#6666)
* Port synapse.replication.tcp to async/await

* Newsfile

* Correctly document type of on_<FOO> functions as async

* Don't be overenthusiastic with the asyncing....
2020-01-16 09:16:12 +00:00
Brendan Abolivier 066b9f52b8
Correctly order when selecting before stream ordering 2020-01-15 19:32:47 +00:00
Brendan Abolivier 8363588237
Fix typo 2020-01-15 19:13:22 +00:00
Brendan Abolivier 855af069a4
Fix instantiation of message retention purge jobs
When figuring out which topological token to start a purge job at, we
need to do the following:

1. Figure out a timestamp before which events will be purged
2. Select the first stream ordering after that timestamp
3. Select info about the first event after that stream ordering
4. Build a topological token from that info

In some situations (e.g. quiet rooms with a short max_lifetime), there
might not be an event after the stream ordering at step 3, therefore we
abort the purge with the error `No event found`. To mitigate that, this
patch fetches the first event _before_ the stream ordering, instead of
after.
2020-01-15 18:56:18 +00:00
Erik Johnston 19a1aac48c
Fix purge_room admin API (#6711) 2020-01-15 18:13:47 +00:00
Andrew Morgan edc244eec4
Remove duplicate session check in web fallback servlet (#6702) 2020-01-15 18:05:18 +00:00
Richard van der Hoff 107f256cd8 Merge branch 'develop' into rav/module_api_extensions 2020-01-15 16:00:24 +00:00
Richard van der Hoff 8f5d7302ac
Implement RedirectException (#6687)
Allow REST endpoint implemnentations to raise a RedirectException, which will
redirect the user's browser to a given location.
2020-01-15 15:58:55 +00:00
Erik Johnston 28c98e51ff
Add `local_current_membership` table (#6655)
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
2020-01-15 14:59:33 +00:00
Erik Johnston b5ce7f5874
Process EDUs in parallel with PDUs. (#6697)
This means that things like to device messages don't get blocked behind
processing PDUs, which can potentially take *ages*.
2020-01-14 14:08:35 +00:00
Erik Johnston e8b68a4e4b
Fixup synapse.replication to pass mypy checks (#6667) 2020-01-14 14:08:06 +00:00
Andrew Morgan 1177d3f3a3
Quarantine media by ID or user ID (#6681) 2020-01-13 18:10:43 +00:00
Richard van der Hoff 326c893d24
Kill off RegistrationError (#6691)
This is pretty pointless. Let's just use SynapseError.
2020-01-13 12:48:22 +00:00
Richard van der Hoff 2d07c73777
Don't assign numeric IDs for empty usernames (#6690)
Fix a bug where we would assign a numeric userid if somebody tried registering
with an empty username
2020-01-13 12:47:30 +00:00
Richard van der Hoff 3cfac9593c
Merge pull request #6689 from matrix-org/rav/saml_mapping_provider_updates
Updates to the SAML mapping provider API
2020-01-13 12:44:55 +00:00
Richard van der Hoff 8039685051
Allow additional_resources to implement Resource directly (#6686)
AdditionalResource really doesn't add any value, and it gets in the way for
resources which want to support child resources or the like. So, if the
resource object already implements the IResource interface, don't bother
wrapping it.
2020-01-13 12:42:44 +00:00
Richard van der Hoff feee819973
Fix exceptions on requests for non-ascii urls (#6682)
Fixes #6402
2020-01-13 12:41:51 +00:00
Richard van der Hoff da4e52544e comment for run_in_background 2020-01-12 21:53:47 +00:00
Richard van der Hoff dc69a1cf43 Pass client redirect URL into SAML mapping providers 2020-01-12 21:40:49 +00:00
Richard van der Hoff 47e63cc67a Pass the module_api into the SamlMappingProvider
... for consistency with other modules, and because we'll need it sooner or
later and it will be a pain to introduce later.
2020-01-12 21:40:49 +00:00
Richard van der Hoff 01243b98e1 Handle `config` not being set for synapse plugin modules
Some modules don't need any config, so having to define a `config` property
just to keep the loader happy is a bit annoying.
2020-01-12 21:34:36 +00:00
Richard van der Hoff 473d3801b6 Cleanups and additions to the module API
Add some useful things, such as error types and logcontext handling, to the
API.

Make `hs` a private member to dissuade people from using it (hopefully
they aren't already).

Add a couple of new methods (`record_user_external_id` and
`generate_short_term_login_token`).
2020-01-12 21:31:44 +00:00
Richard van der Hoff bf46821180 Refuse to start if sqlite is older than 3.11.0 2020-01-09 18:11:04 +00:00
Richard van der Hoff e48ba84e0b Check postgres version in check_database
this saves doing it on each connection, and will allow us to pass extra options
in.
2020-01-09 18:05:59 +00:00
Richard van der Hoff e97d1cf001 Modify check_database to take a connection rather than a cursor
We might not need the cursor at all.
2020-01-09 18:05:50 +00:00
Manuel Stahl d2906fe666 Allow admin users to create or modify users without a shared secret (#6495)
Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
2020-01-09 13:31:00 +00:00
Erik Johnston d773290cb1 Merge branch 'master' into develop 2020-01-09 13:25:48 +00:00
Erik Johnston 24b2c940fb 1.8.0 2020-01-09 11:39:29 +00:00
Erik Johnston 7c232bd98b
Merge pull request #6664 from matrix-org/erikj/media_admin_apis
Fix media repo admin APIs when using a media worker.
2020-01-08 15:50:06 +00:00
Erik Johnston d74054afda Shuffle the code 2020-01-08 14:57:45 +00:00
Erik Johnston bca3455b38 Comments 2020-01-08 14:27:35 +00:00
Erik Johnston 187dc6ad02 Do not rely on streaming events, as media repo doesn't 2020-01-08 14:24:28 +00:00
Richard van der Hoff b064a41291 Merge remote-tracking branch 'origin/release-v1.8.0' into develop 2020-01-08 13:27:17 +00:00
Manuel Stahl 7caaa29daa Fix GET request on /_synapse/admin/v2/users endpoint (#6563)
Fixes #6552
2020-01-08 13:26:40 +00:00
Erik Johnston 1adf27c82a Import RoomStore in media worker to fix admin APIs 2020-01-08 13:26:20 +00:00
Erik Johnston 3cf7d6d5b6 Move media admin store functions to worker store 2020-01-08 13:26:20 +00:00
Richard van der Hoff 573fee759c
Back out ill-advised notary server hackery (#6657)
This was ill-advised. We can't modify verify_keys here, because the response
object has already been signed by the requested key.

Furthermore, it's somewhat unnecessary because existing versions of Synapse
(which get upset that the notary key isn't present in verify_keys) will fall
back to a direct fetch via `/key/v2/server`.

Also: more tests for fetching keys via perspectives: it would be nice if we actually tested when our fetcher can't talk to our notary impl.
2020-01-08 13:24:10 +00:00
Erik Johnston be29ed7ad8
Correctly proxy remote group HTTP errors. (#6654)
e.g. if remote returns a 404 then that shouldn't be treated as an error
but should be proxied through.
2020-01-07 15:36:41 +00:00
Brendan Abolivier 2b6b7f482a
Merge pull request #6621 from matrix-org/babolivier/purge_job_config_typo
Fix a typo in the purge jobs configuration example
2020-01-07 16:17:40 +01:00
Brendan Abolivier 391fb47791
Reword 2020-01-07 14:54:32 +00:00
Brendan Abolivier 3a86477162
Change the example from 5min to 12h
Have a purge job running every 5min is probably not something we want to advise admins to do as a sort-of default.
2020-01-07 14:53:07 +00:00
Erik Johnston 7f0e706ebf 1.8.0rc1 2020-01-07 14:31:13 +00:00
Brendan Abolivier 0ab5853ec9
Merge pull request #6652 from matrix-org/babolivier/depth_missing_events
Fix conditions failing if min_depth = 0
2020-01-07 15:22:07 +01:00
Richard van der Hoff 85db7f73be
Add a background update to clear tombstoned rooms from the directory (#6648)
* Add a background update to clear tombstoned rooms from the directory

* use the ABC metaclass
2020-01-07 14:18:43 +00:00
Richard van der Hoff 9824a39d80
Async/await for background updates (#6647)
so that bg update routines can be async
2020-01-07 14:12:42 +00:00
Richard van der Hoff 1ff5491117
Merge pull request #6645 from matrix-org/rav/fix_synchrotron_error
Fix exceptions in the synchrotron worker log when events are rejected.
2020-01-07 14:02:14 +00:00
Brendan Abolivier cd428a93e2
Fix conditions failing if min_depth = 0
This could result in Synapse not fetching prev_events for new events in the room if it has missed some events.
2020-01-07 12:08:58 +00:00
Richard van der Hoff 1807db5e73
Merge pull request #6629 from matrix-org/rav/kill_event_reference_hashes
Remove a bunch of unused code from event creation
2020-01-06 17:46:31 +00:00
Richard van der Hoff 26c5d3d398 Fix exceptions in log when rejected event is replicated 2020-01-06 17:16:28 +00:00
Richard van der Hoff c74de81bfc async/await for SyncReplicationHandler.process_and_notify 2020-01-06 17:14:28 +00:00
Richard van der Hoff bc42da4ab8 Clarify documentation on get_event* methods
Make it clearer how they behave in the face of rejected and/or missing events.
2020-01-06 17:12:06 +00:00
Richard van der Hoff ba897a7590
Fix some test failures when frozen_dicts are enabled (#6642)
Fixes #4026
2020-01-06 15:22:46 +00:00
Erik Johnston 9f6c1befbb
Add experimental 'databases' config (#6580) 2020-01-06 14:44:01 +00:00