6bbd890f05
make FederationClient._do_send_invite async
2020-02-05 15:50:31 +00:00
146fec0820
Apply suggestions from code review
...
Co-Authored-By: Erik Johnston <erik@matrix.org>
2020-02-05 15:47:00 +00:00
a58860e480
Check sender_key matches on inbound encrypted events. ( #6850 )
...
If they don't then the device lists are probably out of sync.
2020-02-05 14:02:39 +00:00
60d0672426
Merge pull request #6844 from matrix-org/uhoreg/cross_signing_fix_device_fed
...
add device signatures to device key query results
2020-02-05 10:54:49 +00:00
a831d2e4e3
Reduce performance logging to DEBUG ( #6833 )
...
* Reduce tnx performance logging to DEBUG
* Changelog.d
2020-02-05 08:57:37 +00:00
d88e0ec080
Database updates to populate rooms.room_version ( #6847 )
...
We're going to need this so that we can figure out how to handle redactions when fetching events from the database.
2020-02-04 21:31:08 +00:00
6475382d80
Fix detecting unknown devices from remote encrypted events. ( #6848 )
...
We were looking at the wrong event type (`m.room.encryption` vs
`m.room.encrypted`).
Also fixup the duplicate `EvenTypes` entries.
Introduced in #6776 .
2020-02-04 17:25:54 +00:00
74bf3fdbb9
Merge pull request #6844 from matrix-org/uhoreg/cross_signing_fix_device_fed
...
add device signatures to device key query results
2020-02-04 12:03:54 -05:00
5ef91b96f1
Merge remote-tracking branch 'origin/develop' into rav/federation_client_async
2020-02-04 12:07:05 +00:00
c7d6d5c69e
Merge pull request #6837 from matrix-org/rav/federation_async
...
Port much of `synapse.handlers.federation` to async/await.
2020-02-04 12:06:18 +00:00
23d8a55c7a
add device signatures to device key query results
2020-02-04 00:13:12 -05:00
ea23210b2d
make FederationClient.send_invite async
2020-02-03 22:29:49 +00:00
6deeefb68c
make FederationClient.get_missing_events async
2020-02-03 22:28:45 +00:00
abadf44eb2
make FederationClient._do_send_leave async
2020-02-03 22:28:45 +00:00
e88b90aaeb
make FederationClient.send_leave.send_request async
2020-02-03 22:28:45 +00:00
638001116d
make FederationClient._do_send_join async
2020-02-03 22:28:45 +00:00
3960527c2e
make FederationClient.send_join.send_request async
2020-02-03 22:28:45 +00:00
ad09ee9262
make FederationClient.make_membership_event.send_request async
2020-02-03 22:28:45 +00:00
1330c311b7
make FederationClient._try_destination_list async
2020-02-03 22:28:45 +00:00
a46fabf17b
make FederationClient.send_leave async
2020-02-03 22:28:45 +00:00
8af9f11bea
make FederationClient.send_join async
2020-02-03 22:28:45 +00:00
3f11cbb404
make FederationClient.make_membership_event async
2020-02-03 22:28:45 +00:00
24d814ca23
make FederationClient.get_event_auth async
2020-02-03 22:28:45 +00:00
d73683c363
make FederationClient.get_room_state_ids async
2020-02-03 22:28:45 +00:00
0cb0c7bcd5
make FederationClient.get_pdu async
2020-02-03 22:28:45 +00:00
0536d0c9be
make FederationClient.backfill async
2020-02-03 22:28:45 +00:00
5d17c31596
make FederationHandler.send_invite async
2020-02-03 22:28:11 +00:00
e81c093974
make FederationHandler.on_get_missing_events async
2020-02-03 19:15:08 +00:00
b9391c9575
Add typing to SyncHandler ( #6821 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-02-03 18:05:44 +00:00
ae5b3104f0
Fix stacktraces when using ObservableDeferred and async/await ( #6836 )
2020-02-03 17:10:54 +00:00
f64c96662e
make FederationHandler.user_joined_room async
2020-02-03 16:29:30 +00:00
52642860da
make FederationHandler._clean_room_for_join async
2020-02-03 16:29:30 +00:00
814cc00cb9
make FederationHandler._notify_persisted_event async
2020-02-03 16:29:30 +00:00
05299599b6
make FederationHandler.persist_events_and_notify async
2020-02-03 16:29:30 +00:00
3b7e0e002b
make FederationHandler._make_and_verify_event async
2020-02-03 16:22:30 +00:00
4286e429a7
make FederationHandler.do_remotely_reject_invite async
2020-02-03 16:19:18 +00:00
c3f296af32
make FederationHandler._check_for_soft_fail async
2020-02-03 16:16:31 +00:00
dbdf843012
make FederationHandler._persist_auth_tree async
2020-02-03 16:14:58 +00:00
ebd6a15af3
make FederationHandler.do_invite_join async
2020-02-03 16:13:13 +00:00
94f7b4cd54
make FederationHandler.on_event_auth async
2020-02-03 16:06:46 +00:00
863087d186
make FederationHandler.on_exchange_third_party_invite_request async
2020-02-03 16:02:50 +00:00
957129f4a7
make FederationHandler.construct_auth_difference async
2020-02-03 16:00:46 +00:00
0d5f2f4bb0
make FederationHandler._update_context_for_auth_events async
2020-02-03 15:55:35 +00:00
a25ddf26a3
make FederationHandler._update_auth_events_and_context_for_auth async
2020-02-03 15:53:54 +00:00
bc9b75c6f0
make FederationHandler.do_auth async
2020-02-03 15:51:24 +00:00
8033b257a7
make FederationHandler._prep_event async
2020-02-03 15:49:32 +00:00
1cdc253e0a
make FederationHandler._handle_new_event async
2020-02-03 15:48:33 +00:00
c556ed9e15
make FederationHandler._handle_new_events async
2020-02-03 15:43:51 +00:00
6e89ec5e32
make FederationHandler.on_make_leave_request async
2020-02-03 15:40:41 +00:00
d184cbc031
make FederationHandler.on_send_leave_request async
2020-02-03 15:39:24 +00:00
98681f90cb
make FederationHandler.on_make_join_request async
2020-02-03 15:38:02 +00:00
af8ba6b525
make FederationHandler.on_invite_request async
2020-02-03 15:33:42 +00:00
7571bf86f0
make FederationHandler.on_send_join_request async
2020-02-03 15:32:48 +00:00
b3e44f0bdf
make FederationHandler.on_query_auth async
2020-02-03 15:30:23 +00:00
370080531e
Allow URL-encoded user IDs on user admin api paths ( #6825 )
2020-02-03 13:18:42 +00:00
b0d112e78b
Fix `room_version` in `on_invite_request` flow ( #6827 )
...
I messed this up a bit in #6805 , but fortunately we weren't actually doing
anything with the room_version so it didn't matter that it was a str not a RoomVersion.
2020-02-03 13:15:23 +00:00
ac0d45b78b
1.10.0rc1
2020-01-31 15:35:37 +00:00
83b0ea047b
Fix deleting of stale marker for device lists ( #6819 )
...
We were in fact only deleting stale marker when we got an incremental
update, rather than when we did a full resync.
2020-01-31 14:04:15 +00:00
7f93eb1903
pass room_version into compute_event_signature ( #6807 )
2020-01-31 13:47:43 +00:00
a5afdd15e5
Merge pull request #6806 from matrix-org/rav/redact_changes/3
...
Pass room_version into add_hashes_and_signatures
2020-01-31 10:57:03 +00:00
160522e32c
Merge pull request #6820 from matrix-org/rav/get_room_version_id
...
Make `get_room_version` return a RoomVersion object
2020-01-31 10:56:42 +00:00
08f41a6f05
Add `get_room_version` method
...
So that we can start factoring out some of this boilerplatey boilerplate.
2020-01-31 10:28:15 +00:00
d7bf793cc1
s/get_room_version/get_room_version_id/
...
... to make way for a forthcoming get_room_version which returns a RoomVersion
object.
2020-01-31 10:06:21 +00:00
7d846e8704
Fix bug with getting missing auth event during join 500'ed ( #6810 )
2020-01-31 09:49:13 +00:00
2a81393a4b
Pass room_version into add_hashes_and_signatures
2020-01-30 22:15:50 +00:00
54f3f369bd
Pass room_version into create_local_event_from_event_dict
2020-01-30 22:15:50 +00:00
ef6bdafb29
Store the room version in EventBuilder
2020-01-30 22:15:50 +00:00
46a446828d
pass room version into FederationHandler.on_invite_request ( #6805 )
2020-01-30 22:13:02 +00:00
e0992fcc5b
Log when we delete room in bg update ( #6816 )
2020-01-30 17:55:34 +00:00
184303b865
MSC2260: Block direct sends of m.room.aliases events ( #6794 )
...
as per MSC2260
2020-01-30 17:20:55 +00:00
57ad702af0
Backgroud update to clean out rooms from current state ( #6802 )
2020-01-30 17:17:44 +00:00
b660327056
Resync remote device list when detected as stale. ( #6786 )
2020-01-30 17:06:38 +00:00
c3d4ad8afd
Fix sending server up commands from workers ( #6811 )
...
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2020-01-30 16:42:11 +00:00
a5bab2d058
When server leaves room check for stale device lists. ( #6801 )
...
When a server leaves a room it may stop sharing a room with remote
users, and thus not get any updates to their device lists. So we need to
check for this case and delete those device lists from the cache.
We don't need to do this if we stop sharing a room because the remote
user leaves the room, because we track that case via looking at
membership changes.
2020-01-30 16:10:30 +00:00
c80a9fe13d
When a client asks for remote keys check if should resync. ( #6797 )
...
If we detect that the remote users' keys may have changed then we should
attempt to resync against the remote server rather than using the
(potentially) stale local cache.
2020-01-30 15:06:58 +00:00
5a246611e3
Type defintions for use in refactoring for redaction changes ( #6803 )
...
* Bump signedjson to 1.1
... so that we can use the type definitions
* Fix breakage caused by upgrade to signedjson 1.1
Thanks, @illicitonion...
2020-01-30 11:25:59 +00:00
a855b7c3a8
Remove unused DeviceRow class ( #6800 )
2020-01-29 12:06:31 +00:00
dcd85b976d
Make /directory/room/<alias> handle restrictive power levels
...
Fixes a bug where the alias would be added, but `PUT /directory/room/<alias>`
would return a 403.
2020-01-29 11:52:52 +00:00
b36095ae5c
Set the PL for aliases events to 0.
2020-01-29 11:52:52 +00:00
ee42a5513e
Factor out a `copy_power_levels_contents` method
...
I'm going to need another copy (hah!) of this.
2020-01-29 11:52:52 +00:00
6b9e1014cf
Fix race in federation sender that delayed device updates. ( #6799 )
...
We were sending device updates down both the federation stream and
device streams. This mean there was a race if the federation sender
worker processed the federation stream first, as when the sender checked
if there were new device updates the slaved ID generator hadn't been
updated with the new stream IDs and so returned nothing.
This situation is correctly handled by events/receipts/etc by not
sending updates down the federation stream and instead having the
federation sender worker listen on the other streams and poke the
transaction queues as appropriate.
2020-01-29 11:23:01 +00:00
611215a49c
Delete current state when server leaves a room ( #6792 )
...
Otherwise its just stale data, which may get deleted later anyway so
can't be relied on. It's also a bit of a shotgun if we're trying to get
the current state of a room we're not in.
2020-01-29 11:01:32 +00:00
2cad8baa70
Fix bug when querying remote user keys that require a resync. ( #6796 )
...
We ended up only returning a single device, rather than all of them.
2020-01-29 09:56:41 +00:00
fcfb591b31
Fix outbound federation request metrics ( #6795 )
2020-01-28 18:59:48 +00:00
cc109b79dd
Merge pull request #6787 from matrix-org/rav/msc2260
...
Implement updated auth rules from MSC2260
2020-01-28 15:11:22 +00:00
a1f307f7d1
fix bad variable ref
2020-01-28 14:55:22 +00:00
e17a110661
Detect unknown remote devices and mark cache as stale ( #6776 )
...
We just mark the fact that the cache may be stale in the database for
now.
2020-01-28 14:43:21 +00:00
49d3bca37b
Implement updated auth rules from MSC2260
2020-01-28 14:20:10 +00:00
a8ce7aeb43
Pass room version object into event_auth.check and check_redaction ( #6788 )
...
These are easier to work with than the strings and we normally have one around.
This fixes `FederationHander._persist_auth_tree` which was passing a
RoomVersion object into event_auth.check instead of a string.
2020-01-28 14:18:29 +00:00
02b44db922
Warn if postgres database has non-C locale. ( #6734 )
...
As using non-C locale can cause issues on upgrading OS.
2020-01-28 13:44:21 +00:00
33f904835a
Merge branch 'master' into develop
2020-01-28 13:39:39 +00:00
77d9357226
1.9.1
2020-01-28 13:09:36 +00:00
bdbeeb94ec
Fix setting `mau_limit_reserved_threepids` config ( #6793 )
...
Calling the invalidation function during initialisation of the data
stores introduces a circular dependency, causing Synapse to fail to
start.
2020-01-28 13:05:24 +00:00
8df862e45d
Add `rooms.room_version` column ( #6729 )
...
This is so that we don't have to rely on pulling it out from `current_state_events` table.
2020-01-27 14:30:57 +00:00
d5275fc55f
Propagate cache invalidates from workers to other workers. ( #6748 )
...
Currently if a worker invalidates a cache it will be streamed to master, which then didn't forward those to other workers.
2020-01-27 13:47:50 +00:00
9f7aaf90b5
Validate client_secret parameter ( #6767 )
2020-01-24 14:28:40 +00:00
fa4d609e20
Make 'event.redacts' never raise. ( #6771 )
...
There are quite a few places that we assume that a redaction event has a
corresponding `redacts` key, which is not always the case. So lets
cheekily make it so that event.redacts just returns None instead.
2020-01-23 15:19:03 +00:00
51fc3f693e
Merge branch 'master' into develop
2020-01-23 13:45:23 +00:00
f3eac2b3e9
1.9.0
2020-01-23 12:57:55 +00:00
6b7462a13f
a bit of debugging for media storage providers ( #6757 )
...
* a bit of debugging for media storage providers
* changelog
2020-01-23 12:11:44 +00:00
5bd3cb7260
Minor fixes to user admin api ( #6761 )
...
* don't insist on a password (this is valid if you have an SSO login)
* fix reference to undefined `requester`
2020-01-23 12:03:58 +00:00
ce84dd9e20
Remove unnecessary abstractions in admin handler ( #6751 )
2020-01-22 15:09:57 +00:00
0d0f32bc53
1.9.0rc1
2020-01-22 14:03:46 +00:00
90a28fb475
Admin API to list, filter and sort rooms ( #6720 )
2020-01-22 13:36:43 +00:00
ed83c3a018
Fix typo in _select_thumbnail
2020-01-22 12:27:42 +00:00
aa9b00fb2f
Fix and add test to deprecated quarantine media admin api ( #6756 )
2020-01-22 11:05:50 +00:00
5e52d8563b
Allow monthly active user limiting support for worker mode, fixes #4639 . ( #6742 )
2020-01-22 11:05:14 +00:00
5d7a6ad223
Allow streaming cache invalidate all to workers. ( #6749 )
2020-01-22 10:37:00 +00:00
0e68760078
Add a DeltaState to track changes to be made to current state ( #6716 )
2020-01-20 18:07:20 +00:00
b0a66ab83c
Fixup synapse.rest to pass mypy ( #6732 )
2020-01-20 17:38:21 +00:00
74b74462f1
Fix `/events/:event_id` deprecated API. ( #6731 )
2020-01-20 17:38:09 +00:00
0f6e525be3
Fixup synapse.api to pass mypy ( #6733 )
2020-01-20 17:34:13 +00:00
ceecedc68b
Fix changing password via user admin API. ( #6730 )
2020-01-20 17:23:59 +00:00
198d52da3a
Fix empty account_validity config block
2020-01-20 14:05:29 +00:00
a17f64361c
Add more logging around message retention policies support ( #6717 )
...
So we can debug issues like #6683 more easily
2020-01-17 20:51:44 +00:00
0b885d62ef
bump version to v1.9.0.dev2
2020-01-17 14:58:58 +00:00
722b4f302d
Fix syntax error in run_upgrade for schema 57 ( #6728 )
...
Fix #6727
Related #6655
Co-authored-by: Erik Johnston <erikj@jki.re>
2020-01-17 14:30:35 +00:00
3b72bb780a
Merge pull request #6714 from matrix-org/babolivier/retention_select_event
...
Fix instantiation of message retention purge jobs
2020-01-17 14:23:51 +00:00
1dee1e900b
bump version to v1.9.0.dev1
2020-01-17 10:44:12 +00:00
59dc87c618
Merge pull request #6724 from matrix-org/rav/log_saml_attributes
...
Log saml assertions rather than the whole response
2020-01-17 10:33:24 +00:00
2b6a77fcde
Delegate remote_user_id mapping to the saml mapping provider ( #6723 )
...
Turns out that figuring out a remote user id for the SAML user isn't quite as obvious as it seems. Factor it out to the SamlMappingProvider so that it's easy to control.
2020-01-17 10:32:47 +00:00
a8a50f5b57
Wake up transaction queue when remote server comes back online ( #6706 )
...
This will be used to retry outbound transactions to a remote server if
we think it might have come back up.
2020-01-17 10:27:19 +00:00
5ce0b17e38
Clarify the `account_validity` and `email` sections of the sample configuration. ( #6685 )
...
Generally try to make this more comprehensible, and make it match the
conventions.
I've removed the documentation for all the settings which allow you to change
the names of the template files, because I can't really see why they are
useful.
2020-01-17 10:04:15 +00:00
acc7820574
Log saml assertions rather than the whole response
...
... since the whole response is huge.
We even need to break up the assertions, since kibana otherwise truncates them.
2020-01-16 22:26:34 +00:00
14d8f342d5
move batch_iter to a separate module
2020-01-16 22:25:32 +00:00
dac148341b
Fixup diff
2020-01-16 20:25:09 +00:00
842c2cfbf1
Remove get_room_event_after_stream_ordering entirely
2020-01-16 20:24:17 +00:00
d386f2f339
Add StateMap type alias ( #6715 )
2020-01-16 13:31:22 +00:00
e601f35d3b
Lint
2020-01-16 09:55:11 +00:00
38e0e59f42
Add org.matrix.e2e_cross_signing to unstable_features in /versions as per MSC1756 ( #6712 )
2020-01-16 09:46:14 +00:00
48c3a96886
Port synapse.replication.tcp to async/await ( #6666 )
...
* Port synapse.replication.tcp to async/await
* Newsfile
* Correctly document type of on_<FOO> functions as async
* Don't be overenthusiastic with the asyncing....
2020-01-16 09:16:12 +00:00
066b9f52b8
Correctly order when selecting before stream ordering
2020-01-15 19:32:47 +00:00
8363588237
Fix typo
2020-01-15 19:13:22 +00:00
855af069a4
Fix instantiation of message retention purge jobs
...
When figuring out which topological token to start a purge job at, we
need to do the following:
1. Figure out a timestamp before which events will be purged
2. Select the first stream ordering after that timestamp
3. Select info about the first event after that stream ordering
4. Build a topological token from that info
In some situations (e.g. quiet rooms with a short max_lifetime), there
might not be an event after the stream ordering at step 3, therefore we
abort the purge with the error `No event found`. To mitigate that, this
patch fetches the first event _before_ the stream ordering, instead of
after.
2020-01-15 18:56:18 +00:00
19a1aac48c
Fix purge_room admin API ( #6711 )
2020-01-15 18:13:47 +00:00
edc244eec4
Remove duplicate session check in web fallback servlet ( #6702 )
2020-01-15 18:05:18 +00:00
107f256cd8
Merge branch 'develop' into rav/module_api_extensions
2020-01-15 16:00:24 +00:00
8f5d7302ac
Implement RedirectException ( #6687 )
...
Allow REST endpoint implemnentations to raise a RedirectException, which will
redirect the user's browser to a given location.
2020-01-15 15:58:55 +00:00
28c98e51ff
Add `local_current_membership` table ( #6655 )
...
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
2020-01-15 14:59:33 +00:00
b5ce7f5874
Process EDUs in parallel with PDUs. ( #6697 )
...
This means that things like to device messages don't get blocked behind
processing PDUs, which can potentially take *ages*.
2020-01-14 14:08:35 +00:00
e8b68a4e4b
Fixup synapse.replication to pass mypy checks ( #6667 )
2020-01-14 14:08:06 +00:00
1177d3f3a3
Quarantine media by ID or user ID ( #6681 )
2020-01-13 18:10:43 +00:00
326c893d24
Kill off RegistrationError ( #6691 )
...
This is pretty pointless. Let's just use SynapseError.
2020-01-13 12:48:22 +00:00
2d07c73777
Don't assign numeric IDs for empty usernames ( #6690 )
...
Fix a bug where we would assign a numeric userid if somebody tried registering
with an empty username
2020-01-13 12:47:30 +00:00
3cfac9593c
Merge pull request #6689 from matrix-org/rav/saml_mapping_provider_updates
...
Updates to the SAML mapping provider API
2020-01-13 12:44:55 +00:00
8039685051
Allow additional_resources to implement Resource directly ( #6686 )
...
AdditionalResource really doesn't add any value, and it gets in the way for
resources which want to support child resources or the like. So, if the
resource object already implements the IResource interface, don't bother
wrapping it.
2020-01-13 12:42:44 +00:00
feee819973
Fix exceptions on requests for non-ascii urls ( #6682 )
...
Fixes #6402
2020-01-13 12:41:51 +00:00
da4e52544e
comment for run_in_background
2020-01-12 21:53:47 +00:00
dc69a1cf43
Pass client redirect URL into SAML mapping providers
2020-01-12 21:40:49 +00:00
47e63cc67a
Pass the module_api into the SamlMappingProvider
...
... for consistency with other modules, and because we'll need it sooner or
later and it will be a pain to introduce later.
2020-01-12 21:40:49 +00:00
Richard van der Hoff