Commit Graph

389 Commits (c2d4bd62a26ea0f19445aa23367fa7ee1b4e5b08)

Author SHA1 Message Date
AndrewFerr 9bc74743d5
Add configs to make profile data more private (#9203)
Add off-by-default configuration settings to:
- disable putting an invitee's profile info in invite events
- disable profile lookup via federation

Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
2021-02-19 09:50:41 +00:00
Eric Eastwood 0a00b7ff14
Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Travis Ralston b60bb28bbc
Add an admin API to get the current room state (#9168)
This could arguably replace the existing admin API for `/members`, however that is out of scope of this change.

This sort of endpoint is ideal for moderation use cases as well as other applications, such as needing to retrieve various bits of information about a room to perform a task (like syncing power levels between two places). This endpoint exposes nothing more than an admin would be able to access with a `select *` query on their database.
2021-02-02 11:16:29 +00:00
Erik Johnston dd8da8c5f6
Precompute joined hosts and store in Redis (#9198) 2021-01-26 13:57:31 +00:00
Erik Johnston 70586aa63e
Try and drop stale extremities. (#8929)
If we see stale extremities while persisting events, and notice that
they don't change the result of state resolution, we drop them.
2020-12-18 09:49:18 +00:00
David Teller f14428b25c
Allow spam-checker modules to be provide async methods. (#8890)
Spam checker modules can now provide async methods. This is implemented
in a backwards-compatible manner.
2020-12-11 14:05:15 -05:00
Erik Johnston f737368a26
Add admin API for logging in as a user (#8617) 2020-11-17 10:51:25 +00:00
Erik Johnston 243d427fbc
Block clients from sending server ACLs that lock the local server out. (#8708)
Fixes #4042
2020-11-03 12:13:48 +00:00
Richard van der Hoff c97da1e45d
Merge pull request #8678 from matrix-org/rav/fix_frozen_events
Fix serialisation errors when using third-party event rules.
2020-10-28 20:41:42 +00:00
Richard van der Hoff b6ca69e4f1 Remove frozendict_json_encoder and support frozendicts everywhere
Not being able to serialise `frozendicts` is fragile, and it's annoying to have
to think about which serialiser you want. There's no real downside to
supporting frozendicts, so let's just have one json encoder.
2020-10-28 15:56:57 +00:00
Andrew Morgan a699c044b6
Abstract code for stripping room state into a separate method (#8671)
This is a requirement for [knocking](https://github.com/matrix-org/synapse/pull/6739), and is abstracting some code that was originally used by the invite flow. I'm separating it out into this PR as it's a fairly contained change.

For a bit of context: when you invite a user to a room, you send them [stripped state events](https://matrix.org/docs/spec/server_server/unstable#put-matrix-federation-v2-invite-roomid-eventid) as part of `invite_room_state`. This is so that their client can display useful information such as the room name and avatar. The same requirement applies to knocking, as it would be nice for clients to be able to display a list of rooms you've knocked on - room name and avatar included.

The reason we're sending membership events down as well is in the case that you are invited to a room that does not have an avatar or name set. In that case, the client should use the displayname/avatar of the inviter. That information is located in the inviter's membership event.

This is optional as knocks don't really have any user in the room to link up to. When you knock on a room, your knock is sent by you and inserted into the room. It wouldn't *really* make sense to show the avatar of a random user - plus it'd be a data leak. So I've opted not to send membership events to the client here. The UX on the client for when you knock on a room without a name/avatar is a separate problem.

In essence this is just moving some inline code to a reusable store method.
2020-10-27 18:42:46 +00:00
Patrick Cloke 34a5696f93
Fix typos and spelling errors. (#8639) 2020-10-23 12:38:40 -04:00
Richard van der Hoff da0090fdff
Fix modifying events in `ThirdPartyRules` modules (#8564)
EventBuilder.build wants auth events these days
2020-10-16 13:39:46 +01:00
Richard van der Hoff 5649669c3c
Merge pull request #8535 from matrix-org/rav/third_party_events_updates
Support modifying event content from ThirdPartyRules modules
2020-10-15 20:55:41 +01:00
Richard van der Hoff a34b17e492 Simplify `_locally_reject_invite`
Update `EventCreationHandler.create_event` to accept an auth_events param, and
use it in `_locally_reject_invite` instead of reinventing the wheel.
2020-10-13 23:58:48 +01:00
Richard van der Hoff 617e8a4653 Allow ThirdPartyRules modules to replace event content
Support returning a new event dict from `check_event_allowed`.
2020-10-13 23:24:50 +01:00
Richard van der Hoff d9d86c2996 Remove redundant `token_id` parameter to create_event
this is always the same as requester.access_token_id.
2020-10-13 23:06:36 +01:00
Richard van der Hoff 123711ed19 Move third_party_rules check to event creation time
Rather than waiting until we handle the event, call the ThirdPartyRules check
when we fist create the event.
2020-10-13 21:38:48 +01:00
Patrick Cloke 629a951b49
Move additional tasks to the background worker, part 4 (#8513) 2020-10-13 08:20:32 -04:00
Erik Johnston b2486f6656
Fix message duplication if something goes wrong after persisting the event (#8476)
Should fix #3365.
2020-10-13 12:07:56 +01:00
Andrew Morgan 66ac4b1e34
Allow modules to create and send events into rooms (#8479)
This PR allows Synapse modules making use of the `ModuleApi` to create and send non-membership events into a room. This can useful to have modules send messages, or change power levels in a room etc. Note that they must send event through a user that's already in the room.

The non-membership event limitation is currently arbitrary, as it's another chunk of work and not necessary at the moment.
2020-10-09 13:46:36 +01:00
Patrick Cloke c9c0ad5e20
Remove the deprecated Handlers object (#8494)
All handlers now available via get_*_handler() methods on the HomeServer.
2020-10-09 07:24:34 -04:00
Richard van der Hoff 903fcd2d35 update wording 2020-10-07 11:28:05 +01:00
Richard van der Hoff e775b5bb5b kill off `send_nonmember_event`
This is now redundant, and we can just call `handle_new_client_event` directly.
2020-10-05 19:04:10 +01:00
Richard van der Hoff fd0282201e pull up event.sender assertion 2020-10-05 19:00:50 +01:00
Richard van der Hoff 2ee302d016 Move shadow-ban check down into `handle_new_client_event`. 2020-10-05 18:55:06 +01:00
Richard van der Hoff b520a1bf5a De-duplicate duplicate handling
move the "duplicate state event" handling down into `handle_new_client_event`
where it can be shared between multiple call paths.
2020-10-05 18:38:25 +01:00
Richard van der Hoff f31f8e6319
Remove stream ordering from Metadata dict (#8452)
There's no need for it to be in the dict as well as the events table. Instead,
we store it in a separate attribute in the EventInternalMetadata object, and
populate that on load.

This means that we can rely on it being correctly populated for any event which
has been persited to the database.
2020-10-05 14:43:14 +01:00
Erik Johnston ac11fcbbb8
Add EventStreamPosition type (#8388)
The idea is to remove some of the places we pass around `int`, where it can represent one of two things:

1. the position of an event in the stream; or
2. a token that partitions the stream, used as part of the stream tokens.

The valid operations are then:

1. did a position happen before or after a token;
2. get all events that happened before or after a token; and
3. get all events between two tokens.

(Note that we don't want to allow other operations as we want to change the tokens to be vector clocks rather than simple ints)
2020-09-24 13:24:17 +01:00
Richard van der Hoff 2983049a77
Factor out `_send_dummy_event_for_room` (#8370)
this makes it possible to use from the manhole, and seems cleaner anyway.
2020-09-23 18:18:43 +01:00
Erik Johnston 04cc249b43
Add experimental support for sharding event persister. Again. (#8294)
This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
2020-09-14 10:16:41 +01:00
Erik Johnston fe8ed1b46f
Make `StreamToken.room_key` be a `RoomStreamToken` instance. (#8281) 2020-09-11 12:22:55 +01:00
Erik Johnston 5d3e306d9f
Clean up `Notifier.on_new_room_event` code path (#8288)
The idea here is that we pass the `max_stream_id` to everything, and only use the stream ID of the particular event to figure out *when* the max stream position has caught up to the event and we can notify people about it.

This is to maintain the distinction between the position of an item in the stream (i.e. event A has stream ID 513) and a token that can be used to partition the stream (i.e. give me all events after stream ID 352). This distinction becomes important when the tokens are more complicated than a single number, which they will be once we start tracking the position of multiple writers in the tokens.

The valid operations here are:

1. Is a position before or after a token
2. Fetching all events between two tokens
3. Merging multiple tokens to get the "max", i.e. `C = max(A, B)` means that for all positions P where P is before A *or* before B, then P is before C.

Future PR will change the token type to a dedicated type.
2020-09-10 13:24:43 +01:00
Erik Johnston c9dbee50ae
Fixup pusher pool notifications (#8287)
`pusher_pool.on_new_notifications` expected a min and max stream ID, however that was not what we were passing in. Instead, let's just pass it the current max stream ID and have it track the last stream ID it got passed.

I believe that it mostly worked as we called the function for every event. However, it would break for events that got persisted out of order, i.e, that were persisted but the max stream ID wasn't incremented as not all preceding events had finished persisting, and push for that event would be delayed until another event got pushed to the effected users.
2020-09-09 16:56:08 +01:00
Erik Johnston dc9dcdbd59 Revert "Fixup pusher pool notifications"
This reverts commit e7fd336a53.
2020-09-09 16:19:22 +01:00
Erik Johnston e7fd336a53 Fixup pusher pool notifications 2020-09-09 16:17:50 +01:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Brendan Abolivier 9f8abdcc38
Revert "Add experimental support for sharding event persister. (#8170)" (#8242)
* Revert "Add experimental support for sharding event persister. (#8170)"

This reverts commit 82c1ee1c22.

* Changelog
2020-09-04 10:19:42 +01:00
Erik Johnston 82c1ee1c22
Add experimental support for sharding event persister. (#8170)
This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
2020-09-02 15:48:37 +01:00
Patrick Cloke da77520cd1
Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
Andrew Morgan e0d6244beb
Remove unused parameter from, and add safeguard in, get_room_data (#8174)
Small cleanup PR.

* Removed the unused `is_guest` argument
* Added a safeguard to a (currently) impossible code path, fixing static checking at the same time.
2020-08-26 15:07:35 +01:00
Patrick Cloke 5099bd68da
Do not allow send_nonmember_event to be called with shadow-banned users. (#8158) 2020-08-25 10:52:15 -04:00
Patrick Cloke cbd8d83da7
Stop shadow-banned users from sending non-member events. (#8142) 2020-08-24 13:58:56 -04:00
Patrick Cloke eebf52be06
Be stricter about JSON that is accepted by Synapse (#8106) 2020-08-19 07:26:03 -04:00
Patrick Cloke f40645e60b
Convert events worker database to async/await. (#8071) 2020-08-18 16:20:49 -04:00
Patrick Cloke 3c01724b33
Fix the return type of send_nonmember_events. (#8112) 2020-08-18 09:53:13 -04:00
Patrick Cloke 25e55d2598
Return the previous stream token if a non-member event is a duplicate. (#8093) 2020-08-18 07:53:23 -04:00
Richard van der Hoff 53834bb9c4
Run `remove_push_actions_from_staging` in foreground (#8081)
If we got an error persisting an event, we would try to remove the push actions
asynchronously, which would lead to a 'Re-starting finished log context'
warning.

I don't think there's any need for this to be asynchronous.
2020-08-13 17:05:31 +01:00
Erik Johnston 5dd73d029e
Add type hints to handlers.message and events.builder (#8067) 2020-08-12 15:05:50 +01:00
Richard van der Hoff fcbab08cbd
Add an assertion on prev_events in create_new_client_event (#8041)
I think this would have caught all the cases in
https://github.com/matrix-org/synapse/issues/7642 - and I think a 500 makes
more sense here than a 403
2020-08-10 12:29:47 +01:00