Commit Graph

588 Commits (54f2ebbea5e2ca82012fe6f7fbe2246dcad65bc0)

Author SHA1 Message Date
Erik Johnston 43bcb1e54e Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2020-10-13 13:29:50 +01:00
Patrick Cloke 629a951b49
Move additional tasks to the background worker, part 4 (#8513) 2020-10-13 08:20:32 -04:00
Erik Johnston b2486f6656
Fix message duplication if something goes wrong after persisting the event (#8476)
Should fix #3365.
2020-10-13 12:07:56 +01:00
Andrew Morgan 66ac4b1e34
Allow modules to create and send events into rooms (#8479)
This PR allows Synapse modules making use of the `ModuleApi` to create and send non-membership events into a room. This can useful to have modules send messages, or change power levels in a room etc. Note that they must send event through a user that's already in the room.

The non-membership event limitation is currently arbitrary, as it's another chunk of work and not necessary at the moment.
2020-10-09 13:46:36 +01:00
Patrick Cloke c9c0ad5e20
Remove the deprecated Handlers object (#8494)
All handlers now available via get_*_handler() methods on the HomeServer.
2020-10-09 07:24:34 -04:00
Richard van der Hoff 903fcd2d35 update wording 2020-10-07 11:28:05 +01:00
Richard van der Hoff e775b5bb5b kill off `send_nonmember_event`
This is now redundant, and we can just call `handle_new_client_event` directly.
2020-10-05 19:04:10 +01:00
Richard van der Hoff fd0282201e pull up event.sender assertion 2020-10-05 19:00:50 +01:00
Richard van der Hoff 2ee302d016 Move shadow-ban check down into `handle_new_client_event`. 2020-10-05 18:55:06 +01:00
Richard van der Hoff b520a1bf5a De-duplicate duplicate handling
move the "duplicate state event" handling down into `handle_new_client_event`
where it can be shared between multiple call paths.
2020-10-05 18:38:25 +01:00
Richard van der Hoff f31f8e6319
Remove stream ordering from Metadata dict (#8452)
There's no need for it to be in the dict as well as the events table. Instead,
we store it in a separate attribute in the EventInternalMetadata object, and
populate that on load.

This means that we can rely on it being correctly populated for any event which
has been persited to the database.
2020-10-05 14:43:14 +01:00
Richard van der Hoff e675bbcc49 Remove redundant `EventCreationHandler._is_worker_app` attribute
This was added in 1c347c84bf/#7544 as a temporary optimisation. That was never
merged to develop, since it conflicted with #7492. The merge cf92310da forgot
to remove it.
2020-10-01 11:51:57 +01:00
Richard van der Hoff db13a8607e Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2020-10-01 11:22:36 +01:00
Erik Johnston ac11fcbbb8
Add EventStreamPosition type (#8388)
The idea is to remove some of the places we pass around `int`, where it can represent one of two things:

1. the position of an event in the stream; or
2. a token that partitions the stream, used as part of the stream tokens.

The valid operations are then:

1. did a position happen before or after a token;
2. get all events that happened before or after a token; and
3. get all events between two tokens.

(Note that we don't want to allow other operations as we want to change the tokens to be vector clocks rather than simple ints)
2020-09-24 13:24:17 +01:00
Richard van der Hoff 2983049a77
Factor out `_send_dummy_event_for_room` (#8370)
this makes it possible to use from the manhole, and seems cleaner anyway.
2020-09-23 18:18:43 +01:00
Erik Johnston 04cc249b43
Add experimental support for sharding event persister. Again. (#8294)
This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
2020-09-14 10:16:41 +01:00
Erik Johnston fe8ed1b46f
Make `StreamToken.room_key` be a `RoomStreamToken` instance. (#8281) 2020-09-11 12:22:55 +01:00
Erik Johnston 5d3e306d9f
Clean up `Notifier.on_new_room_event` code path (#8288)
The idea here is that we pass the `max_stream_id` to everything, and only use the stream ID of the particular event to figure out *when* the max stream position has caught up to the event and we can notify people about it.

This is to maintain the distinction between the position of an item in the stream (i.e. event A has stream ID 513) and a token that can be used to partition the stream (i.e. give me all events after stream ID 352). This distinction becomes important when the tokens are more complicated than a single number, which they will be once we start tracking the position of multiple writers in the tokens.

The valid operations here are:

1. Is a position before or after a token
2. Fetching all events between two tokens
3. Merging multiple tokens to get the "max", i.e. `C = max(A, B)` means that for all positions P where P is before A *or* before B, then P is before C.

Future PR will change the token type to a dedicated type.
2020-09-10 13:24:43 +01:00
Erik Johnston c9dbee50ae
Fixup pusher pool notifications (#8287)
`pusher_pool.on_new_notifications` expected a min and max stream ID, however that was not what we were passing in. Instead, let's just pass it the current max stream ID and have it track the last stream ID it got passed.

I believe that it mostly worked as we called the function for every event. However, it would break for events that got persisted out of order, i.e, that were persisted but the max stream ID wasn't incremented as not all preceding events had finished persisting, and push for that event would be delayed until another event got pushed to the effected users.
2020-09-09 16:56:08 +01:00
Erik Johnston dc9dcdbd59 Revert "Fixup pusher pool notifications"
This reverts commit e7fd336a53.
2020-09-09 16:19:22 +01:00
Erik Johnston e7fd336a53 Fixup pusher pool notifications 2020-09-09 16:17:50 +01:00
Richard van der Hoff d74e8f2875 Merge branch 'release-v1.20.0' into matrix-org-hotfixes 2020-09-07 13:44:54 +01:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Brendan Abolivier cc23d81a74 Merge branch 'develop' into matrix-org-hotfixes 2020-09-04 11:02:10 +01:00
Brendan Abolivier 9f8abdcc38
Revert "Add experimental support for sharding event persister. (#8170)" (#8242)
* Revert "Add experimental support for sharding event persister. (#8170)"

This reverts commit 82c1ee1c22.

* Changelog
2020-09-04 10:19:42 +01:00
Brendan Abolivier 505ea932f5 Merge branch 'develop' into matrix-org-hotfixes 2020-09-03 15:30:00 +01:00
Erik Johnston 82c1ee1c22
Add experimental support for sharding event persister. (#8170)
This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
2020-09-02 15:48:37 +01:00
Patrick Cloke da77520cd1
Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
Patrick Cloke 3f488bfded Merge branch 'develop' into matrix-org-hotfixes 2020-08-27 10:16:21 -04:00
Andrew Morgan e0d6244beb
Remove unused parameter from, and add safeguard in, get_room_data (#8174)
Small cleanup PR.

* Removed the unused `is_guest` argument
* Added a safeguard to a (currently) impossible code path, fixing static checking at the same time.
2020-08-26 15:07:35 +01:00
Patrick Cloke 5099bd68da
Do not allow send_nonmember_event to be called with shadow-banned users. (#8158) 2020-08-25 10:52:15 -04:00
Patrick Cloke cbd8d83da7
Stop shadow-banned users from sending non-member events. (#8142) 2020-08-24 13:58:56 -04:00
Patrick Cloke eebf52be06
Be stricter about JSON that is accepted by Synapse (#8106) 2020-08-19 07:26:03 -04:00
Patrick Cloke f40645e60b
Convert events worker database to async/await. (#8071) 2020-08-18 16:20:49 -04:00
Richard van der Hoff b4c1cfacc2 Merge branch 'develop' into matrix-org-hotfixes 2020-08-18 18:20:01 +01:00
Richard van der Hoff afe4c4e02e Merge branch 'develop' into matrix-org-hotfixes 2020-08-18 18:13:47 +01:00
Patrick Cloke 3c01724b33
Fix the return type of send_nonmember_events. (#8112) 2020-08-18 09:53:13 -04:00
Patrick Cloke 25e55d2598
Return the previous stream token if a non-member event is a duplicate. (#8093) 2020-08-18 07:53:23 -04:00
Richard van der Hoff 53834bb9c4
Run `remove_push_actions_from_staging` in foreground (#8081)
If we got an error persisting an event, we would try to remove the push actions
asynchronously, which would lead to a 'Re-starting finished log context'
warning.

I don't think there's any need for this to be asynchronous.
2020-08-13 17:05:31 +01:00
Brendan Abolivier 527f73d902 Merge branch 'develop' into matrix-org-hotfixes 2020-08-13 11:45:08 +01:00
Erik Johnston 5dd73d029e
Add type hints to handlers.message and events.builder (#8067) 2020-08-12 15:05:50 +01:00
Richard van der Hoff fcbab08cbd
Add an assertion on prev_events in create_new_client_event (#8041)
I think this would have caught all the cases in
https://github.com/matrix-org/synapse/issues/7642 - and I think a 500 makes
more sense here than a 403
2020-08-10 12:29:47 +01:00
Patrick Cloke d4a7829b12
Convert synapse.api to async/await (#8031) 2020-08-06 08:30:06 -04:00
Erik Johnston a7bdf98d01
Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00
Richard van der Hoff b2ccc72a00 Merge branch 'release-v1.18.0' into matrix-org-hotfixes 2020-07-28 10:15:22 +01:00
Patrick Cloke d8a9cd8d3e
Remove hacky error handling for inlineDeferreds. (#7950) 2020-07-27 08:35:56 -04:00
Richard van der Hoff be777e325d Merge branch 'develop' into matrix-org-hotfixes 2020-07-24 09:57:49 +01:00
Patrick Cloke cc9bb3dc3f
Convert the message handler to async/await. (#7884) 2020-07-22 12:29:15 -04:00
Patrick Cloke 6fca1b3506
Convert _base, profile, and _receipts handlers to async/await (#7860) 2020-07-17 07:08:30 -04:00
Richard van der Hoff cc86fbc9ad Merge branch 'develop' into matrix-org-hotfixes 2020-07-09 11:06:52 +01:00
Richard van der Hoff 2ab0b021f1
Generate real events when we reject invites (#7804)
Fixes #2181. 

The basic premise is that, when we
fail to reject an invite via the remote server, we can generate our own
out-of-band leave event and persist it as an outlier, so that we have something
to send to the client.
2020-07-09 10:40:19 +01:00
Erik Johnston c16bb06d25 Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2020-06-25 09:39:01 +01:00
Patrick Cloke e060bf4462
Convert directory handler to async/await (#7727) 2020-06-22 07:18:00 -04:00
Erik Johnston 8ba1086801 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2020-06-17 14:38:15 +01:00
Dagfinn Ilmari Mannsåker a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Patrick Cloke bd6dc17221
Replace iteritems/itervalues/iterkeys with native versions. (#7692) 2020-06-15 07:03:36 -04:00
Brendan Abolivier d6c7550cf5 Synapse 1.15.0rc1 (2020-06-09)
==============================
 
 Features
 --------
 
 - Advertise support for Client-Server API r0.6.0 and remove related unstable feature flags. ([\#6585](https://github.com/matrix-org/synapse/issues/6585))
 - Add an option to disable autojoining rooms for guest accounts. ([\#6637](https://github.com/matrix-org/synapse/issues/6637))
 - For SAML authentication, add the ability to pass email addresses to be added to new users' accounts via SAML attributes. Contributed by Christopher Cooper. ([\#7385](https://github.com/matrix-org/synapse/issues/7385))
 - Add admin APIs to allow server admins to manage users' devices. Contributed by @dklimpel. ([\#7481](https://github.com/matrix-org/synapse/issues/7481))
 - Add support for generating thumbnails for WebP images. Previously, users would see an empty box instead of preview image. ([\#7586](https://github.com/matrix-org/synapse/issues/7586))
 - Support the standardized `m.login.sso` user-interactive authentication flow. ([\#7630](https://github.com/matrix-org/synapse/issues/7630))
 
 Bugfixes
 --------
 
 - Allow new users to be registered via the admin API even if the monthly active user limit has been reached. Contributed by @dkimpel. ([\#7263](https://github.com/matrix-org/synapse/issues/7263))
 - Fix email notifications not being enabled for new users when created via the Admin API. ([\#7267](https://github.com/matrix-org/synapse/issues/7267))
 - Fix str placeholders in an instance of `PrepareDatabaseException`. Introduced in Synapse v1.8.0. ([\#7575](https://github.com/matrix-org/synapse/issues/7575))
 - Fix a bug in automatic user creation during first time login with `m.login.jwt`. Regression in v1.6.0. Contributed by @olof. ([\#7585](https://github.com/matrix-org/synapse/issues/7585))
 - Fix a bug causing the cross-signing keys to be ignored when resyncing a device list. ([\#7594](https://github.com/matrix-org/synapse/issues/7594))
 - Fix metrics failing when there is a large number of active background processes. ([\#7597](https://github.com/matrix-org/synapse/issues/7597))
 - Fix bug where returning rooms for a group would fail if it included a room that the server was not in. ([\#7599](https://github.com/matrix-org/synapse/issues/7599))
 - Fix duplicate key violation when persisting read markers. ([\#7607](https://github.com/matrix-org/synapse/issues/7607))
 - Prevent an entire iteration of the device list resync loop from failing if one server responds with a malformed result. ([\#7609](https://github.com/matrix-org/synapse/issues/7609))
 - Fix exceptions when fetching events from a remote host fails. ([\#7622](https://github.com/matrix-org/synapse/issues/7622))
 - Make `synctl restart` start synapse if it wasn't running. ([\#7624](https://github.com/matrix-org/synapse/issues/7624))
 - Pass device information through to the login endpoint when using the login fallback. ([\#7629](https://github.com/matrix-org/synapse/issues/7629))
 - Advertise the `m.login.token` login flow when OpenID Connect is enabled. ([\#7631](https://github.com/matrix-org/synapse/issues/7631))
 - Fix bug in account data replication stream. ([\#7656](https://github.com/matrix-org/synapse/issues/7656))
 
 Improved Documentation
 ----------------------
 
 - Update the OpenBSD installation instructions. ([\#7587](https://github.com/matrix-org/synapse/issues/7587))
 - Advertise Python 3.8 support in `setup.py`. ([\#7602](https://github.com/matrix-org/synapse/issues/7602))
 - Add a link to `#synapse:matrix.org` in the troubleshooting section of the README. ([\#7603](https://github.com/matrix-org/synapse/issues/7603))
 - Clarifications to the admin api documentation. ([\#7647](https://github.com/matrix-org/synapse/issues/7647))
 
 Internal Changes
 ----------------
 
 - Convert the identity handler to async/await. ([\#7561](https://github.com/matrix-org/synapse/issues/7561))
 - Improve query performance for fetching state from a PostgreSQL database. ([\#7567](https://github.com/matrix-org/synapse/issues/7567))
 - Speed up processing of federation stream RDATA rows. ([\#7584](https://github.com/matrix-org/synapse/issues/7584))
 - Add comment to systemd example to show postgresql dependency. ([\#7591](https://github.com/matrix-org/synapse/issues/7591))
 - Refactor `Ratelimiter` to limit the amount of expensive config value accesses. ([\#7595](https://github.com/matrix-org/synapse/issues/7595))
 - Convert groups handlers to async/await. ([\#7600](https://github.com/matrix-org/synapse/issues/7600))
 - Clean up exception handling in `SAML2ResponseResource`. ([\#7614](https://github.com/matrix-org/synapse/issues/7614))
 - Check that all asynchronous tasks succeed and general cleanup of `MonthlyActiveUsersTestCase` and `TestMauLimit`. ([\#7619](https://github.com/matrix-org/synapse/issues/7619))
 - Convert `get_user_id_by_threepid` to async/await. ([\#7620](https://github.com/matrix-org/synapse/issues/7620))
 - Switch to upstream `dh-virtualenv` rather than our fork for Debian package builds. ([\#7621](https://github.com/matrix-org/synapse/issues/7621))
 - Update CI scripts to check the number in the newsfile fragment. ([\#7623](https://github.com/matrix-org/synapse/issues/7623))
 - Check if the localpart of a Matrix ID is reserved for guest users earlier in the registration flow, as well as when responding to requests to `/register/available`. ([\#7625](https://github.com/matrix-org/synapse/issues/7625))
 - Minor cleanups to OpenID Connect integration. ([\#7628](https://github.com/matrix-org/synapse/issues/7628))
 - Attempt to fix flaky test: `PhoneHomeStatsTestCase.test_performance_100`. ([\#7634](https://github.com/matrix-org/synapse/issues/7634))
 - Fix typos of `m.olm.curve25519-aes-sha2` and `m.megolm.v1.aes-sha2` in comments, test files. ([\#7637](https://github.com/matrix-org/synapse/issues/7637))
 - Convert user directory, state deltas, and stats handlers to async/await. ([\#7640](https://github.com/matrix-org/synapse/issues/7640))
 - Remove some unused constants. ([\#7644](https://github.com/matrix-org/synapse/issues/7644))
 - Fix type information on `assert_*_is_admin` methods. ([\#7645](https://github.com/matrix-org/synapse/issues/7645))
 - Convert registration handler to async/await. ([\#7649](https://github.com/matrix-org/synapse/issues/7649))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEdVkXOgzrGzds0jtrHgFcFF8ZFs0FAl7fsrsACgkQHgFcFF8Z
 Fs0+4g/+L9rn5pnCdga0D5S2mBsmDzaU6kemNfZqfJCHhtK6e3BTvCbJ2cVwAJ5C
 E/KctWfrrAWqN3DHPd2PetEjUNvDrJSwnVKmkmZCBLB2kybbGd6fBys75N0cj0Tk
 jPedske3J/J9XZhnphJb1Zy+EjFlqQ+9Nm4xAo95G+pSoFhUrcAmAxLXcGU+bci5
 ObFWNSd7eZ7vtHWQXomNIs0d/45iQGmd1ElRcZ/HxW0dSxwQhLUgM3xbSIquAcsK
 q+P9zDvEkyrhSpnVYNdqBuFHzW44PktvpJMSfox0BS7WVIPaIm3MQE2yym44PqU9
 UIEh2I56J/yWywKHE3ABX9Eyh5ROL+k2i6zzMPZYnttMWj4/O9Km8RbJ2pjKm695
 JLaU2p1X67S1nEE8G8h/7GeWVFZh6ZnfyPqRt1F5iwYIg+vIqKH2oEtiRh7lLbW4
 JCsOQHKdN85w+88iehPu5SWSxKETcm+BpeLiQasXXkWVb6hr4ZJ5h9afrwQyXJYY
 i3Q/bBF4DMKMfwp3pMsUYlSdZZHXoX7BdOpkSqKeFK/QDgJBEDORmMU+krIXFObp
 6NGWj2xIIzHKXJx/4HCl/m6J285q/VXGOLVDHEGmOarBfgFGgq3tXn5xSFl1f/Fm
 /Gt11uIrtIwwSgDbGL92KUkySljZ4YbFywEqUR8pPlgTqQZJ2XI=
 =KXx8
 -----END PGP SIGNATURE-----

Merge tag 'v1.15.0rc1' of github.com:matrix-org/synapse into matrix-org-hotfixes

Synapse 1.15.0rc1 (2020-06-09)
==============================

Features
--------

- Advertise support for Client-Server API r0.6.0 and remove related unstable feature flags. ([\#6585](https://github.com/matrix-org/synapse/issues/6585))
- Add an option to disable autojoining rooms for guest accounts. ([\#6637](https://github.com/matrix-org/synapse/issues/6637))
- For SAML authentication, add the ability to pass email addresses to be added to new users' accounts via SAML attributes. Contributed by Christopher Cooper. ([\#7385](https://github.com/matrix-org/synapse/issues/7385))
- Add admin APIs to allow server admins to manage users' devices. Contributed by @dklimpel. ([\#7481](https://github.com/matrix-org/synapse/issues/7481))
- Add support for generating thumbnails for WebP images. Previously, users would see an empty box instead of preview image. ([\#7586](https://github.com/matrix-org/synapse/issues/7586))
- Support the standardized `m.login.sso` user-interactive authentication flow. ([\#7630](https://github.com/matrix-org/synapse/issues/7630))

Bugfixes
--------

- Allow new users to be registered via the admin API even if the monthly active user limit has been reached. Contributed by @dkimpel. ([\#7263](https://github.com/matrix-org/synapse/issues/7263))
- Fix email notifications not being enabled for new users when created via the Admin API. ([\#7267](https://github.com/matrix-org/synapse/issues/7267))
- Fix str placeholders in an instance of `PrepareDatabaseException`. Introduced in Synapse v1.8.0. ([\#7575](https://github.com/matrix-org/synapse/issues/7575))
- Fix a bug in automatic user creation during first time login with `m.login.jwt`. Regression in v1.6.0. Contributed by @olof. ([\#7585](https://github.com/matrix-org/synapse/issues/7585))
- Fix a bug causing the cross-signing keys to be ignored when resyncing a device list. ([\#7594](https://github.com/matrix-org/synapse/issues/7594))
- Fix metrics failing when there is a large number of active background processes. ([\#7597](https://github.com/matrix-org/synapse/issues/7597))
- Fix bug where returning rooms for a group would fail if it included a room that the server was not in. ([\#7599](https://github.com/matrix-org/synapse/issues/7599))
- Fix duplicate key violation when persisting read markers. ([\#7607](https://github.com/matrix-org/synapse/issues/7607))
- Prevent an entire iteration of the device list resync loop from failing if one server responds with a malformed result. ([\#7609](https://github.com/matrix-org/synapse/issues/7609))
- Fix exceptions when fetching events from a remote host fails. ([\#7622](https://github.com/matrix-org/synapse/issues/7622))
- Make `synctl restart` start synapse if it wasn't running. ([\#7624](https://github.com/matrix-org/synapse/issues/7624))
- Pass device information through to the login endpoint when using the login fallback. ([\#7629](https://github.com/matrix-org/synapse/issues/7629))
- Advertise the `m.login.token` login flow when OpenID Connect is enabled. ([\#7631](https://github.com/matrix-org/synapse/issues/7631))
- Fix bug in account data replication stream. ([\#7656](https://github.com/matrix-org/synapse/issues/7656))

Improved Documentation
----------------------

- Update the OpenBSD installation instructions. ([\#7587](https://github.com/matrix-org/synapse/issues/7587))
- Advertise Python 3.8 support in `setup.py`. ([\#7602](https://github.com/matrix-org/synapse/issues/7602))
- Add a link to `#synapse:matrix.org` in the troubleshooting section of the README. ([\#7603](https://github.com/matrix-org/synapse/issues/7603))
- Clarifications to the admin api documentation. ([\#7647](https://github.com/matrix-org/synapse/issues/7647))

Internal Changes
----------------

- Convert the identity handler to async/await. ([\#7561](https://github.com/matrix-org/synapse/issues/7561))
- Improve query performance for fetching state from a PostgreSQL database. ([\#7567](https://github.com/matrix-org/synapse/issues/7567))
- Speed up processing of federation stream RDATA rows. ([\#7584](https://github.com/matrix-org/synapse/issues/7584))
- Add comment to systemd example to show postgresql dependency. ([\#7591](https://github.com/matrix-org/synapse/issues/7591))
- Refactor `Ratelimiter` to limit the amount of expensive config value accesses. ([\#7595](https://github.com/matrix-org/synapse/issues/7595))
- Convert groups handlers to async/await. ([\#7600](https://github.com/matrix-org/synapse/issues/7600))
- Clean up exception handling in `SAML2ResponseResource`. ([\#7614](https://github.com/matrix-org/synapse/issues/7614))
- Check that all asynchronous tasks succeed and general cleanup of `MonthlyActiveUsersTestCase` and `TestMauLimit`. ([\#7619](https://github.com/matrix-org/synapse/issues/7619))
- Convert `get_user_id_by_threepid` to async/await. ([\#7620](https://github.com/matrix-org/synapse/issues/7620))
- Switch to upstream `dh-virtualenv` rather than our fork for Debian package builds. ([\#7621](https://github.com/matrix-org/synapse/issues/7621))
- Update CI scripts to check the number in the newsfile fragment. ([\#7623](https://github.com/matrix-org/synapse/issues/7623))
- Check if the localpart of a Matrix ID is reserved for guest users earlier in the registration flow, as well as when responding to requests to `/register/available`. ([\#7625](https://github.com/matrix-org/synapse/issues/7625))
- Minor cleanups to OpenID Connect integration. ([\#7628](https://github.com/matrix-org/synapse/issues/7628))
- Attempt to fix flaky test: `PhoneHomeStatsTestCase.test_performance_100`. ([\#7634](https://github.com/matrix-org/synapse/issues/7634))
- Fix typos of `m.olm.curve25519-aes-sha2` and `m.megolm.v1.aes-sha2` in comments, test files. ([\#7637](https://github.com/matrix-org/synapse/issues/7637))
- Convert user directory, state deltas, and stats handlers to async/await. ([\#7640](https://github.com/matrix-org/synapse/issues/7640))
- Remove some unused constants. ([\#7644](https://github.com/matrix-org/synapse/issues/7644))
- Fix type information on `assert_*_is_admin` methods. ([\#7645](https://github.com/matrix-org/synapse/issues/7645))
- Convert registration handler to async/await. ([\#7649](https://github.com/matrix-org/synapse/issues/7649))
2020-06-10 10:57:26 +01:00
Andrew Morgan f4e6495b5d
Performance improvements and refactor of Ratelimiter (#7595)
While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both:

* Rather undocumented, and
* causing a *lot* of config checks

This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation. 

Best to be reviewed commit-by-commit.
2020-06-05 10:47:20 +01:00
Erik Johnston 8beca8e21f Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2020-05-26 09:43:21 +01:00
Richard van der Hoff f4269694ce
Optimise some references to hs.config (#7546)
These are surprisingly expensive, and we only really need to do them at startup.
2020-05-22 21:47:07 +01:00
Erik Johnston e5c67d04db
Add option to move event persistence off master (#7517) 2020-05-22 16:11:35 +01:00
Erik Johnston 1531b214fc
Add ability to wait for replication streams (#7542)
The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room).

Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on.

People probably want to look at this commit by commit.
2020-05-22 14:21:54 +01:00
Erik Johnston cf92310da2 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2020-05-21 15:19:00 +01:00
Richard van der Hoff 89f795fe8a Merge branch 'rav/matrix_hacks' into matrix-org-hotfixes 2020-05-20 23:40:22 +01:00
Richard van der Hoff 1c347c84bf inline some config references 2020-05-20 23:33:13 +01:00
Aaron Raimist 250f3eb991
Omit displayname or avatar_url if they aren't set instead of returning null (#7497)
Per https://github.com/matrix-org/matrix-doc/issues/1436#issuecomment-410089470 they should be omitted instead of returning null or "". They aren't marked as required in the spec.

Fixes https://github.com/matrix-org/synapse/issues/7333

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2020-05-19 10:31:25 +01:00
Erik Johnston 1124111a12
Allow censoring of events to happen on workers. (#7492)
This is safe as we can now write to cache invalidation stream on workers, and is required for when we move event persistence off master.
2020-05-13 17:15:40 +01:00
Richard van der Hoff ae4f6140f1 Merge branch 'release-v1.13.0' into matrix-org-hotfixes 2020-05-07 10:42:56 +01:00
Brendan Abolivier d9b8d27494
Add a configuration setting for the dummy event threshold (#7422)
Add dummy_events_threshold which allows configuring the number of forward extremities a room needs for Synapse to send forward extremities in it.
2020-05-07 10:35:23 +01:00
Richard van der Hoff ff20747703 Merge branch 'release-v1.13.0' into matrix-org-hotfixes 2020-05-06 11:57:36 +01:00
Andrew Morgan 6b22921b19
async/await is_server_admin (#7363) 2020-05-01 15:15:36 +01:00
Richard van der Hoff 649e48a799 Merge branch 'develop' into matrix-org-hotfixes 2020-04-24 14:07:47 +01:00
Patrick Cloke 190ab593b7
Use the proper error code when a canonical alias that does not exist is used. (#7109) 2020-03-23 15:21:54 -04:00
Brendan Abolivier 69ce55c510
Don't filter out dummy events when we're checking the visibility of state 2020-03-11 17:52:54 +00:00
Brendan Abolivier 936686ed2d
Don't filter out events when we're checking the visibility of state 2020-03-11 15:21:25 +00:00
Brendan Abolivier 74050d0c1c Merge branch 'develop' into matrix-org-hotfixes 2020-03-09 15:06:56 +00:00
Patrick Cloke 7dcbc33a1b
Validate the alt_aliases property of canonical alias events (#6971) 2020-03-03 07:12:45 -05:00
Richard van der Hoff 69111a8b2a Merge branch 'develop' into matrix-org-hotfixes 2020-02-27 10:46:36 +00:00
Erik Johnston 1f773eec91
Port PresenceHandler to async/await (#6991) 2020-02-26 15:33:26 +00:00
Richard van der Hoff 9dbe34f0d0 Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2020-02-19 11:40:25 +00:00
Richard van der Hoff a0a1fd0bec Add `allow_departed_users` param to `check_in_room_or_world_readable`
... and set it everywhere it's called.

while we're here, rename it for consistency with `check_user_in_room` (and to
help check that I haven't missed any instances)
2020-02-19 08:52:51 +00:00
Erik Johnston bc936b5657 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2020-02-18 16:11:26 +00:00
Richard van der Hoff 5d17c31596 make FederationHandler.send_invite async 2020-02-03 22:28:11 +00:00
Erik Johnston c854d255e5 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2020-01-31 15:06:16 +00:00
Richard van der Hoff d7bf793cc1 s/get_room_version/get_room_version_id/
... to make way for a forthcoming get_room_version which returns a RoomVersion
object.
2020-01-31 10:06:21 +00:00
Richard van der Hoff a8ce7aeb43
Pass room version object into event_auth.check and check_redaction (#6788)
These are easier to work with than the strings and we normally have one around.

This fixes `FederationHander._persist_auth_tree` which was passing a
RoomVersion object into event_auth.check instead of a string.
2020-01-28 14:18:29 +00:00
Erik Johnston 45bf455948 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2020-01-07 14:24:36 +00:00
Richard van der Hoff 859663565c Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2020-01-06 15:43:41 +00:00
Richard van der Hoff 3bef62488e Remove unused hashes and depths from create_event params 2020-01-06 13:45:33 +00:00
Richard van der Hoff 66ca914dc0 Remove unused hashes and depths from create_new_client_event params 2020-01-06 13:45:33 +00:00
Richard van der Hoff 15720092ac replace get_prev_events_and_hashes_for_room with get_prev_events_for_room in create_new_client_event 2020-01-06 13:45:33 +00:00
Richard van der Hoff 5a04781643 rename get_prev_events_for_room to get_prev_events_and_hashes_for_room
... to make way for a new method which just returns the event ids
2020-01-06 13:45:33 +00:00
Erik Johnston fa780e9721
Change EventContext to use the Storage class (#6564) 2019-12-20 10:32:02 +00:00
Andrew Morgan fc316a4894
Prevent redacted events from appearing in message search (#6377) 2019-12-11 13:39:47 +00:00
Neil Johnson 0a522121a0 Merge branch 'release-v1.7.0' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-12-10 11:25:28 +00:00
Erik Johnston 8ad8bcbed0 Pull out room_invite_state_types config option once.
Pulling things out of config is currently surprisingly expensive.
2019-12-09 11:52:19 +00:00
Brendan Abolivier 54dd5dc12b
Add ephemeral messages support (MSC2228) (#6409)
Implement part [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). The parts that differ are:

* the feature is hidden behind a configuration flag (`enable_ephemeral_messages`)
* self-destruction doesn't happen for state events
* only implement support for the `m.self_destruct_after` field (not the `m.self_destruct` one)
* doesn't send synthetic redactions to clients because for this specific case we consider the clients to be able to destroy an event themselves, instead we just censor it (by pruning its JSON) in the database
2019-12-03 19:19:45 +00:00
Brendan Abolivier 708cef88cf
Discard retention policies when retrieving state
Purge jobs don't delete the latest event in a room in order to keep the forward extremity and not break the room. On the other hand, get_state_events, when given an at_token argument calls filter_events_for_client to know if the user can see the event that matches that (sync) token. That function uses the retention policies of the events it's given to filter out those that are too old from a client's view.

Some clients, such as Riot, when loading a room, request the list of members for the latest sync token it knows about, and get confused to the point of refusing to send any message if the server tells it that it can't get that information. This can happen very easily with the message retention feature turned on and a room with low activity so that the last event sent becomes too old according to the room's retention policy.

An easy and clean fix for that issue is to discard the room's retention policies when retrieving state.
2019-11-28 19:26:26 +00:00
Erik Johnston 6d1a3e2bdd Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-11-19 12:59:39 +00:00
Brendan Abolivier 09957ce0e4
Implement per-room message retention policies 2019-11-04 17:09:22 +00:00
Amber Brown 020add5099
Update black to 19.10b0 (#6304)
* update version of black and also fix the mypy config being overridden
2019-11-01 02:43:24 +11:00
Erik Johnston dfe0cd71b6
Merge pull request #6294 from matrix-org/erikj/add_state_storage
Add StateGroupStorage interface
2019-10-31 16:17:53 +01:00
Andrew Morgan 54fef094b3
Remove usage of deprecated logger.warn method from codebase (#6271)
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Erik Johnston 69f0054ce6 Port to use state storage 2019-10-30 14:46:54 +00:00
Erik Johnston 3ca4c7c516 Use new EventPersistenceStore 2019-10-23 16:15:03 +01:00
Richard van der Hoff 6e6b53ed3a Merge branch 'develop' into matrix-org-hotfixes 2019-09-26 15:22:33 +01:00
Neil Johnson 034db2ba21 Fix dummy event insertion consent bug (#6053)
Fixes #5905
2019-09-26 11:47:53 +01:00
Richard van der Hoff 7b61e6f5d6 Merge branch 'develop' into matrix-org-hotfixes 2019-09-18 13:55:25 +01:00
Erik Johnston 57dd41a45b
Fix comments
Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2019-09-11 13:54:50 +01:00
Erik Johnston c64c3bb4c5 Fix how we check for self redaction 2019-09-11 11:18:04 +01:00
Erik Johnston 54ce81c86d Allow use of different ratelimits for admin redactions.
This is useful to allow room admins to quickly deal with a large number
of abusive messages.
2019-09-11 10:46:38 +01:00
Erik Johnston a81e0233e9 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-08-29 11:18:57 +01:00
Will Hunt c998f25006
Apply suggestions from code review
Co-Authored-By: Erik Johnston <erik@matrix.org>
2019-08-23 10:28:54 +01:00
Half-Shot 9ba32f6573 Exempt bot users 2019-08-23 09:56:31 +01:00
Half-Shot 886eceba3e Return user_type in get_user_by_id 2019-08-23 09:14:52 +01:00
Erik Johnston 873ff9522b Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-08-01 14:46:09 +01:00
Erik Johnston cf89266b98 Deny redaction of events in a different room.
We already correctly filter out such redactions, but we should also deny
them over the CS API.
2019-07-31 16:12:27 +01:00
Erik Johnston 65fe31786d Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-07-30 10:12:13 +01:00
Richard van der Hoff f30a71a67b
Stop trying to fetch events with event_id=None. (#5753)
`None` is not a valid event id, so queuing up a database fetch for it seems
like a silly thing to do.

I considered making `get_event` return `None` if `event_id is None`, but then
its interaction with `allow_none` seemed uninituitive, and strong typing ftw.
2019-07-24 13:16:18 +01:00
Amber Brown 4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Richard van der Hoff 4102cb220a Merge branch 'release-v1.2.0' into matrix-org-hotfixes 2019-07-18 15:20:00 +01:00
Richard van der Hoff 9c70a02a9c
Ignore redactions of m.room.create events (#5701) 2019-07-17 19:08:02 +01:00
Erik Johnston 5299707329 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-07-17 10:56:55 +01:00
Amber Brown 463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Erik Johnston 67b73fd147 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-06-21 13:27:04 +01:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Erik Johnston 554609288b Run as background process and fix comments 2019-06-19 11:33:03 +01:00
Erik Johnston b42f90470f Add experimental option to reduce extremities.
Adds new config option `cleanup_extremities_with_dummy_events` which
periodically sends dummy events to rooms with more than 10 extremities.

THIS IS REALLY EXPERIMENTAL.
2019-06-18 15:02:18 +01:00
Erik Johnston c08e4dbadc Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-06-17 14:10:28 +01:00
Brendan Abolivier f874b16b2e Add plugin APIs for implementations of custom event rules. 2019-06-14 18:16:03 +01:00
Erik Johnston bbcd19f2d0 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-05-24 10:53:01 +01:00
Erik Johnston d16f5574b6
Merge pull request #5220 from matrix-org/erikj/dont_bundle_live_events
Don't bundle aggregations with events in /sync or /events or state queries
2019-05-24 10:36:31 +01:00
Erik Johnston 4cb577c23f Don't bundle aggs for /state and /members etc APIs 2019-05-24 09:52:33 +01:00
Erik Johnston 3cd598135f Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-05-23 15:54:13 +01:00
Erik Johnston 44b8ba484e Fix words 2019-05-21 16:51:45 +01:00
Erik Johnston c7ec06e8a6 Block attempts to annotate the same event twice 2019-05-20 17:39:05 +01:00
Richard van der Hoff ca03f90ee7 Merge branch 'develop' into matrix-org-hotfixes 2019-05-20 15:55:39 +01:00
Erik Johnston b54b03f9e1 Allow client event serialization to be async 2019-05-14 11:58:01 +01:00
Richard van der Hoff 9feee29d76 v0.99.4rc1
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEdVkXOgzrGzds0jtrHgFcFF8ZFs0FAlzZihYACgkQHgFcFF8Z
 Fs325w/8DOsFbrvITIYNpHKB8fZ4udrzwL/R+PRV+G5e/piJBumvnCtGqWFIKzwO
 FiF7M+7xPfATxI8sKHcFE7HAbG7/5zCFTp4vgVul6vzt2lhGR6uY0ZmBn7LizIiR
 ++eXAqfsqO4p6PepS5X3Mv17EiIQl+PFfN81va7/t4pk3YXtaucVAzYmlNWmHPiD
 KwyH9OsXdgu00/9QIBh+h2gCeB19e++6b+Ry2ZcMJAOgv8bgRisnjy35d0bN8uGR
 XSGFz9VEH4B8yvCOI9l9L4S+BvRmM+uL8qD5BSq5NIRqKt+YgdE9ioVscy461Xag
 lFjDqjkZxLRDHtLP2gGCM6iLaMIt1wZ3czC2P8YObgtVeskHaqK6rxKs1tP/jz+M
 fd7vXQpqA9zSmNJZ2p/nDFpcP1FRw6/gnYxqemcFOhSCmUeZcznaAkMBOCqW7XFF
 w9EOC5WIWmjHROsOdU59XgWai4igc2kTpflvM8jGWDYTdH4XOnGrde2MKCY+hYc4
 J/dII0sOKlMJzS9cqXkoWhARt+E+OeCbgDjnPnYvLX3AHZJcySGdQMzl+o2TKkYG
 MBGm6DDYsuKMx0Uv18b8WM1dWPbAyOXzxgBYFNuNOZLCZI81LE1jZf86rUnLvDqQ
 JTWBIQJhFiX6YxHMr5Enbtc1qoWp0rhlmCxnXpATSJQMc8pXVPY=
 =9pbU
 -----END PGP SIGNATURE-----

Merge tag 'v0.99.4rc1' into matrix-org-hotfixes

v0.99.4rc1
2019-05-14 11:12:22 +01:00
Brendan Abolivier 1473058b5e
Do checks on aliases for incoming m.room.aliases events (#5128)
Follow-up to #5124

Also added a bunch of checks to make sure everything (both the stuff added on #5124 and this PR) works as intended.
2019-05-08 17:01:30 +01:00
Erik Johnston 8467756dc1 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-04-04 14:43:57 +01:00
Andrew Morgan 4a4d5c4fd6
Fix grammar and document get_current_users_in_room (#4998) 2019-04-03 14:32:20 +01:00
Richard van der Hoff f41c9d37d6 Merge branch 'develop' into matrix-org-hotfixes 2019-04-02 13:47:08 +01:00
Richard van der Hoff 54a87a7b08
Collect room-version variations into one place (#4969)
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
Erik Johnston b0bf1ea7bd Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-03-21 14:10:31 +00:00
Erik Johnston cd62981a6a Revert spurious delete 2019-03-20 17:51:27 +00:00
Erik Johnston aa959a6c07 Use flags 2019-03-20 17:40:29 +00:00
Erik Johnston 7d47cc1305 Move requester check into assert_accepted_privacy_policy 2019-03-20 17:08:36 +00:00
Erik Johnston 74c46d81fa Only require consent for events with an associated request
There are a number of instances where a server or admin may puppet a
user to join/leave rooms, which we don't want to fail if the user has
not consented to the privacy policy. We fix this by adding a check to
test if the requester has an associated access_token, which is used as a
proxy to answer the question of whether the action is being done on
behalf of a real request from the user.
2019-03-20 16:50:23 +00:00
Richard van der Hoff 2561b628af Merge branch 'develop' into matrix-org-hotfixes 2019-03-19 12:19:20 +00:00
Neil Johnson 88f0675967 fix test_auto_create_auto_join_where_no_consent (#4886) 2019-03-19 11:38:59 +00:00
Erik Johnston dc510e0e43 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-03-05 14:41:13 +00:00
Richard van der Hoff 3064952939
Fix incorrect log about not persisting duplicate state event. (#4776)
We were logging this when it was not true.
2019-03-01 16:47:12 +00:00
Erik Johnston d351be1567 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-01-30 11:48:29 +00:00
Erik Johnston a696c48133 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:00:33 +00:00
Erik Johnston 5891a6edc8 Correctly set context.app_service 2019-01-29 12:09:10 +00:00
Erik Johnston 7709d2bd16 Implement rechecking of redactions 2019-01-29 11:56:20 +00:00
Erik Johnston be47cfa9c9 Refactor event building into EventBuilder
This is so that everything is done in one place, making it easier to
change the event format based on room version
2019-01-29 11:13:00 +00:00
Erik Johnston 073f6c2e5e
Merge pull request #4494 from matrix-org/erikj/fixup_event_validator
Split up event validation between event and builder
2019-01-29 10:55:07 +00:00
Andrew Morgan c7f2eaf4f4 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2019-01-29 10:07:13 +00:00
Erik Johnston d758d5310e Correctly use default room version if none is set 2019-01-28 17:26:39 +00:00
Erik Johnston b872c7b1b4 Split up event validation between event and builder
The validator was being run on the EventBuilder objects, and so the
validator only checked a subset of fields. With the upcoming
EventBuilder refactor even fewer fields will be there to validate.

To get around this we split the validation into those that can be run
against an EventBuilder and those run against a fully fledged event.
2019-01-28 17:00:14 +00:00
Erik Johnston ae2a957dba Pass through room version to event auth 2019-01-25 18:31:41 +00:00
Erik Johnston a50cf929c1 Require event format version to parse or create events 2019-01-25 10:32:19 +00:00
Erik Johnston be6a7e47fa
Revert "Require event format version to parse or create events" 2019-01-25 10:23:51 +00:00
Erik Johnston f431ff3fb8 Require event format version to parse or create events 2019-01-23 20:21:33 +00:00
Richard van der Hoff 34406cf22c Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2018-11-06 10:49:20 +00:00
Richard van der Hoff 9b827c40ca
Log some bits about event creation (#4121)
I found these helpful in debugging my room upgrade tests.
2018-10-31 15:42:23 +00:00
Erik Johnston 9f2d14ee26 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2018-10-26 09:52:23 +01:00
Erik Johnston cb53ce9d64
Refactor state group lookup to reduce DB hits (#4011)
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
2018-10-25 17:49:55 +01:00
Richard van der Hoff e7a16c6210 Remove redundant run_as_background_process() from pusherpool
`on_new_notifications` and `on_new_receipts` in `HttpPusher` and `EmailPusher`
now always return synchronously, so we can remove the `defer.gatherResults` on
their results, and the `run_as_background_process` wrappers can be removed too
because the PusherPool methods will now complete quickly enough.
2018-10-22 16:12:11 +01:00
Erik Johnston 7fa156af80 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2018-10-02 14:39:30 +01:00
Richard van der Hoff 8174c6725b Avoid reraise, to improve stacktraces 2018-10-01 18:50:34 +01:00
Erik Johnston f5abc10724 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2018-08-20 11:12:18 +01:00
Richard van der Hoff c144252a8c
Merge pull request #3710 from matrix-org/rav/logcontext_for_pusher_updates
Fix logcontexts for running pushers
2018-08-17 16:21:49 +01:00
Richard van der Hoff 66f7dc8c87 Fix logcontexts for running pushers
First of all, avoid resetting the logcontext before running the pushers, to fix
the "Starting db txn 'get_all_updated_receipts' from sentinel context" warning.

Instead, give them their own "background process" logcontexts.
2018-08-17 00:32:39 +01:00
Neil Johnson 372bf073c1 block event creation and room creation on hitting resource limits 2018-08-16 21:25:16 +01:00
Erik Johnston bb795b56da Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2018-08-16 15:51:16 +01:00
Matthew Hodgson 2f78f432c4 speed up /members and add at= and membership params (#3568) 2018-08-15 16:35:22 +01:00
Erik Johnston 4dd0604f61 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2018-08-15 15:37:05 +01:00
Amber Brown b37c472419
Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678) 2018-08-10 23:50:21 +10:00
Erik Johnston 729b672823 Use new helper base class for ReplicationSendEventRestServlet 2018-07-31 14:32:23 +01:00
Erik Johnston aa1bf10b91 Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2018-07-24 15:49:38 +01:00
Erik Johnston 536bc63a4e
Merge branch 'develop' into erikj/client_apis_move 2018-07-24 09:57:05 +01:00
Erik Johnston e1eb147f2a Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2018-07-23 16:45:22 +01:00
Erik Johnston 0faa3223cd Fix missing attributes on workers.
This was missed during the transition from attribute to getter for
getting state from context.
2018-07-23 16:28:00 +01:00
Erik Johnston b136d7ff8f Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2018-07-23 16:09:40 +01:00
Erik Johnston 0b0b24cb82 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/client_apis_move 2018-07-23 13:21:15 +01:00
Erik Johnston e42510ba63 Use new getters 2018-07-23 13:17:22 +01:00
Erik Johnston 5c88bb722f Move PaginationHandler to its own file 2018-07-20 15:32:23 +01:00
Erik Johnston 0ecf68aedc Move check_in_room_or_world_readable to Auth 2018-07-20 15:30:59 +01:00
Richard van der Hoff 7c712f95bb Combine Limiter and Linearizer
Linearizer was effectively a Limiter with max_count=1, so rather than
maintaining two sets of code, let's combine them.
2018-07-20 13:11:43 +01:00
Richard van der Hoff 8462c26485 Improvements to the Limiter
* give them names, to improve logging
* use a deque rather than a list for efficiency
2018-07-20 12:50:27 +01:00
Erik Johnston 8cb8df55e9 Split MessageHandler into read only and writers
This will let us call the read only parts from workers, and so be able
to move some APIs off of master, e.g. the `/state` API.
2018-07-18 15:33:03 +01:00
Richard van der Hoff f04e35c170 Merge branch 'develop' into matrix-org-hotfixes 2018-07-10 18:04:03 +01:00
Matthew Hodgson ea752bdd99 s/becuase/because/g 2018-07-10 17:58:18 +01:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
hera 82781f5838 Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2018-06-28 21:09:28 +00:00
Amber Brown 6350bf925e
Attempt to be more performant on PyPy (#3462) 2018-06-28 14:49:57 +01:00
Erik Johnston d46450195b Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes 2018-06-25 20:14:34 +01:00