a466b67972
Reduce run-times of tests by advancing the reactor less ( #7757 )
2020-08-27 11:39:53 +01:00
6fe12c9512
Do not propagate typing notifications from shadow-banned users. ( #8176 )
2020-08-26 12:05:36 -04:00
4c6c56dc58
Convert simple_select_one and simple_select_one_onecol to async ( #8162 )
2020-08-26 07:19:32 -04:00
3f91638da6
Allow denying or shadow banning registrations via the spam checker ( #8034 )
2020-08-20 15:42:58 -04:00
050e20e7ca
Convert some of the general database methods to async ( #8100 )
2020-08-17 12:18:01 -04:00
ad6190c925
Convert stream database to async/await. ( #8074 )
2020-08-17 07:24:46 -04:00
5ecc8b5825
Convert devices database to async/await. ( #8069 )
2020-08-12 10:51:42 -04:00
d68e10f308
Convert account data, device inbox, and censor events databases to async/await ( #8063 )
2020-08-12 09:29:06 -04:00
a0acdfa9e9
Converts event_federation and registration databases to async/await ( #8061 )
2020-08-11 17:21:13 -04:00
7f837959ea
Convert directory, e2e_room_keys, end_to_end_keys, monthly_active_users database to async ( #8042 )
2020-08-07 13:36:29 -04:00
d4a7829b12
Convert synapse.api to async/await ( #8031 )
2020-08-06 08:30:06 -04:00
a7bdf98d01
Rename database classes to make some sense ( #8033 )
2020-08-05 21:38:57 +01:00
5d92a1428c
Prevent join->join membership transitions changing member count ( #7977 )
...
`StatsHandler` handles updates to the `current_state_delta_stream`, and updates room stats such as the amount of state events, joined users, etc.
However, it counts every new join membership as a new user entering a room (and that user being in another room), whereas it's possible for a user's membership status to go from join -> join, for instance when they change their per-room profile information.
This PR adds a check for join->join membership transitions, and bails out early, as none of the further checks are necessary at that point.
Due to this bug, membership stats in many rooms have ended up being wildly larger than their true values. I am not sure if we also want to include a migration step which recalculates these statistics (possibly using the `_populate_stats_process_rooms` bg update).
Bug introduced in the initial implementation https://github.com/matrix-org/synapse/pull/4338 .
2020-08-03 21:54:24 +01:00
c978f6c451
Convert federation client to async/await. ( #7975 )
2020-07-30 08:01:33 -04:00
4cce8ef74e
Convert appservice to async. ( #7973 )
2020-07-30 07:27:39 -04:00
6b3ac3b8cd
Convert device handler to async/await ( #7871 )
2020-07-17 07:09:25 -04:00
6fca1b3506
Convert _base, profile, and _receipts handlers to async/await ( #7860 )
2020-07-17 07:08:30 -04:00
9006e125af
Fix tests
2020-07-15 15:47:27 +01:00
b11450dedc
Convert E2E key and room key handlers to async/await. ( #7851 )
2020-07-15 08:48:58 -04:00
6d687ebba1
Convert the appservice handler to async/await. ( #7775 )
2020-07-06 07:40:35 -04:00
62b1ce8539
isort 5 compatibility ( #7786 )
...
The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
2020-07-05 16:32:02 +01:00
71cccf1593
Additional configuration options for auto-join rooms ( #7763 )
2020-06-30 15:41:36 -04:00
3630825612
Convert the typing handler to async/await. ( #7679 )
2020-06-17 10:37:59 -04:00
2b2344652b
Ensure etag is a string for GET room_keys/version response ( #7691 )
2020-06-15 13:42:44 -04:00
fcd6961441
Add option to enable encryption by default for new rooms ( #7639 )
...
Fixes https://github.com/matrix-org/synapse/issues/2431
Adds config option `encryption_enabled_by_default_for_room_type`, which determines whether encryption should be enabled with the default encryption algorithm in private or public rooms upon creation. Whether the room is private or public is decided based upon the room creation preset that is used.
Part of this PR is also pulling out all of the individual instances of `m.megolm.v1.aes-sha2` into a constant variable to eliminate typos ala https://github.com/matrix-org/synapse/pull/7637
Based on #7637
2020-06-10 17:44:34 +01:00
737b4a936e
Convert user directory handler and related classes to async/await. ( #7640 )
2020-06-05 14:42:55 -04:00
09099313e6
Add an option to disable autojoin for guest accounts ( #6637 )
...
Fixes https://github.com/matrix-org/synapse/issues/3177
2020-06-05 18:18:15 +01:00
f4e6495b5d
Performance improvements and refactor of Ratelimiter ( #7595 )
...
While working on https://github.com/matrix-org/synapse/issues/5665 I found myself digging into the `Ratelimiter` class and seeing that it was both:
* Rather undocumented, and
* causing a *lot* of config checks
This PR attempts to refactor and comment the `Ratelimiter` class, as well as encourage config file accesses to only be done at instantiation.
Best to be reviewed commit-by-commit.
2020-06-05 10:47:20 +01:00
c389bfb6ea
Fix encryption algorithm typos in tests/comments ( #7637 )
...
@uhoreg has confirmed these were both typos. They are only in comments and tests though, rather than anything critical.
Introduced in:
* https://github.com/matrix-org/synapse/pull/7157
* https://github.com/matrix-org/synapse/pull/5726
2020-06-04 20:03:40 +01:00
1531b214fc
Add ability to wait for replication streams ( #7542 )
...
The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room).
Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on.
People probably want to look at this commit by commit.
2020-05-22 14:21:54 +01:00
a3cf36f76e
Support UI Authentication for OpenID Connect accounts ( #7457 )
2020-05-15 12:26:02 -04:00
08bc80ef09
Implement room version 6 (MSC2240). ( #7506 )
2020-05-15 09:30:10 -04:00
56b66db78a
Strictly enforce canonicaljson requirements in a new room version ( #7381 )
2020-05-14 13:24:01 -04:00
67feea8044
Extend spam checker to allow for multiple modules ( #7435 )
2020-05-08 19:25:48 +01:00
616af44137
Implement OpenID Connect-based login ( #7256 )
2020-05-08 08:30:40 -04:00
aee9130a83
Stop Auth methods from polling the config on every req. ( #7420 )
2020-05-06 15:54:58 +01:00
6b22921b19
async/await is_server_admin ( #7363 )
2020-05-01 15:15:36 +01:00
eed7c5b89e
Convert auth handler to async/await ( #7261 )
2020-04-15 12:40:18 -04:00
468dcc767b
Allow admins to create aliases when they are not in the room ( #7191 )
2020-04-01 08:27:05 -04:00
6486c96b65
Merge pull request #7157 from matrix-org/rev.outbound_device_pokes_tests
...
Add tests for outbound device pokes
2020-03-30 13:59:07 +01:00
8327eb9280
Add options to prevent users from changing their profile. ( #7096 )
2020-03-27 19:15:23 +00:00
28d9d6e8a9
Remove spurious "name" parameter to `default_config`
...
this is never set to anything other than "test", and is a source of unnecessary
boilerplate.
2020-03-24 18:33:49 +00:00
6a35046363
Revert "Add options to disable setting profile info for prevent changes. ( #7053 )"
...
This reverts commit 54dd28621b6640460d05
2020-03-17 11:25:01 +00:00
54dd28621b
Add options to disable setting profile info for prevent changes. ( #7053 )
2020-03-10 22:23:01 +00:00
885134529f
updates after review
2020-03-09 22:09:29 +01:00
7e5f40e771
fix tests
2020-03-09 21:00:36 +01:00
04f4b5f6f8
add tests
2020-03-09 19:51:31 +01:00
1f5f3ae8b1
Add options to disable setting profile info for prevent changes.
2020-03-08 14:49:33 +01:00
13892776ef
Allow deleting an alias if the user has sufficient power level ( #6986 )
2020-03-04 11:30:46 -05:00
7dcbc33a1b
Validate the alt_aliases property of canonical alias events ( #6971 )
2020-03-03 07:12:45 -05:00
3e99528f2b
Store room version on invite ( #6983 )
...
When we get an invite over federation, store the room version in the rooms table.
The general idea here is that, when we pull the invite out again, we'll want to know what room_version it belongs to (so that we can later redact it if need be). So we need to store it somewhere...
2020-02-26 16:58:33 +00:00
1f773eec91
Port PresenceHandler to async/await ( #6991 )
2020-02-26 15:33:26 +00:00
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 ( #6957 )
...
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
b58d17e44f
Refactor the membership check methods in Auth
...
these were getting a bit unwieldy, so let's combine `check_joined_room` and
`check_user_was_in_room` into a single `check_user_in_room`.
2020-02-18 23:21:44 +00:00
fe3941f6e3
Stop sending events when creating or deleting aliases ( #6904 )
...
Stop sending events when creating or deleting associations (room aliases). Send an updated canonical alias event if one of the alt_aliases is deleted.
2020-02-18 07:29:44 -05:00
02e89021f5
Convert the directory handler tests to use HomeserverTestCase ( #6919 )
...
Convert directory handler tests to use HomeserverTestCase.
2020-02-14 09:05:43 -05:00
49f877d32e
Filter the results of user directory searching via the spam checker ( #6888 )
...
Add a method to the spam checker to filter the user directory results.
2020-02-14 07:17:54 -05:00
a92e703ab9
Reject device display names that are too long ( #6882 )
...
* Reject device display names that are too long.
Too long is currently defined as 100 characters in length.
* Add a regression test for rejecting a too long device display name.
2020-02-10 16:35:26 -05:00
b08b0a22d5
Add typing to synapse.federation.sender ( #6871 )
2020-02-07 13:56:38 +00:00
928edef979
Pass room_version into `event_from_pdu_json`
...
It's called from all over the shop, so this one's a bit messy.
2020-02-06 16:08:27 +00:00
a5afdd15e5
Merge pull request #6806 from matrix-org/rav/redact_changes/3
...
Pass room_version into add_hashes_and_signatures
2020-01-31 10:57:03 +00:00
d7bf793cc1
s/get_room_version/get_room_version_id/
...
... to make way for a forthcoming get_room_version which returns a RoomVersion
object.
2020-01-31 10:06:21 +00:00
ef6bdafb29
Store the room version in EventBuilder
2020-01-30 22:15:50 +00:00
b660327056
Resync remote device list when detected as stale. ( #6786 )
2020-01-30 17:06:38 +00:00
28c98e51ff
Add `local_current_membership` table ( #6655 )
...
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
2020-01-15 14:59:33 +00:00
326c893d24
Kill off RegistrationError ( #6691 )
...
This is pretty pointless. Let's just use SynapseError.
2020-01-13 12:48:22 +00:00
2284eb3a53
Add database config class ( #6513 )
...
This encapsulates config for a given database and is the way to get new
connections.
2019-12-18 10:45:12 +00:00
cb2db17994
look up cross-signing keys from the DB in bulk ( #6486 )
2019-12-12 12:03:28 -05:00
adfdd82b21
Back out perf regression from get_cross_signing_keys_from_cache. ( #6494 )
...
Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression.
2019-12-09 13:59:27 +00:00
a9b393340f
Merge pull request #6484 from matrix-org/erikj/port_sync_handler
...
Port SyncHandler to async/await
2019-12-09 11:32:44 +00:00
b3a4e35ca8
Fixup functions to consistently return deferreds
2019-12-06 10:40:05 +00:00
8437e2383e
Port SyncHandler to async/await
2019-12-05 17:58:25 +00:00
4a33a6dd19
Move background update handling out of store
2019-12-05 11:11:26 +00:00
756d4942f5
Move DB pool and helper functions into dedicated Database class
2019-12-05 10:46:37 +00:00
ee86abb2d6
Remove underscore from SQLBaseStore functions
2019-12-04 16:23:43 +00:00
0f87b912ab
Implementation of MSC2314 ( #6176 )
2019-11-28 08:54:07 +11:00
0d27aba900
add etag and count to key backup endpoints ( #5858 )
2019-11-27 16:14:44 -05:00
807ec3bd99
Fix bug which caused rejected events to be stored with the wrong room state ( #6320 )
...
Fixes a bug where rejected events were persisted with the wrong state group.
Also fixes an occasional internal-server-error when receiving events over
federation which are rejected and (possibly because they are
backwards-extremities) have no prev_group.
Fixes #6289 .
2019-11-06 10:01:39 +00:00
bb6cec27a5
rename get_devices_by_remote to get_device_updates_by_remote
2019-10-30 14:57:34 -04:00
3f33879be4
Port federation_server to async/await
2019-10-29 14:13:08 +00:00
7e7a1461f6
Fix tests
2019-10-25 10:57:37 +01:00
c17efdc01c
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_stores
2019-10-22 18:44:29 +01:00
bb6264be0b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_stores
2019-10-22 10:41:18 +01:00
c66a06ac6b
Move storage classes into a main "data store".
...
This is in preparation for having multiple data stores that offer
different functionality, e.g. splitting out state or event storage.
2019-10-21 16:05:06 +01:00
36adfaedab
Merge branch 'develop' into cross-signing_sig_upload
2019-10-18 18:34:42 +01:00
cfc28325a6
Merge branch 'develop' into uhoreg/e2e_cross-signing_merged
2019-10-18 11:39:40 +01:00
5859a5c569
Fix presence timeouts when synchrotron restarts. ( #6212 )
...
* Fix presence timeouts when synchrotron restarts.
Handling timeouts would fail if there was an external process that had
timed out, e.g. a synchrotron restarting. This was due to a couple of
variable name typoes.
Fixes #3715 .
2019-10-18 06:42:26 +01:00
691dd67fcd
Merge pull request #6189 from matrix-org/uhoreg/e2e_backup_optional_version
...
make version optional in body of e2e backup version update
2019-10-11 10:11:59 -04:00
5373de6cce
change test name to be unique
2019-10-10 08:54:07 -04:00
a139420a3c
Fix races in room stats (and other) updates. ( #6187 )
...
Hopefully this will fix the occasional failures we were seeing in the room directory.
The problem was that events are not necessarily persisted (and `current_state_delta_stream` updated) in the same order as their stream_id. So for instance current_state_delta 9 might be persisted *before* current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it had done everything up to 9, and never came back to do stream_id 8.
We can solve this easily by only processing up to the stream_id where we know all events have been persisted.
2019-10-10 11:29:01 +01:00
4535a07f4a
make version optional in body of e2e backup version update
...
to agree with latest version of the MSC
2019-10-09 17:54:03 -04:00
ae0b78cb1f
Merge pull request #6147 from matrix-org/babolivier/3pid-invite-revoked
...
Don't 500 when trying to exchange a revoked 3PID invite
2019-10-04 12:09:05 +01:00
8a5e8e829b
Lint (again)
2019-10-03 11:30:43 +01:00
ebcb6a30d7
Lint
2019-10-03 11:29:07 +01:00
6527fa18c1
Add test case
2019-10-03 11:24:36 +01:00
a5166e4d5f
Land improved room list based on room stats ( #6019 )
...
Use room_stats and room_state for room directory search
2019-10-02 14:08:35 +01:00
1e19ce00bf
Add 'failure_ts' column to 'destinations' table ( #6016 )
...
Track the time that a server started failing at, for general analysis purposes.
2019-09-17 11:41:54 +01:00
be618e0551
Only count real users when checking for auto-creation of auto-join room
...
Previously if the first registered user was a "support" or "bot" user,
when the first real user registers, the auto-join rooms were not
created.
Fix to exclude non-real (ie users with a special user type) users
when counting how many users there are to determine whether we should
auto-create a room.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-09-09 14:48:08 +03:00
ab729e31cf
use something that's the right type for user_id
2019-09-06 17:52:37 -04:00
e47af0f086
fix test
2019-09-05 17:03:14 -04:00
5914fd09c7
add test
2019-09-04 22:05:02 -04:00
9061b4198a
make isort happy
2019-09-04 22:04:41 -04:00
7d6c70fc7a
make black happy
2019-09-04 22:04:12 -04:00
ac4746ac4b
allow uploading signatures of master key signed by devices
2019-09-04 22:03:41 -04:00
faf72a4c40
Merge branch 'develop' into cross-signing_keys
2019-09-04 19:12:29 -04:00
6e834e94fc
Fix and refactor room and user stats ( #5971 )
...
Previously the stats were not being correctly populated.
2019-09-04 13:04:27 +01:00
4548d1f87e
Remove unnecessary parentheses around return statements ( #5931 )
...
Python will return a tuple whether there are parentheses around the returned values or not.
I'm just sick of my editor complaining about this all over the place :)
2019-08-30 16:28:26 +01:00
8c9adcc95d
fix formatting
2019-08-01 22:09:05 -04:00
336c546d6a
Merge branch 'cross-signing_hidden' into cross-signing_keys
2019-08-01 16:31:40 -04:00
8c97f6414c
Remove non-functional 'expire_access_token' setting ( #5782 )
...
The `expire_access_token` didn't do what it sounded like it should do. What it
actually did was make Synapse enforce the 'time' caveat on macaroons used as
access tokens, but since our access token macaroons never contained such a
caveat, it was always a no-op.
(The code to add 'time' caveats was removed back in v0.18.5, in #1656 )
2019-07-30 08:25:02 +01:00
c659b9f94f
allow uploading keys for cross-signing
2019-07-25 11:08:24 -04:00
4806651744
Replace returnValue with return ( #5736 )
2019-07-23 23:00:55 +10:00
d86321300a
Merge pull request #5589 from matrix-org/erikj/admin_exfiltrate_data
...
Add basic function to get all data for a user out of synapse
2019-07-15 10:04:02 +01:00
5f158ec039
Implement access token expiry ( #5660 )
...
Record how long an access token is valid for, and raise a soft-logout once it
expires.
2019-07-12 17:26:02 +01:00
953dbb7980
Remove access-token support from RegistrationStore.register ( #5642 )
...
The 'token' param is no longer used anywhere except the tests, so let's kill
that off too.
2019-07-10 16:26:49 +01:00
824707383b
Remove access-token support from RegistrationHandler.register ( #5641 )
...
Nothing uses this now, so we can remove the dead code, and clean up the
API.
Since we're changing the shape of the return value anyway, we take the
opportunity to give the method a better name.
2019-07-08 19:01:08 +01:00
1af2fcd492
Move get_or_create_user to test code ( #5628 )
...
This is only used in tests, so...
2019-07-08 23:52:26 +10:00
c061d4f237
Fixup from review comments.
2019-07-04 11:41:06 +01:00
8ee69f299c
Add basic function to get all data for a user out of synapse
2019-07-02 12:09:04 +01:00
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
75538813fc
Fix background updates to handle redactions/rejections ( #5352 )
...
* Fix background updates to handle redactions/rejections
In background updates based on current state delta stream we need to
handle that we may not have all the events (or at least that
`get_events` may raise an exception).
2019-06-06 00:45:46 +10:00
4a30e4acb4
Room Statistics ( #4338 )
2019-05-21 11:36:50 -05:00
3787133c9e
Limit UserIds to a length that fits in a state key ( #5198 )
2019-05-20 11:20:08 +01:00
df2ebd75d3
Migrate all tests to use the dict-based config format instead of hanging items off HomeserverConfig ( #5171 )
2019-05-13 15:01:14 -05:00
b36c82576e
Run Black on the tests again ( #5170 )
2019-05-10 00:12:11 -05:00
12f9d51e82
Add admin api for sending server_notices ( #5121 )
2019-05-02 11:59:16 +01:00
40e576e29c
Move admin api impl to its own package
...
It doesn't really belong under rest/client/v1 any more.
2019-05-01 15:44:30 +01:00
4a4d5c4fd6
Fix grammar and document get_current_users_in_room ( #4998 )
2019-04-03 14:32:20 +01:00
40e56997bc
Review comments
2019-03-28 13:48:41 +00:00
b7fa834c40
Add unit tests
2019-03-27 13:41:36 +00:00
4a125be138
Make federation endpoints more tolerant of trailing slashes v2 ( #4935 )
...
Redo of https://github.com/matrix-org/synapse/pull/4840
2019-03-26 11:35:29 +00:00
a68e00fca8
Some more porting to HomeserverTestCase and remove old RESTHelper ( #4913 )
2019-03-22 02:10:21 +11:00
7bef97dfb7
Remove trailing slashes from outbound federation requests and retry on 400 ( #4840 )
...
As per #3622 , we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
2019-03-21 15:07:28 +00:00
09f991a63d
Merge pull request #4896 from matrix-org/erikj/disable_room_directory
...
Add option to disable search room lists
2019-03-21 10:16:54 +00:00
3660d24ebe
Add test
2019-03-20 15:16:36 +00:00
cc09685830
Add test
2019-03-20 14:53:44 +00:00
b5d48560c7
Fix RegistrationTestCase
...
turns out this relies on there being a `user_consent_version` set.
2019-03-19 12:05:05 +00:00
88f0675967
fix test_auto_create_auto_join_where_no_consent ( #4886 )
2019-03-19 11:38:59 +00:00
282c97327f
Migrate the user directory initial population to a background task ( #4864 )
2019-03-19 04:50:24 +11:00
271cb1998b
Revert "Make federation endpoints more tolerant of trailing slashes for some endpoints ( #4793 )"
...
This reverts commit 290552fd83
2019-03-14 14:30:54 +00:00
9073cfc8bd
Merge pull request #4846 from matrix-org/hawkowl/userdir-search
...
Improve searching in the userdir
2019-03-14 13:54:15 +00:00
09626bfd39
Switch to wrapper function around _send_request
2019-03-13 18:26:06 +00:00
97653ef1f4
Correct argument name
2019-03-12 14:30:26 +00:00
797b6a63fc
fixup
2019-03-13 01:17:51 +11:00
0ea8582f8b
Cleaner way of implementing trailing slashes
2019-03-12 14:11:11 +00:00
10480c4348
fixup
2019-03-12 21:47:14 +11:00
4868b12029
and again
2019-03-11 18:22:26 +00:00
802cb5dcf0
Fix syntax error
2019-03-11 18:08:28 +00:00
66f205e93d
We're calling different functions now
2019-03-11 18:01:58 +00:00
290552fd83
Make federation endpoints more tolerant of trailing slashes for some endpoints ( #4793 )
...
Server side of a solution towards #3622 .
2019-03-11 17:44:03 +00:00
Andrew Morgan