016af01598
Rename VerifyKeyRequest.deferred field ( #5343 )
...
it's a bit confusing
2019-06-05 10:35:13 +01:00
aa530e6800
Call RetryLimiter correctly ( #5340 )
...
Fixes a regression introduced in #5335 .
2019-06-04 22:02:53 +01:00
dae224a73f
Fix failure to fetch batches of PDUs ( #5342 )
...
FederationClient.get_pdu is called in a loop to fetch a batch of PDUs. A
failure to fetch one should not result in a failure of the whole batch. Add the
missing `continue`.
2019-06-04 18:05:06 +01:00
b4189b112f
Rename get_events->get_events_from_store_or_dest ( #5344 )
...
We have too many things called get_event, and it's hard to figure out what we
mean. Also remove some unused params from the signature, and add some logging.
2019-06-04 18:01:09 +01:00
f6dd12d1e2
Merge pull request #5341 from matrix-org/babolivier/email_config
...
Make account validity renewal emails work when email notifs are disabled
2019-06-04 14:49:06 +01:00
2f62e1f6ff
Only parse from email if provided
2019-06-04 14:24:36 +01:00
d1d38081a7
Merge pull request #5324 from matrix-org/erikj/ignore_null
...
Ignore room state with null bytes in for room stats
2019-06-04 14:20:08 +01:00
1cc5fc1f6c
Lint
2019-06-04 13:51:23 +01:00
ac3cc32367
Make account validity renewal emails work when email notifs are disabled
2019-06-04 13:47:44 +01:00
df9c100542
Avoid rapidly backing-off a server if we ignore the retry interval ( #5335 )
2019-06-04 11:53:29 +01:00
4d08b8f30c
Don't do long retries when calling the key notary server. ( #5334 )
...
It takes at least 20 minutes to work through the long_retries schedule (11
attempts, each with a 60 second timeout, and 60 seconds between each request),
so if the notary server isn't returning within the timeout, we'll just end up
blocking whatever request is happening for 20 minutes.
Ain't nobody got time for that.
2019-06-04 11:53:07 +01:00
cb683d3e3c
Merge pull request #5333 from matrix-org/rav/server_keys/09_improve_notary_server
...
Fixes for the key-notary server
2019-06-04 11:48:18 +01:00
5bdb189f86
Improve docstrings on MatrixFederationClient. ( #5332 )
2019-06-04 11:14:16 +01:00
b2b90b7d34
Hawkowl/fix missing auth ( #5328 )
2019-06-04 15:54:27 +10:00
c5d60eadd5
Notary server: make requests to origins in parallel
...
... else we're guaranteed to time out.
2019-06-04 00:16:56 +01:00
def5ea4062
Don't bomb out on direct key fetches as soon as one fails
2019-06-04 00:16:56 +01:00
dce6e9e0c1
Avoid rapidly backing-off a server if we ignore the retry interval
2019-06-03 23:58:42 +01:00
06a1f3e207
Reduce timeout for outbound /key/v2/server requests.
2019-06-03 23:17:38 +01:00
fec2dcb1a5
Enforce validity period on server_keys for fed requests. ( #5321 )
...
When handling incoming federation requests, make sure that we have an
up-to-date copy of the signing key.
We do not yet enforce the validity period for event signatures.
2019-06-03 22:59:51 +01:00
0a56966f7d
Fix
2019-06-03 17:42:52 +01:00
fa4b54aca5
Ignore room state with null bytes in for room stats
2019-06-03 17:06:54 +01:00
6f9f08005c
Merge remote-tracking branch 'origin/develop' into hawkowl/full-schema-v1
2019-06-04 01:16:01 +10:00
2198b7ce2a
add stuff in bg updates
2019-06-04 01:06:00 +10:00
4e75c5e02a
WHY IS THIS CALLED A SLIGHTLY DIFFERENT THING
2019-06-03 22:42:12 +10:00
be452fc9ac
more fix
2019-06-03 22:24:23 +10:00
7f81b967ca
fix schemas
2019-06-03 22:23:40 +10:00
862b2f9ad5
Merge pull request #5307 from matrix-org/rav/server_keys/07-fix-notary-cache-poison
...
Stop overwriting server keys with other keys
2019-06-03 13:19:20 +01:00
dc72b90cd6
full schema
2019-06-03 22:03:28 +10:00
37057d5d60
prepare
2019-06-03 22:02:47 +10:00
2889b05554
Unify v1 and v2 REST client APIs ( #5226 )
2019-06-03 21:28:59 +10:00
220a733d73
Fix handling of failures when calling /event_auth.
...
When processing an incoming event over federation, we may try and
resolve any unexpected differences in auth events. This is a
non-essential process and so should not stop the processing of the event
if it fails (e.g. due to the remote disappearing or not implementing the
necessary endpoints).
Fixes #3330
2019-06-03 09:56:45 +01:00
d828d1dc57
Merge pull request #5309 from matrix-org/rav/limit_displayname_length
...
Limit displaynames and avatar URLs
2019-06-01 11:34:50 +01:00
d16c6375fe
Limit displaynames and avatar URLs
...
These end up in join events everywhere, so let's limit them.
Fixes #5079
2019-06-01 10:44:36 +01:00
37b165620d
Merge pull request #5299 from matrix-org/rav/server_keys/05-rewrite-gsvk-again
...
Rewrite get_server_verify_keys, again.
2019-05-31 17:07:31 +01:00
3600f5568b
Stop overwriting server keys with other keys
...
Fix a bug where we would discard a key result which the origin server is no
longer returning. Fixes #5305 .
2019-05-31 15:58:35 +01:00
58cce39f3a
Merge pull request #5276 from matrix-org/babolivier/account_validity_job_delta
...
Allow configuring a range for the account validity startup job
2019-05-31 12:11:56 +01:00
c605da97bf
Merge remote-tracking branch 'origin/develop' into rav/server_keys/05-rewrite-gsvk-again
2019-05-31 11:38:13 +01:00
fe79b5e521
Merge pull request #5300 from matrix-org/rav/server_keys/06-fix-serverkeys-handling
...
Remove some pointless exception handling
2019-05-31 11:35:29 +01:00
2ae3cc287e
Merge pull request #5296 from matrix-org/rav/server_keys/04-use-attrs-for_verify-request
...
use attr.s for VerifyKeyRequest
2019-05-31 11:34:09 +01:00
4d794dae21
Move delta from +10% to -10%
2019-05-31 11:09:39 +01:00
e9981d58ca
Merge pull request #5293 from Kagamihime/messages-federation-format
...
Fix ignored filter field in `/messages` endpoint
2019-05-31 10:52:59 +01:00
31d44ec4bd
Merge pull request #5294 from matrix-org/erikj/speed_up_room_stats
...
Speed up room stats background update
2019-05-31 10:48:51 +01:00
5037326d66
Add indices. Remove room_ids accidentally added
...
We have to do this by re-inserting a background update and recreating
tables, as the tables only get created during a background update and
will later be deleted.
We also make sure that we remove any entries that should have been
removed but weren't due to a race that has been fixed in a previous
commit.
2019-05-31 10:26:56 +01:00
0c2362861e
Gah python
2019-05-31 09:56:52 +01:00
847b9dcd1c
Make max_delta equal to period * 10%
2019-05-31 09:54:46 +01:00
8ea2f756a9
Remove some pointless exception handling
...
The verify_request deferred already returns a suitable SynapseError, so I don't
really know what we expect to achieve by doing more wrapping, other than log
spam.
Fixes #4278 .
2019-05-30 18:29:56 +01:00
a82c96b87f
Rewrite get_server_verify_keys, again.
...
Attempt to simplify the logic in get_server_verify_keys by splitting it into
two methods.
2019-05-30 18:20:40 +01:00
099829d5a9
use attr.s for VerifyKeyRequest
...
because namedtuple is awful
2019-05-30 17:39:28 +01:00
99113e40ba
Merge branch 'master' into develop
2019-05-30 16:39:49 +01:00
c831748f4d
0.99.5.2
2019-05-30 16:29:47 +01:00
e2c3660a0f
Add index to temp table
2019-05-30 16:18:40 +01:00
06eb408da5
Update synapse/storage/events_bg_updates.py
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-05-30 16:15:37 +01:00
7386c35f58
Rename constant
2019-05-30 16:15:37 +01:00
98f438b52a
Move event background updates to a separate file
2019-05-30 16:15:37 +01:00
9b8cd66524
Fixup comments and logging
2019-05-30 16:15:37 +01:00
1d818fde14
Log actual number of entries deleted
2019-05-30 16:15:37 +01:00
6ebc08c09d
Add DB bg update to cleanup extremities.
...
Due to #5269 we may have extremities in our DB that we shouldn't have,
so lets add a cleanup task such to remove those.
2019-05-30 16:15:37 +01:00
df9d900544
Correctly filter out extremities with soft failed prevs ( #5274 )
...
When we receive a soft failed event we, correctly, *do not* update the
forward extremity table with the event. However, if we later receive an
event that references the soft failed event we then need to remove the
soft failed events prev events from the forward extremities table,
otherwise we just build up forward extremities.
Fixes #5269
2019-05-30 16:12:50 +01:00
8824325b82
Fix ignored filter field in `/messages` endpoint
...
This fixes a bug which were causing the "event_format" field to be
ignored in the filter of requests to the `/messages` endpoint of the
CS API.
Signed-off-by: Eisha Chen-yen-su <chenyensu0@gmail.com>
2019-05-30 16:58:53 +02:00
5ac75fc9a2
Join against events to use its room_id index
2019-05-30 15:26:55 +01:00
e2c46ed851
Move deletion from table inside txn
2019-05-30 15:26:38 +01:00
04710cc2d7
Fetch membership counts all at once
2019-05-30 15:25:41 +01:00
54d50fbfdf
Get events all at once
2019-05-30 15:15:13 +01:00
6cdfb0207e
Add index to temp table
2019-05-30 15:02:31 +01:00
e9e5d3392d
Merge pull request #5278 from matrix-org/erikj/cleanup_bad_extremities
...
Add DB bg update to cleanup extremities.
2019-05-30 14:28:26 +01:00
cb967e2346
Update synapse/storage/events_bg_updates.py
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-05-30 14:06:42 +01:00
45f5d8f3fd
Merge pull request #5256 from aaronraimist/logout-correct-error
...
Show correct error when logging out and access token is missing
2019-05-30 13:33:44 +01:00
468bd090ff
Rename constant
2019-05-30 11:24:42 +01:00
5c1ece0ffc
Move event background updates to a separate file
2019-05-30 11:22:59 +01:00
640fcbb07f
Fixup comments and logging
2019-05-30 10:55:55 +01:00
123918b739
Lint
...
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-05-29 14:44:28 -05:00
8d92329214
Remove spurious debug from MatrixFederationHttpClient.get_json ( #5287 )
...
This is just unhelpful spam
2019-05-29 19:31:52 +01:00
3dcf2feba8
Improve logging for logcontext leaks. ( #5288 )
2019-05-29 19:27:50 +01:00
46c8f7a517
Implement the SHHS complexity API ( #5216 )
2019-05-30 01:47:16 +10:00
532b825ed9
Serve CAS login over r0 ( #5286 )
2019-05-30 00:55:18 +10:00
7e8e683754
Log actual number of entries deleted
2019-05-29 15:11:28 +01:00
d79c9994f4
Add DB bg update to cleanup extremities.
...
Due to #5269 we may have extremities in our DB that we shouldn't have,
so lets add a cleanup task such to remove those.
2019-05-29 15:11:26 +01:00
30858ff461
Fix error when downloading thumbnail with width/height param missing ( #5258 )
...
Fix error when downloading thumbnail with width/height param missing
Fixes #2748
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-05-29 14:27:41 +01:00
58c8ed5b0d
Correctly filter out extremities with soft failed prevs ( #5274 )
...
When we receive a soft failed event we, correctly, *do not* update the
forward extremity table with the event. However, if we later receive an
event that references the soft failed event we then need to remove the
soft failed events prev events from the forward extremities table,
otherwise we just build up forward extremities.
Fixes #5269
2019-05-29 11:56:24 +01:00
f76d407ef3
Fix dropped logcontexts during high outbound traffic. ( #5277 )
...
Fixes #5271 .
2019-05-29 09:17:33 +01:00
9b6f72663e
Fix docs on resetting the user directory ( #5036 )
...
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-05-28 20:54:01 -05:00
540f40f0cd
Merge pull request #5251 from matrix-org/rav/server_keys/01-check_sig
...
Ensure that server_keys fetched via a notary server are correctly signed.
2019-05-28 21:32:17 +01:00
5726378ece
Fix "db txn 'update_presence' from sentinel context" log messages ( #5275 )
...
Fixes #4414 .
2019-05-28 21:20:11 +01:00
52839886d6
Allow configuring a range for the account validity startup job
...
When enabling the account validity feature, Synapse will look at startup for registered account without an expiration date, and will set one equals to 'now + validity_period' for them. On large servers, it can mean that a large number of users will have the same expiration date, which means that they will all be sent a renewal email at the same time, which isn't ideal.
In order to mitigate this, this PR allows server admins to define a 'max_delta' so that the expiration date is a random value in the [now + validity_period ; now + validity_period + max_delta] range. This allows renewal emails to be progressively sent over a configured period instead of being sent all in one big batch.
2019-05-28 16:52:45 +01:00
a97d4e218a
Merge pull request #5268 from matrix-org/babolivier/account_validity_fix_schema
...
Fix schema update for account validity
2019-05-28 10:30:07 +01:00
ba17de7fbc
Fix schema update for account validity
2019-05-28 10:11:38 +01:00
119c9c10b0
Get rid of try except
...
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-05-27 00:13:56 -05:00
d0bba35197
Merge pull request #5260 from matrix-org/travis/fix-room-bg-task
...
Fix logging for room stats background update
2019-05-25 19:59:42 +01:00
bc4b2ecf70
Fix logging for room stats background update
2019-05-25 12:02:48 -06:00
338dca58c0
Merge pull request #5257 from aaronraimist/fix-error-code-publicrooms
...
Fix error code for invalid parameter
2019-05-25 14:09:27 +01:00
2d4853039f
Fix error code for invalid parameter
...
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-05-24 17:13:16 -05:00
56f07d980a
Show correct error when logging out and access token is missing
...
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-05-24 16:32:27 -05:00
fa1b293da2
Simplification to Keyring.wait_for_previous_lookups. ( #5250 )
...
The list of server names was redundant, since it was equivalent to the keys on
the server_to_deferred map. This reduces the number of large lists being passed
around, and has the benefit of deduplicating the entries in `wait_on`.
2019-05-24 22:17:18 +01:00
b825d1c800
Improve error handling/logging for perspectives-key fetching.
...
In particular, don't give up on the first failure.
2019-05-24 15:46:25 +01:00
dd64b9dbdd
Fix appservice timestamp massaging ( #5233 )
...
Signed-off-by: Tulir Asokan <tulir@maunium.net>
2019-05-24 14:44:04 +01:00
dba9152d15
Add missing blank line in config ( #5249 )
2019-05-24 14:12:38 +01:00
d16f5574b6
Merge pull request #5220 from matrix-org/erikj/dont_bundle_live_events
...
Don't bundle aggregations with events in /sync or /events or state queries
2019-05-24 10:36:31 +01:00
4cb577c23f
Don't bundle aggs for /state and /members etc APIs
2019-05-24 09:52:33 +01:00
8c41c04ee4
Merge pull request #5244 from matrix-org/rav/server_keys/00-factor-out-fetchers
...
Factor out KeyFetchers from KeyRing
2019-05-23 16:28:51 +01:00
753b1270da
Require sig from origin server on perspectives responses
2019-05-23 15:01:09 +01:00
Richard van der Hoff