1af2fcd492
Move get_or_create_user to test code ( #5628 )
...
This is only used in tests, so...
2019-07-08 23:52:26 +10:00
f05c7d62bc
Lint
2019-07-08 14:29:27 +01:00
1a807dfe68
Use application/json when querying the IS's /store-invite endpoint
2019-07-08 14:19:39 +01:00
9b1b79f3f5
Add default push rule to ignore reactions ( #5623 )
...
This adds a default push rule following the proposal in
[MSC2153](https://github.com/matrix-org/matrix-doc/pull/2153 ).
See also https://github.com/vector-im/riot-web/issues/10208
See also https://github.com/matrix-org/matrix-js-sdk/pull/976
2019-07-05 17:37:52 +01:00
ad8b909ce9
Add origin_server_ts and sender fields to m.replace ( #5613 )
...
Riot team would like some extra fields as part of m.replace, so here you go.
Fixes : #5598
2019-07-05 17:20:02 +01:00
80cc82a445
Remove support for invite_3pid_guest. ( #5625 )
...
This has never been documented, and I'm not sure it's ever been used outside
sytest.
It's quite a lot of poorly-maintained code, so I'd like to get rid of it.
For now I haven't removed the database table; I suggest we leave that for a
future clearout.
2019-07-05 16:47:58 +01:00
b4f5416dd9
pep8
2019-07-05 14:41:29 +01:00
eadb13d2e9
Remove FileExfiltrationWriter
2019-07-05 14:15:00 +01:00
7f0d8e4288
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_exfiltrate_data
2019-07-05 14:08:21 +01:00
9ccea16d45
Assume key existence. Update docstrings
2019-07-05 14:07:56 +01:00
a6a776f3d8
remove dead transaction persist code ( #5622 )
...
this hasn't done anything for years
2019-07-05 12:59:42 +01:00
9481707a52
Fixes to the federation rate limiter ( #5621 )
...
- Put the default window_size back to 1000ms (broken by #5181 )
- Make the `rc_federation` config actually do something
- fix an off-by-one error in the 'concurrent' limit
- Avoid creating an unused `_PerHostRatelimiter` object for every single
incoming request
2019-07-05 11:10:19 +01:00
0e5434264f
Make errors about email password resets much clearer ( #5616 )
...
The runtime errors that dealt with local email password resets talked about config options that users may not even have in their config file yet (if upgrading). Instead, the cryptic errors are now replaced with hopefully much more helpful ones.
2019-07-05 10:44:12 +01:00
1ee268d33d
Improve the backwards compatibility re-exports of synapse.logging.context ( #5617 )
...
* Improve the backwards compatibility re-exports of synapse.logging.context.
* reexport logformatter too
2019-07-05 02:32:02 +10:00
822a0f0435
Merge branch 'master' of github.com:matrix-org/synapse into develop
2019-07-04 14:00:27 +01:00
20332b278d
1.1.0
2019-07-04 11:44:09 +01:00
c061d4f237
Fixup from review comments.
2019-07-04 11:41:06 +01:00
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ ( #5606 )
2019-07-04 00:07:04 +10:00
d0b849c86d
Apply comment fixups from code review
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-07-03 15:03:38 +01:00
cb8d568cf9
Fix 'utime went backwards' errors on daemonization. ( #5609 )
...
* Fix 'utime went backwards' errors on daemonization.
Fixes #5608
* remove spurious debug
2019-07-03 22:40:45 +10:00
463d5a8fde
1.1.0rc2
2019-07-03 10:51:47 +01:00
91753cae59
Fix a number of "Starting txn from sentinel context" warnings ( #5605 )
...
Fixes #5602 , #5603
2019-07-03 09:31:27 +01:00
0ee9076ffe
Fix media repo breaking ( #5593 )
2019-07-02 19:01:28 +01:00
9f3c0a8556
Add basic admin cmd app
2019-07-02 17:12:48 +01:00
8ee69f299c
Add basic function to get all data for a user out of synapse
2019-07-02 12:09:04 +01:00
a0fa4641c4
prepare v1.1.0rc1
2019-07-02 11:51:55 +01:00
6eecb6e500
Complete the SAML2 implementation ( #5422 )
...
* SAML2 Improvements and redirect stuff
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
* Code cleanups and simplifications.
Also: share the saml client between redirect and response handlers.
* changelog
* Revert redundant changes to static js
* Move all the saml stuff out to a centralised handler
* Add support for tracking SAML2 sessions.
This allows us to correctly handle `allow_unsolicited: False`.
* update sample config
* cleanups
* update sample config
* rename BaseSSORedirectServlet for consistency
* Address review comments
2019-07-02 11:18:11 +01:00
948488e115
Fix JWT login with new users ( #5586 )
...
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
2019-07-02 10:25:37 +01:00
b4914681a5
fix async/await consentresource ( #5585 )
...
Fixes #5582
2019-07-01 23:33:52 +10:00
b4fd86a9b4
Merge branch 'develop' into rav/saml2_client
2019-07-01 14:21:03 +01:00
3bcb13edd0
Address review comments
2019-07-01 12:13:22 +01:00
04196a4dae
Merge pull request #5507 from matrix-org/erikj/presence_sync_tighloop
...
Fix sync tightloop bug.
2019-07-01 11:43:10 +01:00
915280f1ed
Fixup comment
2019-07-01 10:22:42 +01:00
f40a7dc41f
Make the http server handle coroutine-making REST servlets ( #5475 )
2019-06-29 17:06:55 +10:00
c7ff297dde
Merge pull request #5576 from matrix-org/babolivier/3pid-invite-ratelimit
...
Don't update the ratelimiter before sending a 3PID invite
2019-06-28 17:43:48 +01:00
15d9fc31bd
Only ratelimit when sending the email
...
If we do the opposite, an event can arrive after or while sending the email and the 3PID invite event will get ratelimited.
2019-06-28 16:04:05 +01:00
01d0f8e701
Don't update the ratelimiter before sending a 3PID invite
...
This would cause emails being sent, but Synapse responding with a 429 when creating the event. The client would then retry, and with bad timing the same scenario would happen again. Some testing I did ended up sending me 10 emails for one single invite because of this.
2019-06-28 15:22:16 +01:00
071150ce19
Don't log GC 0s at INFO ( #5557 )
2019-06-28 21:45:33 +10:00
be3b901ccd
Update the TLS cipher string and provide configurability for TLS on outgoing federation ( #5550 )
2019-06-28 18:19:09 +10:00
9646a593ac
Added possibilty to disable local password authentication ( #5092 )
...
Signed-off-by: Daniel Hoffend <dh@dotlan.net>
2019-06-27 18:37:29 +01:00
c548dbc4b1
Make it clearer that the template dir is relative to synapse's root dir ( #5543 )
...
Helps address #5444
2019-06-27 18:20:17 +01:00
e79ec03165
Merge pull request #5559 from matrix-org/erikj/refactor_changed_devices
...
Refactor devices changed query to pull less from DB
2019-06-27 16:53:15 +01:00
729f5a4fb6
Review comments
2019-06-27 16:06:23 +01:00
856ea04eb3
Fix JWT login ( #5555 )
...
* Fix JWT login with register
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
* Add pyjwt conditional dependency
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
* Added changelog file
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
* Improved changelog description
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
2019-06-27 12:02:41 +01:00
79b9d9076d
rename BaseSSORedirectServlet for consistency
2019-06-27 00:46:57 +01:00
28db0ae537
cleanups
2019-06-27 00:37:41 +01:00
36f4953dec
Add support for tracking SAML2 sessions.
...
This allows us to correctly handle `allow_unsolicited: False`.
2019-06-26 23:50:55 +01:00
3705322103
Move all the saml stuff out to a centralised handler
2019-06-26 22:52:02 +01:00
0ade403f55
Revert redundant changes to static js
2019-06-26 22:46:23 +01:00
a4daa899ec
Merge branch 'develop' into rav/saml2_client
2019-06-26 22:34:41 +01:00
8624db3194
Refactor and comment sync device list code
2019-06-26 19:39:49 +01:00
f335e77d53
Use batch_iter and correct docstring
2019-06-26 19:39:46 +01:00
806a06daf2
Rename get_users_whose_devices_changed
2019-06-26 19:39:19 +01:00
a2f6d31a63
Refactor get_user_ids_changed to pull less from DB
...
When a client asks for users whose devices have changed since a token we
used to pull *all* users from the database since the token, which could
easily be thousands of rows for old tokens.
This PR changes this to only check for changes for users the client is
actually interested in.
Fixes #5553
2019-06-26 12:03:44 +01:00
ef8c62758c
Prevent multiple upgrades on the same room at once ( #5051 )
...
Closes #4583
Does slightly less than #5045 , which prevented a room from being upgraded multiple times, one after another. This PR still allows that, but just prevents two from happening at the same time.
Mostly just to mitigate the fact that servers are slow and it can take a moment for the room upgrade to actually complete. We don't want people sending another request to upgrade the room when really they just thought the first didn't go through.
2019-06-25 14:19:21 +01:00
4ac7ef4b67
Merge pull request #5524 from matrix-org/rav/new_cmdline_options
...
Add --data-dir and --open-private-ports options.
2019-06-24 17:25:57 +01:00
af8a962905
Merge pull request #5523 from matrix-org/rav/arg_defaults
...
Stop conflating generated config and default config
2019-06-24 17:24:35 +01:00
e59a8cd2e5
Merge pull request #5499 from matrix-org/rav/cleanup_metrics
...
Cleanups and sanity-checking in cpu and db metrics
2019-06-24 17:12:54 +01:00
deb4fe6ef3
Merge pull request #5534 from matrix-org/babolivier/federation-publicrooms
...
Split public rooms directory auth config in two
2019-06-24 16:08:02 +01:00
bfe84e051e
Split public rooms directory auth config in two
2019-06-24 15:42:31 +01:00
25433f212d
Merge pull request #5531 from matrix-org/erikj/workers_pagination_token
...
Fix /messages on workers when no from param specified.
2019-06-24 15:30:10 +01:00
3f8a252dd8
Add "--open-private-ports" cmdline option
...
This is helpful when generating a config file for running synapse under docker.
2019-06-24 14:15:34 +01:00
6a92b06cbb
Add --data-directory commandline argument
...
We don't necessarily want to put the data in the cwd.
2019-06-24 14:15:34 +01:00
16b52642e2
Don't load the generated config as the default.
...
It's too confusing.
2019-06-24 14:14:52 +01:00
7c2f8881a9
Ensure that all config options have sensible defaults
...
This will enable us to skip the unintuitive behaviour where the generated
config and default config are the same thing.
2019-06-24 14:14:52 +01:00
367a8263b3
Remove unused Config.config_dir_path attribute
...
This is no longer used and only serves to confuse.
2019-06-24 13:51:22 +01:00
edea4bb5be
Allow configuration of the path used for ACME account keys.
...
Because sticking it in the same place as the config isn't necessarily the right
thing to do.
2019-06-24 13:51:22 +01:00
c3c6b00d95
Pass config_dir_path and data_dir_path into Config.read_config. ( #5522 )
...
* Pull config_dir_path and data_dir_path calculation out of read_config_files
* Pass config_dir_path and data_dir_path into read_config
2019-06-24 11:34:45 +01:00
21bf4318b5
Factor acme bits out to a separate file ( #5521 )
...
This makes some of the conditional-import hoop-jumping easier.
2019-06-24 11:33:56 +01:00
1793de6c6d
black
2019-06-24 11:16:13 +01:00
dddf20e8e1
Fix /messages on workers when no from param specified.
...
If no `from` param is specified we calculate and use the "current
token" that inlcuded typing, presence, etc. These are unused during
pagination and are not available on workers, so we simply don't
calculate them.
2019-06-24 10:06:51 +01:00
dc94773e60
Avoid raising exceptions in metrics
...
Sentry will catch the errors if they happen, so that should be good enough, and
woun't make things explode if we hit the error condition.
2019-06-24 10:01:16 +01:00
5097aee740
Merge branch 'develop' into rav/cleanup_metrics
2019-06-24 10:00:13 +01:00
c753c098dd
Merge pull request #5498 from matrix-org/rav/fix_clock_reversal
...
Use monotonic clock where possible for metrics
2019-06-24 09:55:12 +01:00
6cda36777b
Drop support for cpu_affinity ( #5525 )
...
This has no useful purpose on python3, and is generally a source of confusion.
2019-06-22 11:01:55 +10:00
e1a795758c
Improve help and cmdline option names for --generate-config options ( #5512 )
...
* group the arguments together into a group
* add new names "--generate-missing-config" and "--config-directory" for
existing cmdline options "--generate-keys" and "--keys-dir", which better
reflect their purposes.
2019-06-21 18:50:43 +01:00
03cea2b0fe
Refactor Config parser and add some comments. ( #5511 )
...
Add some comments, and simplify `read_config_files`.
2019-06-21 17:43:38 +01:00
37933a3bf8
Improve logging when generating config files ( #5510 )
...
Make it a bit clearer what's going on.
2019-06-21 17:14:56 +01:00
5d6644efe0
Only import jinja2 when needed ( #5514 )
...
Fixes https://github.com/matrix-org/synapse/issues/5431
`jinja2` was being imported even when it wasn't strictly necessary. This made it required to run Synapse, even if the functionality that required it wasn't enabled. This was causing new Synapse installations to crash on startup.
Email modules are now required.
2019-06-21 16:52:09 +01:00
0e8b35f7b0
Fix "Unexpected entry in 'full_schemas'" log warning ( #5509 )
...
There is a README.txt which always sets off this warning, which is a bit
alarming when you first start synapse. I don't think we need to warn about
this.
2019-06-21 15:11:57 +01:00
2f8491daef
Fix logging error when a tampered event is detected. ( #5500 )
2019-06-21 15:11:42 +01:00
7eadb74056
Fix /messages on worker when no token supplied
2019-06-21 14:15:59 +01:00
8181e290a9
Fix sync tightloop bug.
...
If, for some reason, presence updates take a while to persist then it
can trigger clients to tightloop calling `/sync` due to the presence
handler returning updates but not advancing the stream token.
Fixes #5503 .
2019-06-21 11:10:27 +01:00
f3ab533374
Support pagination API in client_reader worker
2019-06-21 10:43:12 +01:00
7456698241
Merge pull request #5476 from matrix-org/erikj/histogram_extremities
...
Add metrics for length of new extremities persisted.
2019-06-21 09:58:54 +01:00
f47969f42a
Improve email notification logging ( #5502 )
2019-06-20 15:14:26 +01:00
f8bd30af2a
Black
2019-06-20 13:10:06 +01:00
45f28a9d2f
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/histogram_extremities
2019-06-20 11:59:14 +01:00
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
fe641df770
Sanity-checking for metrics updates
...
Check that our clocks go forward.
2019-06-19 21:18:38 +01:00
f682af052d
Simplify PerformanceCounters.update interface
...
we already have the duration for the update, so may as well use it rather than
passing extra params around and recalculating it.
2019-06-19 21:18:38 +01:00
68128d5626
Remove unused _get_event_counters
...
This has been redundant since cdb3757942
2019-06-19 21:14:25 +01:00
15bf32e062
Use monotonic clock where possible for metrics
...
Fixes intermittent errors observed on Apple hardware which were caused by
time.clock() appearing to go backwards when called from different threads.
Also fixes a bug where database activity times were logged as 1/1000 of their
correct ratio due to confusion between milliseconds and seconds.
2019-06-19 21:09:43 +01:00
7dcf984075
Merge pull request #5042 from matrix-org/erikj/fix_get_missing_events_error
...
Handle the case of `get_missing_events` failing
2019-06-19 13:20:09 +01:00
e0be8d7016
Merge pull request #5480 from matrix-org/erikj/extremities_dummy_events
...
Add experimental option to reduce extremities.
2019-06-19 13:19:18 +01:00
65787b0f7c
Add descriptions and remove redundant set(..)
2019-06-19 11:49:39 +01:00
554609288b
Run as background process and fix comments
2019-06-19 11:33:03 +01:00
092e36457a
Fix typo in deactivation background job
2019-06-19 11:09:41 +01:00
f2d2ae03da
Add some logging to 3pid invite sig verification ( #5015 )
...
I had to add quite a lot of logging to diagnose a problem with 3pid
invites - we only logged the one failure which isn't all that
informative.
NB. I'm not convinced the logic of this loop is right: I think it
should just accept a single valid signature from a trusted source
rather than fail if *any* signature is invalid. Also it should
probably not skip the rest of middle loop if a check fails? However,
I'm deliberately not changing the logic here.
2019-06-18 22:51:24 +01:00
2b20d0fb59
Fix logline
2019-06-18 16:12:53 +01:00
19b80fe68a
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fix_get_missing_events_error
2019-06-18 16:11:43 +01:00
b42f90470f
Add experimental option to reduce extremities.
...
Adds new config option `cleanup_extremities_with_dummy_events` which
periodically sends dummy events to rooms with more than 10 extremities.
THIS IS REALLY EXPERIMENTAL.
2019-06-18 15:02:18 +01:00
16a3124b76
Only count non-cache state resolution
2019-06-18 13:02:06 +01:00
c9385dd238
Use consistent buckets
2019-06-18 12:43:41 +01:00
d6328e03fd
Merge pull request #5477 from matrix-org/babolivier/third_party_rules_3pid
...
Add third party rules hook for 3PID invites
2019-06-17 18:08:31 +01:00
33ea87be39
Make check_threepid_can_be_invited async
2019-06-17 17:39:38 +01:00
112cf5a73a
Add third party rules hook for 3PID invites
2019-06-17 17:39:09 +01:00
8353ddd951
Merge pull request #5479 from matrix-org/erikj/add_create_room_hook_develop
...
Add third party rules hook into create room
2019-06-17 17:30:05 +01:00
ff88d36dcb
Add metric fo number of state groups in resolution
2019-06-17 17:02:53 +01:00
499d4a32cd
Add metrics for len of new extremities persisted.
...
Of new events being persisted add metrics for total size of forward
extremities and number of unchanged, "stale" extremities.
2019-06-17 17:02:48 +01:00
187d2837a9
Add third party rules hook into create room
2019-06-17 16:41:19 +01:00
eba7caf09f
Remove Postgres 9.4 support ( #5448 )
2019-06-18 00:59:00 +10:00
6840ebeef8
Merge pull request #5385 from matrix-org/erikj/reduce_http_exceptions
...
Handle HttpResponseException when using federation client.
2019-06-17 13:54:47 +01:00
dd927b29e1
Merge pull request #5388 from matrix-org/erikj/fix_email_push
...
Fix email notifications for unnamed rooms with multiple people
2019-06-17 13:54:35 +01:00
414d2ca3a6
Merge pull request #5389 from matrix-org/erikj/renew_attestations_on_master
...
Only start background group attestation renewals on master
2019-06-17 13:54:29 +01:00
a9dab970b8
Merge pull request #5464 from matrix-org/erikj/3pid_remote_invite_state
...
Fix 3PID invite room state over federation.
2019-06-17 10:18:28 +01:00
f12e1f029c
Merge pull request #5440 from matrix-org/babolivier/third_party_event_rules
...
Allow server admins to define implementations of extra rules for allowing or denying incoming events
2019-06-14 19:37:59 +01:00
9ca4ae7131
Merge pull request #5461 from matrix-org/erikj/histograms_are_cumalitive
...
Prometheus histograms are cumalative
2019-06-14 18:21:42 +01:00
f874b16b2e
Add plugin APIs for implementations of custom event rules.
2019-06-14 18:16:03 +01:00
14db086428
Merge pull request #5465 from matrix-org/babolivier/fix_deactivation_bg_job
...
Fix background job for deactivated flag
2019-06-14 18:12:56 +01:00
304a1376c2
Fix 3PID invite room state over federation.
...
Fixes that when a user exchanges a 3PID invite for a proper invite over
federation it does not include the `invite_room_state` key.
This was due to synapse incorrectly sending out two invite requests.
2019-06-14 16:19:11 +01:00
e0b77b004d
Fix background job for deactivated flag
2019-06-14 16:00:45 +01:00
9b14a810d2
Merge pull request #5462 from matrix-org/babolivier/account_validity_deactivated_accounts_2
...
Don't send renewal emails to deactivated users (second attempt)
2019-06-14 15:35:31 +01:00
6d56a694f4
Don't send renewal emails to deactivated users
2019-06-14 15:05:56 +01:00
e9344e0dee
Merge pull request #5390 from matrix-org/erikj/dont_log_on_fail_to_get_file
...
Don't log exception when failing to fetch remote content.
2019-06-14 14:25:14 +01:00
3ed595e327
Prometheus histograms are cumalative
2019-06-14 14:07:32 +01:00
d0530382ee
Track deactivated accounts in the database ( #5378 )
2019-06-14 13:18:24 +01:00
a10c8dae85
fix prometheus rendering error
2019-06-14 21:09:33 +10:00
5c15039e06
Clean up code for sending federation EDUs. ( #5381 )
...
This code confused the hell out of me today. Split _get_new_device_messages
into its two (unrelated) parts.
2019-06-13 13:52:08 +01:00
6312d6cc7c
Expose statistics on extrems to prometheus ( #5384 )
2019-06-13 22:40:52 +10:00
09e9a26b71
Remove Python 2.7 support. ( #5425 )
...
* remove 2.7 from CI and publishing
* fill out classifiers and also make it not be installed on 3.5
* some minor bumps so that the old deps work on python 3.5
2019-06-12 21:31:59 +10:00
7e68691ce9
Merge branch 'master' of github.com:matrix-org/synapse into develop
2019-06-11 17:25:16 +01:00
97174780ce
1.0.0
2019-06-11 17:10:01 +01:00
a766c41d25
Bump bleach version so that tests can run on old deps.
2019-06-11 12:34:18 +01:00
10383e6e6f
Change password reset links to /_matrix.
2019-06-11 11:34:33 +01:00
5bc9484537
Merge branch 'release-v1.0.0' of github.com:matrix-org/synapse into develop
2019-06-11 10:37:43 +01:00
b55e8840e4
Merge tag 'v1.0.0rc3' into rav/saml2_client
...
Synapse 1.0.0rc3 (2019-06-10)
=============================
Security: Fix authentication bug introduced in 1.0.0rc1. Please upgrade to rc3 immediately
2019-06-11 00:28:57 +01:00
2ddc13577c
Don't warn user about password reset disabling through config code ( #5387 )
...
Moves the warning about password resets being disabled to the point where a user actually tries to reset their password. Is this an appropriate place for it to happen?
Also removed the disabling of msisdn password resets when you don't have an email config, as that just doesn't make sense.
Also change the error a user receives upon disabled passwords to specify that only email-based password reset is disabled.
2019-06-11 00:25:07 +01:00
6bac9ca6d7
1.0.0rc3
2019-06-11 00:13:02 +01:00
426049247b
Code cleanups and simplifications.
...
Also: share the saml client between redirect and response handlers.
2019-06-11 00:03:57 +01:00
94dac0f3e5
add monthly active users to phonehome stats ( #5252 )
...
* add monthly active users to phonehome stats
2019-06-10 23:33:59 +01:00
49e01e5710
Fix defaults on checking threepids
2019-06-10 23:09:31 +01:00
69a43d9974
Merge remote-tracking branch 'origin/develop' into rav/saml2_client
2019-06-10 20:28:08 +01:00
78e74ab8a1
Merge pull request #5418 from matrix-org/erikj/fix_send_fed_with_limit_zero
...
Fix bug sending federation transactions with lots of EDUs
2019-06-10 20:19:49 +01:00
abce00fc6a
Merge branch 'release-v1.0.0' of github.com:matrix-org/synapse into develop
2019-06-10 18:33:55 +01:00
0167447965
1.0.0rc2
2019-06-10 18:17:43 +01:00
a6b1817940
Merge pull request #5417 from matrix-org/rav/shared_ssl_context
...
Share an SSL context object between SSL connections
2019-06-10 18:16:12 +01:00
81b8fdedf2
rename gutwrenched attr
2019-06-10 17:51:11 +01:00
c413540fb9
Fix bug sending federation transactions with lots of EDUs
...
If we try and send a transaction with lots of EDUs and we run out of
space, we call get_new_device_msgs_for_remote with a limit of 0, which
then failed.
2019-06-10 16:22:45 +01:00
efe7b3176e
Fix federation connections to literal IP addresses
...
turns out we need a shiny version of service_identity to enforce this
correctly.
2019-06-10 15:58:35 +01:00
0382b0ffee
Merge pull request #5415 from matrix-org/erikj/fix_null_valid_until_ms
...
Fix key verification when key stored with null valid_until_ms
2019-06-10 15:56:36 +01:00
8da0d83a54
Merge branch 'erikj/fix_null_valid_until_ms' of github.com:matrix-org/synapse into release-v1.0.0
2019-06-10 15:56:28 +01:00
d11c634ced
clean up impl, and import idna directly
2019-06-10 15:55:12 +01:00
43badd2cd4
Fix key verification when key stored with null valid_until_ms
...
Some keys are stored in the synapse database with a null valid_until_ms
which caused an exception to be thrown when using that key. We fix this
by treating nulls as zeroes, i.e. they keys will match verification
requests with a minimum_valid_until_ms of zero (i.e. don't validate ts)
but will not match requests with a non-zero minimum_valid_until_ms.
Fixes #5391 .
2019-06-10 15:37:10 +01:00
26b62796c2
Merge pull request #5363 from matrix-org/babolivier/account_validity_send_mail_auth
...
Don't check whether the user's account is expired on /send_mail requests
2019-06-10 11:57:02 +01:00
028f674cd3
Better wording
2019-06-10 11:35:54 +01:00
4914a88829
Doc
2019-06-10 11:34:45 +01:00
88d7182ada
Improve startup checks for insecure notary configs ( #5392 )
...
It's not really a problem to trust notary responses signed by the old key so
long as we are also doing TLS validation.
This commit adds a check to the config parsing code at startup to check that
we do not have the insecure matrix.org key without tls validation, and refuses
to start without it.
This allows us to remove the rather alarming-looking warning which happens at
runtime.
2019-06-10 10:33:00 +01:00
c2b6e945e1
Share an SSL context object between SSL connections
...
This involves changing how the info callbacks work.
2019-06-09 14:01:32 +01:00
95d38afe96
Don't log exception when failing to fetch remote content.
...
In particular, let's not log stack traces when we stop processing
becuase the response body was too large.
2019-06-07 12:39:10 +01:00
837340bdce
Only start background group attestation renewals on master
2019-06-07 12:25:06 +01:00
2ebeda48b2
Add test
2019-06-07 12:15:33 +01:00
928d1ccd73
Fix email notifications for large unnamed rooms.
...
When we try and calculate a description for a room for with no name but
multiple other users we threw an exception (due to trying to subscript
result of `dict.values()`).
2019-06-07 12:15:28 +01:00
6745b7de6d
Handle failing to talk to master over replication
2019-06-07 10:47:31 +01:00
a46ef1e3a4
Handle HttpResponseException when using federation client.
...
Otherwise we just log exceptions everywhere.
2019-06-07 10:29:35 +01:00
7c455a86bc
1.0.0rc1
2019-06-07 10:29:32 +01:00
2d1d7b7e6f
Prevent multiple device list updates from breaking a batch send ( #5156 )
...
fixes #5153
2019-06-06 23:54:00 +01:00
a11865016e
Set default room version to v4. ( #5379 )
...
Set default room version to v4.
2019-06-06 20:13:47 +01:00
1b8cb64393
Merge pull request #5320 from matrix-org/hawkowl/full-schema-v1
...
Make a full SQL schema
2019-06-06 18:35:38 +01:00
ed872db8df
fix maybe
2019-06-07 02:53:47 +10:00
3719680ee4
Add ability to perform password reset via email without trusting the identity server ( #5377 )
...
Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option.
This PR is a culmination of 3 smaller PRs which have each been separately reviewed:
* #5308
* #5345
* #5368
2019-06-06 17:34:07 +01:00
9fbb20a531
Stop hardcoding trust of old matrix.org key ( #5374 )
...
There are a few changes going on here:
* We make checking the signature on a key server response optional: if no
verify_keys are specified, we trust to TLS to validate the connection.
* We change the default config so that it does not require responses to be
signed by the old key.
* We replace the old 'perspectives' config with 'trusted_key_servers', which
is also formatted slightly differently.
* We emit a warning to the logs every time we trust a key server response
signed by the old key.
2019-06-06 17:33:11 +01:00
837e32ef55
just user dir?
2019-06-07 01:49:25 +10:00
3b6645d3bf
remove background updates that arent needed
2019-06-07 01:20:58 +10:00
71063a69b8
Fix missing logcontext for PresenceHandler.on_shutdown. ( #5369 )
...
Fixes some warnings, and a scary-looking stacktrace when sytest kills the
process.
2019-06-06 14:45:17 +01:00
89d3d7b2c0
Merge pull request #5221 from matrix-org/erikj/fix_worker_sytest
...
Fix get_max_topological_token to never return None
2019-06-06 13:54:46 +01:00
8f06344e11
Merge pull request #5089 from dnaf/m-heroes-empty-room-name
...
Make /sync return heroes if room name or canonical alias are empty
2019-06-06 11:18:13 +01:00
7f08a3523a
Better words
2019-06-06 11:09:38 +01:00
cb3b381fcb
Merge pull request #5359 from matrix-org/rav/enable_tls_verification
...
Validate federation server TLS certificates by default.
2019-06-06 10:50:42 +01:00
64fa928792
Simplify condition
2019-06-06 10:34:12 +01:00
b9c43c8463
Merge pull request #5355 from matrix-org/babolivier/heroes_left_members
...
Include left members in room summaries' heroes
2019-06-06 10:05:27 +01:00
99d3497949
Merge pull request #5354 from matrix-org/rav/server_keys/99-room-v5
...
Implement room v5 which enforces signing key validity
2019-06-06 09:42:13 +01:00
2eb47e5ee7
Merge pull request #5353 from matrix-org/rav/verify_key_logging
...
Associate a request_name with each verify request, for logging
2019-06-06 09:33:09 +01:00
6362e3af14
add more comments
2019-06-06 04:20:35 +10:00
fe13bd52ac
Don't check whether the user's account is expired on /send_mail requests
2019-06-05 16:35:05 +01:00
7603a706eb
Merge branch 'rav/fix_custom_ca' into rav/enable_tls_verification
2019-06-05 16:32:35 +01:00
f8a45302c9
Fix `federation_custom_ca_list` configuration option.
...
Previously, setting this option would cause an exception at startup.
2019-06-05 16:19:07 +01:00
94f6c674df
Neilj/add r0.5 to versions ( #5360 )
...
* Update _matrix/client/versions to reference support for r0.5.0
2019-06-05 16:11:31 +01:00
75538813fc
Fix background updates to handle redactions/rejections ( #5352 )
...
* Fix background updates to handle redactions/rejections
In background updates based on current state delta stream we need to
handle that we may not have all the events (or at least that
`get_events` may raise an exception).
2019-06-06 00:45:46 +10:00
e2dfb922e1
Validate federation server TLS certificates by default.
2019-06-05 14:17:50 +01:00
0a2f522644
Simplify condition
2019-06-05 14:02:29 +01:00
d53faa40e9
Merge pull request #5317 from matrix-org/erikj/make_do_auth_non_essential
...
Fix handling of failures when calling /event_auth.
2019-06-05 14:01:32 +01:00
40596aec0e
Merge branch 'develop' into m-heroes-empty-room-name
2019-06-05 13:38:01 +01:00
26713515de
Neilj/mau tracking config explainer ( #5284 )
...
Improve documentation of monthly active user blocking and mau_trial_days
2019-06-05 13:16:23 +01:00
dbbaf25dd3
Do user_id != me checks before deciding whether we should pick heroes from the joined members or the parted ones
2019-06-05 11:50:27 +01:00
bc3d6b918b
Add logging when request fails and clarify we ignore errors.
2019-06-05 11:37:51 +01:00
d18e4ea0d4
Implement room v5 which enforces signing key validity
...
Implements [MSC2077](https://github.com/matrix-org/matrix-doc/pull/2077 ) and
fixes #5247 and #4364 .
2019-06-05 11:00:47 +01:00
cea9750d11
Associate a request_name with each verify request, for logging
...
Also:
* rename VerifyKeyRequest->VerifyJsonRequest
* calculate key_ids on VerifyJsonRequest construction
* refactor things to pass around VerifyJsonRequests instead of 4-tuples
2019-06-05 10:46:26 +01:00
14f13babb0
Add a test room version where we enforce key validity ( #5348 )
2019-06-05 10:38:25 +01:00
2615c6bd9e
Clean up debug logging ( #5347 )
...
Remove some spurious stuff, clarify some other stuff
2019-06-05 10:35:40 +01:00
016af01598
Rename VerifyKeyRequest.deferred field ( #5343 )
...
it's a bit confusing
2019-06-05 10:35:13 +01:00
aa530e6800
Call RetryLimiter correctly ( #5340 )
...
Fixes a regression introduced in #5335 .
2019-06-04 22:02:53 +01:00
dae224a73f
Fix failure to fetch batches of PDUs ( #5342 )
...
FederationClient.get_pdu is called in a loop to fetch a batch of PDUs. A
failure to fetch one should not result in a failure of the whole batch. Add the
missing `continue`.
2019-06-04 18:05:06 +01:00
b4189b112f
Rename get_events->get_events_from_store_or_dest ( #5344 )
...
We have too many things called get_event, and it's hard to figure out what we
mean. Also remove some unused params from the signature, and add some logging.
2019-06-04 18:01:09 +01:00
f6dd12d1e2
Merge pull request #5341 from matrix-org/babolivier/email_config
...
Make account validity renewal emails work when email notifs are disabled
2019-06-04 14:49:06 +01:00
2f62e1f6ff
Only parse from email if provided
2019-06-04 14:24:36 +01:00
d1d38081a7
Merge pull request #5324 from matrix-org/erikj/ignore_null
...
Ignore room state with null bytes in for room stats
2019-06-04 14:20:08 +01:00
1cc5fc1f6c
Lint
2019-06-04 13:51:23 +01:00
ac3cc32367
Make account validity renewal emails work when email notifs are disabled
2019-06-04 13:47:44 +01:00
df9c100542
Avoid rapidly backing-off a server if we ignore the retry interval ( #5335 )
2019-06-04 11:53:29 +01:00
4d08b8f30c
Don't do long retries when calling the key notary server. ( #5334 )
...
It takes at least 20 minutes to work through the long_retries schedule (11
attempts, each with a 60 second timeout, and 60 seconds between each request),
so if the notary server isn't returning within the timeout, we'll just end up
blocking whatever request is happening for 20 minutes.
Ain't nobody got time for that.
2019-06-04 11:53:07 +01:00
cb683d3e3c
Merge pull request #5333 from matrix-org/rav/server_keys/09_improve_notary_server
...
Fixes for the key-notary server
2019-06-04 11:48:18 +01:00
5bdb189f86
Improve docstrings on MatrixFederationClient. ( #5332 )
2019-06-04 11:14:16 +01:00
b2b90b7d34
Hawkowl/fix missing auth ( #5328 )
2019-06-04 15:54:27 +10:00
c5d60eadd5
Notary server: make requests to origins in parallel
...
... else we're guaranteed to time out.
2019-06-04 00:16:56 +01:00
def5ea4062
Don't bomb out on direct key fetches as soon as one fails
2019-06-04 00:16:56 +01:00
dce6e9e0c1
Avoid rapidly backing-off a server if we ignore the retry interval
2019-06-03 23:58:42 +01:00
06a1f3e207
Reduce timeout for outbound /key/v2/server requests.
2019-06-03 23:17:38 +01:00
fec2dcb1a5
Enforce validity period on server_keys for fed requests. ( #5321 )
...
When handling incoming federation requests, make sure that we have an
up-to-date copy of the signing key.
We do not yet enforce the validity period for event signatures.
2019-06-03 22:59:51 +01:00
0a56966f7d
Fix
2019-06-03 17:42:52 +01:00
fa4b54aca5
Ignore room state with null bytes in for room stats
2019-06-03 17:06:54 +01:00
6f9f08005c
Merge remote-tracking branch 'origin/develop' into hawkowl/full-schema-v1
2019-06-04 01:16:01 +10:00
2198b7ce2a
add stuff in bg updates
2019-06-04 01:06:00 +10:00
4e75c5e02a
WHY IS THIS CALLED A SLIGHTLY DIFFERENT THING
2019-06-03 22:42:12 +10:00
be452fc9ac
more fix
2019-06-03 22:24:23 +10:00
7f81b967ca
fix schemas
2019-06-03 22:23:40 +10:00
862b2f9ad5
Merge pull request #5307 from matrix-org/rav/server_keys/07-fix-notary-cache-poison
...
Stop overwriting server keys with other keys
2019-06-03 13:19:20 +01:00
dc72b90cd6
full schema
2019-06-03 22:03:28 +10:00
37057d5d60
prepare
2019-06-03 22:02:47 +10:00
2889b05554
Unify v1 and v2 REST client APIs ( #5226 )
2019-06-03 21:28:59 +10:00
220a733d73
Fix handling of failures when calling /event_auth.
...
When processing an incoming event over federation, we may try and
resolve any unexpected differences in auth events. This is a
non-essential process and so should not stop the processing of the event
if it fails (e.g. due to the remote disappearing or not implementing the
necessary endpoints).
Fixes #3330
2019-06-03 09:56:45 +01:00
dc3e586938
SAML2 Improvements and redirect stuff
...
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2019-06-02 18:14:40 +02:00
d828d1dc57
Merge pull request #5309 from matrix-org/rav/limit_displayname_length
...
Limit displaynames and avatar URLs
2019-06-01 11:34:50 +01:00
d16c6375fe
Limit displaynames and avatar URLs
...
These end up in join events everywhere, so let's limit them.
Fixes #5079
2019-06-01 10:44:36 +01:00
37b165620d
Merge pull request #5299 from matrix-org/rav/server_keys/05-rewrite-gsvk-again
...
Rewrite get_server_verify_keys, again.
2019-05-31 17:07:31 +01:00
3600f5568b
Stop overwriting server keys with other keys
...
Fix a bug where we would discard a key result which the origin server is no
longer returning. Fixes #5305 .
2019-05-31 15:58:35 +01:00
58cce39f3a
Merge pull request #5276 from matrix-org/babolivier/account_validity_job_delta
...
Allow configuring a range for the account validity startup job
2019-05-31 12:11:56 +01:00
c605da97bf
Merge remote-tracking branch 'origin/develop' into rav/server_keys/05-rewrite-gsvk-again
2019-05-31 11:38:13 +01:00
fe79b5e521
Merge pull request #5300 from matrix-org/rav/server_keys/06-fix-serverkeys-handling
...
Remove some pointless exception handling
2019-05-31 11:35:29 +01:00
2ae3cc287e
Merge pull request #5296 from matrix-org/rav/server_keys/04-use-attrs-for_verify-request
...
use attr.s for VerifyKeyRequest
2019-05-31 11:34:09 +01:00
4d794dae21
Move delta from +10% to -10%
2019-05-31 11:09:39 +01:00
e9981d58ca
Merge pull request #5293 from Kagamihime/messages-federation-format
...
Fix ignored filter field in `/messages` endpoint
2019-05-31 10:52:59 +01:00
31d44ec4bd
Merge pull request #5294 from matrix-org/erikj/speed_up_room_stats
...
Speed up room stats background update
2019-05-31 10:48:51 +01:00
5037326d66
Add indices. Remove room_ids accidentally added
...
We have to do this by re-inserting a background update and recreating
tables, as the tables only get created during a background update and
will later be deleted.
We also make sure that we remove any entries that should have been
removed but weren't due to a race that has been fixed in a previous
commit.
2019-05-31 10:26:56 +01:00
0c2362861e
Gah python
2019-05-31 09:56:52 +01:00
847b9dcd1c
Make max_delta equal to period * 10%
2019-05-31 09:54:46 +01:00
8ea2f756a9
Remove some pointless exception handling
...
The verify_request deferred already returns a suitable SynapseError, so I don't
really know what we expect to achieve by doing more wrapping, other than log
spam.
Fixes #4278 .
2019-05-30 18:29:56 +01:00
a82c96b87f
Rewrite get_server_verify_keys, again.
...
Attempt to simplify the logic in get_server_verify_keys by splitting it into
two methods.
2019-05-30 18:20:40 +01:00
099829d5a9
use attr.s for VerifyKeyRequest
...
because namedtuple is awful
2019-05-30 17:39:28 +01:00
99113e40ba
Merge branch 'master' into develop
2019-05-30 16:39:49 +01:00
Richard van der Hoff