a6a776f3d8
remove dead transaction persist code ( #5622 )
...
this hasn't done anything for years
2019-07-05 12:59:42 +01:00
9481707a52
Fixes to the federation rate limiter ( #5621 )
...
- Put the default window_size back to 1000ms (broken by #5181 )
- Make the `rc_federation` config actually do something
- fix an off-by-one error in the 'concurrent' limit
- Avoid creating an unused `_PerHostRatelimiter` object for every single
incoming request
2019-07-05 11:10:19 +01:00
0e5434264f
Make errors about email password resets much clearer ( #5616 )
...
The runtime errors that dealt with local email password resets talked about config options that users may not even have in their config file yet (if upgrading). Instead, the cryptic errors are now replaced with hopefully much more helpful ones.
2019-07-05 10:44:12 +01:00
1ee268d33d
Improve the backwards compatibility re-exports of synapse.logging.context ( #5617 )
...
* Improve the backwards compatibility re-exports of synapse.logging.context.
* reexport logformatter too
2019-07-05 02:32:02 +10:00
822a0f0435
Merge branch 'master' of github.com:matrix-org/synapse into develop
2019-07-04 14:00:27 +01:00
20332b278d
1.1.0
2019-07-04 11:44:09 +01:00
c061d4f237
Fixup from review comments.
2019-07-04 11:41:06 +01:00
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ ( #5606 )
2019-07-04 00:07:04 +10:00
d0b849c86d
Apply comment fixups from code review
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-07-03 15:03:38 +01:00
cb8d568cf9
Fix 'utime went backwards' errors on daemonization. ( #5609 )
...
* Fix 'utime went backwards' errors on daemonization.
Fixes #5608
* remove spurious debug
2019-07-03 22:40:45 +10:00
463d5a8fde
1.1.0rc2
2019-07-03 10:51:47 +01:00
91753cae59
Fix a number of "Starting txn from sentinel context" warnings ( #5605 )
...
Fixes #5602 , #5603
2019-07-03 09:31:27 +01:00
0ee9076ffe
Fix media repo breaking ( #5593 )
2019-07-02 19:01:28 +01:00
9f3c0a8556
Add basic admin cmd app
2019-07-02 17:12:48 +01:00
8ee69f299c
Add basic function to get all data for a user out of synapse
2019-07-02 12:09:04 +01:00
a0fa4641c4
prepare v1.1.0rc1
2019-07-02 11:51:55 +01:00
6eecb6e500
Complete the SAML2 implementation ( #5422 )
...
* SAML2 Improvements and redirect stuff
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
* Code cleanups and simplifications.
Also: share the saml client between redirect and response handlers.
* changelog
* Revert redundant changes to static js
* Move all the saml stuff out to a centralised handler
* Add support for tracking SAML2 sessions.
This allows us to correctly handle `allow_unsolicited: False`.
* update sample config
* cleanups
* update sample config
* rename BaseSSORedirectServlet for consistency
* Address review comments
2019-07-02 11:18:11 +01:00
948488e115
Fix JWT login with new users ( #5586 )
...
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
2019-07-02 10:25:37 +01:00
b4914681a5
fix async/await consentresource ( #5585 )
...
Fixes #5582
2019-07-01 23:33:52 +10:00
b4fd86a9b4
Merge branch 'develop' into rav/saml2_client
2019-07-01 14:21:03 +01:00
3bcb13edd0
Address review comments
2019-07-01 12:13:22 +01:00
04196a4dae
Merge pull request #5507 from matrix-org/erikj/presence_sync_tighloop
...
Fix sync tightloop bug.
2019-07-01 11:43:10 +01:00
915280f1ed
Fixup comment
2019-07-01 10:22:42 +01:00
f40a7dc41f
Make the http server handle coroutine-making REST servlets ( #5475 )
2019-06-29 17:06:55 +10:00
c7ff297dde
Merge pull request #5576 from matrix-org/babolivier/3pid-invite-ratelimit
...
Don't update the ratelimiter before sending a 3PID invite
2019-06-28 17:43:48 +01:00
15d9fc31bd
Only ratelimit when sending the email
...
If we do the opposite, an event can arrive after or while sending the email and the 3PID invite event will get ratelimited.
2019-06-28 16:04:05 +01:00
01d0f8e701
Don't update the ratelimiter before sending a 3PID invite
...
This would cause emails being sent, but Synapse responding with a 429 when creating the event. The client would then retry, and with bad timing the same scenario would happen again. Some testing I did ended up sending me 10 emails for one single invite because of this.
2019-06-28 15:22:16 +01:00
071150ce19
Don't log GC 0s at INFO ( #5557 )
2019-06-28 21:45:33 +10:00
be3b901ccd
Update the TLS cipher string and provide configurability for TLS on outgoing federation ( #5550 )
2019-06-28 18:19:09 +10:00
9646a593ac
Added possibilty to disable local password authentication ( #5092 )
...
Signed-off-by: Daniel Hoffend <dh@dotlan.net>
2019-06-27 18:37:29 +01:00
c548dbc4b1
Make it clearer that the template dir is relative to synapse's root dir ( #5543 )
...
Helps address #5444
2019-06-27 18:20:17 +01:00
e79ec03165
Merge pull request #5559 from matrix-org/erikj/refactor_changed_devices
...
Refactor devices changed query to pull less from DB
2019-06-27 16:53:15 +01:00
729f5a4fb6
Review comments
2019-06-27 16:06:23 +01:00
856ea04eb3
Fix JWT login ( #5555 )
...
* Fix JWT login with register
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
* Add pyjwt conditional dependency
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
* Added changelog file
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
* Improved changelog description
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
2019-06-27 12:02:41 +01:00
79b9d9076d
rename BaseSSORedirectServlet for consistency
2019-06-27 00:46:57 +01:00
28db0ae537
cleanups
2019-06-27 00:37:41 +01:00
36f4953dec
Add support for tracking SAML2 sessions.
...
This allows us to correctly handle `allow_unsolicited: False`.
2019-06-26 23:50:55 +01:00
3705322103
Move all the saml stuff out to a centralised handler
2019-06-26 22:52:02 +01:00
0ade403f55
Revert redundant changes to static js
2019-06-26 22:46:23 +01:00
a4daa899ec
Merge branch 'develop' into rav/saml2_client
2019-06-26 22:34:41 +01:00
8624db3194
Refactor and comment sync device list code
2019-06-26 19:39:49 +01:00
f335e77d53
Use batch_iter and correct docstring
2019-06-26 19:39:46 +01:00
806a06daf2
Rename get_users_whose_devices_changed
2019-06-26 19:39:19 +01:00
a2f6d31a63
Refactor get_user_ids_changed to pull less from DB
...
When a client asks for users whose devices have changed since a token we
used to pull *all* users from the database since the token, which could
easily be thousands of rows for old tokens.
This PR changes this to only check for changes for users the client is
actually interested in.
Fixes #5553
2019-06-26 12:03:44 +01:00
ef8c62758c
Prevent multiple upgrades on the same room at once ( #5051 )
...
Closes #4583
Does slightly less than #5045 , which prevented a room from being upgraded multiple times, one after another. This PR still allows that, but just prevents two from happening at the same time.
Mostly just to mitigate the fact that servers are slow and it can take a moment for the room upgrade to actually complete. We don't want people sending another request to upgrade the room when really they just thought the first didn't go through.
2019-06-25 14:19:21 +01:00
4ac7ef4b67
Merge pull request #5524 from matrix-org/rav/new_cmdline_options
...
Add --data-dir and --open-private-ports options.
2019-06-24 17:25:57 +01:00
af8a962905
Merge pull request #5523 from matrix-org/rav/arg_defaults
...
Stop conflating generated config and default config
2019-06-24 17:24:35 +01:00
e59a8cd2e5
Merge pull request #5499 from matrix-org/rav/cleanup_metrics
...
Cleanups and sanity-checking in cpu and db metrics
2019-06-24 17:12:54 +01:00
deb4fe6ef3
Merge pull request #5534 from matrix-org/babolivier/federation-publicrooms
...
Split public rooms directory auth config in two
2019-06-24 16:08:02 +01:00
bfe84e051e
Split public rooms directory auth config in two
2019-06-24 15:42:31 +01:00
25433f212d
Merge pull request #5531 from matrix-org/erikj/workers_pagination_token
...
Fix /messages on workers when no from param specified.
2019-06-24 15:30:10 +01:00
3f8a252dd8
Add "--open-private-ports" cmdline option
...
This is helpful when generating a config file for running synapse under docker.
2019-06-24 14:15:34 +01:00
6a92b06cbb
Add --data-directory commandline argument
...
We don't necessarily want to put the data in the cwd.
2019-06-24 14:15:34 +01:00
16b52642e2
Don't load the generated config as the default.
...
It's too confusing.
2019-06-24 14:14:52 +01:00
7c2f8881a9
Ensure that all config options have sensible defaults
...
This will enable us to skip the unintuitive behaviour where the generated
config and default config are the same thing.
2019-06-24 14:14:52 +01:00
367a8263b3
Remove unused Config.config_dir_path attribute
...
This is no longer used and only serves to confuse.
2019-06-24 13:51:22 +01:00
edea4bb5be
Allow configuration of the path used for ACME account keys.
...
Because sticking it in the same place as the config isn't necessarily the right
thing to do.
2019-06-24 13:51:22 +01:00
c3c6b00d95
Pass config_dir_path and data_dir_path into Config.read_config. ( #5522 )
...
* Pull config_dir_path and data_dir_path calculation out of read_config_files
* Pass config_dir_path and data_dir_path into read_config
2019-06-24 11:34:45 +01:00
21bf4318b5
Factor acme bits out to a separate file ( #5521 )
...
This makes some of the conditional-import hoop-jumping easier.
2019-06-24 11:33:56 +01:00
1793de6c6d
black
2019-06-24 11:16:13 +01:00
dddf20e8e1
Fix /messages on workers when no from param specified.
...
If no `from` param is specified we calculate and use the "current
token" that inlcuded typing, presence, etc. These are unused during
pagination and are not available on workers, so we simply don't
calculate them.
2019-06-24 10:06:51 +01:00
dc94773e60
Avoid raising exceptions in metrics
...
Sentry will catch the errors if they happen, so that should be good enough, and
woun't make things explode if we hit the error condition.
2019-06-24 10:01:16 +01:00
5097aee740
Merge branch 'develop' into rav/cleanup_metrics
2019-06-24 10:00:13 +01:00
c753c098dd
Merge pull request #5498 from matrix-org/rav/fix_clock_reversal
...
Use monotonic clock where possible for metrics
2019-06-24 09:55:12 +01:00
6cda36777b
Drop support for cpu_affinity ( #5525 )
...
This has no useful purpose on python3, and is generally a source of confusion.
2019-06-22 11:01:55 +10:00
e1a795758c
Improve help and cmdline option names for --generate-config options ( #5512 )
...
* group the arguments together into a group
* add new names "--generate-missing-config" and "--config-directory" for
existing cmdline options "--generate-keys" and "--keys-dir", which better
reflect their purposes.
2019-06-21 18:50:43 +01:00
03cea2b0fe
Refactor Config parser and add some comments. ( #5511 )
...
Add some comments, and simplify `read_config_files`.
2019-06-21 17:43:38 +01:00
37933a3bf8
Improve logging when generating config files ( #5510 )
...
Make it a bit clearer what's going on.
2019-06-21 17:14:56 +01:00
5d6644efe0
Only import jinja2 when needed ( #5514 )
...
Fixes https://github.com/matrix-org/synapse/issues/5431
`jinja2` was being imported even when it wasn't strictly necessary. This made it required to run Synapse, even if the functionality that required it wasn't enabled. This was causing new Synapse installations to crash on startup.
Email modules are now required.
2019-06-21 16:52:09 +01:00
0e8b35f7b0
Fix "Unexpected entry in 'full_schemas'" log warning ( #5509 )
...
There is a README.txt which always sets off this warning, which is a bit
alarming when you first start synapse. I don't think we need to warn about
this.
2019-06-21 15:11:57 +01:00
2f8491daef
Fix logging error when a tampered event is detected. ( #5500 )
2019-06-21 15:11:42 +01:00
7eadb74056
Fix /messages on worker when no token supplied
2019-06-21 14:15:59 +01:00
8181e290a9
Fix sync tightloop bug.
...
If, for some reason, presence updates take a while to persist then it
can trigger clients to tightloop calling `/sync` due to the presence
handler returning updates but not advancing the stream token.
Fixes #5503 .
2019-06-21 11:10:27 +01:00
f3ab533374
Support pagination API in client_reader worker
2019-06-21 10:43:12 +01:00
7456698241
Merge pull request #5476 from matrix-org/erikj/histogram_extremities
...
Add metrics for length of new extremities persisted.
2019-06-21 09:58:54 +01:00
f47969f42a
Improve email notification logging ( #5502 )
2019-06-20 15:14:26 +01:00
f8bd30af2a
Black
2019-06-20 13:10:06 +01:00
45f28a9d2f
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/histogram_extremities
2019-06-20 11:59:14 +01:00
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
fe641df770
Sanity-checking for metrics updates
...
Check that our clocks go forward.
2019-06-19 21:18:38 +01:00
f682af052d
Simplify PerformanceCounters.update interface
...
we already have the duration for the update, so may as well use it rather than
passing extra params around and recalculating it.
2019-06-19 21:18:38 +01:00
68128d5626
Remove unused _get_event_counters
...
This has been redundant since cdb3757942
2019-06-19 21:14:25 +01:00
15bf32e062
Use monotonic clock where possible for metrics
...
Fixes intermittent errors observed on Apple hardware which were caused by
time.clock() appearing to go backwards when called from different threads.
Also fixes a bug where database activity times were logged as 1/1000 of their
correct ratio due to confusion between milliseconds and seconds.
2019-06-19 21:09:43 +01:00
7dcf984075
Merge pull request #5042 from matrix-org/erikj/fix_get_missing_events_error
...
Handle the case of `get_missing_events` failing
2019-06-19 13:20:09 +01:00
e0be8d7016
Merge pull request #5480 from matrix-org/erikj/extremities_dummy_events
...
Add experimental option to reduce extremities.
2019-06-19 13:19:18 +01:00
65787b0f7c
Add descriptions and remove redundant set(..)
2019-06-19 11:49:39 +01:00
554609288b
Run as background process and fix comments
2019-06-19 11:33:03 +01:00
092e36457a
Fix typo in deactivation background job
2019-06-19 11:09:41 +01:00
f2d2ae03da
Add some logging to 3pid invite sig verification ( #5015 )
...
I had to add quite a lot of logging to diagnose a problem with 3pid
invites - we only logged the one failure which isn't all that
informative.
NB. I'm not convinced the logic of this loop is right: I think it
should just accept a single valid signature from a trusted source
rather than fail if *any* signature is invalid. Also it should
probably not skip the rest of middle loop if a check fails? However,
I'm deliberately not changing the logic here.
2019-06-18 22:51:24 +01:00
2b20d0fb59
Fix logline
2019-06-18 16:12:53 +01:00
19b80fe68a
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fix_get_missing_events_error
2019-06-18 16:11:43 +01:00
b42f90470f
Add experimental option to reduce extremities.
...
Adds new config option `cleanup_extremities_with_dummy_events` which
periodically sends dummy events to rooms with more than 10 extremities.
THIS IS REALLY EXPERIMENTAL.
2019-06-18 15:02:18 +01:00
16a3124b76
Only count non-cache state resolution
2019-06-18 13:02:06 +01:00
c9385dd238
Use consistent buckets
2019-06-18 12:43:41 +01:00
d6328e03fd
Merge pull request #5477 from matrix-org/babolivier/third_party_rules_3pid
...
Add third party rules hook for 3PID invites
2019-06-17 18:08:31 +01:00
33ea87be39
Make check_threepid_can_be_invited async
2019-06-17 17:39:38 +01:00
112cf5a73a
Add third party rules hook for 3PID invites
2019-06-17 17:39:09 +01:00
8353ddd951
Merge pull request #5479 from matrix-org/erikj/add_create_room_hook_develop
...
Add third party rules hook into create room
2019-06-17 17:30:05 +01:00
ff88d36dcb
Add metric fo number of state groups in resolution
2019-06-17 17:02:53 +01:00
499d4a32cd
Add metrics for len of new extremities persisted.
...
Of new events being persisted add metrics for total size of forward
extremities and number of unchanged, "stale" extremities.
2019-06-17 17:02:48 +01:00
Richard van der Hoff