Commit Graph

10205 Commits (f817fc9ad51eb8b409894136d7eab2b7448a7f09)

Author SHA1 Message Date
Brendan Abolivier 112cf5a73a Add third party rules hook for 3PID invites 2019-06-17 17:39:09 +01:00
Erik Johnston 8353ddd951
Merge pull request #5479 from matrix-org/erikj/add_create_room_hook_develop
Add third party rules hook into create room
2019-06-17 17:30:05 +01:00
Erik Johnston ff88d36dcb Add metric fo number of state groups in resolution 2019-06-17 17:02:53 +01:00
Erik Johnston 499d4a32cd Add metrics for len of new extremities persisted.
Of new events being persisted add metrics for total size of forward
extremities and number of unchanged, "stale" extremities.
2019-06-17 17:02:48 +01:00
Erik Johnston 187d2837a9 Add third party rules hook into create room 2019-06-17 16:41:19 +01:00
Amber Brown eba7caf09f
Remove Postgres 9.4 support (#5448) 2019-06-18 00:59:00 +10:00
Erik Johnston 6840ebeef8
Merge pull request #5385 from matrix-org/erikj/reduce_http_exceptions
Handle HttpResponseException when using federation client.
2019-06-17 13:54:47 +01:00
Erik Johnston dd927b29e1
Merge pull request #5388 from matrix-org/erikj/fix_email_push
Fix email notifications for unnamed rooms with multiple people
2019-06-17 13:54:35 +01:00
Erik Johnston 414d2ca3a6
Merge pull request #5389 from matrix-org/erikj/renew_attestations_on_master
Only start background group attestation renewals on master
2019-06-17 13:54:29 +01:00
Erik Johnston a9dab970b8
Merge pull request #5464 from matrix-org/erikj/3pid_remote_invite_state
Fix 3PID invite room state over federation.
2019-06-17 10:18:28 +01:00
Brendan Abolivier f12e1f029c
Merge pull request #5440 from matrix-org/babolivier/third_party_event_rules
Allow server admins to define implementations of extra rules for allowing or denying incoming events
2019-06-14 19:37:59 +01:00
Erik Johnston 9ca4ae7131
Merge pull request #5461 from matrix-org/erikj/histograms_are_cumalitive
Prometheus histograms are cumalative
2019-06-14 18:21:42 +01:00
Brendan Abolivier f874b16b2e Add plugin APIs for implementations of custom event rules. 2019-06-14 18:16:03 +01:00
Brendan Abolivier 14db086428
Merge pull request #5465 from matrix-org/babolivier/fix_deactivation_bg_job
Fix background job for deactivated flag
2019-06-14 18:12:56 +01:00
Erik Johnston 304a1376c2 Fix 3PID invite room state over federation.
Fixes that when a user exchanges a 3PID invite for a proper invite over
federation it does not include the `invite_room_state` key.

This was due to synapse incorrectly sending out two invite requests.
2019-06-14 16:19:11 +01:00
Brendan Abolivier e0b77b004d
Fix background job for deactivated flag 2019-06-14 16:00:45 +01:00
Brendan Abolivier 9b14a810d2
Merge pull request #5462 from matrix-org/babolivier/account_validity_deactivated_accounts_2
Don't send renewal emails to deactivated users (second attempt)
2019-06-14 15:35:31 +01:00
Brendan Abolivier 6d56a694f4 Don't send renewal emails to deactivated users 2019-06-14 15:05:56 +01:00
Erik Johnston e9344e0dee
Merge pull request #5390 from matrix-org/erikj/dont_log_on_fail_to_get_file
Don't log exception when failing to fetch remote content.
2019-06-14 14:25:14 +01:00
Erik Johnston 3ed595e327 Prometheus histograms are cumalative 2019-06-14 14:07:32 +01:00
Brendan Abolivier d0530382ee
Track deactivated accounts in the database (#5378) 2019-06-14 13:18:24 +01:00
Amber H. Brown a10c8dae85 fix prometheus rendering error 2019-06-14 21:09:33 +10:00
Richard van der Hoff 5c15039e06
Clean up code for sending federation EDUs. (#5381)
This code confused the hell out of me today. Split _get_new_device_messages
into its two (unrelated) parts.
2019-06-13 13:52:08 +01:00
Amber Brown 6312d6cc7c
Expose statistics on extrems to prometheus (#5384) 2019-06-13 22:40:52 +10:00
Amber Brown 09e9a26b71
Remove Python 2.7 support. (#5425)
* remove 2.7 from CI and publishing

* fill out classifiers and also make it not be installed on 3.5

* some minor bumps so that the old deps work on python 3.5
2019-06-12 21:31:59 +10:00
Erik Johnston 7e68691ce9 Merge branch 'master' of github.com:matrix-org/synapse into develop 2019-06-11 17:25:16 +01:00
Erik Johnston 97174780ce 1.0.0 2019-06-11 17:10:01 +01:00
Erik Johnston a766c41d25 Bump bleach version so that tests can run on old deps. 2019-06-11 12:34:18 +01:00
Erik Johnston 10383e6e6f Change password reset links to /_matrix. 2019-06-11 11:34:33 +01:00
Erik Johnston 5bc9484537 Merge branch 'release-v1.0.0' of github.com:matrix-org/synapse into develop 2019-06-11 10:37:43 +01:00
Andrew Morgan 2ddc13577c
Don't warn user about password reset disabling through config code (#5387)
Moves the warning about password resets being disabled to the point where a user actually tries to reset their password. Is this an appropriate place for it to happen?

Also removed the disabling of msisdn password resets when you don't have an email config, as that just doesn't make sense.

Also change the error a user receives upon disabled passwords to specify that only email-based password reset is disabled.
2019-06-11 00:25:07 +01:00
Erik Johnston 6bac9ca6d7 1.0.0rc3 2019-06-11 00:13:02 +01:00
Neil Johnson 94dac0f3e5
add monthly active users to phonehome stats (#5252)
* add monthly active users to phonehome stats
2019-06-10 23:33:59 +01:00
Erik Johnston 49e01e5710 Fix defaults on checking threepids 2019-06-10 23:09:31 +01:00
Erik Johnston 78e74ab8a1
Merge pull request #5418 from matrix-org/erikj/fix_send_fed_with_limit_zero
Fix bug sending federation transactions with lots of EDUs
2019-06-10 20:19:49 +01:00
Erik Johnston abce00fc6a Merge branch 'release-v1.0.0' of github.com:matrix-org/synapse into develop 2019-06-10 18:33:55 +01:00
Erik Johnston 0167447965 1.0.0rc2 2019-06-10 18:17:43 +01:00
Erik Johnston a6b1817940
Merge pull request #5417 from matrix-org/rav/shared_ssl_context
Share an SSL context object between SSL connections
2019-06-10 18:16:12 +01:00
Richard van der Hoff 81b8fdedf2 rename gutwrenched attr 2019-06-10 17:51:11 +01:00
Erik Johnston c413540fb9 Fix bug sending federation transactions with lots of EDUs
If we try and send a transaction with lots of EDUs and we run out of
space, we call get_new_device_msgs_for_remote with a limit of 0, which
then failed.
2019-06-10 16:22:45 +01:00
Richard van der Hoff efe7b3176e Fix federation connections to literal IP addresses
turns out we need a shiny version of service_identity to enforce this
correctly.
2019-06-10 15:58:35 +01:00
Erik Johnston 0382b0ffee
Merge pull request #5415 from matrix-org/erikj/fix_null_valid_until_ms
Fix key verification when key stored with null valid_until_ms
2019-06-10 15:56:36 +01:00
Erik Johnston 8da0d83a54 Merge branch 'erikj/fix_null_valid_until_ms' of github.com:matrix-org/synapse into release-v1.0.0 2019-06-10 15:56:28 +01:00
Richard van der Hoff d11c634ced clean up impl, and import idna directly 2019-06-10 15:55:12 +01:00
Erik Johnston 43badd2cd4 Fix key verification when key stored with null valid_until_ms
Some keys are stored in the synapse database with a null valid_until_ms
which caused an exception to be thrown when using that key. We fix this
by treating nulls as zeroes, i.e. they keys will match verification
requests with a minimum_valid_until_ms of zero (i.e. don't validate ts)
but will not match requests with a non-zero minimum_valid_until_ms.

Fixes #5391.
2019-06-10 15:37:10 +01:00
Brendan Abolivier 26b62796c2
Merge pull request #5363 from matrix-org/babolivier/account_validity_send_mail_auth
Don't check whether the user's account is expired on /send_mail requests
2019-06-10 11:57:02 +01:00
Brendan Abolivier 028f674cd3
Better wording 2019-06-10 11:35:54 +01:00
Brendan Abolivier 4914a88829
Doc 2019-06-10 11:34:45 +01:00
Richard van der Hoff 88d7182ada
Improve startup checks for insecure notary configs (#5392)
It's not really a problem to trust notary responses signed by the old key so
long as we are also doing TLS validation.

This commit adds a check to the config parsing code at startup to check that
we do not have the insecure matrix.org key without tls validation, and refuses
to start without it.

This allows us to remove the rather alarming-looking warning which happens at
runtime.
2019-06-10 10:33:00 +01:00
Richard van der Hoff c2b6e945e1 Share an SSL context object between SSL connections
This involves changing how the info callbacks work.
2019-06-09 14:01:32 +01:00
Erik Johnston 95d38afe96 Don't log exception when failing to fetch remote content.
In particular, let's not log stack traces when we stop processing
becuase the response body was too large.
2019-06-07 12:39:10 +01:00
Erik Johnston 837340bdce Only start background group attestation renewals on master 2019-06-07 12:25:06 +01:00
Erik Johnston 2ebeda48b2 Add test 2019-06-07 12:15:33 +01:00
Erik Johnston 928d1ccd73 Fix email notifications for large unnamed rooms.
When we try and calculate a description for a room for with no name but
multiple other users we threw an exception (due to trying to subscript
result of `dict.values()`).
2019-06-07 12:15:28 +01:00
Erik Johnston 6745b7de6d Handle failing to talk to master over replication 2019-06-07 10:47:31 +01:00
Erik Johnston a46ef1e3a4 Handle HttpResponseException when using federation client.
Otherwise we just log exceptions everywhere.
2019-06-07 10:29:35 +01:00
Neil Johnson 7c455a86bc 1.0.0rc1 2019-06-07 10:29:32 +01:00
Andrew Morgan 2d1d7b7e6f Prevent multiple device list updates from breaking a batch send (#5156)
fixes #5153
2019-06-06 23:54:00 +01:00
Neil Johnson a11865016e
Set default room version to v4. (#5379)
Set default room version to v4.
2019-06-06 20:13:47 +01:00
Erik Johnston 1b8cb64393
Merge pull request #5320 from matrix-org/hawkowl/full-schema-v1
Make a full SQL schema
2019-06-06 18:35:38 +01:00
Amber H. Brown ed872db8df fix maybe 2019-06-07 02:53:47 +10:00
Andrew Morgan 3719680ee4
Add ability to perform password reset via email without trusting the identity server (#5377)
Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option.

This PR is a culmination of 3 smaller PRs which have each been separately reviewed:

* #5308
* #5345
* #5368
2019-06-06 17:34:07 +01:00
Richard van der Hoff 9fbb20a531
Stop hardcoding trust of old matrix.org key (#5374)
There are a few changes going on here:

* We make checking the signature on a key server response optional: if no
  verify_keys are specified, we trust to TLS to validate the connection.

* We change the default config so that it does not require responses to be
  signed by the old key.

* We replace the old 'perspectives' config with 'trusted_key_servers', which
  is also formatted slightly differently.

* We emit a warning to the logs every time we trust a key server response
  signed by the old key.
2019-06-06 17:33:11 +01:00
Amber H. Brown 837e32ef55 just user dir? 2019-06-07 01:49:25 +10:00
Amber H. Brown 3b6645d3bf remove background updates that arent needed 2019-06-07 01:20:58 +10:00
Richard van der Hoff 71063a69b8
Fix missing logcontext for PresenceHandler.on_shutdown. (#5369)
Fixes some warnings, and a scary-looking stacktrace when sytest kills the
process.
2019-06-06 14:45:17 +01:00
Erik Johnston 89d3d7b2c0
Merge pull request #5221 from matrix-org/erikj/fix_worker_sytest
Fix get_max_topological_token to never return None
2019-06-06 13:54:46 +01:00
Brendan Abolivier 8f06344e11
Merge pull request #5089 from dnaf/m-heroes-empty-room-name
Make /sync return heroes if room name or canonical alias are empty
2019-06-06 11:18:13 +01:00
Erik Johnston 7f08a3523a Better words 2019-06-06 11:09:38 +01:00
Richard van der Hoff cb3b381fcb
Merge pull request #5359 from matrix-org/rav/enable_tls_verification
Validate federation server TLS certificates by default.
2019-06-06 10:50:42 +01:00
Brendan Abolivier 64fa928792
Simplify condition 2019-06-06 10:34:12 +01:00
Brendan Abolivier b9c43c8463
Merge pull request #5355 from matrix-org/babolivier/heroes_left_members
Include left members in room summaries' heroes
2019-06-06 10:05:27 +01:00
Richard van der Hoff 99d3497949
Merge pull request #5354 from matrix-org/rav/server_keys/99-room-v5
Implement room v5 which enforces signing key validity
2019-06-06 09:42:13 +01:00
Richard van der Hoff 2eb47e5ee7
Merge pull request #5353 from matrix-org/rav/verify_key_logging
Associate a request_name with each verify request, for logging
2019-06-06 09:33:09 +01:00
Amber Brown 6362e3af14 add more comments 2019-06-06 04:20:35 +10:00
Brendan Abolivier fe13bd52ac
Don't check whether the user's account is expired on /send_mail requests 2019-06-05 16:35:05 +01:00
Richard van der Hoff 7603a706eb Merge branch 'rav/fix_custom_ca' into rav/enable_tls_verification 2019-06-05 16:32:35 +01:00
Richard van der Hoff f8a45302c9 Fix `federation_custom_ca_list` configuration option.
Previously, setting this option would cause an exception at startup.
2019-06-05 16:19:07 +01:00
Neil Johnson 94f6c674df
Neilj/add r0.5 to versions (#5360)
* Update _matrix/client/versions to reference support for r0.5.0
2019-06-05 16:11:31 +01:00
Erik Johnston 75538813fc Fix background updates to handle redactions/rejections (#5352)
* Fix background updates to handle redactions/rejections

In background updates based on current state delta stream we need to
handle that we may not have all the events (or at least that
`get_events` may raise an exception).
2019-06-06 00:45:46 +10:00
Richard van der Hoff e2dfb922e1 Validate federation server TLS certificates by default. 2019-06-05 14:17:50 +01:00
Brendan Abolivier 0a2f522644
Simplify condition 2019-06-05 14:02:29 +01:00
Erik Johnston d53faa40e9
Merge pull request #5317 from matrix-org/erikj/make_do_auth_non_essential
Fix handling of failures when calling /event_auth.
2019-06-05 14:01:32 +01:00
Brendan Abolivier 40596aec0e Merge branch 'develop' into m-heroes-empty-room-name 2019-06-05 13:38:01 +01:00
Neil Johnson 26713515de
Neilj/mau tracking config explainer (#5284)
Improve documentation of monthly active user blocking and mau_trial_days
2019-06-05 13:16:23 +01:00
Brendan Abolivier dbbaf25dd3
Do user_id != me checks before deciding whether we should pick heroes from the joined members or the parted ones 2019-06-05 11:50:27 +01:00
Erik Johnston bc3d6b918b Add logging when request fails and clarify we ignore errors. 2019-06-05 11:37:51 +01:00
Richard van der Hoff d18e4ea0d4 Implement room v5 which enforces signing key validity
Implements [MSC2077](https://github.com/matrix-org/matrix-doc/pull/2077) and
fixes #5247 and #4364.
2019-06-05 11:00:47 +01:00
Richard van der Hoff cea9750d11 Associate a request_name with each verify request, for logging
Also:
* rename VerifyKeyRequest->VerifyJsonRequest
* calculate key_ids on VerifyJsonRequest construction
* refactor things to pass around VerifyJsonRequests instead of 4-tuples
2019-06-05 10:46:26 +01:00
Richard van der Hoff 14f13babb0
Add a test room version where we enforce key validity (#5348) 2019-06-05 10:38:25 +01:00
Richard van der Hoff 2615c6bd9e
Clean up debug logging (#5347)
Remove some spurious stuff, clarify some other stuff
2019-06-05 10:35:40 +01:00
Richard van der Hoff 016af01598
Rename VerifyKeyRequest.deferred field (#5343)
it's a bit confusing
2019-06-05 10:35:13 +01:00
Richard van der Hoff aa530e6800
Call RetryLimiter correctly (#5340)
Fixes a regression introduced in #5335.
2019-06-04 22:02:53 +01:00
Richard van der Hoff dae224a73f
Fix failure to fetch batches of PDUs (#5342)
FederationClient.get_pdu is called in a loop to fetch a batch of PDUs. A
failure to fetch one should not result in a failure of the whole batch. Add the
missing `continue`.
2019-06-04 18:05:06 +01:00
Richard van der Hoff b4189b112f
Rename get_events->get_events_from_store_or_dest (#5344)
We have too many things called get_event, and it's hard to figure out what we
mean. Also remove some unused params from the signature, and add some logging.
2019-06-04 18:01:09 +01:00
Brendan Abolivier f6dd12d1e2
Merge pull request #5341 from matrix-org/babolivier/email_config
Make account validity renewal emails work when email notifs are disabled
2019-06-04 14:49:06 +01:00
Brendan Abolivier 2f62e1f6ff
Only parse from email if provided 2019-06-04 14:24:36 +01:00
Erik Johnston d1d38081a7
Merge pull request #5324 from matrix-org/erikj/ignore_null
Ignore room state with null bytes in for room stats
2019-06-04 14:20:08 +01:00
Brendan Abolivier 1cc5fc1f6c
Lint 2019-06-04 13:51:23 +01:00
Brendan Abolivier ac3cc32367
Make account validity renewal emails work when email notifs are disabled 2019-06-04 13:47:44 +01:00