Commit Graph

155 Commits (dd4778875213d9cb8be7ee71d32751fbd6cdaba2)

Author SHA1 Message Date
Richard van der Hoff 9cd13c5f63
Fix perspectives requests for multiple keys for the same server (#11440)
If we tried to request multiple keys for the same server, we would end up
dropping some of those requests.
2021-11-29 13:15:36 +00:00
Richard van der Hoff a6f7f84570
Fix verification of objects signed with old local keys (#11379)
Fixes a bug introduced in #11129: objects signed by the local server, but with
keys other than the current one, could not be successfully verified.

We need to check the key id in the signature, and track down the right key.
2021-11-19 10:55:09 +00:00
Shay e002faee01
Fetch verify key locally rather than trying to do so over federation if origin and host are the same. (#11129)
* add tests for fetching key locally

* add logic to check if origin server is same as host and fetch verify key locally rather than over federation

* add changelog

* slight refactor, add docstring, change changelog entry

* Make changelog entry one line

* remove verify_json_locally and push locality check to process_request, add function process_request_locally

* remove leftover code reference

* refactor to add common call to 'verify_json and associated handling code

* add type hint to process_json

* add some docstrings + very slight refactor
2021-10-28 10:27:17 -07:00
Patrick Cloke 0f9adc99ad
Add missing type hints to synapse.crypto. (#11146)
And require type hints for this module.
2021-10-21 13:07:07 +00:00
Patrick Cloke 8c7a531e27
Use direct references for some configuration variables (part 2) (#10812) 2021-09-15 08:34:52 -04:00
Jonathan de Jong bf72d10dbf
Use inline type hints in various other places (in `synapse/`) (#10380) 2021-07-15 11:02:43 +01:00
Erik Johnston c842c581ed
When joining a remote room limit the number of events we concurrently check signatures/hashes for (#10117)
If we do hundreds of thousands at once the memory overhead can easily reach 500+ MB.
2021-06-08 11:07:46 +01:00
Erik Johnston fc3d2dc269
Rewrite the KeyRing (#10035) 2021-06-02 16:37:59 +01:00
Erik Johnston 1c6a19002c
Add `Keyring.verify_events_for_server` and reduce memory usage (#10018)
Also add support for giving a callback to generate the JSON object to
verify. This should reduce memory usage, as we no longer have the event
in memory in dict form (which has a large memory footprint) for extend
periods of time.
2021-05-20 16:25:11 +01:00
Richard van der Hoff 5a153772c1
remove `HomeServer.get_config` (#9815)
Every single time I want to access the config object, I have to remember
whether or not we use `get_config`. Let's just get rid of it.
2021-04-14 19:09:08 +01:00
Jonathan de Jong 4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Patrick Cloke b7748d3c00
Import HomeServer from the proper module. (#9665) 2021-03-23 07:12:48 -04:00
Patrick Cloke 1c9a850562
Add type hints to the crypto module. (#8999) 2021-01-04 10:04:50 -05:00
Patrick Cloke 30fba62108
Apply an IP range blacklist to push and key revocation requests. (#8821)
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
2020-12-02 11:09:24 -05:00
Richard van der Hoff fec6f9ac17
Fix occasional "Re-starting finished log context" from keyring (#8398)
* Fix test_verify_json_objects_for_server_awaits_previous_requests

It turns out that this wasn't really testing what it thought it was testing
(in particular, `check_context` was turning failures into success, which was
making the tests pass even though it wasn't clear they should have been.

It was also somewhat overcomplex - we can test what it was trying to test
without mocking out perspectives servers.

* Fix warnings about finished logcontexts in the keyring

We need to make sure that we finish the key fetching magic before we run the
verifying code, to ensure that we don't mess up our logcontexts.
2020-09-25 12:29:54 +01:00
Patrick Cloke 8a4a4186de
Simplify super() calls to Python 3 syntax. (#8344)
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Patrick Cloke 76c43f086a
Do not assume calls to runInteraction return Deferreds. (#8133) 2020-08-20 06:39:55 -04:00
Patrick Cloke 2a89ce8cd4
Convert the crypto module to async/await. (#8003) 2020-08-03 08:29:01 -04:00
Patrick Cloke c978f6c451
Convert federation client to async/await. (#7975) 2020-07-30 08:01:33 -04:00
Dagfinn Ilmari Mannsåker a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Richard van der Hoff 39230d2171
Clean up some LoggingContext stuff (#7120)
* Pull Sentinel out of LoggingContext

... and drop a few unnecessary references to it

* Factor out LoggingContext.current_context

move `current_context` and `set_context` out to top-level functions.

Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.

* move copy-to-parent into `stop`

this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.

* Replace `LoggingContext.alive` with `finished`

Turn `alive` into `finished` and make it a bit better defined.
2020-03-24 14:45:33 +00:00
Patrick Cloke 509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Richard van der Hoff 4b36b482e0
Fix exception when fetching notary server's old keys (#6625)
Lift the restriction that *all* the keys used for signing v2 key responses be
present in verify_keys.

Fixes #6596.
2020-01-06 12:33:56 +00:00
Erik Johnston f5b50d0871
Merge pull request #5895 from matrix-org/erikj/notary_key
Add config option to sign remote key query responses with a separate key.
2019-08-27 11:51:37 +01:00
Erik Johnston 7af5a63063 Fixup review comments 2019-08-23 15:36:28 +01:00
Richard van der Hoff ef1c524bb3
Improve error msg when key-fetch fails (#5896)
There's no point doing a raise_from here, because the exception is always
logged at warn with no stacktrace in the caller. Instead, let's try to give
better messages to reduce confusion.

In particular, this means that we won't log 'Failed to connect to remote
server' when we don't even attempt to connect to the remote server due to
blacklisting.
2019-08-22 10:42:06 +01:00
Erik Johnston 97cbc96093 Only sign when we respond to remote key requests 2019-08-21 11:21:58 +01:00
Erik Johnston 5906be8589 Add config option for keys to use to sign keys
This allows servers to separate keys that are used to sign remote keys
when acting as a notary server.
2019-08-21 10:44:58 +01:00
Amber Brown 4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Richard van der Hoff dcca56baba Add a delay to key lookup lock release to fix stack overflow
A tactical call_later here should fix #5723
2019-07-19 17:57:00 +01:00
Richard van der Hoff c7095be913 Refactor Keyring._start_key_lookups
There's an awful lot of deferreds and dictionaries flying around here. The
whole thing can be made much simpler and achieve the same effect.
2019-07-19 17:49:19 +01:00
Amber Brown 463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Richard van der Hoff 88d7182ada
Improve startup checks for insecure notary configs (#5392)
It's not really a problem to trust notary responses signed by the old key so
long as we are also doing TLS validation.

This commit adds a check to the config parsing code at startup to check that
we do not have the insecure matrix.org key without tls validation, and refuses
to start without it.

This allows us to remove the rather alarming-looking warning which happens at
runtime.
2019-06-10 10:33:00 +01:00
Richard van der Hoff 9fbb20a531
Stop hardcoding trust of old matrix.org key (#5374)
There are a few changes going on here:

* We make checking the signature on a key server response optional: if no
  verify_keys are specified, we trust to TLS to validate the connection.

* We change the default config so that it does not require responses to be
  signed by the old key.

* We replace the old 'perspectives' config with 'trusted_key_servers', which
  is also formatted slightly differently.

* We emit a warning to the logs every time we trust a key server response
  signed by the old key.
2019-06-06 17:33:11 +01:00
Richard van der Hoff cea9750d11 Associate a request_name with each verify request, for logging
Also:
* rename VerifyKeyRequest->VerifyJsonRequest
* calculate key_ids on VerifyJsonRequest construction
* refactor things to pass around VerifyJsonRequests instead of 4-tuples
2019-06-05 10:46:26 +01:00
Richard van der Hoff 2615c6bd9e
Clean up debug logging (#5347)
Remove some spurious stuff, clarify some other stuff
2019-06-05 10:35:40 +01:00
Richard van der Hoff 016af01598
Rename VerifyKeyRequest.deferred field (#5343)
it's a bit confusing
2019-06-05 10:35:13 +01:00
Richard van der Hoff 4d08b8f30c
Don't do long retries when calling the key notary server. (#5334)
It takes at least 20 minutes to work through the long_retries schedule (11
attempts, each with a 60 second timeout, and 60 seconds between each request),
so if the notary server isn't returning within the timeout, we'll just end up
blocking whatever request is happening for 20 minutes.

Ain't nobody got time for that.
2019-06-04 11:53:07 +01:00
Richard van der Hoff c5d60eadd5 Notary server: make requests to origins in parallel
... else we're guaranteed to time out.
2019-06-04 00:16:56 +01:00
Richard van der Hoff def5ea4062 Don't bomb out on direct key fetches as soon as one fails 2019-06-04 00:16:56 +01:00
Richard van der Hoff 06a1f3e207 Reduce timeout for outbound /key/v2/server requests. 2019-06-03 23:17:38 +01:00
Richard van der Hoff fec2dcb1a5
Enforce validity period on server_keys for fed requests. (#5321)
When handling incoming federation requests, make sure that we have an
up-to-date copy of the signing key.

We do not yet enforce the validity period for event signatures.
2019-06-03 22:59:51 +01:00
Richard van der Hoff 862b2f9ad5
Merge pull request #5307 from matrix-org/rav/server_keys/07-fix-notary-cache-poison
Stop overwriting server keys with other keys
2019-06-03 13:19:20 +01:00
Richard van der Hoff 3600f5568b Stop overwriting server keys with other keys
Fix a bug where we would discard a key result which the origin server is no
longer returning. Fixes #5305.
2019-05-31 15:58:35 +01:00
Richard van der Hoff c605da97bf Merge remote-tracking branch 'origin/develop' into rav/server_keys/05-rewrite-gsvk-again 2019-05-31 11:38:13 +01:00
Richard van der Hoff 8ea2f756a9 Remove some pointless exception handling
The verify_request deferred already returns a suitable SynapseError, so I don't
really know what we expect to achieve by doing more wrapping, other than log
spam.

Fixes #4278.
2019-05-30 18:29:56 +01:00
Richard van der Hoff a82c96b87f Rewrite get_server_verify_keys, again.
Attempt to simplify the logic in get_server_verify_keys by splitting it into
two methods.
2019-05-30 18:20:40 +01:00
Richard van der Hoff 099829d5a9 use attr.s for VerifyKeyRequest
because namedtuple is awful
2019-05-30 17:39:28 +01:00