Commit Graph

236 Commits (898196f1cca419c0d2b60529c86ddff3cea83072)

Author SHA1 Message Date
Richard van der Hoff fec6f9ac17
Fix occasional "Re-starting finished log context" from keyring (#8398)
* Fix test_verify_json_objects_for_server_awaits_previous_requests

It turns out that this wasn't really testing what it thought it was testing
(in particular, `check_context` was turning failures into success, which was
making the tests pass even though it wasn't clear they should have been.

It was also somewhat overcomplex - we can test what it was trying to test
without mocking out perspectives servers.

* Fix warnings about finished logcontexts in the keyring

We need to make sure that we finish the key fetching magic before we run the
verifying code, to ensure that we don't mess up our logcontexts.
2020-09-25 12:29:54 +01:00
Richard van der Hoff 4f3096d866 Add a comment re #1691 2020-09-21 12:34:06 +01:00
Patrick Cloke 8a4a4186de
Simplify super() calls to Python 3 syntax. (#8344)
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Patrick Cloke 76c43f086a
Do not assume calls to runInteraction return Deferreds. (#8133) 2020-08-20 06:39:55 -04:00
Richard van der Hoff de6f892065
Add a comment about SSLv23_METHOD (#8043) 2020-08-07 15:14:29 +01:00
Patrick Cloke 2a89ce8cd4
Convert the crypto module to async/await. (#8003) 2020-08-03 08:29:01 -04:00
Patrick Cloke c978f6c451
Convert federation client to async/await. (#7975) 2020-07-30 08:01:33 -04:00
Dagfinn Ilmari Mannsåker a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Richard van der Hoff 39230d2171
Clean up some LoggingContext stuff (#7120)
* Pull Sentinel out of LoggingContext

... and drop a few unnecessary references to it

* Factor out LoggingContext.current_context

move `current_context` and `set_context` out to top-level functions.

Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.

* move copy-to-parent into `stop`

this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.

* Replace `LoggingContext.alive` with `finished`

Turn `alive` into `finished` and make it a bit better defined.
2020-03-24 14:45:33 +00:00
Richard van der Hoff c37db0211e
Share SSL contexts for non-federation requests (#7094)
Extends #5794 etc to the SimpleHttpClient so that it also applies to non-federation requests.

Fixes #7092.
2020-03-17 21:32:25 +00:00
Patrick Cloke 06eb5cae08
Remove special auth and redaction rules for aliases events in experimental room ver. (#7037) 2020-03-09 08:58:25 -04:00
Patrick Cloke 509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Richard van der Hoff 7f93eb1903
pass room_version into compute_event_signature (#6807) 2020-01-31 13:47:43 +00:00
Richard van der Hoff 2a81393a4b Pass room_version into add_hashes_and_signatures 2020-01-30 22:15:50 +00:00
Richard van der Hoff ba897a7590
Fix some test failures when frozen_dicts are enabled (#6642)
Fixes #4026
2020-01-06 15:22:46 +00:00
Richard van der Hoff 4b36b482e0
Fix exception when fetching notary server's old keys (#6625)
Lift the restriction that *all* the keys used for signing v2 key responses be
present in verify_keys.

Fixes #6596.
2020-01-06 12:33:56 +00:00
Michael Kaye f85b9842f0 Don't encode object as UTF-8 string if not needed.
I believe that string formatting ~10-15 sized events will
take a proportion of CPU time.
2019-10-24 18:08:45 +01:00
Amber Brown 850dcfd2d3
Fix well-known lookups with the federation certificate whitelist (#5997) 2019-09-14 04:58:38 +10:00
Andrew Morgan 4548d1f87e
Remove unnecessary parentheses around return statements (#5931)
Python will return a tuple whether there are parentheses around the returned values or not.

I'm just sick of my editor complaining about this all over the place :)
2019-08-30 16:28:26 +01:00
Erik Johnston f5b50d0871
Merge pull request #5895 from matrix-org/erikj/notary_key
Add config option to sign remote key query responses with a separate key.
2019-08-27 11:51:37 +01:00
Erik Johnston 7af5a63063 Fixup review comments 2019-08-23 15:36:28 +01:00
Richard van der Hoff ef1c524bb3
Improve error msg when key-fetch fails (#5896)
There's no point doing a raise_from here, because the exception is always
logged at warn with no stacktrace in the caller. Instead, let's try to give
better messages to reduce confusion.

In particular, this means that we won't log 'Failed to connect to remote
server' when we don't even attempt to connect to the remote server due to
blacklisting.
2019-08-22 10:42:06 +01:00
Erik Johnston 97cbc96093 Only sign when we respond to remote key requests 2019-08-21 11:21:58 +01:00
Erik Johnston 5906be8589 Add config option for keys to use to sign keys
This allows servers to separate keys that are used to sign remote keys
when acting as a notary server.
2019-08-21 10:44:58 +01:00
Erik Johnston a9bcae9f50 Share SSL options for well-known requests 2019-07-31 10:39:24 +01:00
Amber Brown 4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Richard van der Hoff dcca56baba Add a delay to key lookup lock release to fix stack overflow
A tactical call_later here should fix #5723
2019-07-19 17:57:00 +01:00
Richard van der Hoff c7095be913 Refactor Keyring._start_key_lookups
There's an awful lot of deferreds and dictionaries flying around here. The
whole thing can be made much simpler and achieve the same effect.
2019-07-19 17:49:19 +01:00
Amber Brown 463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Amber Brown be3b901ccd
Update the TLS cipher string and provide configurability for TLS on outgoing federation (#5550) 2019-06-28 18:19:09 +10:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Erik Johnston a6b1817940
Merge pull request #5417 from matrix-org/rav/shared_ssl_context
Share an SSL context object between SSL connections
2019-06-10 18:16:12 +01:00
Richard van der Hoff 81b8fdedf2 rename gutwrenched attr 2019-06-10 17:51:11 +01:00
Richard van der Hoff efe7b3176e Fix federation connections to literal IP addresses
turns out we need a shiny version of service_identity to enforce this
correctly.
2019-06-10 15:58:35 +01:00
Richard van der Hoff d11c634ced clean up impl, and import idna directly 2019-06-10 15:55:12 +01:00
Richard van der Hoff 88d7182ada
Improve startup checks for insecure notary configs (#5392)
It's not really a problem to trust notary responses signed by the old key so
long as we are also doing TLS validation.

This commit adds a check to the config parsing code at startup to check that
we do not have the insecure matrix.org key without tls validation, and refuses
to start without it.

This allows us to remove the rather alarming-looking warning which happens at
runtime.
2019-06-10 10:33:00 +01:00
Richard van der Hoff c2b6e945e1 Share an SSL context object between SSL connections
This involves changing how the info callbacks work.
2019-06-09 14:01:32 +01:00
Richard van der Hoff 9fbb20a531
Stop hardcoding trust of old matrix.org key (#5374)
There are a few changes going on here:

* We make checking the signature on a key server response optional: if no
  verify_keys are specified, we trust to TLS to validate the connection.

* We change the default config so that it does not require responses to be
  signed by the old key.

* We replace the old 'perspectives' config with 'trusted_key_servers', which
  is also formatted slightly differently.

* We emit a warning to the logs every time we trust a key server response
  signed by the old key.
2019-06-06 17:33:11 +01:00
Richard van der Hoff cea9750d11 Associate a request_name with each verify request, for logging
Also:
* rename VerifyKeyRequest->VerifyJsonRequest
* calculate key_ids on VerifyJsonRequest construction
* refactor things to pass around VerifyJsonRequests instead of 4-tuples
2019-06-05 10:46:26 +01:00
Richard van der Hoff 2615c6bd9e
Clean up debug logging (#5347)
Remove some spurious stuff, clarify some other stuff
2019-06-05 10:35:40 +01:00
Richard van der Hoff 016af01598
Rename VerifyKeyRequest.deferred field (#5343)
it's a bit confusing
2019-06-05 10:35:13 +01:00
Richard van der Hoff 4d08b8f30c
Don't do long retries when calling the key notary server. (#5334)
It takes at least 20 minutes to work through the long_retries schedule (11
attempts, each with a 60 second timeout, and 60 seconds between each request),
so if the notary server isn't returning within the timeout, we'll just end up
blocking whatever request is happening for 20 minutes.

Ain't nobody got time for that.
2019-06-04 11:53:07 +01:00
Richard van der Hoff c5d60eadd5 Notary server: make requests to origins in parallel
... else we're guaranteed to time out.
2019-06-04 00:16:56 +01:00
Richard van der Hoff def5ea4062 Don't bomb out on direct key fetches as soon as one fails 2019-06-04 00:16:56 +01:00
Richard van der Hoff 06a1f3e207 Reduce timeout for outbound /key/v2/server requests. 2019-06-03 23:17:38 +01:00
Richard van der Hoff fec2dcb1a5
Enforce validity period on server_keys for fed requests. (#5321)
When handling incoming federation requests, make sure that we have an
up-to-date copy of the signing key.

We do not yet enforce the validity period for event signatures.
2019-06-03 22:59:51 +01:00
Richard van der Hoff 862b2f9ad5
Merge pull request #5307 from matrix-org/rav/server_keys/07-fix-notary-cache-poison
Stop overwriting server keys with other keys
2019-06-03 13:19:20 +01:00
Richard van der Hoff 3600f5568b Stop overwriting server keys with other keys
Fix a bug where we would discard a key result which the origin server is no
longer returning. Fixes #5305.
2019-05-31 15:58:35 +01:00
Richard van der Hoff c605da97bf Merge remote-tracking branch 'origin/develop' into rav/server_keys/05-rewrite-gsvk-again 2019-05-31 11:38:13 +01:00