Commit Graph

36 Commits (2d3b4f42f029da54132b26119e9ec0d902505eef)

Author SHA1 Message Date
Erik Johnston 1e4b4d85e7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/reliable_lookups 2019-08-22 13:41:57 +01:00
Erik Johnston 7777d353bf Remove test debugs 2019-08-20 11:46:59 +01:00
Erik Johnston 1dec31560e Change jitter to be a factor rather than absolute value 2019-08-20 11:46:00 +01:00
Erik Johnston f299c5414c Refactor MatrixFederationAgent to retry SRV.
This refactors MatrixFederationAgent to move the SRV lookup into the
endpoint code, this has two benefits:
	1. Its easier to retry different host/ports in the same way as
	   HostnameEndpoint.
	2. We avoid SRV lookups if we have a free connection in the pool
2019-08-15 15:43:22 +01:00
Erik Johnston e6e136decc Retry well known on fail.
If we have recently seen a valid well-known for a domain we want to
retry on (non-final) errors a few times, to handle temporary blips in
networking/etc.
2019-08-15 09:28:58 +01:00
Erik Johnston 17e1e80726 Retry well-known lookup before expiry.
This gives a bit of a grace period where we can attempt to refetch a
remote `well-known`, while still using the cached result if that fails.

Hopefully this will make the well-known resolution a bit more torelant
of failures, rather than it immediately treating failures as "no result"
and caching that for an hour.
2019-08-13 16:20:38 +01:00
Erik Johnston 107ad133fc Move well known lookup into a separate clas 2019-08-07 15:36:38 +01:00
Erik Johnston af9f1c0764 Add a lower bound for TTL on well known results.
It costs both us and the remote server for us to fetch the well known
for every single request we send, so we add a minimum cache period. This
is set to 5m so that we still honour the basic premise of "refetch
frequently".
2019-08-06 17:01:23 +01:00
Erik Johnston a9bcae9f50 Share SSL options for well-known requests 2019-07-31 10:39:24 +01:00
Amber Brown 4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Amber Brown 463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Richard van der Hoff c3c6b00d95
Pass config_dir_path and data_dir_path into Config.read_config. (#5522)
* Pull config_dir_path and data_dir_path calculation out of read_config_files

* Pass config_dir_path and data_dir_path into read_config
2019-06-24 11:34:45 +01:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Richard van der Hoff e016681221 Tests for SSL certs for federation connections
Add some tests for bad certificates for federation and .well-known connections
2019-06-10 16:14:20 +01:00
Richard van der Hoff 9fbb20a531
Stop hardcoding trust of old matrix.org key (#5374)
There are a few changes going on here:

* We make checking the signature on a key server response optional: if no
  verify_keys are specified, we trust to TLS to validate the connection.

* We change the default config so that it does not require responses to be
  signed by the old key.

* We replace the old 'perspectives' config with 'trusted_key_servers', which
  is also formatted slightly differently.

* We emit a warning to the logs every time we trust a key server response
  signed by the old key.
2019-06-06 17:33:11 +01:00
Richard van der Hoff e2dfb922e1 Validate federation server TLS certificates by default. 2019-06-05 14:17:50 +01:00
Amber Brown df2ebd75d3
Migrate all tests to use the dict-based config format instead of hanging items off HomeserverConfig (#5171) 2019-05-13 15:01:14 -05:00
Amber Brown b36c82576e
Run Black on the tests again (#5170) 2019-05-10 00:12:11 -05:00
Andrew Morgan 6824ddd93d Config option for verifying federation certificates (MSC 1711) (#4967) 2019-04-25 14:22:49 +01:00
Richard van der Hoff 8a21b03fba Treat an invalid .well-known the same as an absent one
... basically, carry on and fall back to SRV etc.
2019-02-01 11:37:31 +00:00
Richard van der Hoff d428b46346 Update federation routing logic to check .well-known before SRV 2019-01-31 23:14:18 +00:00
Richard van der Hoff c7b24ac3d0
Follow redirects on .well-known (#4520) 2019-01-30 11:43:33 +00:00
Richard van der Hoff bc5f6e1797
Add a caching layer to .well-known responses (#4516) 2019-01-30 10:55:25 +00:00
Richard van der Hoff cc2d650ef7
Relax requirement for a content-type on .well-known (#4511) 2019-01-29 16:49:17 +00:00
Richard van der Hoff 99e36d5e24
Implement MSC1708 (.well-known lookups for server routing) (#4489) 2019-01-29 13:53:02 +00:00
Richard van der Hoff f2b553d656 Use SimpleResolverComplexifier in tests (#4497)
two reasons for this. One, it saves a bunch of boilerplate. Two, it squashes
unicode to IDNA-in-a-`str` (even on python 3) in a way that it turns out we
rely on to give consistent behaviour between python 2 and 3.
2019-01-29 09:38:29 +00:00
Richard van der Hoff 0fd5b3b53e Handle IP literals explicitly
We don't want to be doing .well-known lookups on these guys.
2019-01-28 10:34:30 +00:00
Richard van der Hoff d840019192
Fix idna and ipv6 literal handling in MatrixFederationAgent (#4487)
Turns out that the library does a better job of parsing URIs than our
reinvented wheel. Who knew.

There are two things going on here. The first is that, unlike
parse_server_name, URI.fromBytes will strip off square brackets from IPv6
literals, which means that it is valid input to ClientTLSOptionsFactory and
HostnameEndpoint.

The second is that we stay in `bytes` throughout (except for the argument to
ClientTLSOptionsFactory), which avoids the weirdness of (sometimes) ending up
with idna-encoded values being held in `unicode` variables. TBH it probably
would have been ok but it made the tests fragile.
2019-01-28 09:56:59 +00:00
Richard van der Hoff 8520bc3109
Fix Host header sent by MatrixFederationAgent (#4468)
Move the Host header logic down here so that (a) it is used if we reuse the
agent elsewhere, and (b) we can mess about with it with .well-known.
2019-01-25 12:38:16 +00:00
Richard van der Hoff afd69a0920 Look up the right SRV record 2019-01-24 13:31:43 +00:00
Richard van der Hoff e1c8440e0c lots more tests for MatrixFederationAgent 2019-01-24 13:28:07 +00:00
Richard van der Hoff 97fd29c019
Don't send IP addresses as SNI (#4452)
The problem here is that we have cut-and-pasted an impl from Twisted, and then
failed to maintain it. It was fixed in Twisted in
https://github.com/twisted/twisted/pull/1047/files; let's do the same here.
2019-01-24 09:34:44 +00:00
Richard van der Hoff d02c4532c0 Add a test for MatrixFederationAgent 2019-01-22 20:35:12 +00:00
Richard van der Hoff 7021784d46 put resolve_service in an object
this makes it easier to stub things out for tests.
2019-01-22 20:35:12 +00:00
Richard van der Hoff 53a327b4d5 Require that service_name be a byte string
it is only ever a bytes now, so let's enforce that.
2019-01-22 20:35:12 +00:00
Richard van der Hoff 33a55289cb
Refactor and bugfix for resove_service (#4427) 2019-01-22 10:59:27 +00:00