e1625d62a8
Add federation room list servlet
2016-05-31 11:55:57 +01:00
f6ebaf4a32
Run transaction queue on reactor
...
This ensures that any CPU work that happens doesn't block message
sending.
2016-05-09 10:10:06 +01:00
9c272da05f
Add an openidish mechanism for proving to third parties that you own a given user_id
2016-05-05 13:42:44 +01:00
62b51b8452
Fix typo in event_auth servlet path
2016-04-29 12:00:51 +01:00
914f1eafac
Lower timeout for make_membership_event
...
Calls to make_membership_event are done in response to client requests,
and so should not be retried over long timeframes.
2016-04-15 11:22:23 +01:00
acdfef7b14
Intern all the things
2016-03-23 16:25:54 +00:00
5244c0b48e
Remove unused backfilled parameter from persist_event
2016-03-21 18:06:08 +00:00
9adf0e92bc
Catch exceptions from EDU handling
2016-03-18 15:12:50 +00:00
3c5f25507b
Yield on EDU handling
2016-03-18 13:55:16 +00:00
e9c1cabac2
Use parse_json_object_from_request to parse JSON out of request bodies
2016-03-11 16:41:03 +00:00
ed61a49169
Add profile information to invites
2016-03-04 14:35:02 +00:00
577951b032
Allow third_party_signed to be specified on /join
2016-02-23 15:11:25 +00:00
2c1fbea531
Fix up logcontexts
2016-02-08 14:26:45 +00:00
d83d004ccd
Fix flake8 warnings for new flake8
2016-02-02 17:18:50 +00:00
9959d9ece8
Remove redundated BaseHomeServer
2016-01-26 13:52:29 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
14d7acfad4
Host /unstable and /r0 versions of r0 APIs
2015-12-01 17:34:32 +00:00
cf4ef5f3c7
Only retry federation requests for a long time for background requests
2015-11-17 18:26:50 +00:00
aca6e5bf46
Don't complain if /make_join response lacks 'prev_state' list (SYN-517)
2015-11-13 17:27:25 +00:00
c0b3554401
Fix missing profile data in federation joins
...
There was a regression where we stopped including profile data in
initial joins for rooms joined over federation.
2015-11-12 16:19:55 +00:00
2cebe53545
Exchange 3pid invites for m.room.member invites
2015-11-05 16:43:19 +00:00
97d792b28f
Don't rearrange transaction_queue
2015-11-03 16:31:08 +00:00
7ce264ce5f
Fix broken cache for getting retry times. This meant we retried remote destinations way more frequently than we should
2015-11-03 16:24:03 +00:00
137fafce4e
Allow rejecting invites
...
This is done by using the same /leave flow as you would use if you had
already accepted the invite and wanted to leave.
2015-10-20 11:58:58 +01:00
643b5fcdc8
Look for keys on the right objects
2015-10-15 13:10:30 +01:00
0c38e8637f
Remove unnecessary class-wrapping
2015-10-13 18:00:38 +01:00
17dffef5ec
Move event contents into third_party_layout field
2015-10-13 15:48:12 +01:00
fcd9ba8802
Fix lint errors
2015-10-06 10:13:05 -05:00
5b3e9713dd
Implement third party identifier invites
2015-10-01 17:49:52 +01:00
8899df13bf
Merge pull request #208 from matrix-org/markjh/end-to-end-key-federation
...
Federation for end-to-end key requests.
2015-08-18 09:12:54 +01:00
0cceb2ac92
Add a few strategic new lines to break up the on_query_client_keys and on_claim_client_keys methods in federation_server.py
2015-08-13 17:27:46 +01:00
2df8dd9b37
Move all the caches into their own package, synapse.util.caches
2015-08-11 18:00:59 +01:00
2da3b1e60b
Get the end-to-end key federation working
2015-07-24 18:26:46 +01:00
62c010283d
Add federation support for end-to-end key requests
2015-07-23 16:03:38 +01:00
b5f55a1d85
Implement bulk verify_signed_json API
2015-06-26 10:39:34 +01:00
cee69441d3
Log more when we have processed the request
2015-06-15 17:11:44 +01:00
8047fd2434
Merge pull request #176 from matrix-org/erikj/backfill_auth
...
Improve backfill.
2015-06-03 17:25:37 +01:00
3483b78d1a
Log where a request came from in federation
2015-06-02 18:15:13 +01:00
d3ded420b1
Rephrase log line
2015-06-02 16:30:52 +01:00
22716774d5
Don't about JSON when warning about content tampering
2015-06-02 16:30:52 +01:00
6eadbfbea0
Remove redundant for loop
2015-05-22 16:12:20 +01:00
74b7de83ec
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/backfill_fixes
2015-05-22 16:10:42 +01:00
36317f3dad
Merge pull request #156 from matrix-org/erikj/join_perf
...
Make joining #matrix:matrix.org over federation quicker
2015-05-22 16:09:54 +01:00
284f55a7fb
Add doc strings
2015-05-22 15:18:04 +01:00
b21d015c55
Log origin and stats of incoming transactions
2015-05-22 14:44:25 +01:00
e70e8e053e
Add txn_id to some log lines
2015-05-22 14:44:02 +01:00
1b446a5d85
Log less lines at INFO level, but include more helpful information
2015-05-22 14:29:57 +01:00
c2633907c5
Merge branch 'erikj/join_perf' of github.com:matrix-org/synapse into erikj/backfill_fixes
2015-05-21 14:58:47 +01:00
ebfdd2eb5b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf
2015-05-21 14:54:52 +01:00
20814fabdd
Actually fetch state for new backwards extremeties when backfilling.
2015-05-20 11:59:02 +01:00
3a653515ec
Add None check
2015-05-19 15:27:09 +01:00
5b1631a4a9
Add a timeout param to get_event
2015-05-19 14:53:32 +01:00
d5cea26d45
Remove pointless newline
2015-05-18 10:16:45 +01:00
c71176858b
Newline, remove debug logging
2015-05-18 10:11:14 +01:00
f8bd4de87d
Remove debug logging
2015-05-18 09:58:03 +01:00
a2c4f3f150
Fix daedlock
2015-05-15 10:54:04 +01:00
95dedb866f
Unwrap defer.gatherResults failures
2015-05-12 13:14:29 +01:00
476899295f
Change the way we do logging contexts so that they survive divergences
2015-05-08 16:32:18 +01:00
0cf7e480b4
And use buffer(...) there as well
2015-05-05 18:20:01 +01:00
977338a7af
Use buffer(...) when inserting into bytea column
2015-05-05 18:12:53 +01:00
3d5a955e08
Missed events are not outliers
2015-05-05 17:36:57 +01:00
1942382246
Don't log enqueue_
2015-05-01 16:14:25 +01:00
23c639ff32
Split a storage function in two so that we don't have to do extra work.
2015-05-01 10:17:19 +01:00
c178e4e6ca
Add missing servlet to list
2015-03-19 17:48:21 +00:00
128cf2daf7
Appease pep8
2015-03-12 16:24:51 +00:00
c782e893ec
Neater metrics from TransactionQueue
2015-03-12 16:24:51 +00:00
2e4f0b2bd7
Replace the @metrics.counted annotations in federation with specifically-written counters and distributions
2015-03-12 16:24:51 +00:00
1748605c5d
Count incoming HTTP requests per servlet that responds
2015-03-12 16:24:51 +00:00
b0cf867319
Use _ instead of . as a metric namespacing separator, for Prometheus
2015-03-12 16:24:51 +00:00
f9478e475b
Rename Metrics' "keys" to "labels"
2015-03-12 16:24:51 +00:00
094803cf82
Put vector gauges on transaction queue pending PDU and EDU dicts
2015-03-12 16:24:50 +00:00
9470412316
Initial attempt at sprinkling some @metrics.counted decorations around the federation code
2015-03-12 16:24:50 +00:00
120b689284
Delete pointless (and unreachable) __init__ method from FederationClient
2015-03-12 16:24:50 +00:00
abaf47bbb6
Fix bug in logging.
2015-03-10 10:28:29 +00:00
98b867f7b7
Fix bug in logging.
2015-03-10 10:16:09 +00:00
019422ebba
Merge pull request #101 from matrix-org/neaten-federation-servlets
...
Neaten federation servlets
2015-03-09 17:39:06 +00:00
d79d91a4a7
Appease pep8
2015-03-05 20:55:40 +00:00
5eab2549ab
Append a $ on PATH at registration time, meaning each PATH attribute doesn't need it
2015-03-05 20:36:05 +00:00
7644cb79b2
Slightly neater(?) arrangement of authentication wrapper for HTTP servlet methods
2015-03-05 20:33:16 +00:00
ba8ac996f9
Remove the dead 'rate_limit_origin' method from TransportLayerServer
2015-03-05 19:43:17 +00:00
a901ed16b5
Move federation API responding code out of weird mix of lambdas into Servlet-style methods on instances
2015-03-05 19:10:57 +00:00
9708f49abf
Docs
2015-03-05 16:35:16 +00:00
96fee64421
Remove unecessary check
2015-03-05 16:31:47 +00:00
39aa968a76
Respect min_depth argument
2015-03-05 16:31:32 +00:00
6dfd8c73fc
Docs.
2015-03-05 16:31:13 +00:00
ae702d161a
Handle if get_missing_pdu returns 400 or not all events.
2015-03-05 16:08:02 +00:00
0a036944bd
Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu
2015-03-02 13:53:30 +00:00
4195e55ccc
Merge branch 'develop' of github.com:matrix-org/synapse into federation_rate_limit
2015-03-02 13:39:22 +00:00
29481690c5
If we're yielding don't add errback
2015-03-02 11:50:43 +00:00
23d9bd1d74
Process transactions serially.
...
Since the events received in a transaction are ordered, later events
might depend on earlier events and so we shouldn't blindly process them
in parellel.
2015-03-02 11:39:57 +00:00
9d9b230501
Make the federation server ratelimiting configurable.
2015-03-02 11:33:45 +00:00
0554d07082
Move federation rate limiting out of transport layer
2015-02-27 15:41:52 +00:00
9dc9118e55
Document FederationRateLimiter
2015-02-27 15:16:47 +00:00
93d90765c4
Initial implementation of federation server rate limiting
2015-02-26 16:15:26 +00:00
59362454dd
Must update pending_transactions map before yield'ing
2015-02-26 15:47:35 +00:00
db215b7e00
Implement and use new batched get missing pdu
2015-02-23 13:58:02 +00:00
0ac2a79faa
Initial stab at implementing a batched get_missing_pdus request
2015-02-19 17:24:14 +00:00
a7925259a1
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.7.1
2015-02-18 13:57:55 +00:00
446ef58992
Add errback to all deferreds in transaction_queue
2015-02-18 12:03:26 +00:00
2462aacd77
Restrict the destinations that synapse can talk to
2015-02-18 11:52:51 +00:00
b68e4a729f
Discard destination 'localhost'
2015-02-18 11:32:39 +00:00
47d3ff4cf8
Don't send failure to self
2015-02-18 11:30:37 +00:00
e482541e1d
Fix pyflakes
2015-02-18 10:44:22 +00:00
1be67eca8a
Merge branch 'keyclient_retry_scheme' of github.com:matrix-org/synapse into develop
2015-02-18 10:34:40 +00:00
fe672a04f7
Merge pull request #77 from matrix-org/failures
...
Failures
2015-02-18 10:29:29 +00:00
08f804208b
Merge pull request #79 from matrix-org/get_pdu_limiting
...
Get pdu limiting
2015-02-18 10:29:10 +00:00
ec847059f3
Rename _fail_fetch_pdu_cache to _get_pdu_cache
2015-02-18 10:14:10 +00:00
9371019133
Try to only back off if we think we failed to connect to the remote
2015-02-17 18:13:34 +00:00
2b8f1a956c
Add per server retry limiting.
...
Factor out the pre destination retry logic from TransactionQueue so it
can be reused in both get_pdu and crypto.keyring
2015-02-17 17:20:56 +00:00
676e8ee78a
Remove debug raise
2015-02-17 15:22:45 +00:00
fa6c93bd26
Merge branch 'consumeErrors' of github.com:matrix-org/synapse into develop
2015-02-17 15:18:17 +00:00
c02da58a9d
Merge branch 'develop' of github.com:matrix-org/synapse into failures
2015-02-17 15:15:07 +00:00
659ead082f
Format the response of transaction request in a nicer way
2015-02-17 15:11:44 +00:00
c82e26ad4b
Actually respond with JSON to incoming transaction
2015-02-17 13:24:13 +00:00
02bfa889de
Handle recieving failures in transactions
2015-02-17 13:13:14 +00:00
72a4de2ce6
Use consumeErrors=True on all DeferredLists.
...
This is so that the DeferredLists actually consume the error instead of
propogating down the non-existent errback chain. This should reduce the
number of unhandled errors we are seeing.
2015-02-17 10:07:01 +00:00
0194e71e99
Merge branch 'develop' of github.com:matrix-org/synapse into get_pdu_limiting
2015-02-17 09:48:23 +00:00
baa5b9a975
Cache results of get_pdu.
2015-02-16 18:02:39 +00:00
91fc5eef1d
Mark old events as outliers.
...
This is to fix the issue where if a remote server sends an event
that references a really "old" event, then the local server will pull
that in and send to all clients.
We decide if an event is old if its depth is less than the minimum depth
of the room.
2015-02-16 14:27:40 +00:00
6138584651
Don't return anything from _handle_new_pdu, since we ignore the return value anyway
2015-02-16 14:08:02 +00:00
789251afa7
Fix logging
2015-02-12 19:29:43 +00:00
58d848adc0
Parrellize fetching of events
2015-02-12 18:35:36 +00:00
963256638d
Correctly handle all the places that can throw exceptions
2015-02-12 18:17:11 +00:00
c52e8d395b
Merge pull request #61 from matrix-org/timeout-federation-requests
...
Timeout federation requests
2015-02-11 17:10:33 +00:00
ef276e8770
Fix so timing out connections to actually work.
2015-02-11 16:48:05 +00:00
ddb816cf60
Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict
2015-02-11 15:44:28 +00:00
4ebbaf0d43
Blunty replace json with simplejson
2015-02-11 14:23:10 +00:00
c8e1da930d
Log all the exits from _attempt_new_transaction
2015-02-10 17:30:46 +00:00
697ab75a34
Sign auth_chains when returned by /state/ requests
2015-02-10 15:46:24 +00:00
e9c85a4d5a
Connection errors in twisted aren't RuntimeErrors
2015-02-05 13:50:15 +00:00
e1515c3e91
Pass through list of room hosts from room alias query to federation so that it can retry against different room hosts
2015-02-05 13:44:42 +00:00
ae46f10fc5
Apply sanity to the transport client interface. Convert 'make_join' and 'send_join' to accept iterables of destinations
2015-02-04 16:28:12 +00:00
ff78eded01
Retry make_join
2015-02-04 13:55:10 +00:00
3c39f42a05
New line
2015-02-03 16:14:19 +00:00
9bace3a367
Actually, the old prune_event function was non-deterministic, so no point keeping it around :(
2015-02-03 15:32:17 +00:00
8dae5c8108
Remove unused imports
2015-02-03 15:01:12 +00:00
7b810e136e
Add new FederationBase
2015-02-03 15:00:42 +00:00
0dd3aea319
Keep around the old (buggy) version of the prune_event function so that we can use it to check signatures for events on old servers
2015-02-03 14:58:30 +00:00
0f48e22ef6
PEP8
2015-02-03 10:43:29 +00:00
40c6fe1b81
Don't bother requesting PDUs with bad signatures from the same server
2015-02-02 17:06:37 +00:00
941f59101b
Don't fail an entire request if one of the returned events fails a signature check. If an event does fail a signature check, look in the local database and request it from the originator.
2015-02-02 16:56:01 +00:00
776ac820f9
Briefly doc structure of query_auth API.
2015-01-30 15:58:28 +00:00
a70a801184
Fix bug where we superfluously asked for current state. Change API of /query_auth/ so that we don't duplicate events in the response.
2015-01-30 13:34:01 +00:00
c1d860870b
Fix regression where we no longer correctly handled the case of gaps in our event graph
2015-01-30 10:48:47 +00:00
78015948a7
Initial implementation of auth conflict resolution
2015-01-29 16:52:33 +00:00
0ef5bfd6a9
Start implementing auth conflict res
2015-01-28 16:16:53 +00:00
c92d64a6c3
Make it the responsibility of the replication layer to check signature and hashes.
2015-01-26 14:33:11 +00:00
7b88619241
Split up replication_layer module into client, server and transaction queue
2015-01-26 10:45:24 +00:00
ca65a9d03e
Split out TransactionQueue from replication layer
2015-01-22 16:37:08 +00:00
5fed042640
Finish renaming "context" to "room_id" in federation codebase
2015-01-16 19:01:03 +00:00
2408c4b0a4
Fold _do_request_for_transaction into the methods that called it since it was a trivial wrapper around client.get_json
2015-01-16 19:01:03 +00:00
602684eac5
Split transport layer into client and server parts
2015-01-16 19:01:03 +00:00
0529a7e2e9
Add some logging for when we are sending transactions.
2015-01-06 14:06:25 +00:00
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
041ac476a5
Supply auth_chain along with current state in '/state/', fetch auth events from a remote server if we are missing some of them
2014-12-18 18:47:13 +00:00
52f99243ab
Use is_outlier() so that we don't get AttributeError
2014-12-16 18:33:50 +00:00
3c77d13aa5
Kill off synapse.api.events.*
2014-12-16 11:29:05 +00:00
aae8a37e63
Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor
2014-12-10 13:18:40 +00:00
95aa903ffa
Try and figure out how and why signatures are being changed.
2014-12-10 11:37:47 +00:00
b8d30899b1
Code style.
2014-12-10 10:16:09 +00:00
71da2bed55
plateau retries after 1h
2014-12-10 00:18:44 +00:00
faf12b64f8
add errbacks to enqueue_pdu deferreds; change logging for failed federation sends to warn rather than exception
2014-12-10 00:12:51 +00:00
2b1acb7671
squidge to 79 columns as per pep8
2014-12-10 00:03:55 +00:00
8529fba02d
fix a million stupid bugs and make it actually work
2014-12-08 19:34:51 +00:00
ee3df06183
More bug fixes
2014-12-08 14:50:48 +00:00
d044121168
Various typos and bug fixes.
2014-12-08 09:08:26 +00:00
9c43b258ec
actually reset retry schedule if we can successfuly talk to it
2014-12-08 00:17:12 +00:00
5cd43d4b9f
fix stupid syntax thinkos
2014-12-07 23:44:16 +00:00
aed62a3583
track replication destination health, and perform exponential back-off when sending transactions. does *not* yet retry transactions, but drops them on the floor if waiting for a server to recover.
2014-12-07 02:26:07 +00:00
c31dba86ec
Convert rest and handlers to use new event structure
2014-12-04 15:50:01 +00:00
5d7c9ab789
Begin converting things to use the new Event structure
2014-12-04 11:27:59 +00:00
d06dfc70b0
Fix bug where did not always resolve all the deferreds in _attempt_new_transaction
2014-12-03 11:50:23 +00:00
07699b5871
Change the way we get missing auth and state events
2014-11-27 14:31:43 +00:00
3598c11c8d
Correctly handle the case where we get an event for an unknown room, which turns out we are actually in
2014-11-26 10:41:08 +00:00
64fc859dac
Fix bugs in invite/join dances.
...
We now do more implement more of the auth on the events so that we
don't reject valid events.
2014-11-25 17:59:49 +00:00
4961a4fab1
Mark the auth events as possible outlier
2014-11-24 13:55:49 +00:00
2bca242fdc
Ask for any auth events that we don't have
2014-11-24 13:46:41 +00:00
4bd0ab76c6
We don't always want to Auth get_persisted_pdu
2014-11-24 12:56:17 +00:00
db9ce032a4
Fix pep8 codestyle warnings
2014-11-20 17:26:36 +00:00
32090aee16
Add a few missing yields, Move deferred lists inside PreserveLoggingContext because they don't interact well with the logging contexts
2014-11-20 16:24:00 +00:00
4eada9a908
Fix backfill request
2014-11-19 17:22:37 +00:00
c5eabe3143
replace user_id with sender
2014-11-19 16:38:40 +00:00
428581dd05
SYN-144: Remove bad keys from pdu json objects, convert age_ts to age
...
for all pdus sent.
2014-11-18 19:20:25 +00:00
db7e8b5619
SYN-141: Decode the query params as UTF-8
2014-11-18 17:17:57 +00:00
cb4b6c844a
Merge PDUs and Events into one object
2014-11-14 21:25:02 +00:00
8d8a133c89
SYN-103: Remove "origin" and "destination" keys from edus
2014-11-13 15:49:03 +00:00
3db0efa69f
Fix pyflake warnings and add a FIXME comment to deal with auth_chains received when joining
2014-11-12 11:27:02 +00:00
6fea478d2e
Fix bugs with invites/joins across federatiom.
...
Both in terms of auth and not trying to fetch missing PDUs for invites,
joins etc.
2014-11-12 11:24:11 +00:00
092979b8cc
Fix bugs which broke federation due to changes in function signatures.
2014-11-11 14:19:13 +00:00
5d439b127b
PEP8
2014-11-10 13:46:44 +00:00
c46088405a
Remove useless comments
2014-11-10 13:39:33 +00:00
003668cfaa
Add auth to the various server-server APIs
2014-11-10 13:37:24 +00:00
6447db063a
Fix backfill to work. Add auth to backfill request
2014-11-10 11:59:51 +00:00
1c06806f90
Finish redaction algorithm.
2014-11-10 10:21:32 +00:00
02c3b1c9e2
Add '/event_auth/' federation api
2014-11-07 15:35:53 +00:00
d2fb2b8095
Implement invite part of invite join dance
2014-11-07 13:41:00 +00:00
328dab2463
Remove /context/ request
2014-11-07 11:40:38 +00:00
3b4dec442d
Return auth chain when handling send_join
2014-11-07 11:22:12 +00:00
a5a4ef3fd7
Fix bug in replication
2014-11-04 15:16:43 +00:00
fc7b2b11a2
PEP8
2014-11-04 15:09:34 +00:00
David Baker