d2688d7f03
Correctly guard against multiple concurrent transactions
2016-09-09 11:44:36 +01:00
b390756150
Update last_device_stream_id_by_dest if there is nothing to send
2016-09-09 11:00:15 +01:00
43954d000e
Add a new method to enqueue the device messages rather than sending a dummy EDU
2016-09-07 16:10:51 +01:00
cb98ac261b
Move the check for federated device_messages.
...
Move the check into _attempt_new_transaction.
Only delete messages if there were messages to delete.
2016-09-07 15:39:13 +01:00
31a07d2335
Add stream change caches for device messages
2016-09-07 15:27:07 +01:00
d4a35ada28
Send device messages over federation
2016-09-06 18:16:20 +01:00
2854ee2a52
Only pull out IDs from DB for /state_ids/ request
2016-09-02 10:53:36 +01:00
f4778d4cd9
Merge branch 'erikj/pdu_check' of github.com:matrix-org/synapse into develop
2016-09-01 14:40:36 +01:00
516a272aca
Ensure we only return a validated pdu in get_pdu
2016-09-01 10:55:02 +01:00
bed10f9880
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-26 14:54:30 +01:00
9219139351
Preserve some logcontexts
2016-08-24 11:58:40 +01:00
c315922b5f
PEP8
2016-08-10 16:34:10 +01:00
ca8abfbf30
Clean up TransactionQueue
2016-08-10 16:24:16 +01:00
5aeadb7414
Merge pull request #999 from matrix-org/erikj/measure_more
...
Measure federation send transaction resources
2016-08-10 14:16:14 +01:00
487bc49bf8
Don't stop on 4xx series errors
2016-08-10 13:39:12 +01:00
ea8c4094db
Also pull out rejected events
2016-08-10 13:26:13 +01:00
f91df1f761
Store if we fail to fetch an event from a destination
2016-08-10 11:31:46 +01:00
3bc9629be5
Measure federation send transaction resources
2016-08-10 10:56:38 +01:00
f5deaff424
Merge pull request #991 from matrix-org/erikj/retry_make
...
Retry joining via other servers if first one failed. Fix some other bugs.
2016-08-05 18:21:27 +01:00
5f360182c6
Fix a couple of python bugs
2016-08-05 18:08:32 +01:00
46453bfc2f
Retry joining via other servers if first one failed
2016-08-05 18:02:03 +01:00
24f36469bc
Add federation /version API
2016-08-05 16:36:07 +01:00
597c79be10
Change the way we specify if we require auth or not
2016-08-05 16:17:04 +01:00
2d4de61fb7
Fix typo
2016-08-05 10:48:56 +01:00
fccadb7719
Check if we already have the events returned by /state/
2016-08-05 10:43:47 +01:00
1515d1b581
Fallback to /state/ on both 400 and 404
2016-08-05 10:24:23 +01:00
a2b7102eea
Tweak integrity error recovery to work as intended
2016-08-04 20:38:08 -07:00
257c41cc2e
Fix typos.
2016-08-04 14:05:45 +01:00
b4e2290d89
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids_api
2016-08-04 14:04:35 +01:00
e3ee63578f
Tidy up get_events
2016-08-04 14:01:18 +01:00
edb33eb163
Rename fields to _ids
2016-08-03 17:19:15 +01:00
bcc9cda8ca
Fix copy + paste fails
2016-08-03 17:17:26 +01:00
4c56bedee3
Actually call get_room_state
2016-08-03 15:04:29 +01:00
520ee9bd2c
Fix syntax error
2016-08-03 15:03:15 +01:00
a60a2eaa02
Comment
2016-08-03 14:52:43 +01:00
e3a720217a
Add /state_ids federation API
...
The new API only returns the event_ids for the state, as most
requesters will already have the vast majority of the events already.
2016-08-03 14:47:37 +01:00
921f17f938
Merge branch 'develop' into rav/refactor_device_query
2016-08-03 11:12:47 +01:00
1efee2f52b
E2E keys: Make federation query share code with client query
...
Refactor the e2e query handler to separate out the local query, and then make
the federation handler use it.
2016-08-02 18:12:00 +01:00
c9154b970c
Don't double wrap 200
2016-08-02 16:45:53 +01:00
b3d5c4ad9d
Fix response cache
2016-08-02 16:42:21 +01:00
248e6770ca
Cache federation state responses
2016-07-21 10:30:12 +01:00
8f4a9bbc16
Linearize some federation endpoints based on (origin, room_id)
2016-06-17 16:43:45 +01:00
9ba2bf1570
Merge pull request #878 from matrix-org/erikj/ujson
...
Disable responding with canonical json for federation
2016-06-17 16:22:12 +01:00
120c238705
Disable responding with canonical json for federation
2016-06-17 16:10:37 +01:00
2884712ca7
Only re-sign our own events
2016-06-17 14:47:33 +01:00
d41a1a91d3
Linearize fetching of gaps on incoming events
...
This potentially stops the server from doing multiple requests for the
same data.
2016-06-15 15:16:14 +01:00
d88faf92d1
Fix up federation PublicRoomList
2016-06-08 14:39:31 +01:00
958c968d02
Merge pull request #856 from matrix-org/erikj/fed_pub_rooms
...
Enable auth on /publicRoom endpoints
2016-06-08 14:36:09 +01:00
1fd6eb695d
Enable auth on federation PublicRoomList
2016-06-08 14:15:18 +01:00
17aab5827a
Add some logging for when servers ask for missing events
2016-06-08 11:55:31 +01:00
6ecb2ca4ec
pep8
2016-06-01 09:48:55 +01:00
963e3ed282
Apparently I am not permitted to have two blank lines here
2016-05-31 17:22:53 +01:00
d240796ded
Basic, un-cached support for secondary_directory_servers
2016-05-31 17:20:07 +01:00
70ecb415f5
Fix c+p fail
2016-05-31 12:00:54 +01:00
e1625d62a8
Add federation room list servlet
2016-05-31 11:55:57 +01:00
f6ebaf4a32
Run transaction queue on reactor
...
This ensures that any CPU work that happens doesn't block message
sending.
2016-05-09 10:10:06 +01:00
9c272da05f
Add an openidish mechanism for proving to third parties that you own a given user_id
2016-05-05 13:42:44 +01:00
62b51b8452
Fix typo in event_auth servlet path
2016-04-29 12:00:51 +01:00
914f1eafac
Lower timeout for make_membership_event
...
Calls to make_membership_event are done in response to client requests,
and so should not be retried over long timeframes.
2016-04-15 11:22:23 +01:00
acdfef7b14
Intern all the things
2016-03-23 16:25:54 +00:00
5244c0b48e
Remove unused backfilled parameter from persist_event
2016-03-21 18:06:08 +00:00
9adf0e92bc
Catch exceptions from EDU handling
2016-03-18 15:12:50 +00:00
3c5f25507b
Yield on EDU handling
2016-03-18 13:55:16 +00:00
e9c1cabac2
Use parse_json_object_from_request to parse JSON out of request bodies
2016-03-11 16:41:03 +00:00
ed61a49169
Add profile information to invites
2016-03-04 14:35:02 +00:00
577951b032
Allow third_party_signed to be specified on /join
2016-02-23 15:11:25 +00:00
2c1fbea531
Fix up logcontexts
2016-02-08 14:26:45 +00:00
d83d004ccd
Fix flake8 warnings for new flake8
2016-02-02 17:18:50 +00:00
9959d9ece8
Remove redundated BaseHomeServer
2016-01-26 13:52:29 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
14d7acfad4
Host /unstable and /r0 versions of r0 APIs
2015-12-01 17:34:32 +00:00
cf4ef5f3c7
Only retry federation requests for a long time for background requests
2015-11-17 18:26:50 +00:00
aca6e5bf46
Don't complain if /make_join response lacks 'prev_state' list (SYN-517)
2015-11-13 17:27:25 +00:00
c0b3554401
Fix missing profile data in federation joins
...
There was a regression where we stopped including profile data in
initial joins for rooms joined over federation.
2015-11-12 16:19:55 +00:00
2cebe53545
Exchange 3pid invites for m.room.member invites
2015-11-05 16:43:19 +00:00
97d792b28f
Don't rearrange transaction_queue
2015-11-03 16:31:08 +00:00
7ce264ce5f
Fix broken cache for getting retry times. This meant we retried remote destinations way more frequently than we should
2015-11-03 16:24:03 +00:00
137fafce4e
Allow rejecting invites
...
This is done by using the same /leave flow as you would use if you had
already accepted the invite and wanted to leave.
2015-10-20 11:58:58 +01:00
643b5fcdc8
Look for keys on the right objects
2015-10-15 13:10:30 +01:00
0c38e8637f
Remove unnecessary class-wrapping
2015-10-13 18:00:38 +01:00
17dffef5ec
Move event contents into third_party_layout field
2015-10-13 15:48:12 +01:00
fcd9ba8802
Fix lint errors
2015-10-06 10:13:05 -05:00
5b3e9713dd
Implement third party identifier invites
2015-10-01 17:49:52 +01:00
8899df13bf
Merge pull request #208 from matrix-org/markjh/end-to-end-key-federation
...
Federation for end-to-end key requests.
2015-08-18 09:12:54 +01:00
0cceb2ac92
Add a few strategic new lines to break up the on_query_client_keys and on_claim_client_keys methods in federation_server.py
2015-08-13 17:27:46 +01:00
2df8dd9b37
Move all the caches into their own package, synapse.util.caches
2015-08-11 18:00:59 +01:00
2da3b1e60b
Get the end-to-end key federation working
2015-07-24 18:26:46 +01:00
62c010283d
Add federation support for end-to-end key requests
2015-07-23 16:03:38 +01:00
b5f55a1d85
Implement bulk verify_signed_json API
2015-06-26 10:39:34 +01:00
cee69441d3
Log more when we have processed the request
2015-06-15 17:11:44 +01:00
8047fd2434
Merge pull request #176 from matrix-org/erikj/backfill_auth
...
Improve backfill.
2015-06-03 17:25:37 +01:00
3483b78d1a
Log where a request came from in federation
2015-06-02 18:15:13 +01:00
d3ded420b1
Rephrase log line
2015-06-02 16:30:52 +01:00
22716774d5
Don't about JSON when warning about content tampering
2015-06-02 16:30:52 +01:00
6eadbfbea0
Remove redundant for loop
2015-05-22 16:12:20 +01:00
74b7de83ec
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/backfill_fixes
2015-05-22 16:10:42 +01:00
36317f3dad
Merge pull request #156 from matrix-org/erikj/join_perf
...
Make joining #matrix:matrix.org over federation quicker
2015-05-22 16:09:54 +01:00
284f55a7fb
Add doc strings
2015-05-22 15:18:04 +01:00
b21d015c55
Log origin and stats of incoming transactions
2015-05-22 14:44:25 +01:00
e70e8e053e
Add txn_id to some log lines
2015-05-22 14:44:02 +01:00
1b446a5d85
Log less lines at INFO level, but include more helpful information
2015-05-22 14:29:57 +01:00
c2633907c5
Merge branch 'erikj/join_perf' of github.com:matrix-org/synapse into erikj/backfill_fixes
2015-05-21 14:58:47 +01:00
ebfdd2eb5b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf
2015-05-21 14:54:52 +01:00
20814fabdd
Actually fetch state for new backwards extremeties when backfilling.
2015-05-20 11:59:02 +01:00
3a653515ec
Add None check
2015-05-19 15:27:09 +01:00
5b1631a4a9
Add a timeout param to get_event
2015-05-19 14:53:32 +01:00
d5cea26d45
Remove pointless newline
2015-05-18 10:16:45 +01:00
c71176858b
Newline, remove debug logging
2015-05-18 10:11:14 +01:00
f8bd4de87d
Remove debug logging
2015-05-18 09:58:03 +01:00
a2c4f3f150
Fix daedlock
2015-05-15 10:54:04 +01:00
95dedb866f
Unwrap defer.gatherResults failures
2015-05-12 13:14:29 +01:00
476899295f
Change the way we do logging contexts so that they survive divergences
2015-05-08 16:32:18 +01:00
0cf7e480b4
And use buffer(...) there as well
2015-05-05 18:20:01 +01:00
977338a7af
Use buffer(...) when inserting into bytea column
2015-05-05 18:12:53 +01:00
3d5a955e08
Missed events are not outliers
2015-05-05 17:36:57 +01:00
1942382246
Don't log enqueue_
2015-05-01 16:14:25 +01:00
23c639ff32
Split a storage function in two so that we don't have to do extra work.
2015-05-01 10:17:19 +01:00
c178e4e6ca
Add missing servlet to list
2015-03-19 17:48:21 +00:00
128cf2daf7
Appease pep8
2015-03-12 16:24:51 +00:00
c782e893ec
Neater metrics from TransactionQueue
2015-03-12 16:24:51 +00:00
2e4f0b2bd7
Replace the @metrics.counted annotations in federation with specifically-written counters and distributions
2015-03-12 16:24:51 +00:00
1748605c5d
Count incoming HTTP requests per servlet that responds
2015-03-12 16:24:51 +00:00
b0cf867319
Use _ instead of . as a metric namespacing separator, for Prometheus
2015-03-12 16:24:51 +00:00
f9478e475b
Rename Metrics' "keys" to "labels"
2015-03-12 16:24:51 +00:00
094803cf82
Put vector gauges on transaction queue pending PDU and EDU dicts
2015-03-12 16:24:50 +00:00
9470412316
Initial attempt at sprinkling some @metrics.counted decorations around the federation code
2015-03-12 16:24:50 +00:00
120b689284
Delete pointless (and unreachable) __init__ method from FederationClient
2015-03-12 16:24:50 +00:00
abaf47bbb6
Fix bug in logging.
2015-03-10 10:28:29 +00:00
98b867f7b7
Fix bug in logging.
2015-03-10 10:16:09 +00:00
019422ebba
Merge pull request #101 from matrix-org/neaten-federation-servlets
...
Neaten federation servlets
2015-03-09 17:39:06 +00:00
d79d91a4a7
Appease pep8
2015-03-05 20:55:40 +00:00
5eab2549ab
Append a $ on PATH at registration time, meaning each PATH attribute doesn't need it
2015-03-05 20:36:05 +00:00
7644cb79b2
Slightly neater(?) arrangement of authentication wrapper for HTTP servlet methods
2015-03-05 20:33:16 +00:00
ba8ac996f9
Remove the dead 'rate_limit_origin' method from TransportLayerServer
2015-03-05 19:43:17 +00:00
a901ed16b5
Move federation API responding code out of weird mix of lambdas into Servlet-style methods on instances
2015-03-05 19:10:57 +00:00
9708f49abf
Docs
2015-03-05 16:35:16 +00:00
96fee64421
Remove unecessary check
2015-03-05 16:31:47 +00:00
39aa968a76
Respect min_depth argument
2015-03-05 16:31:32 +00:00
6dfd8c73fc
Docs.
2015-03-05 16:31:13 +00:00
ae702d161a
Handle if get_missing_pdu returns 400 or not all events.
2015-03-05 16:08:02 +00:00
0a036944bd
Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu
2015-03-02 13:53:30 +00:00
4195e55ccc
Merge branch 'develop' of github.com:matrix-org/synapse into federation_rate_limit
2015-03-02 13:39:22 +00:00
29481690c5
If we're yielding don't add errback
2015-03-02 11:50:43 +00:00
23d9bd1d74
Process transactions serially.
...
Since the events received in a transaction are ordered, later events
might depend on earlier events and so we shouldn't blindly process them
in parellel.
2015-03-02 11:39:57 +00:00
9d9b230501
Make the federation server ratelimiting configurable.
2015-03-02 11:33:45 +00:00
0554d07082
Move federation rate limiting out of transport layer
2015-02-27 15:41:52 +00:00
9dc9118e55
Document FederationRateLimiter
2015-02-27 15:16:47 +00:00
93d90765c4
Initial implementation of federation server rate limiting
2015-02-26 16:15:26 +00:00
59362454dd
Must update pending_transactions map before yield'ing
2015-02-26 15:47:35 +00:00
db215b7e00
Implement and use new batched get missing pdu
2015-02-23 13:58:02 +00:00
0ac2a79faa
Initial stab at implementing a batched get_missing_pdus request
2015-02-19 17:24:14 +00:00
a7925259a1
Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.7.1
2015-02-18 13:57:55 +00:00
446ef58992
Add errback to all deferreds in transaction_queue
2015-02-18 12:03:26 +00:00
2462aacd77
Restrict the destinations that synapse can talk to
2015-02-18 11:52:51 +00:00
b68e4a729f
Discard destination 'localhost'
2015-02-18 11:32:39 +00:00
47d3ff4cf8
Don't send failure to self
2015-02-18 11:30:37 +00:00
e482541e1d
Fix pyflakes
2015-02-18 10:44:22 +00:00
1be67eca8a
Merge branch 'keyclient_retry_scheme' of github.com:matrix-org/synapse into develop
2015-02-18 10:34:40 +00:00
fe672a04f7
Merge pull request #77 from matrix-org/failures
...
Failures
2015-02-18 10:29:29 +00:00
08f804208b
Merge pull request #79 from matrix-org/get_pdu_limiting
...
Get pdu limiting
2015-02-18 10:29:10 +00:00
ec847059f3
Rename _fail_fetch_pdu_cache to _get_pdu_cache
2015-02-18 10:14:10 +00:00
9371019133
Try to only back off if we think we failed to connect to the remote
2015-02-17 18:13:34 +00:00
2b8f1a956c
Add per server retry limiting.
...
Factor out the pre destination retry logic from TransactionQueue so it
can be reused in both get_pdu and crypto.keyring
2015-02-17 17:20:56 +00:00
676e8ee78a
Remove debug raise
2015-02-17 15:22:45 +00:00
fa6c93bd26
Merge branch 'consumeErrors' of github.com:matrix-org/synapse into develop
2015-02-17 15:18:17 +00:00
c02da58a9d
Merge branch 'develop' of github.com:matrix-org/synapse into failures
2015-02-17 15:15:07 +00:00
659ead082f
Format the response of transaction request in a nicer way
2015-02-17 15:11:44 +00:00
c82e26ad4b
Actually respond with JSON to incoming transaction
2015-02-17 13:24:13 +00:00
02bfa889de
Handle recieving failures in transactions
2015-02-17 13:13:14 +00:00
72a4de2ce6
Use consumeErrors=True on all DeferredLists.
...
This is so that the DeferredLists actually consume the error instead of
propogating down the non-existent errback chain. This should reduce the
number of unhandled errors we are seeing.
2015-02-17 10:07:01 +00:00
0194e71e99
Merge branch 'develop' of github.com:matrix-org/synapse into get_pdu_limiting
2015-02-17 09:48:23 +00:00
baa5b9a975
Cache results of get_pdu.
2015-02-16 18:02:39 +00:00
91fc5eef1d
Mark old events as outliers.
...
This is to fix the issue where if a remote server sends an event
that references a really "old" event, then the local server will pull
that in and send to all clients.
We decide if an event is old if its depth is less than the minimum depth
of the room.
2015-02-16 14:27:40 +00:00
6138584651
Don't return anything from _handle_new_pdu, since we ignore the return value anyway
2015-02-16 14:08:02 +00:00
789251afa7
Fix logging
2015-02-12 19:29:43 +00:00
58d848adc0
Parrellize fetching of events
2015-02-12 18:35:36 +00:00
963256638d
Correctly handle all the places that can throw exceptions
2015-02-12 18:17:11 +00:00
c52e8d395b
Merge pull request #61 from matrix-org/timeout-federation-requests
...
Timeout federation requests
2015-02-11 17:10:33 +00:00
ef276e8770
Fix so timing out connections to actually work.
2015-02-11 16:48:05 +00:00
ddb816cf60
Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict
2015-02-11 15:44:28 +00:00
4ebbaf0d43
Blunty replace json with simplejson
2015-02-11 14:23:10 +00:00
c8e1da930d
Log all the exits from _attempt_new_transaction
2015-02-10 17:30:46 +00:00
697ab75a34
Sign auth_chains when returned by /state/ requests
2015-02-10 15:46:24 +00:00
e9c85a4d5a
Connection errors in twisted aren't RuntimeErrors
2015-02-05 13:50:15 +00:00
e1515c3e91
Pass through list of room hosts from room alias query to federation so that it can retry against different room hosts
2015-02-05 13:44:42 +00:00
ae46f10fc5
Apply sanity to the transport client interface. Convert 'make_join' and 'send_join' to accept iterables of destinations
2015-02-04 16:28:12 +00:00
ff78eded01
Retry make_join
2015-02-04 13:55:10 +00:00
3c39f42a05
New line
2015-02-03 16:14:19 +00:00
9bace3a367
Actually, the old prune_event function was non-deterministic, so no point keeping it around :(
2015-02-03 15:32:17 +00:00
8dae5c8108
Remove unused imports
2015-02-03 15:01:12 +00:00
7b810e136e
Add new FederationBase
2015-02-03 15:00:42 +00:00
0dd3aea319
Keep around the old (buggy) version of the prune_event function so that we can use it to check signatures for events on old servers
2015-02-03 14:58:30 +00:00
0f48e22ef6
PEP8
2015-02-03 10:43:29 +00:00
40c6fe1b81
Don't bother requesting PDUs with bad signatures from the same server
2015-02-02 17:06:37 +00:00
941f59101b
Don't fail an entire request if one of the returned events fails a signature check. If an event does fail a signature check, look in the local database and request it from the originator.
2015-02-02 16:56:01 +00:00
776ac820f9
Briefly doc structure of query_auth API.
2015-01-30 15:58:28 +00:00
a70a801184
Fix bug where we superfluously asked for current state. Change API of /query_auth/ so that we don't duplicate events in the response.
2015-01-30 13:34:01 +00:00
c1d860870b
Fix regression where we no longer correctly handled the case of gaps in our event graph
2015-01-30 10:48:47 +00:00
78015948a7
Initial implementation of auth conflict resolution
2015-01-29 16:52:33 +00:00
0ef5bfd6a9
Start implementing auth conflict res
2015-01-28 16:16:53 +00:00
Erik Johnston