Commit Graph

553 Commits (d81602b75afc2c39314da1f9a21be436134e5361)

Author SHA1 Message Date
Richard van der Hoff 233699c42e
Merge pull request #2760 from Valodim/pypy
Synapse on PyPy
2018-04-11 11:20:01 +01:00
Richard van der Hoff b3384232a0 Add metrics for ResponseCache 2018-04-10 23:14:47 +01:00
Erik Johnston 9daf82278f
Merge pull request #3078 from matrix-org/erikj/federation_sender
Send federation events concurrently
2018-04-10 16:43:48 +01:00
Erik Johnston a060dfa132 Use run_in_background instead 2018-04-10 14:25:11 +01:00
Erik Johnston 1246d23710 Preserve log contexts correctly 2018-04-10 12:04:32 +01:00
Erik Johnston d49cbf712f Log event ID on exception 2018-04-10 12:03:41 +01:00
Erik Johnston 11d2609da7 Ensure slashes are escaped 2018-04-10 11:24:40 +01:00
Erik Johnston dab87b84a3 URL quote path segments over federation 2018-04-10 11:16:08 +01:00
Vincent Breitmoser 9fbe70a7dc Use sortedcontainers instead of blist
This commit drop-in replaces blist with SortedContainers. They are
written in pure python so work with pypy, but perform as good as
native implementations, at least in a couple benchmarks:

http://www.grantjenks.com/docs/sortedcontainers/performance.html
2018-04-10 11:29:51 +02:00
Erik Johnston 6e025a97b4 Handle all events in a room correctly 2018-04-09 16:02:48 +01:00
Erik Johnston 11974f3787 Send federation events concurrently 2018-04-09 11:47:10 +01:00
Erik Johnston 145d14656b Handle exceptions in get_hosts_for_room when sending events over federation 2018-04-09 11:47:01 +01:00
Luke Barnard 112c2253e2 pep8 2018-04-06 15:43:27 +01:00
Luke Barnard f8d1917fce Fix federation client `set_group_joinable` typo 2018-04-06 15:43:27 +01:00
David Baker b370fe61c0 Implement group join API 2018-04-06 15:43:27 +01:00
Krombel 1d71f484d4 use PUT instead of POST for federating groups/m.join_policy 2018-04-06 12:54:09 +02:00
Luke Barnard 104c0bc1d5 Use "/settings/" (plural) 2018-04-05 14:07:16 +01:00
Luke Barnard eb8d8d6f57 Use join_policy API instead of joinable
The API is now under
 /groups/$group_id/setting/m.join_policy

and expects a JSON blob of the shape

```json
{
  "m.join_policy": {
    "type": "invite"
  }
}
```

where "invite" could alternatively be "open".
2018-04-03 16:16:40 +01:00
David Baker 32260baa41 pep8 2018-03-28 14:29:42 +01:00
David Baker 79452edeee Add joinability for groups
Adds API to set the 'joinable' flag, and corresponding flag in the
table.
2018-03-28 14:03:37 +01:00
Erik Johnston 95cb401ae0
Merge pull request #2978 from matrix-org/erikj/refactor_replication_layer
Remove ReplicationLayer and user Client/Server directly
2018-03-13 15:45:08 +00:00
Erik Johnston 56e709857c
Merge pull request #2979 from matrix-org/erikj/no_handlers
Don't build handlers on workers unnecessarily
2018-03-13 13:46:38 +00:00
Erik Johnston cea462e285 s/replication_server/federation_server 2018-03-13 13:22:21 +00:00
Erik Johnston 9a2d9b4789
Merge pull request #2977 from matrix-org/erikj/replication_move_props
Move property setting from ReplicationLayer to base classes
2018-03-13 11:45:25 +00:00
Erik Johnston f43b6d6d9b Fix docstring types 2018-03-13 11:29:35 +00:00
Erik Johnston ea7b3c4b1b Remove unused ReplicationLayer 2018-03-13 11:00:04 +00:00
Erik Johnston 265b993b8a Split replication layer into two 2018-03-13 10:55:47 +00:00
Erik Johnston e05bf34117 Move property setting from ReplicationLayer to FederationBase 2018-03-13 10:51:30 +00:00
Erik Johnston c3f79c9da5 Split out edu/query registration to a separate class 2018-03-13 10:24:27 +00:00
Matthew Hodgson ab9f844aaf
Add federation_domain_whitelist option (#2820)
Add federation_domain_whitelist

gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
Richard van der Hoff a027c2af8d Metrics for events processed in appservice and fed sender
More metrics I wished I'd had
2018-01-15 18:23:24 +00:00
Richard van der Hoff bd91857028 Check missing fields in event_from_pdu_json
Return a 400 rather than a 500 when somebody messes up their send_join
2017-12-30 18:40:19 +00:00
Richard van der Hoff 3079f80d4a Factor out `event_from_pdu_json`
turns out we have two copies of this, and neither needs to be an instance
method
2017-12-30 18:40:19 +00:00
Richard van der Hoff 65abc90fb6 federation_server: clean up imports 2017-12-30 18:40:19 +00:00
Richard van der Hoff a7b726ad18 federation_client: clean up imports 2017-12-30 18:40:19 +00:00
Richard van der Hoff d4fb4f7c52 Clear logcontext before starting fed txn queue runner
These processes take a long time compared to the request, so there is lots of
"Entering|Restoring dead context" in the logs. Let's try to shut it up a bit.
2017-11-28 15:26:14 +00:00
Richard van der Hoff 7e6fa29cb5 Remove preserve_context_over_{fn, deferred}
Both of these functions ae known to leak logcontexts. Replace the remaining
calls to them and kill them off.
2017-11-14 11:22:42 +00:00
Erik Johnston 82e4bfb53d Add brackets 2017-11-09 10:06:42 +00:00
Erik Johnston e8814410ef Have an explicit API to update room config 2017-11-08 16:13:27 +00:00
Erik Johnston 94ff2cda73
Revert "Modify group room association API to allow modification of is_public" 2017-11-08 15:43:34 +00:00
Luke Barnard 207fabbc6a Update docs for updating room group association 2017-11-01 09:35:15 +00:00
Luke Barnard 13b3d7b4a0 Flake8 2017-10-31 17:20:11 +00:00
Luke Barnard 20fe347906 Modify group room association API to allow modification of is_public
also includes renamings to make things more consistent.
2017-10-31 17:04:28 +00:00
Erik Johnston 2a7e9faeec Do logcontexts outside ResponseCache 2017-10-25 15:21:08 +01:00
Erik Johnston 39dc52157d Merge branch 'develop' of github.com:matrix-org/synapse into erikj/group_fed_update_profile 2017-10-24 09:16:20 +01:00
Richard van der Hoff eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Erik Johnston ce6d4914f4 Correctly wire in update group profile over federation 2017-10-23 15:21:24 +01:00
Erik Johnston 011d03a0f6 Fix typo 2017-10-19 11:22:48 +01:00
Erik Johnston 9ab859f27b Fix typo in group attestation handling 2017-10-19 10:55:52 +01:00
Richard van der Hoff 582bd19ee9 Fix 500 error when we get an error handling a PDU
FederationServer doesn't have a send_failure (and nor does its subclass,
ReplicationLayer), so this was failing.

I'm not really sure what the idea behind send_failure is, given (a) we don't do
anything at the other end with it except log it, and (b) we also send back the
failure via the transaction response. I suspect there's a whole lot of dead
code around it, but for now I'm just removing the broken bit.
2017-10-17 20:52:40 +01:00
Luke Barnard 85f5674e44 Delint 2017-10-16 15:52:17 +01:00
Luke Barnard 2c5972f87f Implement GET /groups/$groupId/invited_users 2017-10-16 15:31:11 +01:00
Richard van der Hoff 8dd0c85ac5 Merge pull request #2529 from matrix-org/rav/fix_transaction_failure_handling
log pdu_failures from incoming transactions
2017-10-11 17:29:14 +01:00
Erik Johnston 271f5601f3 Fix typo in invite to group 2017-10-11 14:45:33 +01:00
Richard van der Hoff b75d443caf log pdu_failures from incoming transactions
... even if we have no EDUs.

This appears to have been introduced in
476899295f.
2017-10-11 14:36:13 +01:00
Erik Johnston 535cc49f27 Merge pull request #2466 from matrix-org/erikj/groups_merged
Initial Group Implementation
2017-10-11 13:20:07 +01:00
Richard van der Hoff 6a6cc27aee fed server: process PDUs for different rooms in parallel
With luck, this will give a real-time improvement when there are many rooms and
the server ends up calling out to fetch missing events.
2017-10-09 18:30:31 +01:00
Richard van der Hoff 4c7c4d4061 Fed server: use a linearizer for ongoing transactions
We don't want to process the same transaction multiple times concurrently, so
use a linearizer.
2017-10-09 18:30:10 +01:00
Richard van der Hoff ba5b9b80a5 fed server: refactor on_incoming_transaction
Move as much as possible to after the have_responded check, and reduce the
number of times we iterate over the pdu list.
2017-10-09 18:10:53 +01:00
Richard van der Hoff a6e3222fe5 Fed server: Move origin-check code to _handle_received_pdu
The response-building code expects there to be an entry in the `results` list
for each entry in the pdu_list, so the early `continue` was messing this
up. That doesn't really matter, because all that the federation client does is
log any errors, but it's pretty poor form.
2017-10-09 17:53:32 +01:00
Richard van der Hoff 01bbacf3c4 Fix up logcontext handling in (federation) TransactionQueue
Avoid using preserve_context_over_function, which has problems with respect to
logcontexts.
2017-10-06 22:39:25 +01:00
David Baker 27955056e0 Merge branch 'develop' into erikj/groups_merged 2017-10-02 16:20:41 +01:00
David Baker 6cd5fcd536 Make the spam checker a module 2017-09-26 19:20:23 +01:00
Erik Johnston 17b8e2bd02 Add remove room API 2017-09-26 15:52:41 +01:00
Richard van der Hoff 6de74ea6d7 Fix logcontexts in _check_sigs_and_hashes 2017-09-20 01:32:42 +01:00
Richard van der Hoff fcf2c0fd1a Remove redundant `preserve_fn`
preserve_fn is a no-op unless the wrapped function returns a
Deferred. verify_json_objects_for_server returns a list, so this is doing
nothing.
2017-09-20 01:32:42 +01:00
Richard van der Hoff 5ed109d59f PoC for filtering spammy events (#2456)
Demonstration of how you might add some hooks to filter out spammy events.
2017-09-19 12:20:11 +01:00
Erik Johnston ef8e578677 Add bulk group publicised lookup API 2017-08-09 13:36:22 +01:00
Erik Johnston 6f443a74cf Add update group profile API 2017-07-20 09:46:33 +01:00
Erik Johnston 14a34f12d7 Comments 2017-07-18 17:28:42 +01:00
Erik Johnston 3431ec55dc Comments 2017-07-18 17:23:50 +01:00
Erik Johnston 94ecd871a0 Fix typos 2017-07-18 16:38:54 +01:00
Erik Johnston 12ed4ee48e Correctly parse query params 2017-07-18 15:33:09 +01:00
Erik Johnston e5ea6dd021 Add client apis 2017-07-18 14:37:06 +01:00
Erik Johnston cccfcfa7b9 Comments 2017-07-18 10:35:18 +01:00
Erik Johnston 2f9eafdd36 Add local group server support 2017-07-17 12:03:49 +01:00
Erik Johnston cb3aee8219 Ensure category and role ids are non-null 2017-07-14 14:06:55 +01:00
Erik Johnston 8575e3160f Comments 2017-07-13 13:52:41 +01:00
Erik Johnston 7a39da8cc6 Add summary APIs to federation 2017-07-13 11:13:19 +01:00
Erik Johnston a62406aaa5 Add group summary APIs 2017-07-12 11:36:15 +01:00
Erik Johnston fe4e885f54 Add federation API for adding room to group 2017-07-11 14:35:07 +01:00
Erik Johnston 6322fbbd41 Comment 2017-07-11 11:52:03 +01:00
Erik Johnston b8ca494ee9 Initial group server implementation 2017-07-10 15:44:15 +01:00
Erik Johnston 5e49a57ecc Separate federation servlet into different lists 2017-07-05 14:32:24 +01:00
Erik Johnston 6e2a7ee1bc Remove spurious log lines 2017-06-07 11:05:17 +01:00
Erik Johnston dfbda5e025 Faster cache for get_joined_hosts 2017-05-25 17:24:44 +01:00
Erik Johnston ec5c4499f4 Make presence use cached users/hosts in room 2017-05-16 16:01:43 +01:00
Richard van der Hoff de042b3b88 Do some logging when one-time-keys get claimed
might help us figure out if https://github.com/vector-im/riot-web/issues/3868
has happened.
2017-05-09 19:04:56 +01:00
Erik Johnston 27c28eaa27 Merge pull request #2190 from matrix-org/erikj/mark_remote_as_back_more
Always mark remotes as up if we receive a signed request from them
2017-05-05 14:08:12 +01:00
Erik Johnston 310b1ccdc1 Use preserve_fn and add logs 2017-05-05 13:41:19 +01:00
Erik Johnston db7d0c3127 Always mark remotes as up if we receive a signed request from them 2017-05-05 10:34:53 +01:00
Erik Johnston 7166854f41 Add cache for get_current_hosts_in_room 2017-05-02 10:36:35 +01:00
Richard van der Hoff 91b3981800 Try harder when sending leave events
When we're rejecting invites, ignore the backoff data, so that we have a better
chance of not getting the room out of sync.
2017-04-21 01:50:36 +01:00
Richard van der Hoff 4903ccf159 Fix some lies, and other clarifications, in docstrings
The documentation on get_json has been wrong ever since the very first commit
to synapse...
2017-04-21 01:31:09 +01:00
Erik Johnston 247c736b9b Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl
Reduce federation replication traffic
2017-04-12 11:07:13 +01:00
Paul Evans 8fbc0d29ee Merge pull request #2121 from matrix-org/paul/sent-transactions-metric
Add a counter metric for successfully-sent transactions
2017-04-12 11:04:31 +01:00
Erik Johnston 26ae5178a4 Add some comments 2017-04-12 10:36:29 +01:00
Erik Johnston 1745069543 Comment 2017-04-12 10:17:10 +01:00
Erik Johnston c7ddb5ef7a Reuse get_interested_parties 2017-04-12 10:16:26 +01:00
Paul "LeoNerd" Evans 11dbceb761 Add a counter metric for successfully-sent transactions 2017-04-11 17:16:12 +01:00