Commit Graph

2126 Commits (d1d54d60889f94228adff55f64116aa66d07ac3a)

Author SHA1 Message Date
Richard van der Hoff eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Richard van der Hoff 3267b81b81 Merge pull request #2561 from matrix-org/rav/id_checking
Updates to ID checking
2017-10-23 14:39:20 +01:00
Richard van der Hoff 29812c628b Allow = in mxids and groupids
... because the spec says we should.
2017-10-20 23:42:53 +01:00
Richard van der Hoff 58fbbe0f1d Disallow capital letters in userids
Factor out a common function for checking user ids and group ids, which forbids
capitals.
2017-10-20 23:37:22 +01:00
Richard van der Hoff 631d7b87b5 Remove pointless create() method
It just calls the constructor, so we may as well kill it rather than having
random codepaths.
2017-10-20 22:14:55 +01:00
Luke Barnard a3ac4f6b0a _create_rererouter for get_invited_users_in_group 2017-10-16 15:41:03 +01:00
Luke Barnard 2c5972f87f Implement GET /groups/$groupId/invited_users 2017-10-16 15:31:11 +01:00
Luke Barnard 6079d0027a Log a warning when no profile for invited member
And return empty profile
2017-10-16 14:20:45 +01:00
David Baker b8d8ed1ba9 Merge pull request #2531 from matrix-org/dbkr/spamcheck_error_messages
Allow error strings from spam checker
2017-10-12 10:31:03 +01:00
Erik Johnston b752507b48 Fix fetching remote summaries 2017-10-11 16:59:18 +01:00
Richard van der Hoff 68fd82e840 Merge pull request #2530 from matrix-org/rav/fix_receipt_logcontext
fix a logcontext leak in read receipt handling
2017-10-11 15:08:53 +01:00
David Baker b78bae2d51 fix isinstance 2017-10-11 14:49:09 +01:00
David Baker c3b7a45e84 Allow error strings from spam checker 2017-10-11 14:39:22 +01:00
Richard van der Hoff c3e190ce67 fix a logcontext leak in read receipt handling 2017-10-11 14:37:20 +01:00
Erik Johnston 4ce4379235 Fix attestations to check correct server name 2017-10-11 14:11:43 +01:00
Erik Johnston 535cc49f27 Merge pull request #2466 from matrix-org/erikj/groups_merged
Initial Group Implementation
2017-10-11 13:20:07 +01:00
Erik Johnston bc7f3eb32f Merge pull request #2483 from jeremycline/unfreeze-ujson-dump
Unfreeze event before serializing with ujson
2017-10-11 13:18:52 +01:00
Erik Johnston 84e27a592d Merge pull request #2490 from matrix-org/erikj/drop_left_room_events
Ignore incoming events for rooms that we have left
2017-10-10 11:58:32 +01:00
Richard van der Hoff c8f568ddf9 Fix up deferred handling in federation.py
* Avoid preserve_context_over_deferred, which is broken

* set consumeErrors=True on defer.gatherResults, to avoid spurious "unhandled
  failure" erros
2017-10-06 22:14:24 +01:00
David Baker f878e6f8af Spam checking: add the invitee to user_may_invite 2017-10-05 14:02:28 +01:00
David Baker d8ce68b09b spam check room publishing 2017-10-04 14:29:33 +01:00
David Baker 78d4ced829 un-double indent 2017-10-04 12:44:27 +01:00
David Baker 197c14dbcf Add room creation checks to spam checker
Lets the spam checker deny attempts to create rooms and add aliases
to them.
2017-10-04 10:47:54 +01:00
David Baker 1e375468de pass room id too 2017-10-03 17:13:14 +01:00
David Baker c2c188b699 Federation was passing strings anyway
so pass string everywhere
2017-10-03 15:46:19 +01:00
David Baker c46a0d7eb4 this shouldn't be debug 2017-10-03 15:20:14 +01:00
David Baker bd769a81e1 better logging 2017-10-03 15:16:40 +01:00
David Baker 41fd9989a2 Skip spam check for admin users 2017-10-03 14:17:44 +01:00
Erik Johnston e4ab96021e Update comments 2017-10-03 14:10:41 +01:00
David Baker 2a7ed700d5 Fix param name & lint 2017-10-03 14:04:10 +01:00
David Baker 84716d267c Allow spam checker to reject invites too 2017-10-03 13:56:43 +01:00
Erik Johnston f2da6df568 Remove spurious line feed 2017-10-03 11:31:06 +01:00
Erik Johnston 30848c0fcd Ignore incoming events for rooms that we have left
When synapse receives an event for a room its not in over federation, it
double checks with the remote server to see if it is in fact in the
room. This is done so that if the server has forgotten about the room
(usually as a result of the database being dropped) it can recover from
it.

However, in the presence of state resets in large rooms, this can cause
a lot of work for servers that have legitimately left. As a hacky
solution that supports both cases we drop incoming events for rooms that
we have explicitly left.

This means that we no longer support the case of servers having
forgotten that they've rejoined a room, but that is sufficiently rare
that we're not going to support it for now.
2017-10-03 11:18:21 +01:00
David Baker 27955056e0 Merge branch 'develop' into erikj/groups_merged 2017-10-02 16:20:41 +01:00
Jeremy Cline cafb8de132
Unfreeze event before serializing with ujson
In newer versions of https://github.com/esnme/ultrajson, ujson does not
serialize frozendicts (introduced in esnme/ultrajson@53f85b1). Although
the PyPI version is still 1.35, Fedora ships with a build from commit
esnme/ultrajson@2f1d487. This causes the serialization to fail if the
distribution-provided package is used.

This runs the event through the unfreeze utility before serializing it.

Thanks to @ignatenkobrain for tracking down the root cause.

fixes #2351

Signed-off-by: Jeremy Cline <jeremy@jcline.org>
2017-09-30 11:22:37 -04:00
Erik Johnston 8090fd4664 Fix /joined_members to work with AS users 2017-09-28 10:09:32 +01:00
Erik Johnston adec03395d Fix bug where /joined_members didn't check user was in room 2017-09-27 15:14:39 +01:00
David Baker 6cd5fcd536 Make the spam checker a module 2017-09-26 19:20:23 +01:00
Erik Johnston 17b8e2bd02 Add remove room API 2017-09-26 15:52:41 +01:00
Erik Johnston 95298783bb Add is_publicised to group summary 2017-09-26 11:04:37 +01:00
Erik Johnston e3edca3b5d Refactor to speed up incremental syncs 2017-09-25 17:35:39 +01:00
Erik Johnston 3166ed55b2 Fix device list when rejoining room (#2461) 2017-09-22 14:44:17 +01:00
Richard van der Hoff aa620d09a0 Add a config option to block all room invites (#2457)
- allows sysadmins the ability to lock down their servers so that people can't
send their users room invites.
2017-09-19 16:08:14 +01:00
Richard van der Hoff 2eabdf3f98 add some comments to on_exchange_third_party_invite_request 2017-09-19 12:20:36 +01:00
Richard van der Hoff 5ed109d59f PoC for filtering spammy events (#2456)
Demonstration of how you might add some hooks to filter out spammy events.
2017-09-19 12:20:11 +01:00
Erik Johnston 93e504d04e Ensure that creator of group sees group down /sync 2017-09-19 11:08:16 +01:00
Erik Johnston b5feaa5a49 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged 2017-09-19 11:07:45 +01:00
Richard van der Hoff 3f405b34e9 Fix overzealous kicking of guest users (#2453)
We should only kick guest users if the guest access event is authorised.
2017-09-19 08:52:52 +01:00
Erik Johnston 2d1b7955ae Don't filter out current state events from timeline 2017-09-18 17:13:03 +01:00
Erik Johnston d6dadd95ac Correctly handle leaving room in /key/changes 2017-09-18 15:38:22 +01:00
Erik Johnston 4a94eb3ea4 Fix typo 2017-09-15 09:56:54 +01:00
Erik Johnston 3a0cee28d6 Actually hook leave notifs up 2017-09-14 11:49:37 +01:00
Erik Johnston 4f845a0713 Handle joining/leaving rooms in /keys/changes 2017-09-13 16:28:08 +01:00
Erik Johnston 473700f016 Get left rooms 2017-09-13 15:13:41 +01:00
Erik Johnston 9ce866ed4f In sync handle device lists for newly joined/left rooms 2017-09-12 16:44:26 +01:00
Erik Johnston 69ef4987a6 Add left section to /keys/changes 2017-09-08 14:44:36 +01:00
Erik Johnston 53cc8ad35a Send down device list change notif when member leaves/rejoins room 2017-09-07 15:08:39 +01:00
Erik Johnston 4a9b1cf253 Add user profiles to summary from group server 2017-08-25 16:23:58 +01:00
Erik Johnston 258409ef61 Fix typos and reinherit 2017-08-25 14:45:20 +01:00
Erik Johnston bf81f3cf2c Split out profile handler to fix tests 2017-08-25 14:34:56 +01:00
Erik Johnston 27ebc5c8f2 Add remote profile cache 2017-08-25 11:25:47 +01:00
Erik Johnston ba3ff7918b Fixup 2017-08-11 13:42:42 +01:00
Erik Johnston ef8e578677 Add bulk group publicised lookup API 2017-08-09 13:36:22 +01:00
Erik Johnston 05e21285aa Store whether the user wants to publicise their membership of a group 2017-08-08 13:01:46 +01:00
Erik Johnston a1e67bcb97 Remove stale TODO comments 2017-08-04 10:07:10 +01:00
Erik Johnston d5e32c843f Correctly add joins to correct segment 2017-07-24 13:31:26 +01:00
Erik Johnston 96917d5552 Merge pull request #2378 from matrix-org/erikj/group_sync_support
Add groups to sync stream
2017-07-21 11:05:39 +01:00
Erik Johnston 960dae3340 Add notifier 2017-07-20 17:14:44 +01:00
Erik Johnston 2cc998fed8 Fix replication. And notify 2017-07-20 17:13:18 +01:00
Erik Johnston 139fe30f47 Remember to cast to bool 2017-07-20 16:47:35 +01:00
Erik Johnston c544188ee3 Add groups to sync stream 2017-07-20 16:36:42 +01:00
Erik Johnston 6f443a74cf Add update group profile API 2017-07-20 09:46:33 +01:00
Erik Johnston 14a34f12d7 Comments 2017-07-18 17:28:42 +01:00
Erik Johnston 94ecd871a0 Fix typos 2017-07-18 16:38:54 +01:00
Erik Johnston 332839f6ea Update federation client pokes 2017-07-18 14:45:37 +01:00
Erik Johnston e5ea6dd021 Add client apis 2017-07-18 14:37:06 +01:00
Erik Johnston cccfcfa7b9 Comments 2017-07-18 10:35:18 +01:00
Erik Johnston 68f34e85ce Use transport client directly 2017-07-18 10:29:57 +01:00
Erik Johnston 2f9eafdd36 Add local group server support 2017-07-17 12:03:49 +01:00
Erik Johnston e9aec001f4 Use less DB for device list handling in sync 2017-07-12 10:30:10 +01:00
Erik Johnston 28e8c46f29 Merge pull request #2352 from matrix-org/erikj/group_server_split
Initial Group Server
2017-07-12 10:26:29 +01:00
Erik Johnston 9a6fd3ef29 Don't compute push actions for backfilled events 2017-07-11 10:02:21 +01:00
Erik Johnston b8ca494ee9 Initial group server implementation 2017-07-10 15:44:15 +01:00
Erik Johnston 5a7f561a9b Fix bug where pusherpool didn't start and broke some rooms
Since we didn't instansiate the PusherPool at start time it could fail
at run time, which it did for some users.

This may or may not fix things for those users, but it should happen at
start time and stop the server from starting.
2017-07-06 17:55:51 +01:00
Mark Haines 3e279411fe Improve the error handling for bad invites received over federation 2017-06-30 16:20:30 +01:00
Erik Johnston c72058bcc6 Use an ExpiringCache for storing registration sessions
This is because pruning them was a significant performance drain on
matrix.org
2017-06-29 14:08:37 +01:00
Erik Johnston 816605a137 Merge pull request #2307 from matrix-org/erikj/user_ip_batch
Batch upsert user ips
2017-06-27 15:08:32 +01:00
Erik Johnston a0a561ae85 Fix up client ips to read from pending data 2017-06-27 14:46:12 +01:00
Erik Johnston 1bce3e6b35 Remove unused variables 2017-06-26 14:03:27 +01:00
Erik Johnston e3cbec10c1 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ensure_round_trip 2017-06-26 14:02:44 +01:00
Erik Johnston 44c722931b Make some more params configurable 2017-06-22 14:59:52 +01:00
Erik Johnston 2d520a9826 Typo. ARGH. 2017-06-22 14:42:39 +01:00
Erik Johnston 24d894e2e2 Fix thinko in unhandled user spam 2017-06-22 14:39:05 +01:00
Erik Johnston 1217c7da91 Don't work out users who share room with appservice users 2017-06-21 12:00:41 +01:00
Erik Johnston e5ae386ea4 Handle all cases of sending membership events 2017-06-19 16:07:54 +01:00
Erik Johnston b490299a3b Change to create new room and join other users 2017-06-19 14:10:13 +01:00
Erik Johnston d7fe6b356c Add shutdown room API 2017-06-19 12:37:27 +01:00
Erik Johnston fcf01dd88e Reject local events that don't round trip the DB 2017-06-19 11:33:40 +01:00
Erik Johnston 6aa5bc8635 Initial worker impl 2017-06-16 11:47:11 +01:00
Erik Johnston a9d6fa8b2b Include users who share room with requester in user directory 2017-06-15 10:17:21 +01:00
Erik Johnston 4564b05483 Implement updating users who share rooms on the fly 2017-06-15 10:17:17 +01:00
Erik Johnston 72613bc379 Implement initial population of users who share rooms table 2017-06-15 09:59:04 +01:00
Erik Johnston d9fd937e39 Fix user directory insertion due to missing room_id 2017-06-13 11:50:24 +01:00
Erik Johnston 6ae8373d40 Don't assume existance of events when updating user directory 2017-06-13 10:19:26 +01:00
Erik Johnston a837765e8c Merge pull request #2266 from matrix-org/erikj/host_in_room
Change is_host_joined to use current_state table
2017-06-12 09:49:51 +01:00
Erik Johnston 1664948e41 Comment 2017-06-09 13:05:05 +01:00
Erik Johnston 0185b75381 Change is_host_joined to use current_state table
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
2017-06-09 10:52:26 +01:00
Erik Johnston 7132e5cdff Mark remote invite rejections as outliers 2017-06-09 10:08:18 +01:00
Erik Johnston f45f07ab86 Merge pull request #2258 from matrix-org/erikj/user_dir
Don't start user_directory handling on workers
2017-06-07 14:04:50 +01:00
Erik Johnston a053ff3979 Merge pull request #2248 from matrix-org/erikj/state_fixup
Faster cache for get_joined_hosts
2017-06-07 14:01:06 +01:00
Erik Johnston ecdd2a3658 Don't start user_directory handling on workers 2017-06-07 12:02:53 +01:00
Erik Johnston 2f34ad31ac Add some logging to user directory 2017-06-07 11:50:44 +01:00
Erik Johnston 09e4bc0501 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup 2017-06-07 11:05:23 +01:00
Erik Johnston 1a01af079e Handle profile updates in user directory 2017-06-01 15:39:51 +01:00
Erik Johnston 21e255a8f1 Split the table in two 2017-06-01 14:50:46 +01:00
Erik Johnston 7233341eac Comments 2017-06-01 13:11:38 +01:00
Erik Johnston 8be6fd95a3 Check if host is still in room 2017-06-01 13:05:39 +01:00
Erik Johnston 9c7db2491b Fix removing users 2017-06-01 11:36:50 +01:00
Erik Johnston 0fe6f3c521 Bug fixes and logging
- Check if room is public when a user joins before adding to user dir
- Fix typo of field name "content.join_rules" -> "content.join_rule"
2017-06-01 11:09:49 +01:00
Erik Johnston cc7609aa9f Comment briefly on how we keep user_directory up to date 2017-05-31 17:11:18 +01:00
Erik Johnston b2d8d07109 Lifts things into separate function 2017-05-31 17:00:24 +01:00
Erik Johnston f9791498ae Typos 2017-05-31 16:50:57 +01:00
Erik Johnston f091061711 Fix tests 2017-05-31 16:34:40 +01:00
Erik Johnston dc51af3d03 Pull max id from correct table 2017-05-31 15:13:49 +01:00
Erik Johnston 350622a107 Handle the server leaving a public room 2017-05-31 15:11:36 +01:00
Erik Johnston 63fda37e20 Add comments 2017-05-31 15:00:29 +01:00
Erik Johnston 3b5f22ca40 Add search 2017-05-31 14:00:01 +01:00
Erik Johnston b5db4ed5f6 Update room column when room becomes unpublic 2017-05-31 13:40:28 +01:00
Erik Johnston 168524543f Add call later 2017-05-31 11:59:36 +01:00
Erik Johnston 3e123b8497 Start later 2017-05-31 11:56:27 +01:00
Erik Johnston 42137efde7 Don't go round in circles 2017-05-31 11:55:13 +01:00
Erik Johnston eeb2f9e546 Add user_directory to database 2017-05-31 11:51:01 +01:00
Erik Johnston 23da638360 Fix typing tests 2017-05-26 10:02:04 +01:00
Erik Johnston 11f139a647 Merge pull request #2247 from matrix-org/erikj/auth_event
Only store event_auth for state events
2017-05-24 16:46:34 +01:00
Erik Johnston c049472b8a Only store event_auth for state events 2017-05-24 15:23:31 +01:00
Erik Johnston b4f59c7e27 Add count of one time keys to sync stream 2017-05-19 15:47:55 +01:00
Erik Johnston cafe659f72 Store ActionGenerator in HomeServer 2017-05-19 10:09:56 +01:00
Erik Johnston 66d8ffabbd Faster push rule calculation via push specific cache
We add a push rule specific cache that ensures that we can reuse
calculated push rules appropriately when a user join/leaves.
2017-05-17 16:55:40 +01:00
Erik Johnston 6fa8148ccb Merge pull request #2228 from matrix-org/erikj/speed_up_get_hosts
Speed up get_joined_hosts
2017-05-16 17:40:55 +01:00
Erik Johnston 13f540ef1b Speed up get_joined_hosts 2017-05-16 16:05:22 +01:00
Erik Johnston ec5c4499f4 Make presence use cached users/hosts in room 2017-05-16 16:01:43 +01:00
Luke Barnard ccad2ed824 Modify condition on empty localpart 2017-05-10 17:34:30 +01:00
Luke Barnard 369195caa5 Modify register/available to be GET with query param
- GET is now the method for register/available
- a query parameter "username" is now used

Also, empty usernames are now handled with an error message on registration or via register/available: `User ID cannot be empty`
2017-05-10 17:23:55 +01:00
Erik Johnston a3648f84b2 Merge pull request #2208 from matrix-org/erikj/ratelimit_overrid
Add per user ratelimiting overrides
2017-05-10 15:54:48 +01:00
Erik Johnston b990b2fce5 Add per user ratelimiting overrides 2017-05-10 11:05:43 +01:00
Richard van der Hoff aedaba018f Replace some instances of preserve_context_over_deferred 2017-05-09 19:04:56 +01:00
Richard van der Hoff de042b3b88 Do some logging when one-time-keys get claimed
might help us figure out if https://github.com/vector-im/riot-web/issues/3868
has happened.
2017-05-09 19:04:56 +01:00
Richard van der Hoff a7e9d8762d Allow clients to upload one-time-keys with new sigs
When a client retries a key upload, don't give an error if the signature has
changed (but the key is the same).

Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.
2017-05-09 19:04:56 +01:00
Erik Johnston 27c28eaa27 Merge pull request #2190 from matrix-org/erikj/mark_remote_as_back_more
Always mark remotes as up if we receive a signed request from them
2017-05-05 14:08:12 +01:00
Erik Johnston 653d90c1a5 Comment 2017-05-05 14:01:17 +01:00
Erik Johnston 7b222fc56e Remove redundant reset of destination timers 2017-05-05 11:14:09 +01:00
Erik Johnston b843631d71 Add comment and TODO 2017-05-05 10:59:32 +01:00
Erik Johnston f346048a6e Handle exceptions thrown in handling remote device list updates 2017-05-05 10:34:10 +01:00
David Baker 60833c8978 Merge pull request #2147 from matrix-org/dbkr/http_request_propagate_error
Propagate errors sensibly from proxied IS requests
2017-05-03 11:23:25 +01:00
Erik Johnston 0c27383dd7 Merge pull request #2170 from matrix-org/erikj/fed_hole_state
Don't fetch state for missing events that we fetched
2017-05-03 10:49:37 +01:00
Erik Johnston ef862186dd Merge together redundant calculations/logging 2017-05-03 10:06:43 +01:00
Erik Johnston 3e5a62ecd8 Add more granular event send metrics 2017-05-02 14:23:26 +01:00
Erik Johnston 2347efc065 Fixup 2017-04-28 12:46:53 +01:00
Erik Johnston 9b147cd730 Remove unncessary call in _get_missing_events_for_pdu 2017-04-28 11:55:25 +01:00
Erik Johnston 3a9f5bf6dd Don't fetch state for missing events that we fetched 2017-04-28 11:26:46 +01:00
Erik Johnston 25a96e0c63 Merge pull request #2163 from matrix-org/erikj/fix_invite_state
Fix invite state to always include all events
2017-04-27 17:36:30 +01:00
Erik Johnston 87ae59f5e9 Typo 2017-04-27 15:16:21 +01:00
Erik Johnston e42b4ebf0f Add some extra logging for edge cases of federation 2017-04-27 14:38:21 +01:00
Erik Johnston 34e682d385 Fix invite state to always include all events 2017-04-26 16:18:08 +01:00
David Baker 82ae0238f9 Revert accidental commit 2017-04-26 11:43:16 +01:00
David Baker 81804909d3 Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error 2017-04-26 11:31:55 +01:00
David Baker 1a9255c12e Use CodeMessageException subclass instead
Parse json errors from get_json client methods and throw special
errors.
2017-04-25 19:30:55 +01:00
David Baker 2e165295b7 Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error 2017-04-21 11:35:52 +01:00
David Baker a90a0f5c8a Propagate errors sensibly from proxied IS requests
When we're proxying Matrix endpoints, parse out Matrix error
responses and turn them into SynapseErrors so they can be
propagated sensibly upstream.
2017-04-21 11:32:48 +01:00
Richard van der Hoff 0cdb32fc43 Remove redundant try/except clauses
The `except SynapseError` clauses were pointless because the wrapped functions
would never throw a `SynapseError` (they either throw a `CodeMessageException`
or a `RuntimeError`).

The `except CodeMessageException` is now also pointless because the caller
treats all exceptions equally, so we may as well just throw the
`CodeMessageException`.
2017-04-21 01:32:01 +01:00
Richard van der Hoff 838810b76a Broaden the conditions for locally_rejecting invites
The logic for marking invites as locally rejected was all well and good, but
didn't happen when the remote server returned a 500, or wasn't reachable, or
had no DNS, or whatever.

Just expand the except clause to catch everything.

Fixes https://github.com/matrix-org/synapse/issues/761.
2017-04-21 01:31:37 +01:00
Richard van der Hoff 736b9a4784 Remove redundant function
inline `reject_remote_invite`, which only existed to make tracing the callflow
more difficult.
2017-04-21 01:31:09 +01:00
Luke Barnard 3fb8784c92 m.read_marker -> m.fully_read (#2128)
Also:
 - change the REST endpoint to have a "S" on the end (so it's now /read_markers)
 - change the content of the m.read_up_to event to have the key "event_id" instead of "marker".
2017-04-18 17:46:15 +01:00
Luke Barnard 78f0ddbfad Merge pull request #2120 from matrix-org/luke/read-markers
Implement Read Marker API
2017-04-13 14:21:31 +01:00
Luke Barnard b9557064bf Simplify is_event_after logic 2017-04-12 14:36:20 +01:00
Erik Johnston 247c736b9b Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl
Reduce federation replication traffic
2017-04-12 11:07:13 +01:00
Luke Barnard 69a18514e9 Only notify user, not entire room 2017-04-12 10:50:37 +01:00
Luke Barnard 122cd52ce4 Remove comment, simplify null-guard 2017-04-12 10:48:32 +01:00
Erik Johnston c7ddb5ef7a Reuse get_interested_parties 2017-04-12 10:16:26 +01:00
Luke Barnard 77fb2b72ae Handle no previous RM 2017-04-12 09:47:29 +01:00
Luke Barnard 867822fa1e flake8 2017-04-11 17:36:04 +01:00
Luke Barnard 73880268ef Refactor event ordering check to events store 2017-04-11 17:34:09 +01:00
Luke Barnard 131485ef66 Copyright 2017-04-11 17:33:51 +01:00
Luke Barnard 0127423027 flake8 2017-04-11 17:07:07 +01:00
Erik Johnston 414522aed5 Move get_interested_parties 2017-04-11 15:33:26 +01:00
Erik Johnston 2be8a281d2 Comments 2017-04-11 15:28:24 +01:00
Erik Johnston 6308ac45b0 Move get_interested_remotes back to presence handler 2017-04-11 15:19:26 +01:00
Luke Barnard d892079844 Finish implementing RM endpoint
- This change causes a 405 to be sent if "m.read_marker" is set via /account_data
 - This also fixes-up the RM endpoint so that it actually Works.
2017-04-11 15:01:39 +01:00
lukebarnard e263c26690 Initial commit of RM server-side impl
(See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u)
2017-04-11 11:55:30 +01:00
Erik Johnston f3cf3ff8b6 Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-04-11 11:13:32 +01:00
Erik Johnston 29574fd5b3 Reduce federation presence replication traffic
This is mainly done by moving the calculation of where to send presence
updates from the presence handler to the transaction queue, so we only
need to send the presence event (and not the destinations) across the
replication connection. Before we were duplicating by sending the full
state across once per destination.
2017-04-10 16:48:30 +01:00
Richard van der Hoff 64765e5199 When we do an invite rejection, save the signed leave event to the db
During a rejection of an invite received over federation, we ask a remote
server to make us a `leave` event, then sign it, then send that with
`send_leave`.

We were saving the *unsigned* version of the event (which has a different event
id to the signed version) to our db (and sending it to the clients), whereas
other servers in the room will have seen the *signed* version. We're not aware
of any actual problems that caused, except that it makes the database confusing
to look at and generally leaves the room in a weird state.
2017-04-07 14:39:32 +01:00
Erik Johnston 98ce212093 Merge pull request #2103 from matrix-org/erikj/no-double-encode
Don't double encode replication data
2017-04-07 09:39:52 +01:00
Erik Johnston 877c029c16 Use iteritems 2017-04-06 15:51:22 +01:00
Erik Johnston 96b9b6c127 Don't double json encode typing replication data 2017-04-05 11:34:20 +01:00
Erik Johnston 62b89daac6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server 2017-04-04 09:46:16 +01:00
Richard van der Hoff 773e64cc1a Merge pull request #2095 from matrix-org/rav/cull_log_preserves
Cull spurious PreserveLoggingContexts
2017-04-03 17:02:25 +01:00
Richard van der Hoff 30bcbf775a Accept join events from all servers
Make sure that we accept join events from any server, rather than just the
origin server, to make the federation join dance work correctly.

(Fixes #1893).
2017-04-03 15:58:07 +01:00
Richard van der Hoff 7eb9f34cc3 Remove spurious yield
In `MessageHandler`, remove `yield` on call to `Notifier.on_new_room_event`:
it doesn't return anything anyway.
2017-04-03 15:44:19 +01:00
Richard van der Hoff 0b08c48fc5 Remove more spurious `PreserveLoggingContext`s
Remove `PreserveLoggingContext` around calls to `Notifier.on_new_room_event`;
there is no problem if the logcontext is set when calling it.
2017-04-03 15:43:37 +01:00
Erik Johnston 36d2b66f90 Add a timestamp to USER_SYNC command
This timestamp is used to indicate when the user last sync'd
2017-03-31 15:42:22 +01:00
Erik Johnston 9d0170ac6c Fix up presence 2017-03-31 11:36:32 +01:00
Erik Johnston 63fcc42990 Remove user from process_presence when stops syncing 2017-03-30 14:26:08 +01:00
Erik Johnston e9dd8370b0 Add functions to presence to support remote syncs
The TCP replication protocol streams deltas of who has started or
stopped syncing. This is different from the HTTP API which periodically
sends the full list of users who are syncing. This commit adds support
for the new TCP style of sending deltas.
2017-03-30 13:25:14 +01:00
Erik Johnston 24d35ab47b Add new storage functions for new replication
The new replication protocol will keep all the streams separate, rather
than muxing multiple streams into one.
2017-03-30 11:48:35 +01:00
Erik Johnston 7fc1f1e2b6 Cache hosts in room 2017-03-24 11:46:24 +00:00
Richard van der Hoff 9397edb28b Merge pull request #2050 from matrix-org/rav/federation_backoff
push federation retry limiter down to matrixfederationclient
2017-03-23 22:27:01 +00:00
Richard van der Hoff 5a16cb4bf0 Ignore backoff history for invites, aliases, and roomdirs
Add a param to the federation client which lets us ignore historical backoff
data for federation queries, and set it for a handful of operations.
2017-03-23 12:23:22 +00:00
Richard van der Hoff 4bd597d9fc push federation retry limiter down to matrixfederationclient
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
Richard van der Hoff e08f81d96a Add a missing yield in device key upload
(this would only very very rarely actually be a useful thing, so the main
problem was the logcontext leak...)
2017-03-23 00:16:43 +00:00
Richard van der Hoff eddce9d74a Merge pull request #2027 from matrix-org/rav/logcontext_leaks
A few fixes to logcontext things
2017-03-20 11:53:36 +00:00
Richard van der Hoff f40c2db05a Stop preserve_fn leaking context into the reactor
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.

Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
2017-03-18 00:07:43 +00:00
Richard van der Hoff 5068fb16a5 Refactoring and cleanups
A few non-functional changes:

* A bunch of docstrings to document types
* Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more
  readable.
* Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid
  mind-bending conditional.
* Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid
  mind-bending conditional.
2017-03-17 15:06:07 +00:00
Richard van der Hoff 2abe85d50e Merge pull request #2016 from matrix-org/rav/queue_pdus_during_join
Queue up federation PDUs while a room join is in progress
2017-03-17 11:32:44 +00:00
Erik Johnston da146657c9 Comments 2017-03-16 13:04:07 +00:00
Erik Johnston a158c36a8a Comment 2017-03-16 11:57:45 +00:00
Erik Johnston 6957bfdca6 Don't recreate so many sets 2017-03-16 11:54:26 +00:00
Erik Johnston 2ccf3b241c Implement no op for room stream in sync 2017-03-16 11:06:41 +00:00
Richard van der Hoff 9ce53a3861 Queue up federation PDUs while a room join is in progress
This just takes the existing `room_queues` logic and moves it out to
`on_receive_pdu` instead of `_process_received_pdu`, which ensures that we
don't start trying to fetch prev_events and whathaveyou until the join has
completed.
2017-03-15 18:01:11 +00:00
Erik Johnston 54d2b7e596 Merge pull request #2014 from Half-Shot/hs/fix-appservice-presence
Add fallback to last_active_ts if it beats the last sync time on a presence timeout.
2017-03-15 17:37:15 +00:00
Erik Johnston f83ac78201 Cache set of users whose presence the other user should see 2017-03-15 15:29:19 +00:00
Will Hunt e6032054bf Add a great comment to handle_timeout for active vs sync times. 2017-03-15 15:24:48 +00:00
Will Hunt ebf5a6b14c Add fallback to last_active_ts if it beats the last sync time. 2017-03-15 15:17:16 +00:00
Erik Johnston e892457a03 Comment 2017-03-15 15:01:39 +00:00
Erik Johnston 6c82de5100 Format presence events on the edges instead of reformatting them multiple times 2017-03-15 14:27:34 +00:00
Erik Johnston 7827251daf Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2
Phone number registration / login support v2
2017-03-15 09:59:54 +00:00
Erik Johnston bb256ac96f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_list_speed 2017-03-14 11:35:05 +00:00
Luke Barnard bbeeb97f75 Implement _simple_delete_many_txn, use it to delete devices
(But this doesn't implement the same for deleting access tokens or e2e keys.

Also respond to code review.
2017-03-13 17:53:23 +00:00
David Baker 73a5f06652 Support registration / login with phone number
Changes from https://github.com/matrix-org/synapse/pull/1971
2017-03-13 17:27:51 +00:00
Richard van der Hoff 3c69f32402 Merge remote-tracking branch 'origin/develop' into rav/refactor_received_pdu 2017-03-13 12:20:47 +00:00
Erik Johnston 0162994983 Comments 2017-03-13 11:53:26 +00:00
Erik Johnston 7eae6eaa2f Revert "Support registration & login with phone number" 2017-03-13 09:59:33 +00:00
Erik Johnston 79926e016e Assume rooms likely haven't changed 2017-03-13 09:50:10 +00:00
Erik Johnston 8ffbe43ba1 Get current state by using current_state_events table 2017-03-10 17:39:35 +00:00
Erik Johnston 64d62e41b8 Noop repated delete device inbox calls from /sync 2017-03-10 10:36:43 +00:00
Richard van der Hoff 29235901b8 Move FederationServer._handle_new_pdu to FederationHandler
Unfortunately this significantly increases the size of the already-rather-big
FederationHandler, but the code fits more naturally here, and it paves the way
for the tighter integration that I need between handling incoming PDUs and
doing the join dance.

Other than renaming the existing `FederationHandler.on_receive_pdu` to
`_process_received_pdu` to make way for it, this just consists of the move, and
replacing `self.handler` with `self` and `self` with `self.replication_layer`.
2017-03-09 16:20:13 +00:00
David Baker 663396e45d Merge pull request #1971 from matrix-org/dbkr/msisdn_signin
Support registration & login with phone number
2017-03-09 10:18:51 +00:00
David Baker 0e0aee25c4 Fix log line 2017-03-08 11:46:22 +00:00
David Baker 88df6c0c9a Factor out msisdn canonicalisation
Plus a couple of other minor fixes
2017-03-08 11:03:39 +00:00
David Baker 402a7bf63d Fix pep8 2017-03-08 09:33:40 +00:00
Erik Johnston f2581ee8b8 Don't keep around old stream IDs forever 2017-03-03 16:02:53 +00:00
Erik Johnston 9834367eea Spelling 2017-03-03 15:31:57 +00:00
David Baker ad882cd54d Just return the deferred straight off
defer.returnValue doth not maketh a generator: it would need a
yield to be a generator, and this doesn't need a yield.
2017-03-01 18:08:51 +00:00
David Baker 3557cf34dc Merge remote-tracking branch 'origin/develop' into dbkr/msisdn_signin 2017-03-01 17:20:37 +00:00
Erik Johnston 36be39b8b3 Fix device list update to not constantly resync 2017-03-01 14:12:11 +00:00
Erik Johnston 848cf95ea0 Pop with default value to stop throwing 2017-02-28 10:02:54 +00:00
Erik Johnston 64a2cef9bb Pop rather than del from dict 2017-02-27 19:15:36 +00:00
Erik Johnston 17673404fb Remove unused param 2017-02-20 15:02:01 +00:00
Erik Johnston 7f026792e1 Fix /context/ visibiltiy rules 2017-02-20 14:54:50 +00:00
David Baker 8c87bb550e Merge pull request #1922 from matrix-org/dbkr/allow_forget_for_ban
Allow forgetting rooms you're banned from
2017-02-17 10:52:30 +00:00