Commit Graph

1274 Commits (c1267d04c5fdd900ea917682056b1239495f90ee)

Author SHA1 Message Date
Richard van der Hoff a92b4ea76f Make sure we add all invited members before returning from createRoom
add a missing yield.
2016-03-04 00:06:03 +00:00
Richard van der Hoff a85179aff3 Merge remote-tracking branch 'origin/develop' into rav/SYN-642 2016-03-03 19:05:54 +00:00
Daniel Wagner-Hall 0b3083c75b Merge pull request #621 from matrix-org/daniel/ratelimiting
Pass whole requester to ratelimiting
2016-03-03 17:02:48 +00:00
Daniel Wagner-Hall b4022cc487 Pass whole requester to ratelimiting
This will enable more detailed decisions
2016-03-03 16:43:42 +00:00
Mark Haines ddf9e7b302 Hook up the push rules to the notifier 2016-03-03 14:57:45 +00:00
Richard van der Hoff 79f34bdbc2 Merge branch 'develop' into rav/SYN-642 2016-03-03 11:39:25 +00:00
Richard van der Hoff 74cd80e530 Fix typo 2016-03-03 10:28:58 +00:00
David Baker ff8b87118d Stop using checkpw as it seems to have vanished from bcrypt. Use `bcrypt.hashpw(password, hashed) == hashed` as per the bcrypt README. 2016-03-02 18:06:45 +00:00
Richard van der Hoff fc1f932cc0 Move arg default to the start of the function
Also don't overwrite the list that gets passed in.
2016-03-02 16:44:14 +00:00
Richard van der Hoff 863d3f26b3 fix pyflakes quibble 2016-03-02 15:52:50 +00:00
Richard van der Hoff 9ff940a0ef Address review comments 2016-03-02 15:40:30 +00:00
Erik Johnston 2a78dac60d Merge pull request #612 from matrix-org/erikj/cache_size
Add environment variable SYNAPSE_CACHE_FACTOR, default it to 0.1
2016-03-02 14:36:10 +00:00
Richard van der Hoff 05ea111c47 Fix pyflakes warning 2016-03-01 17:45:24 +00:00
Richard van der Hoff 8a1d3b86af Handle rejections of invites from local users locally
Slightly hacky fix to SYN-642, which avoids the federation codepath when trying
to reject invites from local users.
2016-03-01 17:27:22 +00:00
Mark Haines 60a0f81c7a Add a /replication API for extracting the updates that happened on
synapse

This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.

This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.

Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.

Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.

The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
2016-03-01 14:49:41 +00:00
Erik Johnston f9af8962f8 Allow alias creators to delete aliases 2016-03-01 14:46:31 +00:00
Erik Johnston 910fc0f28f Add enviroment variable SYNAPSE_CACHE_FACTOR, default it to 0.1 2016-03-01 12:56:39 +00:00
Patrik Oldsberg 9c48f1ed22 handlers/register: make sure another user id is generated when a collision occurs
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-02-29 23:12:37 +01:00
David Baker 0f0b011440 Send the invier's member event in room invite state so the invitee has their display name and avatar. 2016-02-25 18:12:09 +00:00
Daniel Wagner-Hall 33300673b7 Generate guest access token on 3pid invites
This means that following the same link across multiple sessions or
devices can re-use the same guest account.

Note that this is somewhat of an abuse vector; we can't throw up
captchas on this flow, so this is a way of registering ephemeral
accounts for spam, whose sign-up we don't rate limit.
2016-02-24 14:41:25 +00:00
Erik Johnston b5f77eb12a Check presence token interval is less than 100, rather than the token itself 2016-02-23 15:47:37 +00:00
Erik Johnston e3e0ac6ec7 Merge pull request #602 from matrix-org/erikj/presence
Change the way we figure out presence updates for small deltas
2016-02-23 15:18:41 +00:00
Daniel Wagner-Hall 577951b032 Allow third_party_signed to be specified on /join 2016-02-23 15:11:25 +00:00
Erik Johnston 13f86c3489 Handle get_all_entities_changed returning None 2016-02-23 15:05:37 +00:00
Erik Johnston 6e0209112b Add comments 2016-02-23 14:57:45 +00:00
Erik Johnston c77dae7a1a Change the way we figure out presence updates for small deltas 2016-02-23 14:54:40 +00:00
Erik Johnston a7b2ce32f7 Merge pull request #600 from matrix-org/erikj/presence
Measure PresenceEventSource.get_new_events
2016-02-23 14:54:26 +00:00
Erik Johnston 02e928cf9b Don't include defer.returnValue in Measure block 2016-02-23 14:52:16 +00:00
Erik Johnston 56a94ccd9e Measure PresenceEventSource.get_new_events 2016-02-23 14:52:16 +00:00
Patrik Oldsberg baf056bae8 handlers/sync: style fix, line too long
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-02-23 15:49:36 +01:00
Erik Johnston 138c405974 Pick up currently_active across federation 2016-02-23 10:40:11 +00:00
Patrik Oldsberg 210b7d8e00 handlers/_base: don't allow room create event to be changed
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-02-23 00:22:41 +01:00
Patrik Oldsberg 1dcfb201c4 handlers/sync: fix SyncResult not counting account_data change when converting to bool
This fixes account_data events not triggering an immediate /sync response

Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-02-23 00:21:59 +01:00
Richard van der Hoff f7e3de02ef Merge pull request #587 from matrix-org/rav/guest_access_after_room_join
Allow guest users access to messages in rooms they have joined
2016-02-22 16:36:06 +00:00
Richard van der Hoff 5be3944730 address review comments
drop commented-out special casing for historyvisibility event
s/he/they/ for users
2016-02-22 15:27:44 +00:00
Erik Johnston 9e696bd6a3 Remove superfluous call to get_state_at when we already have an event for that stream position 2016-02-22 13:54:46 +00:00
Richard van der Hoff 6c5b147a39 Interpret unknown visibilities the same as shared 2016-02-19 17:11:11 +00:00
Erik Johnston 3dbaeef58c Correctly filter states 2016-02-19 12:27:35 +00:00
Erik Johnston 42ac5f0c1a Only send presence updates to remote hosts if user is ours 2016-02-19 12:19:56 +00:00
Richard van der Hoff 05aee12652 Merge branch 'develop' into rav/guest_access_after_room_join 2016-02-19 12:00:16 +00:00
Erik Johnston 24d9f2c140 Add Measures to presence 2016-02-19 11:50:48 +00:00
Richard van der Hoff b71ca2b014 Allow guest users access to messages in rooms they have joined
There should be no difference between guest users and non-guest users in terms
of access to messages. Define the semantics of the is_peeking argument to
filter_events_for_clients (slightly) better; interpret it appropriately, and
set it correctly from /sync.
2016-02-19 11:41:02 +00:00
Erik Johnston 4a95eb0a12 Add presence metric 2016-02-19 11:32:04 +00:00
Erik Johnston be799453aa Remove spurious extra metrics 2016-02-19 11:29:33 +00:00
Erik Johnston ea7786e8ca Merge pull request #586 from matrix-org/erikj/presence
Fix presence `currently_active`. Add presence metrics.
2016-02-19 11:26:32 +00:00
Erik Johnston 929cb0ed7d Don't set currently_active for remote presence 2016-02-19 10:58:27 +00:00
Erik Johnston 5f4eca3816 More metrics 2016-02-19 10:21:41 +00:00
Erik Johnston 5614b4dafb Add presence metrics 2016-02-19 09:50:54 +00:00
Erik Johnston e5ad2e5267 Merge pull request #582 from matrix-org/erikj/presence
Rewrite presence for performance.
2016-02-19 09:37:50 +00:00
Erik Johnston 220231d8e3 Merge pull request #573 from matrix-org/erikj/sync_fix
Mitigate against incorrect old state in /sync.
2016-02-18 16:40:58 +00:00
Erik Johnston 48b652bcbe Remove invalid arg. 2016-02-18 14:57:09 +00:00
Erik Johnston b4796a62ee Add unit test 2016-02-18 11:52:33 +00:00
Daniel Wagner-Hall f8d21e1431 Review comments 2016-02-18 11:02:14 +00:00
Erik Johnston fe95f2217c Add stuff pulled from the DB to the cache 2016-02-18 10:26:24 +00:00
Erik Johnston 8351538873 PEP8 2016-02-18 10:12:12 +00:00
Erik Johnston 112283e230 Prefix TS fields with _ts 2016-02-18 10:11:43 +00:00
Erik Johnston b31ec214a5 Remove status_msg when going offline. Don't offline -> online if you send a message 2016-02-18 09:54:08 +00:00
Erik Johnston 114b929f8b Check presence state is a valid one 2016-02-18 09:16:32 +00:00
Erik Johnston ddca9c56fc Move if statement 2016-02-18 09:11:53 +00:00
Erik Johnston 58371fa263 Comment 2016-02-18 09:09:50 +00:00
Daniel Wagner-Hall 591af2d074 Some cleanup
I'm not particularly happy with the "action" switching, but there's no
convenient way to defer the work that needs to happen after it, so... :(
2016-02-17 15:50:13 +00:00
Erik Johnston c229c87398 Remove spurious comment 2016-02-17 15:48:29 +00:00
Erik Johnston e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall a4e278bfe7 Respond to federated invite with non-empty context
Currently, we magically perform an extra database hit to find the
inviter, and use this to guess where we should send the event. Instead,
fill in a valid context, so that other callers relying on the context
actually have one.
2016-02-17 15:25:12 +00:00
Daniel Wagner-Hall 6605adf669 Some cleanup, some TODOs, more to do 2016-02-16 19:05:02 +00:00
Daniel Wagner-Hall d1fb790818 Some cleanup 2016-02-16 14:25:23 +00:00
Daniel Wagner-Hall 1f403325ac Tidy? up room creation event sending 2016-02-16 12:00:50 +00:00
Daniel Wagner-Hall 04686df17a Add comment 2016-02-16 11:52:46 +00:00
Daniel Wagner-Hall feedaa37fa Merge branch 'develop' into daniel/roomcleanupincremental
Conflicts:
	synapse/handlers/room.py
2016-02-16 11:34:48 +00:00
Daniel Wagner-Hall 4bfb32f685 Branch off member and non member sends
Unclean, needs tidy-up, but works
2016-02-15 18:21:30 +00:00
Daniel Wagner-Hall 1a2197d7bf Simplify room creation code 2016-02-15 18:19:01 +00:00
Daniel Wagner-Hall e560045cfd Simplify room creation code 2016-02-15 18:18:39 +00:00
Daniel Wagner-Hall 8168341e9b Use update_membership for profile updates 2016-02-15 17:14:34 +00:00
Daniel Wagner-Hall 1bbb67c452 Use update_membership to kick guests 2016-02-15 16:40:22 +00:00
Daniel Wagner-Hall 150fcde0dc Reuse update_membership from /join 2016-02-15 16:16:03 +00:00
Daniel Wagner-Hall 73e616df2a Inline _do_local_membership_update 2016-02-15 16:02:22 +00:00
Daniel Wagner-Hall f318d4f2a4 Inline _do_join as it now only has one caller
Also, consistently apply rate limiting.

Again, ugly, but a step in the right direction.
2016-02-15 15:57:10 +00:00
Daniel Wagner-Hall e71095801f Merge implementation of /join by alias or ID
This code is kind of rough (passing the remote servers down a long
chain), but is a step towards improvement.
2016-02-15 15:39:16 +00:00
Daniel Wagner-Hall dbeed36dec Merge some room joining codepaths
Force joining by alias to go through the send_membership_event checks,
rather than bypassing them straight into _do_join. This is the first of
many stages of cleanup.
2016-02-15 14:38:27 +00:00
Daniel Wagner-Hall 4de08a4672 Revert "Merge two of the room join codepaths"
This reverts commit cf81375b94.

It subtly violates a guest joining auth check
2016-02-12 16:17:24 +00:00
Daniel Wagner-Hall cf81375b94 Merge two of the room join codepaths
There's at least one more to merge in.

Side-effects:
 * Stop reporting None as displayname and avatar_url in some cases
 * Joining a room by alias populates guest-ness in join event
 * Remove unspec'd PUT version of /join/<room_id_or_alias> which has not
   been called on matrix.org according to logs
 * Stop recording access_token_id on /join/room_id - currently we don't
   record it on /join/room_alias; I can try to thread it through at some
   point.
2016-02-12 15:11:49 +00:00
Erik Johnston 4d54d87c3e Mitigate against incorrect old state in /sync.
There have been reports from the field that servers occasionally have
incorrect notions of what the old state of a room is. This proves
problematic as /sync relies on a correct old state.
This patch makes /sync specifically include in the 'state' key any
current state events that haven't been correctly included.
2016-02-12 11:13:06 +00:00
Daniel Wagner-Hall dc6da63e30 Enforce user_id exclusivity for AS registrations
This whole set of checks is kind of an ugly mess, which I may clean up
at some point, but for now let's be correct.
2016-02-11 17:37:38 +00:00
Erik Johnston 0eff740523 Return events in correct order for /events 2016-02-11 10:07:27 +00:00
Erik Johnston 24f00a6c33 Use _simple_select_many for _get_state_group_for_events 2016-02-10 12:57:50 +00:00
Erik Johnston 8e49892b21 Only calculate initial sync for 10 rooms at a time
This helps to ensure we don't completely starve other requests.
2016-02-10 11:42:07 +00:00
Erik Johnston 7b0d846407 Atomically persit push actions when we persist the event 2016-02-09 16:19:15 +00:00
Erik Johnston f28cc45183 Pass in current state to push action handler 2016-02-09 16:01:40 +00:00
Erik Johnston feb294d552 Remove dead code 2016-02-09 14:32:17 +00:00
Erik Johnston eff12e838c Don't load all ephemeral state for a room on every sync 2016-02-09 13:55:59 +00:00
Erik Johnston 82631c5f94 Fix unit tests 2016-02-09 13:50:37 +00:00
Erik Johnston 6c558ee8bc Measure some /sync related things 2016-02-09 11:31:42 +00:00
Erik Johnston 2c1fbea531 Fix up logcontexts 2016-02-08 14:26:45 +00:00
Daniel Wagner-Hall 79a1c0574b Allocate guest user IDs numericcally
The current random IDs are ugly and confusing when presented in UIs.
This makes them prettier and easier to read.

Also, disable non-automated registration of numeric IDs so that we don't
need to worry so much about people carving out our automated address
space and us needing to keep retrying ID registration.
2016-02-05 11:22:30 +00:00
Erik Johnston c8e4d5de7f Merge pull request #555 from matrix-org/erikj/logcontext
Allowing tagging log contexts
2016-02-03 15:20:00 +00:00
Erik Johnston 8450114098 Merge pull request #554 from matrix-org/erikj/event_push
Change event_push_actions_rm_tokens schema
2016-02-03 15:02:47 +00:00
Erik Johnston d4f72a5bfb Allowing tagging log contexts 2016-02-03 13:52:27 +00:00
Erik Johnston 9cd80a7b5c PEP8 2016-02-03 11:52:57 +00:00
Erik Johnston 772b45c745 Remove unused method 2016-02-03 11:43:26 +00:00
Erik Johnston 6f52e90065 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_room_fix 2016-02-03 11:06:29 +00:00
Erik Johnston 771528ab13 Change event_push_actions_rm_tokens schema 2016-02-03 10:50:49 +00:00
Daniel Wagner-Hall d83d004ccd Fix flake8 warnings for new flake8 2016-02-02 17:18:50 +00:00
Erik Johnston 8a391e33ae s/get_room_changes_for_user/get_membership_changes_for_user/ 2016-02-02 16:12:10 +00:00
Erik Johnston 04ad93e6fd Merge pull request #545 from matrix-org/erikj/sync
Move /sync state calculations from rest to handler
2016-02-02 15:28:43 +00:00
Erik Johnston 65e92eca49 Change the way we do public room list fetching 2016-02-02 15:21:10 +00:00
David Baker 69214ea671 Pass make_guest whne we autogen a user ID 2016-02-02 14:42:31 +00:00
Erik Johnston b023995538 WARN if we get a topo token instead of stream. 2016-02-02 14:11:14 +00:00
Erik Johnston 854ca32f10 Comments 2016-02-01 16:52:27 +00:00
Erik Johnston 89b40b225c Order things correctly 2016-02-01 16:32:46 +00:00
Erik Johnston 4bf448be25 Switch over /events to use per room caches 2016-02-01 16:26:51 +00:00
Erik Johnston fa48020a52 Move state calculations from rest to handler 2016-02-01 15:59:40 +00:00
Mark Haines 6927d0e091 Add missing param to the log line 2016-01-29 15:01:26 +00:00
Mark Haines 47374a33fc Merge remote-tracking branch 'origin/develop' into markjh/3pid 2016-01-29 14:15:12 +00:00
Mark Haines 0fcafbece8 Add config option for setting the trusted id servers, disabling checking the ID server in integration tests 2016-01-29 14:12:26 +00:00
Erik Johnston ea320d3464 Don't work out unread_notifs_for_room_id unless needed 2016-01-29 13:34:48 +00:00
Erik Johnston 571a566399 Change load limit params 2016-01-28 14:11:16 +00:00
Erik Johnston 3c6518ddbf Amalgamate incremental and full sync for user 2016-01-28 14:03:48 +00:00
Erik Johnston 4e7948b47a Allow paginating backwards from stream token 2016-01-28 11:52:34 +00:00
Erik Johnston 61eaa6ec64 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync 2016-01-28 10:50:57 +00:00
Richard van der Hoff 5cba88ea7c Make it possible to paginate forwards from stream tokens
In order that we can fill the gap after a /sync, make it possible to paginate
forwards from a stream token.
2016-01-27 17:42:45 +00:00
Erik Johnston aca3193efb Use the same path for incremental with gap or without gap 2016-01-27 17:33:27 +00:00
Erik Johnston b97f6626b6 Add cache to room stream 2016-01-27 17:33:26 +00:00
Richard van der Hoff d6d60b4d6c Federation: drop events which cause SynapseErrors
... rather than rejecting any attempt to federate channels which contain such
events.
2016-01-27 17:02:10 +00:00
Erik Johnston c887c4cbd5 Merge pull request #524 from matrix-org/erikj/sync
Move some sync logic from rest to handlers pacakege
2016-01-25 16:58:39 +00:00
Erik Johnston dc2647cd3d PEP8 2016-01-25 15:48:54 +00:00
Erik Johnston d59c58bc95 Remove weird stuff 2016-01-25 13:38:53 +00:00
Erik Johnston 4021f95261 Move logic from rest/ to handlers/ 2016-01-25 10:10:44 +00:00
Erik Johnston 7959e8b764 Underscores are allowed in user ids 2016-01-22 14:59:49 +00:00
Erik Johnston 0e07f2e15d Only fetch events for rooms and receipts 2016-01-21 16:10:37 +00:00
Erik Johnston b088291f14 Don't explode when given a unicode username in /register/ 2016-01-20 15:40:25 +00:00
Daniel Wagner-Hall da417aa56d Allow non-guests to peek on rooms using /events 2016-01-20 15:34:07 +00:00
Erik Johnston af30140621 Merge pull request #506 from matrix-org/erikj/push_fast
Only compute unread notifications for rooms we send down stream
2016-01-19 17:31:10 +00:00
Erik Johnston ac2842ff1e Only compute unread notifications for rooms we send down stream 2016-01-19 17:19:53 +00:00
Erik Johnston 892ee473d9 Don't use form of get_state_for_events with None state_key 2016-01-19 17:14:46 +00:00
Erik Johnston 5a7d1ecffc Add regex cache. Only caculate push actions for users that have sent read receipts, and are on that server 2016-01-19 16:01:05 +00:00
Erik Johnston b8518ffe65 Use all_ephemeral_by_room in incremental_sync_with_gap_for_room 2016-01-19 14:26:58 +00:00
Erik Johnston 3adcc4c86a Return highlight_count in /sync 2016-01-19 11:35:50 +00:00
Erik Johnston 63485b3029 Re-enable urnead notifications 2016-01-18 14:48:30 +00:00
Erik Johnston cc66a9a5e3 Allow filtering events for multiple users at once 2016-01-18 14:43:50 +00:00
Daniel Wagner-Hall ac5a4477ad Require unbanning before other membership changes 2016-01-15 16:27:26 +00:00
Daniel Wagner-Hall b5ce4f0427 Remove unused parameters 2016-01-15 13:54:03 +00:00
David Baker 5819b7a78c M_INVALID_USERNAME to be consistent with the parameter name 2016-01-15 10:06:34 +00:00
David Baker 3f8db3d597 Add specific error code for invalid user names. 2016-01-14 17:21:04 +00:00
Richard van der Hoff 2978053d16 Merge branch 'release-v0.12.1' into develop 2016-01-14 15:04:08 +00:00
Erik Johnston bce602eb4e Use logger not logging 2016-01-14 09:56:26 +00:00
Erik Johnston 2655d61d70 Don't change signature. Return empty list 2016-01-13 17:43:39 +00:00
Erik Johnston fcb05b4c82 Temporarily disable notification branch 2016-01-13 17:39:58 +00:00
Richard van der Hoff 49f33f6438 Add 'event' result to 'context' endpoint
... because the context isn't much use without the event.
2016-01-13 16:42:14 +00:00
Daniel Wagner-Hall 93afb40cd4 Skip, rather than erroring, invalid guest requests
Erroring causes problems when people make illegal requests, because they
don't know what limit parameter they should pass.

This is definitely buggy. It leaks message counts for rooms people don't
have permission to see, via tokens. But apparently we already
consciously decided to allow that as a team, so this preserves that
behaviour.
2016-01-13 16:41:42 +00:00
Erik Johnston 72ba26679b Merge pull request #480 from matrix-org/erikj/guest_event_tightloop
Dont fire user_joined_room when guest hits /events
2016-01-13 11:00:50 +00:00