Commit Graph

5556 Commits (0b1e287e81cb81261e45e94d5acf618976c7bd29)

Author SHA1 Message Date
Erik Johnston f96020550f Update comments 2016-09-22 12:54:22 +01:00
Erik Johnston 81964aeb90 Merge pull request #1132 from matrix-org/erikj/initial_sync_split
Support /initialSync in synchrotron worker
2016-09-22 12:45:02 +01:00
Erik Johnston 2e9ee30969 Add comments 2016-09-22 11:59:46 +01:00
Erik Johnston a61e4522b5 Shuffle things around to make unit tests work 2016-09-22 11:08:12 +01:00
Erik Johnston 1168cbd54d Allow invites via 3pid to bypass sender sig check
When a server sends a third party invite another server may be the one
that the inviting user registers with. In this case it is that remote
server that will issue an actual invitation, and wants to do it "in the
name of" the original invitee. However, the new proper invite will not
be signed by the original server, and thus other servers would reject
the invite if it was seen as coming from the original user.

To fix this, a special case has been added to the auth rules whereby
another server can send an invite "in the name of" another server's
user, so long as that user had previously issued a third party invite
that is now being accepted.
2016-09-22 10:56:53 +01:00
Erik Johnston bbc0d9617f Merge pull request #1134 from matrix-org/erikj/fix_stream_public_deletion
Fix _delete_old_forward_extrem_cache query
2016-09-21 17:04:04 +01:00
Erik Johnston 8009d84364 Match against event_id, rather than room_id 2016-09-21 16:46:59 +01:00
Erik Johnston dc692556d6 Remove spurious AS clause 2016-09-21 16:28:47 +01:00
Erik Johnston dc78db8c56 Update correct table 2016-09-21 15:52:44 +01:00
Erik Johnston 4f78108d8c Readd entries to public_room_list_stream that were deleted 2016-09-21 15:24:22 +01:00
Erik Johnston 0b78d8adf2 Fix _delete_old_forward_extrem_cache query 2016-09-21 15:20:56 +01:00
Erik Johnston 90c070c850 Add total_room_count_estimate to /publicRooms 2016-09-21 13:30:05 +01:00
Erik Johnston 87528f0756 Support /initialSync in synchrotron worker 2016-09-21 11:46:28 +01:00
Erik Johnston 2b8ff4659f Bump version and changelog 2016-09-19 17:16:56 +01:00
Erik Johnston ddfcdd4778 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.18.0 2016-09-19 17:15:24 +01:00
Erik Johnston 6f0c5e5d9b Merge pull request #1131 from matrix-org/matthew/e2e-notifs
Notify on e2e events
2016-09-19 10:50:26 +01:00
Erik Johnston 49cf205dc7 _id field must uniquely identify different conditions 2016-09-19 10:34:01 +01:00
Erik Johnston 39af634dd2 Merge pull request #1130 from matrix-org/erikj/fix_pubroom_pag
Handle fact that _generate_room_entry may not return a room entry
2016-09-19 10:13:59 +01:00
Matthew Hodgson 3f6ec271ba proposal for notifying on e2e events 2016-09-17 22:05:06 +01:00
Erik Johnston 4d49e0bdfd PEP8 2016-09-17 18:09:22 +01:00
Erik Johnston 81570abfb2 Handle fact that _generate_room_entry may not return a room entry 2016-09-17 18:01:54 +01:00
Erik Johnston ddc89df89d Enable guest access to POST /publicRooms 2016-09-17 15:55:24 +01:00
Erik Johnston eb24aecf8c Merge pull request #1129 from matrix-org/erikj/fix_pubroom_pag
Fix and clean up publicRooms pagination
2016-09-17 15:30:34 +01:00
Erik Johnston e1ba98d724 Merge pull request #1127 from matrix-org/dbkr/publicroom_search_case_insensitive
Make public room search case insensitive
2016-09-17 15:01:17 +01:00
Erik Johnston a298331de4 Spelling 2016-09-17 14:59:40 +01:00
Erik Johnston 71edaae981 Fix and clean up publicRooms pagination 2016-09-17 14:46:19 +01:00
Matthew Hodgson 883df2e983 fix logger for client_reader worker 2016-09-17 14:12:04 +01:00
David Baker 5336acd46f Make public room search case insensitive 2016-09-16 19:02:42 +01:00
Erik Johnston 19fe990476 Update changelog and bump version 2016-09-16 17:30:59 +01:00
Erik Johnston 995f2f032f Fix public room pagination for client_reader app 2016-09-16 14:48:21 +01:00
Erik Johnston a68807d426 Comment 2016-09-16 11:36:20 +01:00
Erik Johnston 2e67cabd7f Make POST /publicRooms require auth 2016-09-16 11:32:51 +01:00
Erik Johnston b7b62bf9ea Comment 2016-09-16 11:00:29 +01:00
Erik Johnston d84319ae10 Add remote reoom cache 2016-09-16 10:31:59 +01:00
Erik Johnston 23b6701a28 Support filtering remote room lists 2016-09-16 10:24:15 +01:00
Erik Johnston e58a9d781c Filter remote rooms lists locally 2016-09-16 10:19:32 +01:00
Erik Johnston 74d4cdee25 Don't cache searched in /publicRooms 2016-09-16 09:05:11 +01:00
Erik Johnston 418bcd4309 Add new storage function to slave store 2016-09-16 08:37:39 +01:00
Erik Johnston 098db4aa52 Add very basic filter API to /publicRooms 2016-09-15 17:50:16 +01:00
Erik Johnston c33b25fd8d Change the way we calculate new_limit in /publicRooms and add POST API 2016-09-15 17:35:20 +01:00
Erik Johnston de4f798f01 Handling expiring stream extrems correctly. 2016-09-15 17:34:59 +01:00
Erik Johnston ea6dc356b0 Merge pull request #1125 from matrix-org/erikj/public_room_cache
Change get_pos_of_last_change to return upper bound
2016-09-15 15:48:53 +01:00
Erik Johnston 955f34d23e Change get_pos_of_last_change to return upper bound 2016-09-15 15:12:07 +01:00
Erik Johnston 241d7d2d62 Merge pull request #1124 from matrix-org/erikj/enable_state_caching_workers
Enable state caches on workers
2016-09-15 15:01:47 +01:00
Erik Johnston 4be85281f9 Enable state caches on workers 2016-09-15 14:31:22 +01:00
Erik Johnston cb3edec6af Use stream_change cache to make get_forward_extremeties_for_room cache more effective 2016-09-15 14:28:13 +01:00
Erik Johnston 55e6fc917c Add cache to get_forward_extremeties_for_room 2016-09-15 14:04:28 +01:00
Erik Johnston 68c1ed4d1a Remove default public rooms limit 2016-09-15 13:56:20 +01:00
Erik Johnston b82fa849c8 Merge pull request #1120 from matrix-org/erikj/push_invite_cache
Ensure we don't mutate the cache of push rules
2016-09-15 13:27:18 +01:00
Erik Johnston e457034e99 Merge pull request #1121 from matrix-org/erikj/public_room_paginate
Add pagination support to publicRooms
2016-09-15 13:27:09 +01:00
Erik Johnston 1d98cf26be By default limit /publicRooms to 100 entries 2016-09-15 13:18:35 +01:00
Erik Johnston 211786ecd6 Stream public room changes down replication 2016-09-15 11:47:23 +01:00
Erik Johnston 4fb65a1091 Base public room list off of public_rooms stream 2016-09-15 11:27:04 +01:00
Erik Johnston 5810cffd33 Pass since/from parameters over federation 2016-09-15 10:36:19 +01:00
Erik Johnston f3eead0660 Allow paginating both forwards and backwards 2016-09-15 10:15:37 +01:00
Erik Johnston 4131381123 Remove support for aggregate room lists 2016-09-15 09:28:15 +01:00
Erik Johnston 6a5ded5988 Ensure we don't mutate the cache of push rules 2016-09-15 09:16:13 +01:00
Erik Johnston 4f181f361d Accept optional token to public room list 2016-09-15 09:08:57 +01:00
Erik Johnston c566f0ee17 Calculate the public room list from a stream_ordering 2016-09-14 17:42:47 +01:00
Erik Johnston 772c6067a3 Refactor public rooms to not pull out the full state for each room 2016-09-14 17:29:25 +01:00
Erik Johnston baffe96d95 Add a room visibility stream 2016-09-14 17:29:19 +01:00
Erik Johnston 264a48aedf Merge pull request #1117 from matrix-org/erikj/fix_state
Ensure we don't mutate state cache entries
2016-09-14 16:50:37 +01:00
Erik Johnston ed992ae6ba Add a DB index to figure out past state at a stream ordering in a room 2016-09-14 16:20:27 +01:00
Erik Johnston e0b6db29ed Split out public room list into a worker process 2016-09-14 14:42:51 +01:00
Erik Johnston a70a43bc51 Move RoomListHandler into a separate file 2016-09-14 14:07:37 +01:00
Erik Johnston f2b2cd8eb4 Amalgamate two identical consecutive if statements 2016-09-14 11:16:22 +01:00
Erik Johnston 00f51493f5 Fix reindex 2016-09-14 10:18:30 +01:00
Erik Johnston d5ae1f1291 Ensure we don't mutate state cache entries 2016-09-14 10:03:48 +01:00
Matthew Hodgson 1b01488d27 Merge pull request #1111 from matrix-org/matthew/device-ids
make device IDs more useful for human disambiguation
2016-09-14 01:18:02 +01:00
Paul Evans 0f73f0e70e Merge pull request #1116 from matrix-org/paul/tiny-fixes
Fix typo "persiting"
2016-09-13 13:27:42 +01:00
Paul "LeoNerd" Evans ca35e54d6b Fix typo "persiting" 2016-09-13 13:26:33 +01:00
Mark Haines ad816b0add Limit how often we ask for keys from dead servers 2016-09-13 11:53:50 +01:00
Erik Johnston 43253c10b8 Remove redundant event_auth index 2016-09-13 11:47:48 +01:00
Mark Haines 18ab019a4a Move the E2E key handling into the e2e handler 2016-09-13 11:35:35 +01:00
Mark Haines 76b09c29b0 Merge pull request #1110 from matrix-org/markjh/e2e_timeout
Add a timeout parameter for end2end key queries.
2016-09-13 10:50:45 +01:00
Erik Johnston ba6bc2faa0 Merge pull request #1109 from matrix-org/erikj/partial_indices
Add WHERE clause support to index creation
2016-09-13 09:06:16 +01:00
Matthew Hodgson edbcb4152b make device IDs more useful for human disambiguation 2016-09-13 00:02:39 +01:00
Mark Haines 949c2c5435 Add a timeout parameter for end2end key queries.
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.

Fetch keys from different servers in parallel rather than in series.

Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
Erik Johnston b17af156c7 Remove where clause 2016-09-12 17:05:54 +01:00
Erik Johnston 1c9da43a95 Merge pull request #1108 from matrix-org/erikj/create_dm
Add is_direct param to /createRoom
2016-09-12 16:57:16 +01:00
Erik Johnston 0b32bb20bb Index contains_url for file search queries 2016-09-12 16:57:05 +01:00
Erik Johnston c94de0ab60 Add WHERE clause support to index creation 2016-09-12 16:55:01 +01:00
Erik Johnston 502c901e11 Merge pull request #1107 from matrix-org/erikj/backfill_none
Fix backfill when cannot find an event.
2016-09-12 16:48:01 +01:00
Erik Johnston 48a5a7552d Add is_direct param to /createRoom 2016-09-12 16:34:20 +01:00
Erik Johnston 706b5d76ed Fix backfill when cannot find an event.
`get_pdu` can succeed but return None.
2016-09-12 14:59:51 +01:00
Erik Johnston 7c679b1118 Merge pull request #1106 from matrix-org/erikj/state_reindex_concurrent
Create new index concurrently
2016-09-12 14:38:40 +01:00
Erik Johnston d080b3425c Merge pull request #1105 from matrix-org/erikj/make_notif_highlight_query_fast
Optimise /notifications query
2016-09-12 14:34:12 +01:00
Erik Johnston 03a98aff3c Create new index concurrently 2016-09-12 14:27:01 +01:00
Erik Johnston fa20c9ce94 Change the index to be stream_ordering, highlight 2016-09-12 14:04:08 +01:00
Erik Johnston 5ef5435529 Remove unused import 2016-09-12 13:32:58 +01:00
Mark Haines aa7b890cfe Merge pull request #1104 from matrix-org/markjh/direct_to_device_federation_sync
Fix direct to device messages recieved over federation to notify sync
2016-09-12 13:25:23 +01:00
Erik Johnston 7cd6edb947 Use register_background_index_update 2016-09-12 12:54:48 +01:00
Erik Johnston 0294c14ec4 Add back in query change 2016-09-12 12:43:56 +01:00
Erik Johnston 7fe42cf949 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_notif_highlight_query_fast 2016-09-12 12:37:09 +01:00
Erik Johnston 15ca0c6a4d Make reindex happen in bg 2016-09-12 12:36:36 +01:00
Mark Haines a232e06100 Fix direct to device messages recieved over federation to notify sync 2016-09-12 12:30:46 +01:00
Mark Haines 4a32d25d4c Merge branch 'develop' into markjh/bearer_token 2016-09-12 11:14:56 +01:00
Mark Haines 31f85f9db9 Add comments to existing schema deltas that used "CREATE INDEX" directly 2016-09-12 11:00:26 +01:00
Mark Haines ec609f8094 Fix unit tests 2016-09-12 10:46:02 +01:00
Erik Johnston 54417999b6 Revert "Add index to event_push_actions" 2016-09-12 10:39:55 +01:00
Erik Johnston 45dc260060 Merge pull request #1101 from matrix-org/erikj/state_types_idx
Change state fetch query for postgres to be faster
2016-09-12 10:20:38 +01:00
Erik Johnston d1c217c823 Merge pull request #1097 from matrix-org/erikj/replication_typing_rest
Correctly handle typing stream id resetting
2016-09-12 10:10:15 +01:00
Erik Johnston 897d57bc58 Change state fetch query for postgres to be faster
It turns out that postgres doesn't like doing a list of OR's and is
about 1000x slower, so we just issue a query for each specific type
seperately.
2016-09-12 10:05:07 +01:00
Erik Johnston 555460ae1b Merge pull request #1095 from matrix-org/erikj/batch_edus
Clobber EDUs in send queue
2016-09-12 08:04:15 +01:00
Richard van der Hoff 4162f820ff Merge pull request #1100 from VShell/fix-cas
Conform better to the CAS protocol specification
2016-09-09 21:57:42 +01:00
Shell Turner 29205e9596 Conform better to the CAS protocol specification
Redirect to CAS's /login endpoint properly, and
don't require an <attributes> element.

Signed-off-by: Shell Turner <cam.turn@gmail.com>
2016-09-09 21:20:14 +01:00
David Baker b91e2833b3 Merge remote-tracking branch 'origin/develop' into dbkr/make_notif_highlight_query_fast 2016-09-09 19:11:34 +01:00
David Baker f2acc3dcf9 Add index to event_push_actions
and remove room_id caluse so it uses it

Mostly from @negativemjark
2016-09-09 18:54:54 +01:00
Mark Haines 3ddec016ff Merge branch 'develop' into markjh/bearer_token 2016-09-09 18:51:22 +01:00
Mark Haines 8e01263587 Allow clients to supply access_tokens as headers
Clients can continue to supply access tokens as query parameters
or can supply the token as a header:

   Authorization: Bearer <access_token_goes_here>

This matches the ouath2 format of
https://tools.ietf.org/html/rfc6750#section-2.1
2016-09-09 18:17:42 +01:00
Erik Johnston 3265def8c7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/batch_edus 2016-09-09 18:06:01 +01:00
Erik Johnston af4701b311 Fix incorrect attribute name 2016-09-09 17:36:56 +01:00
Erik Johnston 44330a21e9 Comment 2016-09-09 17:22:07 +01:00
Erik Johnston 464ffd1b5e Comment 2016-09-09 17:17:23 +01:00
Erik Johnston 327425764e Add edu.type as part of key. Remove debug logging 2016-09-09 17:13:30 +01:00
Mark Haines dbff7e9436 Merge pull request #1096 from matrix-org/markjh/get_access_token
Add helper function for getting access_tokens from requests
2016-09-09 17:09:27 +01:00
Erik Johnston a4339de9de Correctly handle typing stream id resetting 2016-09-09 16:44:26 +01:00
Mark Haines 8aee5aa068 Add helper function for getting access_tokens from requests
Rather than reimplementing the token parsing in the various places.
This will make it easier to change the token parsing to allow access_tokens
in HTTP headers.
2016-09-09 16:33:15 +01:00
Erik Johnston 52b2318777 Clobber EDUs in send queue 2016-09-09 15:59:08 +01:00
Paul Evans 56f38d1776 Merge pull request #1091 from matrix-org/paul/third-party-lookup
Improvements to 3PE lookup API
2016-09-09 15:43:11 +01:00
Paul Evans 8cb252d00c Merge pull request #1094 from matrix-org/paul/get-state-whole-event
Allow clients to ask for the whole of a single state event
2016-09-09 15:26:17 +01:00
Paul "LeoNerd" Evans 776594f99d Log if rejecting 3PE query metadata result due to type check 2016-09-09 15:11:31 +01:00
Paul "LeoNerd" Evans ed44c475d8 Reject malformed 3PE query metadata results earlier in AS API handling code 2016-09-09 15:07:04 +01:00
Erik Johnston ab80d5e0a9 Drop replication log levels 2016-09-09 14:56:50 +01:00
Paul "LeoNerd" Evans f25d74f69c Minor fixes from PR comments 2016-09-09 14:54:16 +01:00
Paul "LeoNerd" Evans d271383e63 Filter returned events for client-facing format 2016-09-09 14:40:15 +01:00
Paul "LeoNerd" Evans 0fc0a3bdff Allow clients to specify the format a room state event is returned in 2016-09-09 14:34:29 +01:00
Erik Johnston 6c4d582144 Deduplicate presence in _update_states 2016-09-09 14:28:22 +01:00
Erik Johnston 685da5a3b0 Merge pull request #1092 from matrix-org/erikj/transaction_queue_check
Check if destination is ready for retry earlier
2016-09-09 13:49:56 +01:00
Erik Johnston a6c6750166 Check if destination is ready for retry earlier 2016-09-09 13:46:05 +01:00
Paul "LeoNerd" Evans bdbcfc2a80 appease pep8 2016-09-09 13:31:39 +01:00
Paul "LeoNerd" Evans 6eb0c8a2e4 Python isn't JavaScript; have to quote dict keys 2016-09-09 13:31:17 +01:00
Mark Haines 6b54fa81de Merge pull request #1089 from matrix-org/markjh/direct_to_device_stream
Track the max device stream_id in a separate table,
2016-09-09 13:31:07 +01:00
Paul "LeoNerd" Evans 25eb769b26 Efficiency fix for lookups of a single protocol 2016-09-09 13:25:02 +01:00
Paul "LeoNerd" Evans 3328428d05 Allow lookup of a single 3PE protocol query metadata 2016-09-09 13:19:04 +01:00
Erik Johnston 4598682b43 Fix tightloop on sending transaction 2016-09-09 13:12:53 +01:00
Paul "LeoNerd" Evans 033d43e419 Don't corrupt shared cache on subsequent protocol requests 2016-09-09 13:10:36 +01:00
Mark Haines 647c724573 Use the previous MAX value if any to set the stream_id 2016-09-09 11:52:44 +01:00
Erik Johnston a15ba15e64 Merge pull request #1088 from matrix-org/erikj/transaction_queue_check
Correctly guard against multiple concurrent transactions
2016-09-09 11:49:06 +01:00
Mark Haines 6a6cbfcf1e Track the max_stream_device_id in a separate table, since we delete from the inbox table 2016-09-09 11:48:23 +01:00
Erik Johnston d2688d7f03 Correctly guard against multiple concurrent transactions 2016-09-09 11:44:36 +01:00
Mark Haines 303b6f29f0 Merge pull request #1087 from matrix-org/markjh/reapply_delta
Reapply 34/device_outbox in 35/device_outbox_again.py since the schem…
2016-09-09 11:24:27 +01:00
Erik Johnston 1fe7ca1362 Merge branch 'release-v0.17.3' of github.com:matrix-org/synapse into develop 2016-09-09 11:15:40 +01:00
Erik Johnston 66efcbbff1 Bump changelog and version 2016-09-09 11:07:39 +01:00
Mark Haines 0877157353 Just move the schema and add some DROPs 2016-09-09 11:04:47 +01:00
Erik Johnston 2ffec928e2 Reduce batch size to be under SQL limit 2016-09-09 11:03:31 +01:00
Erik Johnston b390756150 Update last_device_stream_id_by_dest if there is nothing to send 2016-09-09 11:00:15 +01:00
Matthew Hodgson b8f84f99ff Merge pull request #1081 from matrix-org/dbkr/notifications_only_highlight
Implement `only=highlight` on `/notifications`
2016-09-09 00:09:51 +01:00
Mark Haines 43b77c5d97 Only catch databas errors 2016-09-08 17:44:21 +01:00
Paul "LeoNerd" Evans 2f267ee160 Collect up all the "instances" lists of individual AS protocol results into one combined answer to the client 2016-09-08 17:43:53 +01:00
Mark Haines 7d5b142547 Add a stub run_upgrade 2016-09-08 17:39:11 +01:00
David Baker c3276aef25 Merge pull request #1080 from matrix-org/dbkr/fix_notifications_api_with_from
Fix /notifications API when used with `from` param
2016-09-08 17:35:35 +01:00
Mark Haines fa722a699c Reapply 34/device_outbox in 35/device_outbox_again.py since the schema was bumped before it landed on develop 2016-09-08 17:35:16 +01:00
Erik Johnston 5c688739d6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/reindex_state_groups 2016-09-08 16:52:09 +01:00
Erik Johnston ebb46497ba Add delta file 2016-09-08 16:38:54 +01:00
Mark Haines 91ec972277 Merge pull request #1084 from matrix-org/markjh/direct_to_device_wildcard
Support wildcard device_ids for direct to device messages
2016-09-08 16:28:08 +01:00
Erik Johnston 5beda10bbd Reindex state_groups_state after pruning 2016-09-08 16:18:01 +01:00
Erik Johnston 257025ac89 Merge pull request #1082 from matrix-org/erikj/remote_public_rooms
Add server param to /publicRooms
2016-09-08 16:04:22 +01:00
Erik Johnston 3f9889bfd6 Use parse_string 2016-09-08 15:51:10 +01:00
Erik Johnston caa22334b3 Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-09-08 15:33:00 +01:00
Erik Johnston b152ee71fe Bump version and changelog 2016-09-08 15:23:44 +01:00
Mark Haines a1c8f268e5 Support wildcard device_ids for direct to device messages 2016-09-08 15:13:05 +01:00
Erik Johnston 8b93af662d Check the user_id for presence/typing matches origin 2016-09-08 15:07:38 +01:00
Mark Haines fa9d36e050 Merge branch 'develop' into markjh/direct_to_device_federation 2016-09-08 13:43:43 +01:00
David Baker 4ef222ab61 Implement `only=highlight` on `/notifications` 2016-09-08 13:43:35 +01:00
Erik Johnston 61cd9af09b Log delta files we're applying 2016-09-08 13:40:46 +01:00
Erik Johnston 791658b576 Add server param to /publicRooms 2016-09-08 11:53:05 +01:00
David Baker c5b49eb7ca Fix /notifications API when used with `from` param 2016-09-08 09:40:10 +01:00
Erik Johnston b568ca309c Temporarily disable sequential scans for state fetching 2016-09-08 09:38:54 +01:00
Mark Haines 3c320c006c Merge pull request #1077 from matrix-org/markjh/device_logging
Log the types and values when failing to store devices
2016-09-07 18:24:24 +01:00
Mark Haines 85b51fdd6b Log the types and values when failing to store devices 2016-09-07 17:19:18 +01:00
Mark Haines 43954d000e Add a new method to enqueue the device messages rather than sending a dummy EDU 2016-09-07 16:10:51 +01:00
Mark Haines 2a0159b8ae Fix the stream change cache to work over replication 2016-09-07 15:58:00 +01:00
Mark Haines cb98ac261b Move the check for federated device_messages.
Move the check into _attempt_new_transaction.
Only delete messages if there were messages to delete.
2016-09-07 15:39:13 +01:00
Mark Haines 31a07d2335 Add stream change caches for device messages 2016-09-07 15:27:07 +01:00
Erik Johnston 513188aa56 Comment 2016-09-07 14:53:23 +01:00
Erik Johnston fadb01551a Add appopriate framing clause 2016-09-07 14:39:01 +01:00
Erik Johnston d25c20ccbe Use windowing function to make use of index 2016-09-07 14:22:22 +01:00
Mark Haines 7d893beebe Comment the add_messages storage functions 2016-09-07 12:03:37 +01:00
Erik Johnston 94a83b534f Merge pull request #1065 from matrix-org/erikj/state_storage
Move to storing state_groups_state as deltas
2016-09-07 09:39:58 +01:00
Mark Haines d4a35ada28 Send device messages over federation 2016-09-06 18:16:20 +01:00
Mark Haines e020834e4f Add storage methods for federated device messages 2016-09-06 15:12:13 +01:00
Mark Haines 2ad72da931 Add tables for federated device messages
Adds tables for storing the messages that need to be sent to a
remote device and for deduplicating messages received.
2016-09-06 15:10:40 +01:00
Erik Johnston 3c4208a057 Record counts of state changes 2016-09-06 11:31:01 +01:00
Mark Haines f4164edb70 Move _add_messages_to_device_inbox_txn into a separate method 2016-09-06 11:26:37 +01:00
Erik Johnston 2eed4d7af4 Merge pull request #1072 from matrix-org/erikj/presence_fiddle
Fiddle should_notify to better report stats
2016-09-06 10:57:48 +01:00
Erik Johnston 438ef47637 Short circuit if presence is the same 2016-09-06 10:28:35 +01:00
Erik Johnston 74a3b4a650 Fiddle should_notify to better report stats 2016-09-06 10:23:38 +01:00
Erik Johnston d51b8a1674 Add quotes and be explicity about script-src 2016-09-05 17:35:01 +01:00
Erik Johnston 662b031a30 Allow PDF to be rendered from media repo 2016-09-05 17:25:26 +01:00
Erik Johnston 0595413c0f Scale the batch size so that we're not bitten by the minimum 2016-09-05 15:49:57 +01:00
Erik Johnston a7032abb2e Correctly handle reindexing state groups that already have an edge 2016-09-05 15:07:23 +01:00
Erik Johnston 9e6d88f4e2 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage 2016-09-05 15:01:33 +01:00
Erik Johnston 8c93e0bae7 Merge pull request #1070 from matrix-org/erikj/presence_stats
Record why we have chosen to notify
2016-09-05 15:01:30 +01:00
Erik Johnston 70332a12dd Take value in a better way 2016-09-05 14:57:14 +01:00
Erik Johnston 373654c635 Comment about sqlite and WITH RECURSIVE 2016-09-05 14:50:36 +01:00
Erik Johnston 485d999c8a Correctly delete old state groups in purge history API 2016-09-05 14:49:08 +01:00
Erik Johnston 69054e3d4c Record why we have chosen to notify 2016-09-05 14:12:11 +01:00
Erik Johnston 69a2d4e38c Use get_joined_users_from_context instead of manually looking up hosts 2016-09-05 13:44:40 +01:00
Erik Johnston f12993ec16 Bump changelog and version 2016-09-05 13:01:14 +01:00
Erik Johnston bb36b93f71 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage 2016-09-05 11:53:11 +01:00
Erik Johnston d87b87adf7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/bulk_push 2016-09-05 11:52:49 +01:00
Erik Johnston caed150363 Remove unused imports 2016-09-05 10:52:01 +01:00
Erik Johnston 80a6a445fa Only fetch local pushers 2016-09-05 10:43:32 +01:00
Erik Johnston 628e65721b Add comments 2016-09-05 10:41:27 +01:00
Mark Haines 274c2f50a5 Merge pull request #1067 from matrix-org/markjh/idempotent
Fix membership changes to be idempotent
2016-09-05 10:21:25 +01:00
Erik Johnston a99e933550 Add upgrade script that will slowly prune state_groups_state entries 2016-09-05 10:05:36 +01:00
Erik Johnston 3847fa38c4 Make bulk_get_push_rules_for_room use get_joined_users_from_context cache 2016-09-05 10:02:38 +01:00
Mark Haines f2690c6423 Fix membership changes to be idempotent 2016-09-02 19:23:22 +01:00
Mark Haines 81b94c5750 Merge pull request #1066 from matrix-org/markjh/direct_to_device_lowerbound
Only return new device messages in /sync
2016-09-02 16:18:34 +01:00
Mark Haines 65fa37ac5e Only return new device messages in /sync 2016-09-02 15:50:37 +01:00
Erik Johnston 3baf641a48 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage 2016-09-02 14:54:07 +01:00
Erik Johnston c0238ecbed Explicitly specify state_key for history_visibility fetching 2016-09-02 14:53:46 +01:00
Erik Johnston 273b6bcf22 Merge pull request #1064 from matrix-org/erikj/on_receive_check
Only check if host is in room if we have state and auth_chain
2016-09-02 14:43:35 +01:00
Erik Johnston f7f1027d3d Comment on when auth chain and state are None 2016-09-02 14:42:38 +01:00
Erik Johnston 34e5e17f91 Comment 2016-09-02 14:26:07 +01:00
Erik Johnston b96c6c3185 Docstrings 2016-09-02 14:19:22 +01:00
Erik Johnston 1ffe9578d1 Merge pull request #1063 from matrix-org/erikj/pull_out_ids_only
Only pull out IDs from DB for /state_ids/ request
2016-09-02 14:16:28 +01:00
Erik Johnston cce957e254 Bump max_entries on get_destination_retry_timings 2016-09-02 14:08:33 +01:00
Erik Johnston bd9b8d87ae Only check if host is in room if we have state and auth_chain 2016-09-02 13:40:28 +01:00
Erik Johnston 657847e4c6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage 2016-09-02 11:04:48 +01:00
Mark Haines 965168a842 Merge branch 'develop' into markjh/direct_to_device_synchrotron 2016-09-02 10:59:24 +01:00
Erik Johnston 2854ee2a52 Only pull out IDs from DB for /state_ids/ request 2016-09-02 10:53:36 +01:00
Erik Johnston 598317927c Limit the length of state chains 2016-09-02 10:41:38 +01:00
Mark Haines 7ed5acacf4 Fix up the calls to the notifier for device messages 2016-09-01 18:08:40 +01:00
Erik Johnston 051a9ea921 Linearize state resolution to help caches 2016-09-01 14:55:03 +01:00
Erik Johnston 265d847ffd Fix typo in log line 2016-09-01 14:50:06 +01:00
Erik Johnston f4778d4cd9 Merge branch 'erikj/pdu_check' of github.com:matrix-org/synapse into develop 2016-09-01 14:40:36 +01:00
Erik Johnston 9e25443db8 Move to storing state_groups_state as deltas 2016-09-01 14:31:26 +01:00
Erik Johnston 44982606ee Merge pull request #1060 from matrix-org/erikj/state_ids
Assign state groups in state handler.
2016-09-01 14:20:42 +01:00
Erik Johnston 516a272aca Ensure we only return a validated pdu in get_pdu 2016-09-01 10:55:02 +01:00
Erik Johnston 0cfd6c3161 Use state_groups table to test existence 2016-08-31 16:25:57 +01:00
Erik Johnston 5405351b14 Lower get_linearized_receipts_for_room cache size 2016-08-31 16:19:44 +01:00
Erik Johnston ed7a703d4c Handle the fact that workers can't generate state groups 2016-08-31 15:53:19 +01:00
Erik Johnston 1671913287 Merge pull request #1059 from matrix-org/erikj/sent_transaction_delete
Clean up old sent transactions
2016-08-31 15:07:33 +01:00
Erik Johnston f51888530d Always specify state_group so that its in the cache 2016-08-31 14:58:11 +01:00
Erik Johnston 826ca61745 Add storage function to SlaveStore 2016-08-31 14:45:04 +01:00
Erik Johnston fbd2615de4 Merge pull request #1057 from matrix-org/erikj/fix_email_name
Fix email notifs by adding missing param
2016-08-31 14:33:45 +01:00
Erik Johnston c10cb581c6 Correctly handle the difference between prev and current state 2016-08-31 14:26:22 +01:00
Erik Johnston ef0cc648cf Clean up old sent transactions 2016-08-31 11:12:02 +01:00
Mark Haines 761f9fccff Merge pull request #1058 from matrix-org/markjh/direct_to_device_synchrotron
Add a replication stream for direct to device messages
2016-08-31 11:08:18 +01:00
Mark Haines a662252758 Return the current stream position from add_messages_to_device_inbox 2016-08-31 10:42:52 +01:00
Mark Haines 1aa3e1d287 Add a replication stream for direct to device messages 2016-08-31 10:38:58 +01:00
Erik Johnston 1bb8ec296d Generate state group ids in state layer 2016-08-31 10:09:46 +01:00
Erik Johnston d80f64d370 Fix email notifs by adding missing param 2016-08-30 21:46:39 +01:00
Kegan Dougal c882783535 flake8 2016-08-30 17:20:31 +01:00
Kegan Dougal 572acde483 Use None instead of the empty string
Change how we validate the 'url' field as a result.
2016-08-30 17:16:00 +01:00
Erik Johnston 5dc2a702cf Make _state_groups_id_gen a normal IdGenerator 2016-08-30 16:55:11 +01:00
Erik Johnston 3e784eff74 Remove state replication stream 2016-08-30 16:51:36 +01:00
Kegan Dougal 16b652f0a3 Flake8 2016-08-30 16:30:12 +01:00