418bcd4309
Add new storage function to slave store
2016-09-16 08:37:39 +01:00
098db4aa52
Add very basic filter API to /publicRooms
2016-09-15 17:50:16 +01:00
c33b25fd8d
Change the way we calculate new_limit in /publicRooms and add POST API
2016-09-15 17:35:20 +01:00
de4f798f01
Handling expiring stream extrems correctly.
2016-09-15 17:34:59 +01:00
ea6dc356b0
Merge pull request #1125 from matrix-org/erikj/public_room_cache
...
Change get_pos_of_last_change to return upper bound
2016-09-15 15:48:53 +01:00
955f34d23e
Change get_pos_of_last_change to return upper bound
2016-09-15 15:12:07 +01:00
241d7d2d62
Merge pull request #1124 from matrix-org/erikj/enable_state_caching_workers
...
Enable state caches on workers
2016-09-15 15:01:47 +01:00
4be85281f9
Enable state caches on workers
2016-09-15 14:31:22 +01:00
cb3edec6af
Use stream_change cache to make get_forward_extremeties_for_room cache more effective
2016-09-15 14:28:13 +01:00
55e6fc917c
Add cache to get_forward_extremeties_for_room
2016-09-15 14:04:28 +01:00
68c1ed4d1a
Remove default public rooms limit
2016-09-15 13:56:20 +01:00
b82fa849c8
Merge pull request #1120 from matrix-org/erikj/push_invite_cache
...
Ensure we don't mutate the cache of push rules
2016-09-15 13:27:18 +01:00
e457034e99
Merge pull request #1121 from matrix-org/erikj/public_room_paginate
...
Add pagination support to publicRooms
2016-09-15 13:27:09 +01:00
1d98cf26be
By default limit /publicRooms to 100 entries
2016-09-15 13:18:35 +01:00
211786ecd6
Stream public room changes down replication
2016-09-15 11:47:23 +01:00
4fb65a1091
Base public room list off of public_rooms stream
2016-09-15 11:27:04 +01:00
5810cffd33
Pass since/from parameters over federation
2016-09-15 10:36:19 +01:00
f3eead0660
Allow paginating both forwards and backwards
2016-09-15 10:15:37 +01:00
4131381123
Remove support for aggregate room lists
2016-09-15 09:28:15 +01:00
6a5ded5988
Ensure we don't mutate the cache of push rules
2016-09-15 09:16:13 +01:00
4f181f361d
Accept optional token to public room list
2016-09-15 09:08:57 +01:00
c566f0ee17
Calculate the public room list from a stream_ordering
2016-09-14 17:42:47 +01:00
772c6067a3
Refactor public rooms to not pull out the full state for each room
2016-09-14 17:29:25 +01:00
baffe96d95
Add a room visibility stream
2016-09-14 17:29:19 +01:00
264a48aedf
Merge pull request #1117 from matrix-org/erikj/fix_state
...
Ensure we don't mutate state cache entries
2016-09-14 16:50:37 +01:00
ed992ae6ba
Add a DB index to figure out past state at a stream ordering in a room
2016-09-14 16:20:27 +01:00
e0b6db29ed
Split out public room list into a worker process
2016-09-14 14:42:51 +01:00
a70a43bc51
Move RoomListHandler into a separate file
2016-09-14 14:07:37 +01:00
f2b2cd8eb4
Amalgamate two identical consecutive if statements
2016-09-14 11:16:22 +01:00
00f51493f5
Fix reindex
2016-09-14 10:18:30 +01:00
d5ae1f1291
Ensure we don't mutate state cache entries
2016-09-14 10:03:48 +01:00
1b01488d27
Merge pull request #1111 from matrix-org/matthew/device-ids
...
make device IDs more useful for human disambiguation
2016-09-14 01:18:02 +01:00
0f73f0e70e
Merge pull request #1116 from matrix-org/paul/tiny-fixes
...
Fix typo "persiting"
2016-09-13 13:27:42 +01:00
ca35e54d6b
Fix typo "persiting"
2016-09-13 13:26:33 +01:00
ad816b0add
Limit how often we ask for keys from dead servers
2016-09-13 11:53:50 +01:00
43253c10b8
Remove redundant event_auth index
2016-09-13 11:47:48 +01:00
18ab019a4a
Move the E2E key handling into the e2e handler
2016-09-13 11:35:35 +01:00
76b09c29b0
Merge pull request #1110 from matrix-org/markjh/e2e_timeout
...
Add a timeout parameter for end2end key queries.
2016-09-13 10:50:45 +01:00
ba6bc2faa0
Merge pull request #1109 from matrix-org/erikj/partial_indices
...
Add WHERE clause support to index creation
2016-09-13 09:06:16 +01:00
edbcb4152b
make device IDs more useful for human disambiguation
2016-09-13 00:02:39 +01:00
949c2c5435
Add a timeout parameter for end2end key queries.
...
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.
Fetch keys from different servers in parallel rather than in series.
Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
b17af156c7
Remove where clause
2016-09-12 17:05:54 +01:00
1c9da43a95
Merge pull request #1108 from matrix-org/erikj/create_dm
...
Add is_direct param to /createRoom
2016-09-12 16:57:16 +01:00
0b32bb20bb
Index contains_url for file search queries
2016-09-12 16:57:05 +01:00
c94de0ab60
Add WHERE clause support to index creation
2016-09-12 16:55:01 +01:00
502c901e11
Merge pull request #1107 from matrix-org/erikj/backfill_none
...
Fix backfill when cannot find an event.
2016-09-12 16:48:01 +01:00
48a5a7552d
Add is_direct param to /createRoom
2016-09-12 16:34:20 +01:00
706b5d76ed
Fix backfill when cannot find an event.
...
`get_pdu` can succeed but return None.
2016-09-12 14:59:51 +01:00
7c679b1118
Merge pull request #1106 from matrix-org/erikj/state_reindex_concurrent
...
Create new index concurrently
2016-09-12 14:38:40 +01:00
d080b3425c
Merge pull request #1105 from matrix-org/erikj/make_notif_highlight_query_fast
...
Optimise /notifications query
2016-09-12 14:34:12 +01:00
03a98aff3c
Create new index concurrently
2016-09-12 14:27:01 +01:00
fa20c9ce94
Change the index to be stream_ordering, highlight
2016-09-12 14:04:08 +01:00
5ef5435529
Remove unused import
2016-09-12 13:32:58 +01:00
aa7b890cfe
Merge pull request #1104 from matrix-org/markjh/direct_to_device_federation_sync
...
Fix direct to device messages recieved over federation to notify sync
2016-09-12 13:25:23 +01:00
7cd6edb947
Use register_background_index_update
2016-09-12 12:54:48 +01:00
0294c14ec4
Add back in query change
2016-09-12 12:43:56 +01:00
7fe42cf949
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_notif_highlight_query_fast
2016-09-12 12:37:09 +01:00
15ca0c6a4d
Make reindex happen in bg
2016-09-12 12:36:36 +01:00
a232e06100
Fix direct to device messages recieved over federation to notify sync
2016-09-12 12:30:46 +01:00
31f85f9db9
Add comments to existing schema deltas that used "CREATE INDEX" directly
2016-09-12 11:00:26 +01:00
54417999b6
Revert "Add index to event_push_actions"
2016-09-12 10:39:55 +01:00
45dc260060
Merge pull request #1101 from matrix-org/erikj/state_types_idx
...
Change state fetch query for postgres to be faster
2016-09-12 10:20:38 +01:00
d1c217c823
Merge pull request #1097 from matrix-org/erikj/replication_typing_rest
...
Correctly handle typing stream id resetting
2016-09-12 10:10:15 +01:00
897d57bc58
Change state fetch query for postgres to be faster
...
It turns out that postgres doesn't like doing a list of OR's and is
about 1000x slower, so we just issue a query for each specific type
seperately.
2016-09-12 10:05:07 +01:00
555460ae1b
Merge pull request #1095 from matrix-org/erikj/batch_edus
...
Clobber EDUs in send queue
2016-09-12 08:04:15 +01:00
4162f820ff
Merge pull request #1100 from VShell/fix-cas
...
Conform better to the CAS protocol specification
2016-09-09 21:57:42 +01:00
29205e9596
Conform better to the CAS protocol specification
...
Redirect to CAS's /login endpoint properly, and
don't require an <attributes> element.
Signed-off-by: Shell Turner <cam.turn@gmail.com>
2016-09-09 21:20:14 +01:00
b91e2833b3
Merge remote-tracking branch 'origin/develop' into dbkr/make_notif_highlight_query_fast
2016-09-09 19:11:34 +01:00
f2acc3dcf9
Add index to event_push_actions
...
and remove room_id caluse so it uses it
Mostly from @negativemjark
2016-09-09 18:54:54 +01:00
3265def8c7
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/batch_edus
2016-09-09 18:06:01 +01:00
af4701b311
Fix incorrect attribute name
2016-09-09 17:36:56 +01:00
44330a21e9
Comment
2016-09-09 17:22:07 +01:00
464ffd1b5e
Comment
2016-09-09 17:17:23 +01:00
327425764e
Add edu.type as part of key. Remove debug logging
2016-09-09 17:13:30 +01:00
dbff7e9436
Merge pull request #1096 from matrix-org/markjh/get_access_token
...
Add helper function for getting access_tokens from requests
2016-09-09 17:09:27 +01:00
a4339de9de
Correctly handle typing stream id resetting
2016-09-09 16:44:26 +01:00
8aee5aa068
Add helper function for getting access_tokens from requests
...
Rather than reimplementing the token parsing in the various places.
This will make it easier to change the token parsing to allow access_tokens
in HTTP headers.
2016-09-09 16:33:15 +01:00
52b2318777
Clobber EDUs in send queue
2016-09-09 15:59:08 +01:00
56f38d1776
Merge pull request #1091 from matrix-org/paul/third-party-lookup
...
Improvements to 3PE lookup API
2016-09-09 15:43:11 +01:00
8cb252d00c
Merge pull request #1094 from matrix-org/paul/get-state-whole-event
...
Allow clients to ask for the whole of a single state event
2016-09-09 15:26:17 +01:00
776594f99d
Log if rejecting 3PE query metadata result due to type check
2016-09-09 15:11:31 +01:00
ed44c475d8
Reject malformed 3PE query metadata results earlier in AS API handling code
2016-09-09 15:07:04 +01:00
ab80d5e0a9
Drop replication log levels
2016-09-09 14:56:50 +01:00
f25d74f69c
Minor fixes from PR comments
2016-09-09 14:54:16 +01:00
d271383e63
Filter returned events for client-facing format
2016-09-09 14:40:15 +01:00
0fc0a3bdff
Allow clients to specify the format a room state event is returned in
2016-09-09 14:34:29 +01:00
6c4d582144
Deduplicate presence in _update_states
2016-09-09 14:28:22 +01:00
685da5a3b0
Merge pull request #1092 from matrix-org/erikj/transaction_queue_check
...
Check if destination is ready for retry earlier
2016-09-09 13:49:56 +01:00
a6c6750166
Check if destination is ready for retry earlier
2016-09-09 13:46:05 +01:00
bdbcfc2a80
appease pep8
2016-09-09 13:31:39 +01:00
6eb0c8a2e4
Python isn't JavaScript; have to quote dict keys
2016-09-09 13:31:17 +01:00
6b54fa81de
Merge pull request #1089 from matrix-org/markjh/direct_to_device_stream
...
Track the max device stream_id in a separate table,
2016-09-09 13:31:07 +01:00
25eb769b26
Efficiency fix for lookups of a single protocol
2016-09-09 13:25:02 +01:00
3328428d05
Allow lookup of a single 3PE protocol query metadata
2016-09-09 13:19:04 +01:00
4598682b43
Fix tightloop on sending transaction
2016-09-09 13:12:53 +01:00
033d43e419
Don't corrupt shared cache on subsequent protocol requests
2016-09-09 13:10:36 +01:00
647c724573
Use the previous MAX value if any to set the stream_id
2016-09-09 11:52:44 +01:00
a15ba15e64
Merge pull request #1088 from matrix-org/erikj/transaction_queue_check
...
Correctly guard against multiple concurrent transactions
2016-09-09 11:49:06 +01:00
6a6cbfcf1e
Track the max_stream_device_id in a separate table, since we delete from the inbox table
2016-09-09 11:48:23 +01:00
d2688d7f03
Correctly guard against multiple concurrent transactions
2016-09-09 11:44:36 +01:00
303b6f29f0
Merge pull request #1087 from matrix-org/markjh/reapply_delta
...
Reapply 34/device_outbox in 35/device_outbox_again.py since the schem…
2016-09-09 11:24:27 +01:00
1fe7ca1362
Merge branch 'release-v0.17.3' of github.com:matrix-org/synapse into develop
2016-09-09 11:15:40 +01:00
66efcbbff1
Bump changelog and version
2016-09-09 11:07:39 +01:00
0877157353
Just move the schema and add some DROPs
2016-09-09 11:04:47 +01:00
2ffec928e2
Reduce batch size to be under SQL limit
2016-09-09 11:03:31 +01:00
b390756150
Update last_device_stream_id_by_dest if there is nothing to send
2016-09-09 11:00:15 +01:00
b8f84f99ff
Merge pull request #1081 from matrix-org/dbkr/notifications_only_highlight
...
Implement `only=highlight` on `/notifications`
2016-09-09 00:09:51 +01:00
43b77c5d97
Only catch databas errors
2016-09-08 17:44:21 +01:00
2f267ee160
Collect up all the "instances" lists of individual AS protocol results into one combined answer to the client
2016-09-08 17:43:53 +01:00
7d5b142547
Add a stub run_upgrade
2016-09-08 17:39:11 +01:00
c3276aef25
Merge pull request #1080 from matrix-org/dbkr/fix_notifications_api_with_from
...
Fix /notifications API when used with `from` param
2016-09-08 17:35:35 +01:00
fa722a699c
Reapply 34/device_outbox in 35/device_outbox_again.py since the schema was bumped before it landed on develop
2016-09-08 17:35:16 +01:00
5c688739d6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/reindex_state_groups
2016-09-08 16:52:09 +01:00
ebb46497ba
Add delta file
2016-09-08 16:38:54 +01:00
91ec972277
Merge pull request #1084 from matrix-org/markjh/direct_to_device_wildcard
...
Support wildcard device_ids for direct to device messages
2016-09-08 16:28:08 +01:00
5beda10bbd
Reindex state_groups_state after pruning
2016-09-08 16:18:01 +01:00
257025ac89
Merge pull request #1082 from matrix-org/erikj/remote_public_rooms
...
Add server param to /publicRooms
2016-09-08 16:04:22 +01:00
3f9889bfd6
Use parse_string
2016-09-08 15:51:10 +01:00
caa22334b3
Merge branch 'master' of github.com:matrix-org/synapse into develop
2016-09-08 15:33:00 +01:00
b152ee71fe
Bump version and changelog
2016-09-08 15:23:44 +01:00
a1c8f268e5
Support wildcard device_ids for direct to device messages
2016-09-08 15:13:05 +01:00
8b93af662d
Check the user_id for presence/typing matches origin
2016-09-08 15:07:38 +01:00
fa9d36e050
Merge branch 'develop' into markjh/direct_to_device_federation
2016-09-08 13:43:43 +01:00
4ef222ab61
Implement `only=highlight` on `/notifications`
2016-09-08 13:43:35 +01:00
61cd9af09b
Log delta files we're applying
2016-09-08 13:40:46 +01:00
791658b576
Add server param to /publicRooms
2016-09-08 11:53:05 +01:00
c5b49eb7ca
Fix /notifications API when used with `from` param
2016-09-08 09:40:10 +01:00
b568ca309c
Temporarily disable sequential scans for state fetching
2016-09-08 09:38:54 +01:00
3c320c006c
Merge pull request #1077 from matrix-org/markjh/device_logging
...
Log the types and values when failing to store devices
2016-09-07 18:24:24 +01:00
85b51fdd6b
Log the types and values when failing to store devices
2016-09-07 17:19:18 +01:00
43954d000e
Add a new method to enqueue the device messages rather than sending a dummy EDU
2016-09-07 16:10:51 +01:00
2a0159b8ae
Fix the stream change cache to work over replication
2016-09-07 15:58:00 +01:00
cb98ac261b
Move the check for federated device_messages.
...
Move the check into _attempt_new_transaction.
Only delete messages if there were messages to delete.
2016-09-07 15:39:13 +01:00
31a07d2335
Add stream change caches for device messages
2016-09-07 15:27:07 +01:00
513188aa56
Comment
2016-09-07 14:53:23 +01:00
fadb01551a
Add appopriate framing clause
2016-09-07 14:39:01 +01:00
d25c20ccbe
Use windowing function to make use of index
2016-09-07 14:22:22 +01:00
7d893beebe
Comment the add_messages storage functions
2016-09-07 12:03:37 +01:00
94a83b534f
Merge pull request #1065 from matrix-org/erikj/state_storage
...
Move to storing state_groups_state as deltas
2016-09-07 09:39:58 +01:00
d4a35ada28
Send device messages over federation
2016-09-06 18:16:20 +01:00
e020834e4f
Add storage methods for federated device messages
2016-09-06 15:12:13 +01:00
2ad72da931
Add tables for federated device messages
...
Adds tables for storing the messages that need to be sent to a
remote device and for deduplicating messages received.
2016-09-06 15:10:40 +01:00
3c4208a057
Record counts of state changes
2016-09-06 11:31:01 +01:00
f4164edb70
Move _add_messages_to_device_inbox_txn into a separate method
2016-09-06 11:26:37 +01:00
2eed4d7af4
Merge pull request #1072 from matrix-org/erikj/presence_fiddle
...
Fiddle should_notify to better report stats
2016-09-06 10:57:48 +01:00
438ef47637
Short circuit if presence is the same
2016-09-06 10:28:35 +01:00
74a3b4a650
Fiddle should_notify to better report stats
2016-09-06 10:23:38 +01:00
d51b8a1674
Add quotes and be explicity about script-src
2016-09-05 17:35:01 +01:00
662b031a30
Allow PDF to be rendered from media repo
2016-09-05 17:25:26 +01:00
0595413c0f
Scale the batch size so that we're not bitten by the minimum
2016-09-05 15:49:57 +01:00
Erik Johnston