b4017539d4
Make the pushers lang field column longer
...
To accommodate things like zh-Hans-CN
Fixes https://github.com/vector-im/riot-ios/issues/1031
2017-02-17 10:42:57 +00:00
138e030cfe
Comment
2017-02-16 15:03:36 +00:00
502ae6c663
Comment
2017-02-16 14:47:11 +00:00
e6acf0c399
Store the default push actions in a more efficient manner
2017-02-16 14:40:24 +00:00
04eca2589d
Merge pull request #1916 from matrix-org/erikj/push_actions_delete
...
Aggregate event push actions
2017-02-16 15:28:58 +01:00
7dcbcca68c
Merge pull request #1921 from matrix-org/rav/fix_key_changes
...
Fix bugs in the /keys/changes api
2017-02-15 11:25:16 +00:00
fc2f29c1d0
Fix bugs in the /keys/changes api
...
* `get_forward_extremeties_for_room` takes a numeric `stream_ordering`. We were
passing a `RoomStreamToken`, which meant that it returned the *current*
extremities, rather than those corresponding to the `from_token`. However:
* `get_state_ids_for_events` required a second ('types') parameter; this meant
that a `TypeError` was thrown and we ended up acting as though there was *no*
prev state.
* `get_state_ids_for_events` actually returns a map from event_id to state
dictionary - just looking up the state keys in it again meant that we acted
as though there was no prev state. We now check if each member's state has
changed since *any* of the extremities.
Also add/fix some comments.
2017-02-14 13:59:50 +00:00
ce3c8df6df
Less aggressive timers
2017-02-14 13:41:24 +00:00
095b45c165
Aggregate event push actions
2017-02-14 13:39:41 +00:00
795f8e3fe7
Merge pull request #1873 from matrix-org/erikj/delete_push_actions
...
Be more agressive about purging old room event_push_actions
2017-02-14 14:29:04 +01:00
9e617cd4c2
Cache get_presence storage
2017-02-13 13:50:03 +00:00
52cd019a54
Make None check explicit
2017-02-08 16:04:29 +00:00
f20cd34858
Merge pull request #1892 from matrix-org/erikj/rejection_fwd_extrem
...
Ignore new rejected events when working out forward extremities.
2017-02-08 16:59:06 +01:00
7723b4caa4
Ignore new rejected events when working out forward extremeties.
2017-02-08 14:48:06 +00:00
af6da6db2d
Merge pull request #1784 from morteza-araby/user-admin
...
Administration functionalities
2017-02-06 16:21:10 +01:00
38258a0976
Bump cache sizes for common membership queries
2017-02-02 18:45:55 +00:00
832e9c52ca
Comment
2017-02-02 13:09:56 +00:00
54a79c1d37
Make presence.get_new_events a bit faster
...
We do this by caching the set of users a user shares rooms with.
2017-02-02 13:07:18 +00:00
2849d3f29d
admin,storage: added more administrator functionalities
...
administrators can now:
- Set displayname of users
- Update user avatars
- Search for users by user_id
- Browse all users in a paginated API
- Reset user passwords
- Deactivate users
Helpers for doing paginated queries has also been added to storage
Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>
2017-02-02 14:02:26 +01:00
fac3c03087
Be more agressive about purging old room event_push_actions
2017-02-01 18:27:24 +00:00
fbfe44bb4d
Doc args
2017-02-01 17:52:57 +00:00
d61a04583e
Comment
2017-02-01 17:35:23 +00:00
7e919bdbd0
Include newly joined users in /keys/changes API
2017-02-01 17:33:16 +00:00
ebfaff84c9
Merge pull request #1870 from matrix-org/erikj/cache_get_all_new_events
...
Add a small cache get_all_new_events
2017-02-01 13:22:02 +00:00
62f6b86ba7
Merge pull request #1868 from matrix-org/erikj/replication_cache
...
Only invalidate membership caches based on the cache stream
2017-02-01 13:12:30 +00:00
f6124311fd
Add m.room.member type to query
2017-02-01 11:59:17 +00:00
88a4d54883
Merge pull request #1867 from matrix-org/erikj/member_index
...
Add an index to make membership queries faster
2017-02-01 11:44:27 +00:00
368c88c487
Add a small cache get_all_new_events
2017-02-01 10:50:44 +00:00
5deaf9e30b
Up get_latest_event_ids_in_room cache
2017-02-01 10:39:41 +00:00
458b6f4733
Only invalidate membership caches based on the cache stream
...
Before we completely invalidated get_users_in_room whenever we updated
any current_state_events table. This was way too aggressive.
2017-01-31 16:09:03 +00:00
21b7375778
Add an index to make membership queries faster
2017-01-31 15:15:57 +00:00
ac001dabdc
Merge pull request #1864 from matrix-org/erikj/device_list_fixes
...
Fix clearing out old device list outbound pokes
2017-01-31 13:35:35 +00:00
bfb3d255b1
Merge pull request #1862 from matrix-org/erikj/presence_update
...
Use DB cache of joined users for presence
2017-01-31 13:23:24 +00:00
ab55794b6f
Fix deletion of old sent devices correctly
2017-01-31 13:22:41 +00:00
d3169e8d28
Only fetch with row ts and count > 1
2017-01-31 11:20:03 +00:00
05b9f48ee5
Fix clearing out old device list outbound pokes
2017-01-31 10:08:55 +00:00
4b3403ca9b
Stream cache invalidations for room membership storage functions
2017-01-30 17:28:22 +00:00
1c13c9f6b6
Don't have such a large cache
2017-01-30 17:12:14 +00:00
c7a26b7c32
Fix unit tests
2017-01-30 17:11:24 +00:00
c2c9a78db9
Noop device key changes if they're the same
2017-01-30 16:55:04 +00:00
e75a779d9e
Fix query
2017-01-30 16:38:20 +00:00
828db669ec
Use get_users_in_room and declare it iterable
2017-01-30 16:37:22 +00:00
3670025e64
Rename func
2017-01-30 14:11:31 +00:00
d360c97ae1
Clear out old destination pokes.
2017-01-30 10:14:37 +00:00
76100203ab
Always use the latest stream_id, sent or unsent
2017-01-30 10:14:25 +00:00
d1e1fd6210
Add ts column to device_lists_outbound_pokes
2017-01-27 15:23:48 +00:00
252b503fc8
Hook device list updates to replication
2017-01-27 14:31:35 +00:00
84a35f32c7
Comment
2017-01-27 10:35:12 +00:00
c517a19c2d
Comment
2017-01-27 10:33:26 +00:00
738a2867c8
SQL param ordering
2017-01-27 10:31:29 +00:00
755adff0e4
User if rather than for
2017-01-27 10:31:06 +00:00
888c59c955
Better name
2017-01-27 10:29:47 +00:00
f25a4a4692
Remove unused param
2017-01-27 10:27:39 +00:00
31aca5589c
Fix on sqlite: use left rather than outer join
2017-01-26 16:55:50 +00:00
76d40f4904
Handle users leaving rooms
2017-01-26 16:39:33 +00:00
fbfad76c03
Add comments
2017-01-26 16:33:21 +00:00
c974116f19
Implement device key caching over federation
2017-01-26 16:07:24 +00:00
51e9fe36e4
Fix up sending of m.device_list_update edus
2017-01-25 16:55:21 +00:00
2367c5568c
Add basic implementation of local device list changes
2017-01-25 14:27:27 +00:00
fdf2a31a51
Typo
2017-01-23 16:14:14 +00:00
c77b24c092
Refactor to calculate state delta outside transaction
2017-01-23 14:51:33 +00:00
5d2134d485
Comments
2017-01-20 17:13:24 +00:00
a55fa2047f
Insert delta of current_state_events to be more efficient
2017-01-20 17:10:18 +00:00
d0897dead5
Spelling
2017-01-20 15:05:11 +00:00
567aa35b67
Update all call sites after rename
2017-01-20 14:40:31 +00:00
f2f40e64a9
Comments
2017-01-20 14:38:13 +00:00
4c6a31cd6e
Calculate the forward extremeties once
2017-01-20 14:28:53 +00:00
09eb08f910
Derive current_state_events from state groups
2017-01-20 11:52:51 +00:00
4ec1cf49e2
Lower loading events log to DEBUG
2017-01-17 17:28:32 +00:00
d906206049
Increase state_group_cache_size
2017-01-17 11:31:08 +00:00
897f8752da
Up cache max entries for state
2017-01-16 15:08:17 +00:00
01521299c7
Increase cache size limit
2017-01-16 11:56:51 +00:00
2fae34bd2c
Optionally measure size of cache by sum of length of values
2017-01-13 17:46:17 +00:00
f0325a9ccc
Merge pull request #1793 from matrix-org/erikj/change_device_inbox_index
...
Change device_inbox stream index to include user
2017-01-13 15:14:51 +00:00
c050f493dd
Add comment
2017-01-13 15:14:41 +00:00
ab655dca33
Explicitly close the cursor
2017-01-10 15:15:25 +00:00
5a32e9273e
Don't disable autocommit
2017-01-10 15:11:27 +00:00
caddadfc5a
Change device_inbox stream index to include user
...
This makes fetching the nost recently changed users much tricker, and
brings it in line with e.g. presence_stream indices.
2017-01-10 15:04:57 +00:00
dd52d4de4c
Limit number of entries to prefill from cache
...
Some tables, like device_inbox, take a long time to query at startup for
the stream change cache prefills. This is likely because they are slower
growing streams and so are more fragmented on disk. For now, lets pull
fewer entries out to make startup quicker.
In future, we should add a better index to make it even faster.
2017-01-10 14:34:50 +00:00
bd85434cb3
Merge branch 'release-v0.18.6' into develop
2017-01-05 13:58:19 +00:00
8cfc0165e9
fix annoying typos
2017-01-05 13:39:43 +00:00
b31ed22738
Merge branch 'release-v0.18.6' into develop
2017-01-05 13:03:02 +00:00
dd3df11c55
More logging for the linearizer and for get_events
2017-01-05 12:32:47 +00:00
0c88ab1844
Add /account/3pid/delete endpoint
...
Also fix a typo in a comment
2016-12-20 18:27:30 +00:00
edc6a1e4f9
Add some logging for syncing to_device events
...
Attempt to track down the loss of to_device events
(https://github.com/vector-im/riot-web/issues/2711 etc).
2016-12-15 18:16:10 +00:00
c3208e45c9
Fixup membership query
2016-12-14 10:46:58 +00:00
8588d0eb3d
Merge pull request #1697 from matrix-org/erikj/fix_bg_member
...
Fix background update that prematurely stopped
2016-12-12 17:20:17 +00:00
1574b839e0
Merge pull request #1676 from matrix-org/erikj/room_list
...
Add new API appservice specific public room list
2016-12-12 17:00:10 +00:00
7ec2bf9b77
Fix background update that prematurely stopped
2016-12-12 16:54:58 +00:00
09cbcb78d3
Add cache to get_public_room_ids_at_stream_id
2016-12-12 14:41:51 +00:00
631376e2ac
Notify replication. Use correct network_id
2016-12-12 14:28:15 +00:00
d45c984653
Docstring
2016-12-12 11:00:27 +00:00
1697f6a323
Merge pull request #1680 from matrix-org/erikj/joined_rooms
...
Add new room membership APIs
2016-12-09 11:35:48 +00:00
52d12ca782
Add /room/<room_id>/joined_members API
...
This returns the currently joined members in the room with their display
names and avatar urls. This is more efficient than /members for large
rooms where you don't need the full events.
2016-12-08 13:32:07 +00:00
c45d8e9ba2
Add profile data to the room_membership table for joins
2016-12-08 13:08:41 +00:00
bd07a35c29
Fix result of insert_receipt
...
This should fix the absence of notifications when new receipts arrive.
2016-12-08 12:11:34 +00:00
2687af82d4
Comments
2016-12-07 09:58:33 +00:00
f32fb65552
Add new API appservice specific public room list
2016-12-06 16:12:27 +00:00
aa09d6b8f0
Rip out more refresh_token code
...
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.
Still TODO: removing the table from the schema.
2016-11-30 17:40:18 +00:00
46ca345b06
Don't send old events as federation
2016-11-24 12:29:02 +00:00
1985860c6e
Comment
2016-11-23 15:59:59 +00:00
2ac516850b
More efficient notif count queries
2016-11-23 15:57:04 +00:00
302fbd218d
Merge pull request #1635 from matrix-org/erikj/split_out_fed_txn
...
Split out federation transaction sending to a worker
2016-11-23 15:39:12 +00:00
b69f76c106
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_fed_txn
2016-11-23 11:31:53 +00:00
0163466d72
Ignore AS users when fetching push rules
...
By ignoring AS users early on when fetching push rules for a room we can
avoid needlessly hitting the DB and filling up the caches.
2016-11-23 11:01:01 +00:00
54fed21c04
Fix tests and flake8
2016-11-22 18:18:31 +00:00
90565d015e
Invalidate retry cache in both directions
2016-11-22 17:45:44 +00:00
a2a6c1c22f
Fail with a coherent error message if `/sync?filter=` is invalid
2016-11-21 13:15:25 +00:00
524d61bf7e
Fix tests
2016-11-21 11:53:02 +00:00
7c9cdb2245
Store federation stream positions in the database
2016-11-21 11:33:08 +00:00
f8ee66250a
Handle sending events and device messages over federation
2016-11-17 15:48:04 +00:00
ed787cf09e
Hook up the send queue and create a federation sender worker
2016-11-16 17:34:44 +00:00
c7546b3cdb
Merge pull request #1617 from matrix-org/erikj/intern_state_dict
...
Correctly intern keys in state cache
2016-11-15 16:45:55 +00:00
f9d156d270
New Flake8 fixes
2016-11-15 11:22:29 +00:00
587d8ac60f
Correctly intern keys in state cache
2016-11-08 11:53:25 +00:00
63772443e6
Comment
2016-11-04 10:53:42 +00:00
a3f6576084
Remove unused but buggy function
2016-11-04 10:48:20 +00:00
64c6566980
Remove spurious comment
2016-11-03 15:04:32 +00:00
8fd4d9129f
Replace postgres GIN with GIST
...
This is because GIN can be slow to write too, especially when the table
gets large.
2016-11-03 15:00:03 +00:00
760469c812
Continue to clean up received_transactions
2016-11-01 11:42:08 +00:00
f09d2b692f
Removed unused stuff
2016-10-31 17:10:56 +00:00
4c3eb14d68
Increase batching of sent transaction inserts
...
This should further reduce the number of individual inserts,
transactions and updates that are required for keeping sent_transactions
up to date.
2016-10-31 16:07:45 +00:00
a9111786f9
Use most recently added binding, not most recently seen user.
2016-10-27 14:32:45 +01:00
1fc1bc2a51
Fix user_threepids schema delta
...
The delta `37/user_threepids.sql` aimed to update all the email
addresses to be lower case, however duplicate emails may exist in the
table already.
This commit adds a step where the delta moves the duplicate emails to a
new `medium` `email_old`. Only the most recently used account keeps the
binding intact. We move rather than delete so that we retain some record
of which emails were associated with which account.
2016-10-27 14:14:44 +01:00
d04e2ff3a4
Fix incredubly slow back pagination query
...
If a client didn't specify a from token when paginating backwards
synapse would attempt to query the (global) maximum topological token.
This a) doesn't make much sense since they're room specific and b) there
are no indices that lets postgres do this efficiently.
2016-10-24 13:35:51 +01:00
78c083f159
Merge pull request #1164 from pik/error-codes
...
Clarify Error codes for GET /filter/
2016-10-19 14:26:17 +01:00
0108ed8ae6
Latest delta is now 37
2016-10-19 11:40:35 +01:00
a7f48320b1
Merge remote-tracking branch 'origin/develop' into dbkr/password_reset_case_insensitive
2016-10-19 11:28:56 +01:00
df2a616c7b
Convert emails to lowercase when storing
...
And db migration sql to convert existing addresses.
2016-10-19 11:13:55 +01:00
e8b1d2a452
Refactor test_filter to use real DataStore
...
* add tests for filter api errors
2016-10-18 12:17:38 -05:00
6942d68247
Bump schema version
2016-10-17 11:17:45 +01:00
b59994b454
Remove TODO
2016-10-17 11:17:02 +01:00
816988baaa
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/remove_auth
2016-10-17 11:10:37 +01:00
2869a29fd7
Drop some unused indices
2016-10-17 11:08:19 +01:00
29c5922021
Revert part of 6207399
...
older sqlite doesn't support indexes on expressions, lets just
store things lowercase in the db
2016-10-14 16:20:24 +01:00
bcb1245a2d
Merge remote-tracking branch 'origin/develop' into dbkr/password_reset_case_insensitive
2016-10-14 15:10:38 +01:00
62073992c5
Make password reset email field case insensitive
2016-10-14 13:56:53 +01:00
3061dac53e
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/replication_noop
2016-10-11 14:08:29 +01:00
9bfc617791
storage/appservice: make appservice methods only relying on the cache synchronous
2016-10-06 15:24:59 +02:00
5875a65253
Merge pull request #1145 from matrix-org/erikj/fix_reindex
...
Fix background reindex of origin_server_ts
2016-09-29 13:53:48 +01:00
9040c9ffa1
Fix background reindex of origin_server_ts
...
The storage function `_get_events_txn` was removed everywhere except
from this background reindex. The function was removed due to it being
(almost) completely unused while also being large and complex.
Therefore, instead of resurrecting `_get_events_txn` we manually
reimplement the bits that are needed directly.
2016-09-27 11:23:49 +01:00
4974147aa3
Remove duplication
2016-09-27 09:27:54 +01:00
13122e5e24
Remove unused variable
2016-09-27 09:21:51 +01:00
cf3e1cc200
Fix perf of fetching state in SQLite
2016-09-26 17:16:24 +01:00
748d8fdc7b
Reduce DB hits for replication
...
Some streams will occaisonally advance their positions without actually
having any new rows to send over federation. Currently this means that
the token will not advance on the workers, leading to them repeatedly
sending a slightly out of date token. This in turns requires the master
to hit the DB to check if there are any new rows, rather than hitting
the no op logic where we check if the given token matches the current
token.
This commit changes the API to always return an entry if the position
for a stream has changed, allowing workers to advance their tokens
correctly.
2016-09-23 16:49:21 +01:00
8009d84364
Match against event_id, rather than room_id
2016-09-21 16:46:59 +01:00
dc692556d6
Remove spurious AS clause
2016-09-21 16:28:47 +01:00
dc78db8c56
Update correct table
2016-09-21 15:52:44 +01:00
4f78108d8c
Readd entries to public_room_list_stream that were deleted
2016-09-21 15:24:22 +01:00
0b78d8adf2
Fix _delete_old_forward_extrem_cache query
2016-09-21 15:20:56 +01:00
David Baker