fc2f29c1d0
Fix bugs in the /keys/changes api
...
* `get_forward_extremeties_for_room` takes a numeric `stream_ordering`. We were
passing a `RoomStreamToken`, which meant that it returned the *current*
extremities, rather than those corresponding to the `from_token`. However:
* `get_state_ids_for_events` required a second ('types') parameter; this meant
that a `TypeError` was thrown and we ended up acting as though there was *no*
prev state.
* `get_state_ids_for_events` actually returns a map from event_id to state
dictionary - just looking up the state keys in it again meant that we acted
as though there was no prev state. We now check if each member's state has
changed since *any* of the extremities.
Also add/fix some comments.
2017-02-14 13:59:50 +00:00
21b7375778
Add an index to make membership queries faster
2017-01-31 15:15:57 +00:00
a55fa2047f
Insert delta of current_state_events to be more efficient
2017-01-20 17:10:18 +00:00
897f8752da
Up cache max entries for state
2017-01-16 15:08:17 +00:00
01521299c7
Increase cache size limit
2017-01-16 11:56:51 +00:00
2fae34bd2c
Optionally measure size of cache by sum of length of values
2017-01-13 17:46:17 +00:00
8cfc0165e9
fix annoying typos
2017-01-05 13:39:43 +00:00
587d8ac60f
Correctly intern keys in state cache
2016-11-08 11:53:25 +00:00
4974147aa3
Remove duplication
2016-09-27 09:27:54 +01:00
13122e5e24
Remove unused variable
2016-09-27 09:21:51 +01:00
cf3e1cc200
Fix perf of fetching state in SQLite
2016-09-26 17:16:24 +01:00
00f51493f5
Fix reindex
2016-09-14 10:18:30 +01:00
d5ae1f1291
Ensure we don't mutate state cache entries
2016-09-14 10:03:48 +01:00
03a98aff3c
Create new index concurrently
2016-09-12 14:27:01 +01:00
897d57bc58
Change state fetch query for postgres to be faster
...
It turns out that postgres doesn't like doing a list of OR's and is
about 1000x slower, so we just issue a query for each specific type
seperately.
2016-09-12 10:05:07 +01:00
5beda10bbd
Reindex state_groups_state after pruning
2016-09-08 16:18:01 +01:00
b568ca309c
Temporarily disable sequential scans for state fetching
2016-09-08 09:38:54 +01:00
513188aa56
Comment
2016-09-07 14:53:23 +01:00
fadb01551a
Add appopriate framing clause
2016-09-07 14:39:01 +01:00
d25c20ccbe
Use windowing function to make use of index
2016-09-07 14:22:22 +01:00
0595413c0f
Scale the batch size so that we're not bitten by the minimum
2016-09-05 15:49:57 +01:00
a7032abb2e
Correctly handle reindexing state groups that already have an edge
2016-09-05 15:07:23 +01:00
70332a12dd
Take value in a better way
2016-09-05 14:57:14 +01:00
373654c635
Comment about sqlite and WITH RECURSIVE
2016-09-05 14:50:36 +01:00
628e65721b
Add comments
2016-09-05 10:41:27 +01:00
a99e933550
Add upgrade script that will slowly prune state_groups_state entries
2016-09-05 10:05:36 +01:00
598317927c
Limit the length of state chains
2016-09-02 10:41:38 +01:00
9e25443db8
Move to storing state_groups_state as deltas
2016-09-01 14:31:26 +01:00
0cfd6c3161
Use state_groups table to test existence
2016-08-31 16:25:57 +01:00
c10cb581c6
Correctly handle the difference between prev and current state
2016-08-31 14:26:22 +01:00
1bb8ec296d
Generate state group ids in state layer
2016-08-31 10:09:46 +01:00
5dc2a702cf
Make _state_groups_id_gen a normal IdGenerator
2016-08-30 16:55:11 +01:00
778fa85f47
Make sync not pull out full state
2016-08-25 18:59:44 +01:00
a3dc1e9cbe
Replace context.current_state with context.current_state_ids
2016-08-25 17:32:22 +01:00
17f4f14df7
Pull out event ids rather than full events for state
2016-08-25 13:42:44 +01:00
ba214a5e32
Remove lru option
2016-08-19 14:17:11 +01:00
61c7edfd34
Add cache to _get_state_groups_from_groups
2016-04-19 17:22:03 +01:00
87f2dec8d4
Make the cache objects be per instance rather than being global
2016-04-06 13:08:05 +01:00
89e6839a48
Merge pull request #686 from matrix-org/markjh/doc_strings
...
Use google style doc strings.
2016-04-01 16:20:09 +01:00
2a37467fa1
Use google style doc strings.
...
pycharm supports them so there is no need to use the other format.
Might as well convert the existing strings to reduce the risk of
people accidentally cargo culting the wrong doc string format.
2016-04-01 16:12:07 +01:00
e36bfbab38
Use a stream id generator for backfilled ids
2016-04-01 13:29:05 +01:00
31a9eceda5
Add a replication stream for state groups
2016-03-30 16:01:58 +01:00
1e25f62ee6
Use a stream id generator to assign state group ids
2016-03-30 12:55:02 +01:00
2f0180b09e
Don't bother interning keys that are already interned
2016-03-23 16:29:46 +00:00
acdfef7b14
Intern all the things
2016-03-23 16:25:54 +00:00
75daede92f
String intern
2016-03-23 14:53:53 +00:00
8b0dfc9fc4
Don't cache events in get_current_state_for_key
2016-03-23 11:42:17 +00:00
2c86187a1b
Don't cache events in _state_group_cache
...
Instead, simply cache the event ids, relying on the event cache to cache
the actual events.
The problem was that while the state groups cache was limited in the
number of groups it could hold, each individual group could consist of
thousands of events.
2016-03-22 12:00:09 +00:00
54172924c8
Load the current id in the IdGenerator constructor
...
Rather than loading them lazily. This allows us to remove all
the yield statements and spurious arguments for the get_next
methods.
It also allows us to replace all instances of get_next_txn with
get_next since get_next no longer needs to access the db.
2016-03-01 14:32:56 +00:00
5189bfdef4
Batch fetch _get_state_groups_from_groups
2016-02-10 13:24:42 +00:00
Richard van der Hoff