724aef9a87
Opt out of cache expiry for `get_users_who_share_room_with_user` ( #10826 )
...
* Allow LruCaches to opt out of time-based expiry
* Don't expire `get_users_who_share_room` & friends
2021-09-22 14:21:58 +01:00
a2d7195e01
Track why we're evicting from caches ( #10829 )
...
So we can see distinguish between "evicting because the cache is too big" and "evicting because the cache entries haven't been recently used".
2021-09-22 10:59:52 +01:00
524b8ead77
Add types to synapse.util. ( #10601 )
2021-09-10 17:03:18 +01:00
95e47b2e78
[pyupgrade] `synapse/` ( #10348 )
...
This PR is tantamount to running
```
pyupgrade --py36-plus --keep-percent-format `find synapse/ -type f -name "*.py"`
```
Part of #9744
2021-07-19 15:28:05 +01:00
bdfde6dca1
Use inline type hints in `http/federation/`, `storage/` and `util/` ( #10381 )
2021-07-15 12:46:54 -04:00
7a5873277e
Add support for evicting cache entries based on last access time. ( #10205 )
2021-07-05 16:32:12 +01:00
224f2f949b
Combine `LruCache.invalidate` and `invalidate_many` ( #9973 )
...
* Make `invalidate` and `invalidate_many` do the same thing
... so that we can do either over the invalidation replication stream, and also
because they always confused me a bit.
* Kill off `invalidate_many`
* changelog
2021-05-27 10:33:56 +01:00
c0df6bae06
Remove `keylen` from `LruCache`. ( #9993 )
...
`keylen` seems to be a thing that is frequently incorrectly set, and we don't really need it.
The only time it was used was to figure out if we had removed a subtree in `del_multi`, which we can do better by changing `TreeCache.pop` to return a different type (`TreeCacheNode`).
Commits should be independently reviewable.
2021-05-24 14:02:01 +01:00
ef889c98a6
Optionally track memory usage of each LruCache ( #9881 )
...
This will double count slightly in the presence of interned strings. It's off by default as it can consume a lot of resources.
2021-05-05 16:54:36 +01:00
391bfe9a7b
Reduce memory footprint of caches ( #9886 )
2021-04-28 11:59:28 +01:00
4b965c862d
Remove redundant "coding: utf-8" lines ( #9786 )
...
Part of #9744
Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.
`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
2ca4e349e9
Bugbear: Add Mutable Parameter fixes ( #9682 )
...
Part of #9366
Adds in fixes for B006 and B008, both relating to mutable parameter lint errors.
Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
2021-04-08 22:38:54 +01:00
96e7d3c4a0
Fix 'LruCache' object has no attribute '_on_resize' ( #8591 )
...
We need to make sure we are readu for the `set_cache_factor` callback.
2020-10-19 21:13:50 +01:00
97647b33c2
Replace DeferredCache with LruCache where possible ( #8563 )
...
Most of these uses don't need a full-blown DeferredCache; LruCache is lighter and more appropriate.
2020-10-19 12:20:29 +01:00
6d7b22041d
review comments
2020-10-16 16:25:15 +01:00
995cc615a0
Apply suggestions from code review
...
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2020-10-16 16:14:42 +01:00
0ec0bc3886
type annotations for LruCache
2020-10-16 15:56:39 +01:00
3ee17585cd
Make LruCache register its own metrics ( #8561 )
...
rather than have everything that instantiates an LruCache manage metrics
separately, have LruCache do it itself.
2020-10-16 15:51:57 +01:00
7eff59ec91
Add some more type annotations to Cache
2020-10-14 23:38:14 +01:00
c619253db8
Stop sub-classing object ( #8249 )
2020-09-04 06:54:56 -04:00
eefc6b3a0d
Don't apply cache factor to event cache. ( #7578 )
...
This is already correctly done when we instansiate the cache, but wasn't
when it got reloaded (which always happens at least once on startup).
2020-05-27 12:04:37 +01:00
7cb8b4bc67
Allow configuration of Synapse's cache without using synctl or environment variables ( #6391 )
2020-05-11 18:45:23 +01:00
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
49af402019
run isort
2018-07-09 16:09:20 +10:00
9a0d783c11
Add comments
2018-03-19 11:35:53 +00:00
7c7706f42b
Fix bug where state cache used lots of memory
...
The state cache bases its size on the sum of the size of entries. The
size of the entry is calculated once on insertion, so it is important
that the size of entries does not change.
The DictionaryCache modified the entries size, which caused the state
cache to incorrectly think it was smaller than it actually was.
2018-03-15 15:46:54 +00:00
bc496df192
report metrics on number of cache evictions
2018-02-05 15:34:01 +00:00
c430111d0e
Update LruCache size estimate on clear
2017-01-18 14:55:23 +00:00
1ccd5676e3
Remove needless call to evict()
2017-01-17 11:42:26 +00:00
f85b6ca494
Speed up cache size calculation
...
Instead of calculating the size of the cache repeatedly, which can take
a long time now that it can use a callback, instead cache the size and
update that on insertion and deletion.
This requires changing the cache descriptors to have two caches, one for
pending deferreds and the other for the actual values. There's no reason
to evict from the pending deferreds as they won't take up any more
memory.
2017-01-17 11:18:13 +00:00
2fae34bd2c
Optionally measure size of cache by sum of length of values
2017-01-13 17:46:17 +00:00
45fd2c8942
Ensure invalidation list does not grow unboundedly
2016-08-19 16:09:16 +01:00
4161ff2fc4
Add concept of cache contexts
2016-08-19 14:17:07 +01:00
c4a8cbd15a
Make LruCache use a dedicated _Node class
2016-03-22 16:06:21 +00:00
c046630c33
Remove spurious self.size
2016-01-29 11:17:54 +00:00
766526e114
Make TreeCache keep track of its own size.
2016-01-29 10:11:21 +00:00
50e18938a9
Reset size on clear
2016-01-29 10:00:45 +00:00
7cd418d38e
Don't add the member functiopn if we're not using treecache
2016-01-22 13:40:37 +00:00
cd80019eec
docs
2016-01-22 12:21:13 +00:00
d552861346
Revert all the bits changing keys of eeverything that used LRUCaches to tuples
2016-01-22 12:18:14 +00:00
10f76dc5da
Make LRU cache not default to treecache & add options to use it
2016-01-22 12:10:33 +00:00
f1f8122120
Change LRUCache to be tree-based so we can delete subtrees.
2016-01-21 19:16:25 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
2df8dd9b37
Move all the caches into their own package, synapse.util.caches
2015-08-11 18:00:59 +01:00
David Robertson