Commit Graph

148 Commits (738ccf61c01df04e1aef521ea7d1ae2844784214)

Author SHA1 Message Date
Erik Johnston 66f371e8b8 Merge pull request #2176 from matrix-org/erikj/faster_get_joined
Make get_joined_users faster
2017-05-05 10:59:55 +01:00
Erik Johnston e3aa8a7aa8 Merge pull request #2185 from matrix-org/erikj/smaller_caches
Optimise caches for single key
2017-05-05 10:19:05 +01:00
Erik Johnston cf589f2c1e Fixes 2017-05-05 10:17:56 +01:00
Erik Johnston aa93cb9f44 Add comment 2017-05-04 14:59:28 +01:00
Erik Johnston 537dbadea0 Intern host strings 2017-05-04 14:55:28 +01:00
Erik Johnston 07a07588a0 Make caches bigger 2017-05-04 14:52:28 +01:00
Erik Johnston dfaa58f72d Fix comment and num args 2017-05-04 14:50:24 +01:00
Erik Johnston 5d8290429c Reduce size of get_users_in_room 2017-05-04 13:43:19 +01:00
Erik Johnston 7ebf518c02 Make get_joined_users faster 2017-05-03 15:55:54 +01:00
Erik Johnston 7166854f41 Add cache for get_current_hosts_in_room 2017-05-02 10:36:35 +01:00
Erik Johnston f144365281 Comment 2017-04-25 15:18:26 +01:00
Erik Johnston d9aa645f86 Reduce size of joined_user cache
The _get_joined_users_from_context cache stores a mapping from user_id
to avatar_url and display_name. Instead of storing those in a dict,
store them in a namedtuple as that uses much less memory.

We also try converting the string to ascii to further reduce the size.
2017-04-25 14:38:51 +01:00
Erik Johnston 7fc1f1e2b6 Cache hosts in room 2017-03-24 11:46:24 +00:00
Erik Johnston d58b1ffe94 Replace some calls to cursor_to_dict
cursor_to_dict can be surprisinglh expensive for large result sets, so lets
only call it when we need to.
2017-03-24 11:07:02 +00:00
Erik Johnston da146657c9 Comments 2017-03-16 13:04:07 +00:00
Erik Johnston 6957bfdca6 Don't recreate so many sets 2017-03-16 11:54:26 +00:00
Erik Johnston 38258a0976 Bump cache sizes for common membership queries 2017-02-02 18:45:55 +00:00
Erik Johnston 832e9c52ca Comment 2017-02-02 13:09:56 +00:00
Erik Johnston 54a79c1d37 Make presence.get_new_events a bit faster
We do this by caching the set of users a user shares rooms with.
2017-02-02 13:07:18 +00:00
Erik Johnston 62f6b86ba7 Merge pull request #1868 from matrix-org/erikj/replication_cache
Only invalidate membership caches based on the cache stream
2017-02-01 13:12:30 +00:00
Erik Johnston f6124311fd Add m.room.member type to query 2017-02-01 11:59:17 +00:00
Erik Johnston 458b6f4733 Only invalidate membership caches based on the cache stream
Before we completely invalidated get_users_in_room whenever we updated
any current_state_events table. This was way too aggressive.
2017-01-31 16:09:03 +00:00
Erik Johnston 21b7375778 Add an index to make membership queries faster 2017-01-31 15:15:57 +00:00
Erik Johnston 1c13c9f6b6 Don't have such a large cache 2017-01-30 17:12:14 +00:00
Erik Johnston 828db669ec Use get_users_in_room and declare it iterable 2017-01-30 16:37:22 +00:00
Erik Johnston 897f8752da Up cache max entries for state 2017-01-16 15:08:17 +00:00
Erik Johnston 01521299c7 Increase cache size limit 2017-01-16 11:56:51 +00:00
Erik Johnston 2fae34bd2c Optionally measure size of cache by sum of length of values 2017-01-13 17:46:17 +00:00
Matthew Hodgson 8cfc0165e9 fix annoying typos 2017-01-05 13:39:43 +00:00
Erik Johnston c3208e45c9 Fixup membership query 2016-12-14 10:46:58 +00:00
Erik Johnston 7ec2bf9b77 Fix background update that prematurely stopped 2016-12-12 16:54:58 +00:00
Erik Johnston 52d12ca782 Add /room/<room_id>/joined_members API
This returns the currently joined members in the room with their display
names and avatar urls. This is more efficient than /members for large
rooms where you don't need the full events.
2016-12-08 13:32:07 +00:00
Erik Johnston c45d8e9ba2 Add profile data to the room_membership table for joins 2016-12-08 13:08:41 +00:00
Erik Johnston 2ffec928e2 Reduce batch size to be under SQL limit 2016-09-09 11:03:31 +01:00
Erik Johnston c10cb581c6 Correctly handle the difference between prev and current state 2016-08-31 14:26:22 +01:00
Erik Johnston bed10f9880 Use state handler instead of get_users_in_room/get_joined_hosts 2016-08-26 14:54:30 +01:00
Erik Johnston 4daa397a00 Add is_host_joined to slave storage 2016-08-26 13:10:56 +01:00
Erik Johnston 1ccdc1e93a Cache check_host_in_room 2016-08-26 10:59:40 +01:00
Erik Johnston 721414d98a Add desc 2016-08-25 17:49:05 +01:00
Erik Johnston a3dc1e9cbe Replace context.current_state with context.current_state_ids 2016-08-25 17:32:22 +01:00
Erik Johnston 4d70d1f80e Add some invalidations to a cache_stream 2016-08-15 11:15:17 +01:00
Mark Haines 0b2158719c Remove dead code.
Loading push rules now happens in the datastore, so we can remove
the methods that loaded them outside the datastore.

The ``waiting_for_join_list`` in federation handler is populated by
anything, so can be removed.

The ``_get_members_events_txn`` method isn't called from anywhere
so can be removed.
2016-06-07 15:07:11 +01:00
Erik Johnston ccb56fc24b Make get_joined_hosts_for_room use get_users_in_room 2016-06-03 11:20:23 +01:00
Erik Johnston c8285564a3 Use state to calculate get_users_in_room 2016-06-01 15:25:25 +01:00
Mark Haines 816df9f267 get_room_members is unused now 2016-05-16 19:51:43 +01:00
Mark Haines 1a3a2002ff Spell "domain" correctly
s/domian/domain/g
2016-05-16 19:17:23 +01:00
Erik Johnston 08dfa8eee2 Add and use get_domian_from_id 2016-05-09 10:36:03 +01:00
David Baker 2547dffccc Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions
Change pushers to use the event_actions table
2016-04-11 12:58:55 +01:00
Mark Haines 7e2f971c08 Remove some unused functions (#711)
* Remove some unused functions

* get_room_events_stream is only used in tests

* is_exclusive_room might actually be something we want
2016-04-08 14:01:56 +01:00
David Baker 0fd1cd2400 pep8 2016-04-06 16:50:47 +01:00