Commit Graph

883 Commits (erikj/persist_event_perf)

Author SHA1 Message Date
Erik Johnston 9d6e735344 PEP8 2015-06-25 14:15:53 +01:00
Erik Johnston 584a1c0dca Parallelise key storage 2015-06-25 14:13:47 +01:00
Erik Johnston 3cf2203c5c Don't parallelise persist_events 2015-06-25 14:11:18 +01:00
Erik Johnston 9e6daddaf5 Persist events in parrellel if possible 2015-06-25 11:56:10 +01:00
Erik Johnston e2a241af08 Make _persist_event_txn call _persist_events_txn 2015-06-25 11:45:05 +01:00
Erik Johnston 74f7b44955 Add desc to store_server_keys_json _simple_upsert 2015-06-24 13:15:52 +01:00
Erik Johnston a29319fefa Implement a batch API for verify_json_objects_for_server 2015-06-24 11:21:35 +01:00
Erik Johnston f859e3ca37 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/persist_event_perf 2015-06-23 16:41:58 +01:00
Erik Johnston 6924852592 Batch SELECTs in _get_auth_chain_ids_txn 2015-06-23 11:01:04 +01:00
Erik Johnston 0e1d2d0628 Remove debug logging 2015-06-22 18:09:16 +01:00
Erik Johnston 19f0497ac3 Batch outside transactions 2015-06-22 18:06:13 +01:00
Erik Johnston be29e152d1 Don't update current_state_events for outliers 2015-06-22 17:41:42 +01:00
Erik Johnston a4ad9b9556 Only persist a maximum of 100 events at a time 2015-06-22 17:17:25 +01:00
Erik Johnston 96ce61b2b1 Implement persist_event*s* 2015-06-22 17:03:34 +01:00
Erik Johnston 03a9e4436b Add a many version of _handle_prev_events 2015-06-22 14:09:39 +01:00
Erik Johnston a76e217d8e Add a many version of store_state_groups_txn 2015-06-22 14:09:25 +01:00
Erik Johnston 96be533f1f Use new store.persist_events function in federation handler 2015-06-22 11:42:10 +01:00
Erik Johnston b39b294d1f Properly cache get_server_verify_keys 2015-06-19 17:20:58 +01:00
Erik Johnston 83df0aef58 PEP8 2015-06-19 15:08:01 +01:00
Erik Johnston efe2785cab Cache get_server_verify_key 2015-06-19 15:06:51 +01:00
Erik Johnston 6df6354f1c Don't bother storing things we don't need 2015-06-18 17:47:06 +01:00
Erik Johnston 2eda996a63 Add a dummy.sql into delta/20 as pip isn't packinging the pushers.py 2015-06-13 12:21:58 +01:00
Erik Johnston 1e365e88bd Bump schema version 2015-06-11 15:50:39 +01:00
David Baker da84946de4 pep8 2015-06-04 16:43:45 +01:00
David Baker 63a7b3ad1e Add script to (re)convert the pushers table to changing the unique key. Also give the python db upgrade scripts the database engine so they can convert parameter strings, and add *args **kwargs to the upgrade function so we can add more args in future and previous scripts will ignore them. 2015-06-04 16:16:01 +01:00
Erik Johnston 0d7d9c37b6 Add cache to get_state_groups 2015-06-03 14:45:55 +01:00
Erik Johnston d8866d7277 Caches should be bound to instances.
Before, caches were global and so different instances of the stores
would share caches. This caused problems in the unit tests.
2015-06-03 14:45:17 +01:00
Erik Johnston 6e17463228 Don't explode if we don't have the event 2015-06-01 16:39:43 +01:00
Erik Johnston a7b65bdedf Add config option to turn off freezing events. Use new encode_json api and ujson.loads 2015-05-29 12:17:33 +01:00
Mark Haines d9f60e8dc8 Merge pull request #163 from matrix-org/markjh/presence_list_cache
Add a cache for the presence list
2015-05-22 17:02:23 +01:00
Mark Haines 0e42dfbe22 Merge pull request #164 from matrix-org/markjh/pusher_performance_2
Add a cache for get_push rules for user, fix cache invalidation
2015-05-22 17:01:56 +01:00
Mark Haines 1a9a9abcc7 Add a cache for getting the presence list for a user 2015-05-22 16:11:17 +01:00
Erik Johnston 74b7de83ec Merge branch 'develop' of github.com:matrix-org/synapse into erikj/backfill_fixes 2015-05-22 16:10:42 +01:00
Mark Haines 36317f3dad Merge pull request #156 from matrix-org/erikj/join_perf
Make joining #matrix:matrix.org over federation quicker
2015-05-22 16:09:54 +01:00
Mark Haines b6adfc59f5 Invalidate the get_latest_event_ids_in_room cache when deleting from event_forward_extremities 2015-05-22 13:01:03 +01:00
Mark Haines a04cde613e Add a cache for get_push rules for user, fix cache invalidation 2015-05-22 10:39:45 +01:00
Erik Johnston 4429e720ae Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-05-22 10:33:00 +01:00
Mark Haines 2043527b9b Don't try to use a txn when not in one, remove spurious debug logging 2015-05-21 16:53:03 +01:00
Mark Haines 53447e9cd3 Add caches for things requested by the pushers 2015-05-21 16:41:39 +01:00
Mark Haines d61ce3f670 Add a cache for get_current_state with state_key 2015-05-21 16:41:39 +01:00
Erik Johnston a910984b58 Actually return something from lambda 2015-05-21 15:58:41 +01:00
Erik Johnston e309b1045d Sort backfill events 2015-05-21 15:57:35 +01:00
Erik Johnston 0180bfe4aa Remove dead code 2015-05-21 15:53:41 +01:00
Erik Johnston 1f3d1d85a9 Only get non-state 2015-05-21 15:52:29 +01:00
Erik Johnston 39a3340f73 Skip events we've already seen 2015-05-21 15:48:56 +01:00
Erik Johnston ae3bff3491 Correctly prepopulate queue 2015-05-21 15:46:07 +01:00
Erik Johnston dc085ddf8c Don't prepopulate event_results 2015-05-21 15:44:05 +01:00
Erik Johnston 73d23c6ae8 Don't readd things that are already in event_results 2015-05-21 15:40:22 +01:00
Erik Johnston 6189d8e54d PriorityQueue gives lowest first 2015-05-21 15:38:08 +01:00
Erik Johnston 115ef3ddac Correctly capture Queue.Empty exception 2015-05-21 15:37:43 +01:00