Commit Graph

202 Commits (96c5b9f87cd5d959d0976487399d4b913082598d)

Author SHA1 Message Date
Erik Johnston 7cd6a6f6cf Awful idea for speeding up fetching of events 2015-05-14 15:34:02 +01:00
Erik Johnston 386b7330d2 Move from _base to events 2015-05-14 14:45:22 +01:00
Erik Johnston 7d6a1dae31 Jump out early 2015-05-14 14:27:58 +01:00
Erik Johnston 656223fbd3 Actually, we probably want to run this in a transaction 2015-05-14 14:26:35 +01:00
Erik Johnston 2f7f8e1c2b Preemptively jump into a transaction if we ask for get_prev_content 2015-05-14 14:17:36 +01:00
Erik Johnston e1e9f0c5b2 loop -> gatherResults 2015-05-14 13:58:49 +01:00
Erik Johnston ab78a8926e Err, we probably want a bigger limit 2015-05-14 13:47:16 +01:00
Erik Johnston cdb3757942 Refactor _get_events 2015-05-14 13:31:55 +01:00
Erik Johnston 36ea26c5c0 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/perf 2015-05-14 12:01:38 +01:00
David Baker 7c549dd557 Add ID generator for push_rules_enable to #resolve SYN-378 2015-05-14 11:44:03 +01:00
Erik Johnston 968b01a91a Actually use async method 2015-05-13 17:02:46 +01:00
Erik Johnston 4071f29653 Fetch events from events_id in their own transactions 2015-05-13 16:59:41 +01:00
Erik Johnston a988361aea Typo 2015-05-13 15:44:15 +01:00
Erik Johnston 8888982db3 Don't insert None 2015-05-13 15:43:32 +01:00
Erik Johnston cf706cc6ef Don't return None 2015-05-13 15:31:25 +01:00
Erik Johnston 5971d240d4 Limit batch size 2015-05-13 15:26:49 +01:00
Erik Johnston ca4f458787 Fetch events in bulk 2015-05-13 15:13:42 +01:00
Erik Johnston 6edff11a88 Don't fetch redaction and rejection stuff for each event, so we can use index only scan 2015-05-13 14:39:05 +01:00
Erik Johnston 409bcc76bd Load events for state group seperately 2015-05-13 11:13:31 +01:00
Erik Johnston da6a7bbdde Merge branch 'develop' of github.com:matrix-org/synapse into erikj/logging_context 2015-05-12 13:10:42 +01:00
Erik Johnston cd525c0f5a push_rules table expects an 'id' field 2015-05-11 11:24:01 +01:00
Erik Johnston 476899295f Change the way we do logging contexts so that they survive divergences 2015-05-08 16:32:18 +01:00
Erik Johnston ed2584050f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/executemany 2015-05-05 18:15:20 +01:00
Mark Haines 31049c4d72 Merge pull request #139 from matrix-org/bugs/SYN-369
Fix race with cache invalidation. SYN-369
2015-05-05 17:46:13 +01:00
Mark Haines deb0237166 Add some doc-string 2015-05-05 17:45:11 +01:00
Mark Haines d18f37e026 Collect the invalidate callbacks on the transaction object rather than passing around a separate list 2015-05-05 17:32:21 +01:00
Erik Johnston 9951542393 Add a comment about the zip(*[zip(sorted(...),...)]) 2015-05-05 17:06:55 +01:00
Mark Haines 041b6cba61 SYN-369: Add comments to the sequence number logic in the cache 2015-05-05 16:32:44 +01:00
Mark Haines 63075118a5 Add debug flag in synapse/storage/_base.py for debugging the cache logic by comparing what is in the cache with what was in the database on every access 2015-05-05 16:24:04 +01:00
Erik Johnston 43c2e8deae Add support for using executemany 2015-05-05 15:13:25 +01:00
Erik Johnston 1692dc019d Don't call 'encode_parameter' no-op 2015-05-05 15:00:30 +01:00
Mark Haines 261d809a47 Sequence the modifications to the cache so that selects don't race with inserts 2015-05-05 14:13:50 +01:00
Erik Johnston 657298cebd Don't lock user_ips table for upsert. 2015-05-01 10:46:48 +01:00
Erik Johnston fabb7acd45 Fix bug where we reconnected to the database on every query. 2015-05-01 10:24:24 +01:00
Erik Johnston 2732be83d9 Shuffle operations so that locking upsert happens last in the txn. This ensures the lock is held for the least amount of time possible. 2015-04-27 13:22:30 +01:00
Erik Johnston e4c4664d73 Handle the fact that postgres databases can be restarted from under us 2015-04-27 12:40:49 +01:00
Erik Johnston b8092fbc82 Go back to storing JSON in TEXT 2015-04-16 11:17:52 +01:00
Erik Johnston 5b31afcbd1 Remove debug logging 2015-04-15 16:27:04 +01:00
Erik Johnston ed26e4012b pushers table requires a unique id. 2015-04-15 16:24:14 +01:00
Erik Johnston a5c72780e6 Don't pass in removed flag 2015-04-15 15:13:22 +01:00
Erik Johnston 4af32a2817 Postgres does not allow you to continue using a cursor after a DB exception has been raised, so move _simple_insert or_ignore flag out of transaction 2015-04-15 14:51:21 +01:00
Erik Johnston 58d8339966 Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS. 2015-04-14 13:53:20 +01:00
Erik Johnston 7ed2ec3061 Handle the fact that in sqlite binary data might be stored as unicode or bytes 2015-04-10 13:41:54 +01:00
Erik Johnston 8bf285e082 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-09 13:13:26 +01:00
Erik Johnston 8ad0f4912e
Stream ordering and out of order insertions.
Handle the fact that events can be persisted out of order, and so to get
the "current max" stream token becomes non trivial - as we need to make
sure that *all* stream tokens less than the current max have also
successfully been persisted.
2015-04-09 11:41:36 +01:00
Mark Haines 83f5125d52 Merge pull request #114 from matrix-org/improve_get_event_cache
Improve get event cache
2015-04-08 16:50:27 +01:00
Erik Johnston 3887350e47 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-08 14:11:07 +01:00
Erik Johnston c1b34af441 Move database timer logging to seperate logger 2015-04-08 13:31:19 +01:00
Erik Johnston 9a05795619 Retry transaction, not SQL query 2015-04-08 13:11:28 +01:00
Erik Johnston 4fe95094d1 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:05:39 +01:00