a0b8e5f2fe
Merge pull request #211 from matrix-org/email_in_use
...
Changes for unique emails
2015-08-20 10:04:04 +01:00
e624cdec64
Merge pull request #228 from matrix-org/erikj/_get_state_for_groups
...
Ensure we never return a None event from _get_state_for_groups
2015-08-18 16:30:17 +01:00
38a965b816
Merge pull request #227 from matrix-org/erikj/receipts_take2
...
Re-enable receipts API.
2015-08-18 16:30:04 +01:00
a82938416d
Remove newline because vertical whitespace makes mjark sad
2015-08-18 16:28:13 +01:00
0bfdaf1f4f
Rejig the code to make it nicer
2015-08-18 16:26:07 +01:00
128ed32e6b
Bump size of get_presence_state cache
2015-08-18 15:51:23 +01:00
f704c10f29
Rename unhelpful variable name
2015-08-18 11:54:03 +01:00
8199475ce0
Ensure we never return a None event from _get_state_for_groups
2015-08-18 11:44:10 +01:00
85d0bc3bdc
Reduce cache size from obscenely large to quite large
2015-08-18 11:00:38 +01:00
cfc503681f
Comments
2015-08-18 10:49:23 +01:00
8f4165628b
Add index receipts_linearized_room_stream
2015-08-17 14:43:54 +01:00
d3d582bc1c
Remove unused import
2015-08-17 13:38:09 +01:00
afef6f5d16
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts_take2
2015-08-17 13:23:44 +01:00
2d97e65558
Remember to invalidate caches
2015-08-17 10:46:55 +01:00
1a9510bb84
Implement a batched presence_handler.get_state and use it
2015-08-17 10:40:23 +01:00
47abebfd6d
Add batched version of store.get_presence_state
2015-08-17 09:50:50 +01:00
f9d4da7f45
Fix bug where we were leaking None into state event lists
2015-08-17 09:39:45 +01:00
30883d8409
Merge pull request #221 from matrix-org/auth
...
Simplify LoginHander and AuthHandler
2015-08-14 17:02:22 +01:00
68b255c5a1
Batch _get_linearized_receipts_for_rooms
2015-08-14 15:06:22 +01:00
129ee4e149
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts_take2
2015-08-13 17:28:43 +01:00
2bb2c02571
Remove some vertical space
2015-08-13 17:11:30 +01:00
57877b01d7
Replace list comprehension
2015-08-13 17:00:17 +01:00
5db5677969
Add metrics to the receipts cache
2015-08-13 16:58:23 +01:00
0fbed2a8fa
Comment
2015-08-12 17:22:54 +01:00
c10ac7806e
Explain why we're prefilling dict with Nones
2015-08-12 17:16:30 +01:00
101ee3fd00
Better variable name
2015-08-12 17:08:05 +01:00
df361d08f7
Split _get_state_for_group_from_cache into two
2015-08-12 17:06:21 +01:00
415c2f0549
Simplify LoginHander and AuthHandler
...
* Merge LoginHandler -> AuthHandler
* Add a bunch of documentation
* Improve some naming
* Remove unused branches
I will start merging the actual logic of the two handlers shortly
2015-08-12 15:49:37 +01:00
2df8dd9b37
Move all the caches into their own package, synapse.util.caches
2015-08-11 18:00:59 +01:00
53a817518b
Comments
2015-08-11 11:40:40 +01:00
6eaa116867
Comment
2015-08-11 11:35:24 +01:00
4762c276cb
Docs
2015-08-11 11:33:41 +01:00
1b994a97dd
Fix application of ACLs
2015-08-11 10:41:40 +01:00
10b874067b
Fix state cache
2015-08-11 09:12:41 +01:00
017b798e4f
Clean up StateStore
2015-08-10 15:01:06 +01:00
2c019eea11
Remove unused function
2015-08-10 14:44:41 +01:00
bb0a475c30
Comments
2015-08-10 14:27:38 +01:00
559c51debc
Use TypeError instead of ValueError and give a nicer error mesasge
...
when someone calls Cache.invalidate with the wrong type.
2015-08-10 14:07:17 +01:00
7ce71f2ffc
Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-10 13:47:51 +01:00
8c3a62b5c7
Merge pull request #215 from matrix-org/erikj/cache_varargs_interface
...
Change Cache to not use *args in its interface
2015-08-10 13:47:45 +01:00
86eaaa885b
Rename keyargs to args in CacheDescriptor
2015-08-10 13:44:44 +01:00
e0b6e49466
Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-10 10:39:22 +01:00
2cd6cb9f65
Rename keyargs to args in Cache
2015-08-10 10:38:47 +01:00
aa88582e00
Do bounds check
2015-08-10 10:08:15 +01:00
5119e416e8
Line length
2015-08-10 10:05:30 +01:00
8f04b6fa7a
Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-07 19:30:25 +01:00
7dec0b2bee
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-07 19:28:39 +01:00
06218ab125
Merge pull request #212 from matrix-org/erikj/cache_deferreds
...
Make CacheDescriptor cache deferreds rather than the deferreds' values
2015-08-07 19:28:05 +01:00
2352974aab
Merge branch 'erikj/cache_deferreds' of github.com:matrix-org/synapse into erikj/cache_varargs_interface
2015-08-07 19:26:54 +01:00
9c5385b53a
s/observed/observer/
2015-08-07 19:26:38 +01:00
ffab798a38
Merge branch 'erikj/cache_deferreds' of github.com:matrix-org/synapse into erikj/cache_varargs_interface
2015-08-07 19:18:47 +01:00
62126c996c
Propogate stale cache errors to calling functions
2015-08-07 19:17:58 +01:00
3213ff630c
Remove unnecessary cache
2015-08-07 19:14:05 +01:00
20addfa358
Change Cache to not use *args in its interface
2015-08-07 18:32:47 +01:00
9eb5b23d3a
Batch up various DB requests for event -> state
2015-08-07 18:16:02 +01:00
0211890134
Implement a CacheListDescriptor
2015-08-07 18:14:49 +01:00
0db40d3e93
Don't complain about extra .pyc files we find while hunting for database schemas
2015-08-07 17:22:11 +01:00
b2c7bd4b09
Cache get_recent_events_for_room
2015-08-07 14:42:34 +01:00
b3768ec10a
Remove unncessary cache
2015-08-07 13:41:05 +01:00
b8e386db59
Change Cache to not use *args in its interface
2015-08-07 11:52:21 +01:00
fe994e728f
Store absence of state in cache
2015-08-07 10:17:38 +01:00
1d08bf7c17
Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache
2015-08-06 14:03:15 +01:00
63b1eaf32c
Docs
2015-08-06 14:02:50 +01:00
b811c98574
Remove failed deferreds from cache
2015-08-06 14:01:27 +01:00
433314cc34
Re-implement DEBUG_CACHES flag
2015-08-06 14:01:05 +01:00
f596ff402e
Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache
2015-08-06 13:37:56 +01:00
2efb93af52
Merge branch 'erikj/cached_keyword_args' into erikj/cache_deferreds
2015-08-06 13:35:28 +01:00
953dbd28a7
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cached_keyword_args
2015-08-06 13:35:03 +01:00
7eea3e356f
Make @cached cache deferreds rather than the deferreds' values
2015-08-06 13:33:34 +01:00
3e1b77efc2
Merge branch 'erikj/cached_keyword_args' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-05 16:45:56 +01:00
b52b4a84ec
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-05 15:41:20 +01:00
1e62a3d3a9
Up the cache size for 'get_joined_hosts_for_room' and 'get_users_in_room'
2015-08-05 15:40:40 +01:00
a89559d797
Use LRU cache by default
2015-08-05 15:39:47 +01:00
07507643cb
Use dictionary cache to do group -> state fetching
2015-08-05 15:11:42 +01:00
c67ba143fa
Move DictionaryCache
2015-08-04 15:58:28 +01:00
e7768e77f5
Add basic dictionary cache
2015-08-04 15:56:56 +01:00
e124128542
Bump schema version
2015-08-04 14:50:31 +01:00
c77048e12f
Add endpoint that proxies ID server request token and errors if the given email is in use on this Home Server.
2015-08-04 14:37:09 +01:00
413a4c289b
Add comment
2015-08-04 11:08:07 +01:00
4d6cb8814e
Speed up event filtering (for ACL) logic
2015-08-04 09:32:23 +01:00
39e21ea51c
Add support for using keyword arguments with cached functions
2015-07-27 13:57:29 +01:00
20c0324e9c
Dodesn't seem to make any difference: guess it does work with the object reference
2015-07-21 16:21:37 -07:00
cf7a40b08a
I think this was what was intended...
2015-07-21 16:08:00 -07:00
3b5823c74d
s/take/claim/ for end to end key APIs
2015-07-20 18:23:54 +01:00
4f973eb657
Up default cache size for _RoomStreamChangeCache
2015-07-18 19:07:33 +01:00
b6d4a4c6d8
Merge pull request #199 from matrix-org/erikj/receipts
...
Implement read receipts.
2015-07-16 18:18:36 +01:00
baa55fb69e
Merge pull request #193 from matrix-org/erikj/bulk_persist_event
...
Add bulk insert events API
2015-07-14 10:49:24 +01:00
4624d6035e
Docs
2015-07-14 10:19:07 +01:00
5989637f37
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts
2015-07-13 13:50:57 +01:00
e5991af629
Comments
2015-07-13 13:30:43 +01:00
17bb9a7eb9
Remove commented out code
2015-07-10 14:07:57 +01:00
532fcc997a
Merge pull request #196 from matrix-org/erikj/room_history
...
Add ability to restrict room history.
2015-07-10 13:47:04 +01:00
0b3389bcd2
Merge pull request #194 from matrix-org/erikj/bulk_verify_sigs
...
Implement bulk verify_signed_json API
2015-07-10 13:46:53 +01:00
0d7f0febf4
Uniquely name unique constraint
2015-07-10 13:43:03 +01:00
ed88720952
Handle error slightly better
2015-07-09 16:14:46 +01:00
bf0d59ed30
Don't bother with a timeout for one time keys on the server.
2015-07-09 14:04:03 +01:00
1af188209a
Change format of receipts to allow inclusion of data
2015-07-09 11:39:30 +01:00
8fb79eeea4
Only remove one time keys when new one time keys are added
2015-07-08 17:04:29 +01:00
304343f4d7
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts
2015-07-08 15:37:33 +01:00
af812b68dd
Add a cache to fetching of receipt streams
2015-07-08 15:35:00 +01:00
87311d1b8c
Hook up receipts to v1 initialSync
2015-07-08 11:02:04 +01:00
ca041d5526
Wire together receipts and the notifer/federation
2015-07-07 15:25:30 +01:00
716e426933
Fix various typos
2015-07-07 10:55:31 +01:00
b5770f8947
Add store for client end to end keys
2015-07-06 18:46:47 +01:00
65e69dec8b
Don't explode if we don't recognize one of the event_ids in the backfill request
2015-07-06 09:33:03 +01:00
1a60545626
Add basic impl for room history ACL on GET /messages client API
2015-07-02 16:20:10 +01:00
ac78e60de6
Add stream_id index
2015-07-02 13:18:41 +01:00
80a61330ee
Add basic storage functions for handling of receipts
2015-07-01 17:19:12 +01:00
901f56fa63
Add tables for receipts
2015-06-30 15:29:47 +01:00
b5f55a1d85
Implement bulk verify_signed_json API
2015-06-26 10:39:34 +01:00
5130d80d79
Add bulk insert events API
2015-06-25 17:29:34 +01:00
6924852592
Batch SELECTs in _get_auth_chain_ids_txn
2015-06-23 11:01:04 +01:00
2eda996a63
Add a dummy.sql into delta/20 as pip isn't packinging the pushers.py
2015-06-13 12:21:58 +01:00
1e365e88bd
Bump schema version
2015-06-11 15:50:39 +01:00
da84946de4
pep8
2015-06-04 16:43:45 +01:00
63a7b3ad1e
Add script to (re)convert the pushers table to changing the unique key. Also give the python db upgrade scripts the database engine so they can convert parameter strings, and add *args **kwargs to the upgrade function so we can add more args in future and previous scripts will ignore them.
2015-06-04 16:16:01 +01:00
0d7d9c37b6
Add cache to get_state_groups
2015-06-03 14:45:55 +01:00
d8866d7277
Caches should be bound to instances.
...
Before, caches were global and so different instances of the stores
would share caches. This caused problems in the unit tests.
2015-06-03 14:45:17 +01:00
6e17463228
Don't explode if we don't have the event
2015-06-01 16:39:43 +01:00
a7b65bdedf
Add config option to turn off freezing events. Use new encode_json api and ujson.loads
2015-05-29 12:17:33 +01:00
d9f60e8dc8
Merge pull request #163 from matrix-org/markjh/presence_list_cache
...
Add a cache for the presence list
2015-05-22 17:02:23 +01:00
0e42dfbe22
Merge pull request #164 from matrix-org/markjh/pusher_performance_2
...
Add a cache for get_push rules for user, fix cache invalidation
2015-05-22 17:01:56 +01:00
1a9a9abcc7
Add a cache for getting the presence list for a user
2015-05-22 16:11:17 +01:00
74b7de83ec
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/backfill_fixes
2015-05-22 16:10:42 +01:00
36317f3dad
Merge pull request #156 from matrix-org/erikj/join_perf
...
Make joining #matrix:matrix.org over federation quicker
2015-05-22 16:09:54 +01:00
b6adfc59f5
Invalidate the get_latest_event_ids_in_room cache when deleting from event_forward_extremities
2015-05-22 13:01:03 +01:00
a04cde613e
Add a cache for get_push rules for user, fix cache invalidation
2015-05-22 10:39:45 +01:00
4429e720ae
Merge branch 'master' of github.com:matrix-org/synapse into develop
2015-05-22 10:33:00 +01:00
2043527b9b
Don't try to use a txn when not in one, remove spurious debug logging
2015-05-21 16:53:03 +01:00
53447e9cd3
Add caches for things requested by the pushers
2015-05-21 16:41:39 +01:00
d61ce3f670
Add a cache for get_current_state with state_key
2015-05-21 16:41:39 +01:00
a910984b58
Actually return something from lambda
2015-05-21 15:58:41 +01:00
e309b1045d
Sort backfill events
2015-05-21 15:57:35 +01:00
0180bfe4aa
Remove dead code
2015-05-21 15:53:41 +01:00
1f3d1d85a9
Only get non-state
2015-05-21 15:52:29 +01:00
39a3340f73
Skip events we've already seen
2015-05-21 15:48:56 +01:00
ae3bff3491
Correctly prepopulate queue
2015-05-21 15:46:07 +01:00
dc085ddf8c
Don't prepopulate event_results
2015-05-21 15:44:05 +01:00
73d23c6ae8
Don't readd things that are already in event_results
2015-05-21 15:40:22 +01:00
6189d8e54d
PriorityQueue gives lowest first
2015-05-21 15:38:08 +01:00
115ef3ddac
Correctly capture Queue.Empty exception
2015-05-21 15:37:43 +01:00
c2633907c5
Merge branch 'erikj/join_perf' of github.com:matrix-org/synapse into erikj/backfill_fixes
2015-05-21 14:58:47 +01:00
ebfdd2eb5b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/join_perf
2015-05-21 14:54:52 +01:00
a551c5dad7
Merge pull request #155 from matrix-org/erikj/perf
...
Bulk and batch retrieval of events.
2015-05-21 14:54:40 +01:00
27e4b45c06
s/for events/for requests for events/
2015-05-21 14:52:23 +01:00
ac5f2bf9db
s/for events/for requests for events/
2015-05-21 14:50:57 +01:00
80a167b1f0
Add comments
2015-05-21 11:19:04 +01:00
9118a92862
Split up _get_events into defer and txn versions
2015-05-20 13:27:16 +01:00
ab45e12d31
Make not return a deferred _get_event_from_row_txn
2015-05-20 13:07:19 +01:00
f407cbd2f1
PEP8
2015-05-20 13:02:01 +01:00
227f8ef031
Split out _get_event_from_row back into defer and _txn version
2015-05-20 13:00:57 +01:00
2bc60c55af
Fix _get_backfill_events to return events in the correct order
2015-05-20 12:57:00 +01:00
9084cdd70f
Ensure event_results is a set
2015-05-19 16:34:31 +01:00
aa729349dd
Fix event_backwards_extrem insertion to ignore outliers
2015-05-19 15:27:00 +01:00
99914ec9f8
Merge pull request #152 from matrix-org/notifier_performance
...
Notifier performance
2015-05-18 17:49:59 +01:00
165eb2dbe6
Comments and shuffle of functions
2015-05-18 15:18:41 +01:00
65878a2319
Remove unused metric
2015-05-18 14:06:30 +01:00
4d1b6f4ad1
Remove rejected events if we don't want rejected events
2015-05-18 14:03:46 +01:00
0b0033c40b
Merge branch 'develop' into notifier_performance
2015-05-18 13:50:01 +01:00
10f1bdb9a2
Move get_events functions to storage.events
2015-05-18 10:21:40 +01:00
c71176858b
Newline, remove debug logging
2015-05-18 10:11:14 +01:00
c3b37abdfd
PEP8
2015-05-15 16:59:58 +01:00
6c74fd62a0
Revert limiting of fetching, it didn't help perf.
2015-05-15 16:45:35 +01:00
9ff7f66a2b
init j
2015-05-15 16:36:03 +01:00
70f272f71c
Don't completely drain the list
2015-05-15 16:34:17 +01:00
8763dd80ef
Don't fetch prev_content for current_state
2015-05-15 15:33:01 +01:00
acb12cc811
Make store.get_current_state fetch events asyncly
2015-05-15 15:20:05 +01:00
d62dee7eae
Remove more debug logging
2015-05-15 15:06:37 +01:00
0f29cfabc3
Remove debug logging
2015-05-15 14:06:42 +01:00
e275a9c0d9
preserve log context
2015-05-15 11:54:51 +01:00
aa32bd38e4
Add a wait
2015-05-15 11:35:04 +01:00
372d4c6d7b
Srsly. Don't use closures. Baaaaaad
2015-05-15 11:26:00 +01:00
575ec91d82
Correctly pass through params
2015-05-15 11:15:10 +01:00
de01438a57
Sort out error handling
2015-05-15 11:00:50 +01:00
a2c4f3f150
Fix daedlock
2015-05-15 10:54:04 +01:00
1d566edb81
Remove race condition
2015-05-14 16:54:35 +01:00
ef3d8754f5
Call from right thread
2015-05-14 15:41:55 +01:00
142934084a
Count and loop
2015-05-14 15:40:21 +01:00
96c5b9f87c
Don't start up more fetch_events
2015-05-14 15:36:04 +01:00
7cd6a6f6cf
Awful idea for speeding up fetching of events
2015-05-14 15:34:02 +01:00
7f4105a5c9
Turn off preemptive transactions
2015-05-14 14:51:06 +01:00
f4d58deba1
PEP8
2015-05-14 14:45:42 +01:00
386b7330d2
Move from _base to events
2015-05-14 14:45:22 +01:00
7d6a1dae31
Jump out early
2015-05-14 14:27:58 +01:00
656223fbd3
Actually, we probably want to run this in a transaction
2015-05-14 14:26:35 +01:00
2f7f8e1c2b
Preemptively jump into a transaction if we ask for get_prev_content
2015-05-14 14:17:36 +01:00
e1e9f0c5b2
loop -> gatherResults
2015-05-14 13:58:49 +01:00
ab78a8926e
Err, we probably want a bigger limit
2015-05-14 13:47:16 +01:00
f6f902d459
Move fetching of events into their own transactions
2015-05-14 13:45:48 +01:00
cdb3757942
Refactor _get_events
2015-05-14 13:31:55 +01:00
36ea26c5c0
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/perf
2015-05-14 12:01:38 +01:00
7c549dd557
Add ID generator for push_rules_enable to #resolve SYN-378
2015-05-14 11:44:03 +01:00
899d4675dd
Merge branch 'notifier_unify' into notifier_performance
2015-05-14 11:36:44 +01:00
4f1d984e56
Add index on events
2015-05-13 17:22:26 +01:00
968b01a91a
Actually use async method
2015-05-13 17:02:46 +01:00
4071f29653
Fetch events from events_id in their own transactions
2015-05-13 16:59:41 +01:00
a988361aea
Typo
2015-05-13 15:44:15 +01:00
8888982db3
Don't insert None
2015-05-13 15:43:32 +01:00
cf706cc6ef
Don't return None
2015-05-13 15:31:25 +01:00
5971d240d4
Limit batch size
2015-05-13 15:26:49 +01:00
Mark Haines