Commit Graph

342 Commits (cf823389307bf8e78dec0b018c44abc8eec91ddf)

Author SHA1 Message Date
Luke Barnard bbeeb97f75 Implement _simple_delete_many_txn, use it to delete devices
(But this doesn't implement the same for deleting access tokens or e2e keys.

Also respond to code review.
2017-03-13 17:53:23 +00:00
Richard van der Hoff 6ad71cc29d Remove spurious SQL logging (#1972)
looks like the upsert function was accidentally sending sql logging to the
general logger. We already log the sql in `txn.execute`.
2017-03-08 18:00:44 +00:00
Erik Johnston b84907bdbb Intern table column names once 2017-02-28 14:38:16 +00:00
Erik Johnston 1a4f8022e6 Strip newlines from SQL queries 2017-02-23 11:15:31 +00:00
Morteza Araby 2849d3f29d admin,storage: added more administrator functionalities
administrators can now:
 - Set displayname of users
 - Update user avatars
 - Search for users by user_id
 - Browse all users in a paginated API
 - Reset user passwords
 - Deactivate users

Helpers for doing paginated queries has also been added to storage

Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>
2017-02-02 14:02:26 +01:00
Erik Johnston 2367c5568c Add basic implementation of local device list changes 2017-01-25 14:27:27 +00:00
Erik Johnston d906206049 Increase state_group_cache_size 2017-01-17 11:31:08 +00:00
Erik Johnston dd52d4de4c Limit number of entries to prefill from cache
Some tables, like device_inbox, take a long time to query at startup for
the stream change cache prefills. This is likely because they are slower
growing streams and so are more fragmented on disk. For now, lets pull
fewer entries out to make startup quicker.

In future, we should add a better index to make it even faster.
2017-01-10 14:34:50 +00:00
Erik Johnston 524d61bf7e Fix tests 2016-11-21 11:53:02 +00:00
Erik Johnston 7c9cdb2245 Store federation stream positions in the database 2016-11-21 11:33:08 +00:00
pik e8b1d2a452 Refactor test_filter to use real DataStore
* add tests for filter api errors
2016-10-18 12:17:38 -05:00
Erik Johnston ba214a5e32 Remove lru option 2016-08-19 14:17:11 +01:00
Erik Johnston 949629291c Do it in storage function 2016-08-16 17:05:34 +01:00
Erik Johnston a2427981b7 Use cached get_user_by_access_token in slaves 2016-08-16 11:24:32 +01:00
Erik Johnston 89e786bd85 Doc get_next() context manager usage 2016-08-15 13:45:26 +01:00
Erik Johnston d9664344ec Rename table. Add docs. 2016-08-15 11:45:57 +01:00
Erik Johnston 64e7e11853 Implement cache replication stream 2016-08-15 11:16:45 +01:00
Erik Johnston 4d70d1f80e Add some invalidations to a cache_stream 2016-08-15 11:15:17 +01:00
Erik Johnston 99bbd90b0d Always run txn.after_callbacks 2016-08-15 09:45:44 +01:00
Richard van der Hoff 053e83dafb More doc-comments
Fix some more comments on some things
2016-07-20 16:40:28 +01:00
Erik Johnston f328d95cef Feature: Add deactivate account admin API
Allows server admins to "deactivate" accounts, which:

- Revokes all access tokens
- Removes all threepids
- Removes password

The API is a POST to `/admin/deactivate/<user_id>`
2016-06-30 15:40:58 +01:00
Mark Haines 80f34d7b57 Fix setting the _clock in SQLBaseStore 2016-06-02 15:23:56 +01:00
David Baker d4503e25ed Make deleting push actions more efficient
There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.
2016-05-20 17:56:10 +01:00
Erik Johnston 0c11c1be88 Spelling 2016-05-13 14:42:25 +01:00
Erik Johnston 13d37c3c56 Fixup add_pusher 2016-05-13 11:25:02 +01:00
Mark Haines 7e2f971c08 Remove some unused functions (#711)
* Remove some unused functions

* get_room_events_stream is only used in tests

* is_exclusive_room might actually be something we want
2016-04-08 14:01:56 +01:00
Mark Haines a1e0d316ea Move _get_cache_dict into the SQLBaseStore 2016-04-06 13:05:19 +01:00
Erik Johnston acdfef7b14 Intern all the things 2016-03-23 16:25:54 +00:00
Erik Johnston b591277620 Make stateGroupCache honour CACHE_SIZE_FACTOR 2016-03-22 10:32:50 +00:00
Mark Haines a1cf9e3bf3 Add a stream for push rule updates 2016-03-01 18:16:37 +00:00
Erik Johnston 2c1fbea531 Fix up logcontexts 2016-02-08 14:26:45 +00:00
Daniel Wagner-Hall d83d004ccd Fix flake8 warnings for new flake8 2016-02-02 17:18:50 +00:00
Erik Johnston 87f9477b10 Add a Homeserver.setup method.
This is for setting up dependencies that require work on startup. This
is useful for the DataStore that wants to read a bunch from the database
before initiliazing.
2016-01-26 15:51:06 +00:00
Erik Johnston aea5da0ef6 Guard against empty iterables 2016-01-25 15:59:29 +00:00
Erik Johnston 1ebf5e3d03 Correct docstring 2016-01-25 15:53:36 +00:00
Erik Johnston ddd25def01 Implement a _simple_select_many_batch 2016-01-25 13:36:02 +00:00
Matthew Hodgson 6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
Mark Haines 6a5ff5f223 Track the time spent in the database per request.
and track the number of transactions that request started.
2015-12-07 17:56:11 +00:00
Erik Johnston c85c912562 Add basic full text search impl. 2015-10-09 15:48:31 +01:00
Mark Haines 396834f1c0 synapse/storage/_base.py:_simple_max_id was unused 2015-09-23 10:30:38 +01:00
Mark Haines 1d9036aff2 synapse/storage/_base.py:_simple_delete was unused 2015-09-23 10:30:25 +01:00
Mark Haines 1ee3d26432 synapse/storage/_base.py:_simple_selectupdate_one was unused 2015-09-23 10:30:03 +01:00
Mark Haines 7dd4f79c49 synapse/storage/_base.py:_execute_and_decode was unused 2015-09-22 18:37:07 +01:00
Mark Haines 527d95dea0 synapse/storage/_base.py:Table was unused 2015-09-22 18:14:15 +01:00
Erik Johnston b62c1395d6 Merge branch 'release-v0.10.0' of github.com:matrix-org/synapse into develop 2015-09-01 13:11:55 +01:00
Erik Johnston cd800ad99a Lower size of 'stateGroupCache' now that we have data from matrix.org to support doing so 2015-09-01 10:09:03 +01:00
Daniel Wagner-Hall cecbd636e9 /tokenrefresh POST endpoint
This allows refresh tokens to be exchanged for (access_token,
refresh_token).

It also starts issuing them on login, though no clients currently
interpret them.
2015-08-20 16:21:35 +01:00
Erik Johnston 2df8dd9b37 Move all the caches into their own package, synapse.util.caches 2015-08-11 18:00:59 +01:00
Erik Johnston 53a817518b Comments 2015-08-11 11:40:40 +01:00
Erik Johnston 6eaa116867 Comment 2015-08-11 11:35:24 +01:00
Erik Johnston 4762c276cb Docs 2015-08-11 11:33:41 +01:00
Erik Johnston bb0a475c30 Comments 2015-08-10 14:27:38 +01:00
Erik Johnston 7ce71f2ffc Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-10 13:47:51 +01:00
Erik Johnston 86eaaa885b Rename keyargs to args in CacheDescriptor 2015-08-10 13:44:44 +01:00
Erik Johnston e0b6e49466 Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-10 10:39:22 +01:00
Erik Johnston 2cd6cb9f65 Rename keyargs to args in Cache 2015-08-10 10:38:47 +01:00
Erik Johnston 7dec0b2bee Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-07 19:28:39 +01:00
Erik Johnston 2352974aab Merge branch 'erikj/cache_deferreds' of github.com:matrix-org/synapse into erikj/cache_varargs_interface 2015-08-07 19:26:54 +01:00
Erik Johnston 9c5385b53a s/observed/observer/ 2015-08-07 19:26:38 +01:00
Erik Johnston ffab798a38 Merge branch 'erikj/cache_deferreds' of github.com:matrix-org/synapse into erikj/cache_varargs_interface 2015-08-07 19:18:47 +01:00
Erik Johnston 62126c996c Propogate stale cache errors to calling functions 2015-08-07 19:17:58 +01:00
Erik Johnston 20addfa358 Change Cache to not use *args in its interface 2015-08-07 18:32:47 +01:00
Erik Johnston 0211890134 Implement a CacheListDescriptor 2015-08-07 18:14:49 +01:00
Erik Johnston b8e386db59 Change Cache to not use *args in its interface 2015-08-07 11:52:21 +01:00
Erik Johnston 1d08bf7c17 Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache 2015-08-06 14:03:15 +01:00
Erik Johnston 63b1eaf32c Docs 2015-08-06 14:02:50 +01:00
Erik Johnston b811c98574 Remove failed deferreds from cache 2015-08-06 14:01:27 +01:00
Erik Johnston 433314cc34 Re-implement DEBUG_CACHES flag 2015-08-06 14:01:05 +01:00
Erik Johnston f596ff402e Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache 2015-08-06 13:37:56 +01:00
Erik Johnston 2efb93af52 Merge branch 'erikj/cached_keyword_args' into erikj/cache_deferreds 2015-08-06 13:35:28 +01:00
Erik Johnston 953dbd28a7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cached_keyword_args 2015-08-06 13:35:03 +01:00
Erik Johnston 7eea3e356f Make @cached cache deferreds rather than the deferreds' values 2015-08-06 13:33:34 +01:00
Erik Johnston 3e1b77efc2 Merge branch 'erikj/cached_keyword_args' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-05 16:45:56 +01:00
Erik Johnston b52b4a84ec Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-05 15:41:20 +01:00
Erik Johnston a89559d797 Use LRU cache by default 2015-08-05 15:39:47 +01:00
Erik Johnston 07507643cb Use dictionary cache to do group -> state fetching 2015-08-05 15:11:42 +01:00
Erik Johnston 4d6cb8814e Speed up event filtering (for ACL) logic 2015-08-04 09:32:23 +01:00
Erik Johnston 39e21ea51c Add support for using keyword arguments with cached functions 2015-07-27 13:57:29 +01:00
Erik Johnston 80a61330ee Add basic storage functions for handling of receipts 2015-07-01 17:19:12 +01:00
Erik Johnston d8866d7277 Caches should be bound to instances.
Before, caches were global and so different instances of the stores
would share caches. This caused problems in the unit tests.
2015-06-03 14:45:17 +01:00
Erik Johnston 4429e720ae Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-05-22 10:33:00 +01:00
Mark Haines 53447e9cd3 Add caches for things requested by the pushers 2015-05-21 16:41:39 +01:00
Mark Haines d61ce3f670 Add a cache for get_current_state with state_key 2015-05-21 16:41:39 +01:00
Erik Johnston 65878a2319 Remove unused metric 2015-05-18 14:06:30 +01:00
Erik Johnston c71176858b Newline, remove debug logging 2015-05-18 10:11:14 +01:00
Erik Johnston c3b37abdfd PEP8 2015-05-15 16:59:58 +01:00
Erik Johnston aa32bd38e4 Add a wait 2015-05-15 11:35:04 +01:00
Erik Johnston a2c4f3f150 Fix daedlock 2015-05-15 10:54:04 +01:00
Erik Johnston 1d566edb81 Remove race condition 2015-05-14 16:54:35 +01:00
Erik Johnston 142934084a Count and loop 2015-05-14 15:40:21 +01:00
Erik Johnston 7cd6a6f6cf Awful idea for speeding up fetching of events 2015-05-14 15:34:02 +01:00
Erik Johnston 386b7330d2 Move from _base to events 2015-05-14 14:45:22 +01:00
Erik Johnston 7d6a1dae31 Jump out early 2015-05-14 14:27:58 +01:00
Erik Johnston 656223fbd3 Actually, we probably want to run this in a transaction 2015-05-14 14:26:35 +01:00
Erik Johnston 2f7f8e1c2b Preemptively jump into a transaction if we ask for get_prev_content 2015-05-14 14:17:36 +01:00
Erik Johnston e1e9f0c5b2 loop -> gatherResults 2015-05-14 13:58:49 +01:00
Erik Johnston ab78a8926e Err, we probably want a bigger limit 2015-05-14 13:47:16 +01:00
Erik Johnston cdb3757942 Refactor _get_events 2015-05-14 13:31:55 +01:00
Erik Johnston 36ea26c5c0 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/perf 2015-05-14 12:01:38 +01:00
David Baker 7c549dd557 Add ID generator for push_rules_enable to #resolve SYN-378 2015-05-14 11:44:03 +01:00