644aac5f73
Tighter filtering for user_daily_visits
2018-05-18 17:10:35 +01:00
ef466b3a13
fix psql compatability bug
2018-05-18 15:51:21 +01:00
be11a02c4f
remove empty line
2018-05-16 10:45:40 +01:00
31c2502ca8
style and further contraining query
2018-05-16 09:46:43 +01:00
05ac15ae82
Limit query load of generate_user_daily_visits
...
The aim is to keep track of when it was last called and only query from that point in time
2018-05-15 17:01:33 +01:00
f077e97914
instead of inserting user daily visit data at the end of the day, instead insert incrementally through the day
2018-05-14 13:50:58 +01:00
dd1a832419
remove user agent from data model, will just join on user_ips
2018-05-01 12:13:49 +01:00
fb6015d0a6
pep8
2018-04-25 17:56:11 +01:00
617bf40924
Generate user daily stats
2018-04-25 17:37:29 +01:00
d54cfbb7a8
fix typo
2018-04-10 17:38:16 +01:00
41e0611895
remove errant print
2018-04-09 18:44:20 +01:00
61b439c904
Fix msec to sec, again
2018-04-09 18:43:48 +01:00
87770300d5
Fix msec to sec
2018-04-09 18:38:59 +01:00
64bc2162ef
Fix psycopg2 interpolation
2018-04-09 17:50:36 +01:00
0e5f479fc0
Review comments
...
Use iteritems over item to loop over dict
formatting
2018-04-05 12:16:46 +01:00
b4e37c6f50
pep8
2018-03-29 17:27:39 +01:00
9ee44a372d
Remove need for sqlite specific query
2018-03-29 16:45:34 +01:00
dc7c020b33
fix pep8 errors
2018-03-28 17:25:15 +01:00
241e4e8687
remove twisted deferral cruft
2018-03-28 16:25:53 +01:00
86932be2cb
Support multi client R30 for psql
2018-03-28 14:36:53 +01:00
0f890f477e
No need to cast in count_daily_users
2018-03-28 11:49:57 +01:00
a32d2548d9
query and call for r30 stats
2018-03-28 10:39:13 +01:00
9187e0762f
count_daily_users failed if db was sqlite due to type failure - presumably this prevcented all sqlite homeservers reporting home
2018-03-28 10:02:32 +01:00
64346be26d
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_stream_store
2018-03-01 16:26:42 +00:00
884b26ae41
Remove unused variables
2018-03-01 16:23:48 +00:00
f793bc3877
Split out stream store
2018-03-01 15:13:08 +00:00
6411f725be
Calculate stream_ordering_month_ago correctly on workers
2018-03-01 14:20:53 +00:00
d0957753bf
Merge pull request #2898 from matrix-org/erikj/split_push_rules_store
...
Split PushRulesStore
2018-02-23 11:23:23 +00:00
7e6cf89dc2
Update copyright
2018-02-23 10:39:19 +00:00
cbaad969f9
Split PushRulesStore
2018-02-21 10:43:31 +00:00
ca9b9d9703
Split AccountDataStore and TagStore
2018-02-21 10:15:04 +00:00
f5ac4dc2d4
Split ReceiptsStore
2018-02-20 16:28:28 +00:00
3d33eef6fc
Store state groups separately from events ( #2784 )
...
* Split state group persist into seperate storage func
* Add per database engine code for state group id gen
* Move store_state_group to StateReadStore
This allows other workers to use it, and so resolve state.
* Hook up store_state_group
* Fix tests
* Rename _store_mult_state_groups_txn
* Rename StateGroupReadStore
* Remove redundant _have_persisted_state_group_txn
* Update comments
* Comment compute_event_context
* Set start val for state_group_id_seq
... otherwise we try to recreate old state groups
* Update comments
* Don't store state for outliers
* Update comment
* Update docstring as state groups are ints
2018-02-06 14:31:24 +00:00
6cfee09be9
Make __init__ consitstent across Store heirarchy
...
Add db_conn parameters to the `__init__` methods of the *Store classes, so that
they are all consistent, which makes the multiple inheritance work correctly
(and so that we can later extract mixins which can be used in the slavedstores)
2017-11-13 10:46:07 +00:00
c544188ee3
Add groups to sync stream
2017-07-20 16:36:42 +01:00
b8ca494ee9
Initial group server implementation
2017-07-10 15:44:15 +01:00
a0a561ae85
Fix up client ips to read from pending data
2017-06-27 14:46:12 +01:00
052c5d19d5
Merge pull request #2281 from matrix-org/erikj/phone_home_stats
...
Fix phone home stats
2017-06-15 12:46:23 +01:00
617304b2cf
Fix phone home stats
2017-06-14 19:47:15 +01:00
c62c480dc6
Merge pull request #2259 from matrix-org/erikj/fix_state_woes
...
Fix bug where state_group tables got corrupted
2017-06-07 17:51:25 +01:00
197bd126f0
Fix bug where state_group tables got corrupted
...
This is due to the fact that we prefilled caches using txn.call_after,
which always gets called including on error.
We fix this by making txn.call_after only fire when a transaction
completes successfully, which is what we want most of the time anyway.
2017-06-07 17:39:36 +01:00
304880d185
Add stream change cache
2017-05-31 15:46:36 +01:00
eeb2f9e546
Add user_directory to database
2017-05-31 11:51:01 +01:00
2849d3f29d
admin,storage: added more administrator functionalities
...
administrators can now:
- Set displayname of users
- Update user avatars
- Search for users by user_id
- Browse all users in a paginated API
- Reset user passwords
- Deactivate users
Helpers for doing paginated queries has also been added to storage
Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>
2017-02-02 14:02:26 +01:00
2367c5568c
Add basic implementation of local device list changes
2017-01-25 14:27:27 +00:00
dd52d4de4c
Limit number of entries to prefill from cache
...
Some tables, like device_inbox, take a long time to query at startup for
the stream change cache prefills. This is likely because they are slower
growing streams and so are more fragmented on disk. For now, lets pull
fewer entries out to make startup quicker.
In future, we should add a better index to make it even faster.
2017-01-10 14:34:50 +00:00
c45d8e9ba2
Add profile data to the room_membership table for joins
2016-12-08 13:08:41 +00:00
aa09d6b8f0
Rip out more refresh_token code
...
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.
Still TODO: removing the table from the schema.
2016-11-30 17:40:18 +00:00
de4f798f01
Handling expiring stream extrems correctly.
2016-09-15 17:34:59 +01:00
baffe96d95
Add a room visibility stream
2016-09-14 17:29:19 +01:00
6a6cbfcf1e
Track the max_stream_device_id in a separate table, since we delete from the inbox table
2016-09-09 11:48:23 +01:00
2a0159b8ae
Fix the stream change cache to work over replication
2016-09-07 15:58:00 +01:00
31a07d2335
Add stream change caches for device messages
2016-09-07 15:27:07 +01:00
5dc2a702cf
Make _state_groups_id_gen a normal IdGenerator
2016-08-30 16:55:11 +01:00
e993925279
Add store-and-forward direct-to-device messaging
2016-08-25 17:35:37 +01:00
d9664344ec
Rename table. Add docs.
2016-08-15 11:45:57 +01:00
64e7e11853
Implement cache replication stream
2016-08-15 11:16:45 +01:00
4d70d1f80e
Add some invalidations to a cache_stream
2016-08-15 11:15:17 +01:00
f863a52cea
Add device_id support to /login
...
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.
Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.
Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
978fa53cc2
Pull out min stream_ordering from ex_outlier_stream
2016-07-15 10:22:30 +01:00
eef541a291
Move insert_client_ip to a separate class
2016-06-03 14:42:35 +01:00
3ae915b27e
Add a slaved store for presence
2016-06-03 11:05:53 +01:00
85b992f621
Fix to allow start with postgres
2016-05-27 10:44:44 +01:00
d4503e25ed
Make deleting push actions more efficient
...
There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.
2016-05-20 17:56:10 +01:00
9c272da05f
Add an openidish mechanism for proving to third parties that you own a given user_id
2016-05-05 13:42:44 +01:00
8e6a163f27
Add timestamp and auto incrementing ID
2016-05-04 15:19:12 +01:00
a1e0d316ea
Move _get_cache_dict into the SQLBaseStore
2016-04-06 13:05:19 +01:00
df727f2126
Fix stuck invites
...
If rejecting a remote invite fails with an error response don't fail
the entire request; instead mark the invite as locally rejected.
This fixes the bug where users can get stuck invites which they can
neither accept nor reject.
2016-04-05 11:13:24 +01:00
a2866e2e6a
Rename direction to step, apply checks consistently
2016-04-01 13:50:54 +01:00
e36bfbab38
Use a stream id generator for backfilled ids
2016-04-01 13:29:05 +01:00
1e25f62ee6
Use a stream id generator to assign state group ids
2016-03-30 12:55:02 +01:00
b6e8420aee
Add replication stream for pushers
2016-03-15 17:33:10 +00:00
7076082ae6
Fix relative imports so they work in both py3 and py27
2016-03-08 11:45:50 +00:00
9848b54cac
Prefill from the correct stream
2016-03-04 16:20:22 +00:00
deda48068c
prefill the push rules stream change cache
2016-03-04 16:19:42 +00:00
1b4f4a936f
Hook up the push rules stream to account_data in /sync
2016-03-04 14:44:01 +00:00
a1cf9e3bf3
Add a stream for push rule updates
2016-03-01 18:16:37 +00:00
54172924c8
Load the current id in the IdGenerator constructor
...
Rather than loading them lazily. This allows us to remove all
the yield statements and spurious arguments for the get_next
methods.
It also allows us to replace all instances of get_next_txn with
get_next since get_next no longer needs to access the db.
2016-03-01 14:32:56 +00:00
e6c5e3f28a
Close cursor
2016-02-18 16:39:28 +00:00
112283e230
Prefix TS fields with _ts
2016-02-18 10:11:43 +00:00
e5999bfb1a
Initial cut
2016-02-17 15:40:50 +00:00
d83d004ccd
Fix flake8 warnings for new flake8
2016-02-02 17:18:50 +00:00
25c311eaf6
Cache get_room_changes_for_user
2016-01-29 16:52:48 +00:00
8da95b6f1b
Comment. Remove superfluous order by
2016-01-29 15:39:17 +00:00
b5dbced938
Don't prefill account data
2016-01-29 14:53:59 +00:00
3d60686c0c
Actually use cache
2016-01-29 14:49:11 +00:00
45488e0ffa
Max is not a function
2016-01-29 14:42:01 +00:00
f67d60496a
Convert param style
2016-01-29 14:41:16 +00:00
18579534ea
Prefill stream change caches
2016-01-29 14:37:59 +00:00
7ed2bbeb11
Clean up a bit. Add comment
2016-01-28 14:32:05 +00:00
87f9477b10
Add a Homeserver.setup method.
...
This is for setting up dependencies that require work on startup. This
is useful for the DataStore that wants to read a bunch from the database
before initiliazing.
2016-01-26 15:51:06 +00:00
c232780081
Merge pull request #456 from matrix-org/store_event_actions
...
Send unread notification counts
2016-01-08 14:47:15 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
c914d67cda
Rename event-actions to event_push_actions as per PR request
2016-01-04 14:05:37 +00:00
aa667ee396
Save event actions to the db
2015-12-10 17:51:15 +00:00
95f30ecd1f
Add API for setting account_data globaly or on a per room basis
2015-12-01 18:41:32 +00:00
892e70ec84
Add APIs for adding and removing tags from rooms
2015-10-28 16:06:57 +00:00
f9340ea0d5
Merge branch 'erikj/store_engine' into erikj/search
2015-10-13 13:29:02 +01:00
ec398af41c
Expose error more nicely
2015-10-13 11:43:43 +01:00
40b6a5aad1
Split out the schema preparation and update logic into its own module
2015-10-13 11:38:48 +01:00
Neil Johnson