8abdd7b553
Fix up indices for users_who_share_rooms
2017-06-26 14:01:30 +01:00
7fe8ed1787
Store URL cache preview downloads seperately
...
This makes it easier to clear old media out at a later date
2017-06-23 11:14:11 +01:00
b8b936a6ea
Add API to quarantine media
2017-06-19 17:39:21 +01:00
d7fe6b356c
Add shutdown room API
2017-06-19 12:37:27 +01:00
ebcd55d641
Add DB schema for tracking users who share rooms
2017-06-15 09:45:48 +01:00
65f0513a33
Split up device_lists_outbound_pokes table for faster updates.
2017-06-07 11:02:38 +01:00
21e255a8f1
Split the table in two
2017-06-01 14:50:46 +01:00
5dd1b2c525
Use unique indices
2017-05-31 17:29:12 +01:00
5d79d728f5
Split out directory and search tables
2017-05-31 15:23:49 +01:00
350622a107
Handle the server leaving a public room
2017-05-31 15:11:36 +01:00
63fda37e20
Add comments
2017-05-31 15:00:29 +01:00
eeb2f9e546
Add user_directory to database
2017-05-31 11:51:01 +01:00
dd48f7204c
Add comment
2017-05-30 15:01:22 +01:00
04095f7581
Add clobbered event_id
2017-05-30 14:53:01 +01:00
a584a81b3e
Add current_state_delta_stream table
2017-05-30 14:44:09 +01:00
6e614e9e10
Add background task to clear out old event_auth
2017-05-24 15:23:34 +01:00
ff3d810ea8
Add a comment to old delta
2017-05-11 12:48:50 +01:00
baafb85ba4
Add an index to event_search
...
- to make the purge API quicker
2017-05-11 12:05:22 +01:00
b990b2fce5
Add per user ratelimiting overrides
2017-05-10 11:05:43 +01:00
d766343668
Add index to device_lists_stream
2017-03-01 15:56:30 +00:00
e4919b9329
Add stream_id index to device_lists_outbound_pokes
...
As this is used for replication streaming
2017-02-28 11:19:06 +00:00
4aa29508af
Use TEXT rather than VARCHAR
...
While we're changing anyway
2017-02-17 10:51:49 +00:00
b4017539d4
Make the pushers lang field column longer
...
To accommodate things like zh-Hans-CN
Fixes https://github.com/vector-im/riot-ios/issues/1031
2017-02-17 10:42:57 +00:00
095b45c165
Aggregate event push actions
2017-02-14 13:39:41 +00:00
21b7375778
Add an index to make membership queries faster
2017-01-31 15:15:57 +00:00
d1e1fd6210
Add ts column to device_lists_outbound_pokes
2017-01-27 15:23:48 +00:00
fbfad76c03
Add comments
2017-01-26 16:33:21 +00:00
c974116f19
Implement device key caching over federation
2017-01-26 16:07:24 +00:00
2367c5568c
Add basic implementation of local device list changes
2017-01-25 14:27:27 +00:00
c050f493dd
Add comment
2017-01-13 15:14:41 +00:00
caddadfc5a
Change device_inbox stream index to include user
...
This makes fetching the nost recently changed users much tricker, and
brings it in line with e.g. presence_stream indices.
2017-01-10 15:04:57 +00:00
1574b839e0
Merge pull request #1676 from matrix-org/erikj/room_list
...
Add new API appservice specific public room list
2016-12-12 17:00:10 +00:00
c45d8e9ba2
Add profile data to the room_membership table for joins
2016-12-08 13:08:41 +00:00
2687af82d4
Comments
2016-12-07 09:58:33 +00:00
f32fb65552
Add new API appservice specific public room list
2016-12-06 16:12:27 +00:00
46ca345b06
Don't send old events as federation
2016-11-24 12:29:02 +00:00
2ac516850b
More efficient notif count queries
2016-11-23 15:57:04 +00:00
7c9cdb2245
Store federation stream positions in the database
2016-11-21 11:33:08 +00:00
f8ee66250a
Handle sending events and device messages over federation
2016-11-17 15:48:04 +00:00
8fd4d9129f
Replace postgres GIN with GIST
...
This is because GIN can be slow to write too, especially when the table
gets large.
2016-11-03 15:00:03 +00:00
a9111786f9
Use most recently added binding, not most recently seen user.
2016-10-27 14:32:45 +01:00
1fc1bc2a51
Fix user_threepids schema delta
...
The delta `37/user_threepids.sql` aimed to update all the email
addresses to be lower case, however duplicate emails may exist in the
table already.
This commit adds a step where the delta moves the duplicate emails to a
new `medium` `email_old`. Only the most recently used account keeps the
binding intact. We move rather than delete so that we retain some record
of which emails were associated with which account.
2016-10-27 14:14:44 +01:00
0108ed8ae6
Latest delta is now 37
2016-10-19 11:40:35 +01:00
a7f48320b1
Merge remote-tracking branch 'origin/develop' into dbkr/password_reset_case_insensitive
2016-10-19 11:28:56 +01:00
df2a616c7b
Convert emails to lowercase when storing
...
And db migration sql to convert existing addresses.
2016-10-19 11:13:55 +01:00
6942d68247
Bump schema version
2016-10-17 11:17:45 +01:00
b59994b454
Remove TODO
2016-10-17 11:17:02 +01:00
816988baaa
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/remove_auth
2016-10-17 11:10:37 +01:00
2869a29fd7
Drop some unused indices
2016-10-17 11:08:19 +01:00
62073992c5
Make password reset email field case insensitive
2016-10-14 13:56:53 +01:00
8009d84364
Match against event_id, rather than room_id
2016-09-21 16:46:59 +01:00
dc78db8c56
Update correct table
2016-09-21 15:52:44 +01:00
4f78108d8c
Readd entries to public_room_list_stream that were deleted
2016-09-21 15:24:22 +01:00
baffe96d95
Add a room visibility stream
2016-09-14 17:29:19 +01:00
ed992ae6ba
Add a DB index to figure out past state at a stream ordering in a room
2016-09-14 16:20:27 +01:00
43253c10b8
Remove redundant event_auth index
2016-09-13 11:47:48 +01:00
0b32bb20bb
Index contains_url for file search queries
2016-09-12 16:57:05 +01:00
7fe42cf949
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_notif_highlight_query_fast
2016-09-12 12:37:09 +01:00
15ca0c6a4d
Make reindex happen in bg
2016-09-12 12:36:36 +01:00
31f85f9db9
Add comments to existing schema deltas that used "CREATE INDEX" directly
2016-09-12 11:00:26 +01:00
54417999b6
Revert "Add index to event_push_actions"
2016-09-12 10:39:55 +01:00
b91e2833b3
Merge remote-tracking branch 'origin/develop' into dbkr/make_notif_highlight_query_fast
2016-09-09 19:11:34 +01:00
f2acc3dcf9
Add index to event_push_actions
...
and remove room_id caluse so it uses it
Mostly from @negativemjark
2016-09-09 18:54:54 +01:00
647c724573
Use the previous MAX value if any to set the stream_id
2016-09-09 11:52:44 +01:00
6a6cbfcf1e
Track the max_stream_device_id in a separate table, since we delete from the inbox table
2016-09-09 11:48:23 +01:00
303b6f29f0
Merge pull request #1087 from matrix-org/markjh/reapply_delta
...
Reapply 34/device_outbox in 35/device_outbox_again.py since the schem…
2016-09-09 11:24:27 +01:00
0877157353
Just move the schema and add some DROPs
2016-09-09 11:04:47 +01:00
43b77c5d97
Only catch databas errors
2016-09-08 17:44:21 +01:00
7d5b142547
Add a stub run_upgrade
2016-09-08 17:39:11 +01:00
fa722a699c
Reapply 34/device_outbox in 35/device_outbox_again.py since the schema was bumped before it landed on develop
2016-09-08 17:35:16 +01:00
ebb46497ba
Add delta file
2016-09-08 16:38:54 +01:00
fa9d36e050
Merge branch 'develop' into markjh/direct_to_device_federation
2016-09-08 13:43:43 +01:00
d4a35ada28
Send device messages over federation
2016-09-06 18:16:20 +01:00
2ad72da931
Add tables for federated device messages
...
Adds tables for storing the messages that need to be sent to a
remote device and for deduplicating messages received.
2016-09-06 15:10:40 +01:00
485d999c8a
Correctly delete old state groups in purge history API
2016-09-05 14:49:08 +01:00
a99e933550
Add upgrade script that will slowly prune state_groups_state entries
2016-09-05 10:05:36 +01:00
657847e4c6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage
2016-09-02 11:04:48 +01:00
9e25443db8
Move to storing state_groups_state as deltas
2016-09-01 14:31:26 +01:00
ef0cc648cf
Clean up old sent transactions
2016-08-31 11:12:02 +01:00
e993925279
Add store-and-forward direct-to-device messaging
2016-08-25 17:35:37 +01:00
aad8a1a825
Delete old received_transactions
2016-08-22 16:29:46 +01:00
9da84a9a1e
Make AppserviceHandler stream events from database
...
This is for two reasons:
1. Suppresses duplicates correctly, as the notifier doesn't do any
duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
stream.
2016-08-18 11:54:41 +01:00
732cf72b86
Fix push_display_name_rename schema update
2016-08-17 18:12:21 +01:00
ad42322257
Add migration script
...
To port existing rule actions & enable entries to the new name
2016-08-16 16:56:30 +01:00
d9664344ec
Rename table. Add docs.
2016-08-15 11:45:57 +01:00
4d70d1f80e
Add some invalidations to a cache_stream
2016-08-15 11:15:17 +01:00
4fec5e57be
Default device_display_name to null
...
It turns out that it's more useful to return a null device display name (and
let clients decide how to handle it: eg, falling back to device_id) than using
a constant string like "unknown device".
2016-08-03 11:53:00 +01:00
26cb0efa88
SQL syntax fix
2016-07-27 12:30:22 +01:00
2e3d90d67c
Make the device id on e2e key upload optional
...
We should now be able to get our device_id from the access_token, so the
device_id on the upload request is optional. Where it is supplied, we should
check that it matches.
For active access_tokens without an associated device_id, we ought to register
the device in the devices table.
Also update the table on upgrade so that all of the existing e2e keys are
associated with real devices.
2016-07-26 23:38:12 +01:00
436bffd15f
Implement deleting devices
2016-07-26 07:35:48 +01:00
ec5717caf5
Create index on user_ips in the background
...
user_ips is kinda big, so really we want to add the index in the background
once we're running. Replace the schema delta with one which will do that.
I've done this in a way that's reasonably easy to reuse as there a few other
indexes I need, and I don't suppose they will be the last.
2016-07-22 13:16:39 +01:00
bc8f265f0a
GET /devices endpoint
...
implement a GET /devices endpoint which lists all of the user's devices.
It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
1e2a7f18a1
Merge pull request #922 from matrix-org/erikj/file_api2
...
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
022b9176fe
schema fix
...
device_id should be text, not bigint.
2016-07-19 11:44:05 +01:00
f863a52cea
Add device_id support to /login
...
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.
Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.
Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
b64aa6d687
Add sender and contains_url field to events table
2016-07-14 15:35:43 +01:00
314b146b2e
Track approximate last access time for remote media
2016-06-29 11:41:20 +01:00
c8c5bf950a
Fix synapse/storage/schema/delta/30/as_users.py
2016-05-31 17:10:40 +01:00
c9aff0736c
Remove topics table
2016-05-13 14:40:38 +01:00
9295fa30a8
Annotate the removed indicies with why they were removed.
2016-05-13 14:16:57 +01:00
5e50058473
Remove unused indices
...
This includes removing both unused indices and indices that are subsets
of other indices.
2016-05-13 13:28:07 +01:00
3b1930e8ec
unbreak schema
2016-05-10 16:42:37 +01:00
997db04648
Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs
2016-05-10 14:40:19 +02:00
9c272da05f
Add an openidish mechanism for proving to third parties that you own a given user_id
2016-05-05 13:42:44 +01:00
fcd1eb642d
Add primary key
2016-05-04 16:51:51 +01:00
8e6a163f27
Add timestamp and auto incrementing ID
2016-05-04 15:19:12 +01:00
984d4a2c0f
Add /report endpoint
2016-05-04 11:28:10 +01:00
acded821c4
Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs
2016-04-29 10:05:20 +01:00
ae571810f2
Order NULLs first
2016-04-21 18:14:18 +01:00
8fae3d7b1e
Use special UPDATE syntax
2016-04-21 18:01:49 +01:00
51bb339ab2
Create index concurrently
2016-04-21 17:16:11 +01:00
b743c1237e
Add missing run_upgrade
2016-04-21 17:12:04 +01:00
c877f0f034
Optimise event_search in postgres
2016-04-21 16:56:14 +01:00
07d765209d
First bits of emailpusher
...
Mostly logic of when to send an email
2016-04-19 14:24:36 +01:00
96bcfb29c7
Add index
2016-04-14 11:26:33 +01:00
2547dffccc
Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions
...
Change pushers to use the event_actions table
2016-04-11 12:58:55 +01:00
6ff7a79308
move local_media_repository_url_cache.sql to schema v31
2016-04-08 19:09:02 +01:00
d6e7333ae4
Merge branch 'develop' into matthew/preview_urls
2016-04-07 17:26:44 +01:00
05d044aac3
pep8
2016-04-07 16:45:38 +01:00
2d5c693fd3
Fix port script for changes merged from develop
2016-04-07 16:43:54 +01:00
9c99ab4572
Merge remote-tracking branch 'origin/develop' into dbkr/pushers_use_event_actions
2016-04-07 16:35:22 +01:00
7e2c89a37f
Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do:
...
* Make badges work again
* Remove old, unused code
2016-04-06 15:42:15 +01:00
8aab9d87fa
Don't require config to create database
2016-04-06 14:15:45 +01:00
0c53d750e7
Docs and indents
2016-04-04 18:02:48 +01:00
92ab45a330
Add upgrade path, rename table
2016-04-04 17:07:43 +01:00
3d76b7cb2b
Store invites in a separate table.
2016-04-04 16:30:15 +01:00
9f7dc2bef7
Merge branch 'develop' into matthew/preview_urls
2016-04-04 00:38:21 +01:00
7426c86eb8
add a persistent cache of URL lookups, and fix up the in-memory one to work
2016-04-03 00:31:57 +01:00
1fbb094c6f
Add replication streams for ex outliers and current state resets
2016-03-30 17:19:56 +01:00
b275765545
Comment about weird SQL
2016-03-23 14:15:32 +00:00
0c1a27b787
SQLite and postgres doesn't share a true literal
2016-03-23 14:10:49 +00:00
84afeb41f3
Ensure all old public rooms have aliases
2016-03-23 13:59:34 +00:00
ba660ecde2
Add a comment to offer a hint to an explanation for why we have a unique constraint on (app_id, pushkey, user_id)
2016-03-16 10:35:00 +00:00
b6e8420aee
Add replication stream for pushers
2016-03-15 17:33:10 +00:00
9669a99d1a
Update users table in a batched manner
2016-03-10 15:12:19 +00:00
3b97797c8d
Merge branch 'develop' into daniel/ick
2016-03-08 17:35:09 +00:00
edca2d9891
Idempotent-ise schema update script
...
If any ASes don't have an ID, the schema will fail, and then it will
error when trying to add the column again.
2016-03-08 17:32:29 +00:00
ebcbb23226
s/stream_ordering/event_stream_ordering/ in push
2016-03-04 16:15:23 +00:00
ec7460b4f2
Merge branch 'develop' into markjh/pushrule_stream
2016-03-04 14:44:34 +00:00
dd463e246d
Merge pull request #614 from matrix-org/erikj/alias_delete
...
Allow alias creators to delete aliases
2016-03-04 09:02:33 +00:00
95481e7ba7
Merge pull request #571 from matrix-org/daniel/asids
...
Mark AS users with their AS's ID
2016-03-03 13:56:28 +00:00
a1cf9e3bf3
Add a stream for push rule updates
2016-03-01 18:16:37 +00:00
f9af8962f8
Allow alias creators to delete aliases
2016-03-01 14:46:31 +00:00
33300673b7
Generate guest access token on 3pid invites
...
This means that following the same link across multiple sessions or
devices can re-use the same guest account.
Note that this is somewhat of an abuse vector; we can't throw up
captchas on this flow, so this is a way of registering ephemeral
accounts for spam, whose sign-up we don't rate limit.
2016-02-24 14:41:25 +00:00
112283e230
Prefix TS fields with _ts
2016-02-18 10:11:43 +00:00
e5999bfb1a
Initial cut
2016-02-17 15:40:50 +00:00
763360594d
Mark AS users with their AS's ID
2016-02-11 17:26:42 +00:00
8450114098
Merge pull request #554 from matrix-org/erikj/event_push
...
Change event_push_actions_rm_tokens schema
2016-02-03 15:02:47 +00:00
771528ab13
Change event_push_actions_rm_tokens schema
2016-02-03 10:50:49 +00:00
65e92eca49
Change the way we do public room list fetching
2016-02-02 15:21:10 +00:00
fdca8ec418
Add events index
2016-01-28 16:41:59 +00:00
86896408b0
Add index to event_push_actions
2016-01-25 15:30:32 +00:00
d4315bbf6b
Add index by user id on receipts_linearized
2016-01-20 15:33:27 +00:00
fe56138142
Remove rogue 'admin'
2016-01-11 17:09:03 +00:00
c232780081
Merge pull request #456 from matrix-org/store_event_actions
...
Send unread notification counts
2016-01-08 14:47:15 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
ae1262a241
Add schema change file for is_guest flag
2016-01-06 11:58:20 +00:00
c79f221192
Add is_guest flag to users db to track whether a user is a guest user or not. Use this so we can run _filter_events_for_client when calculating event_push_actions.
2016-01-06 11:38:09 +00:00
eb03625626
Merge remote-tracking branch 'origin/develop' into store_event_actions
2016-01-05 18:39:50 +00:00
cfd07aafff
Allow guests to upgrade their accounts
2016-01-05 18:01:18 +00:00
c914d67cda
Rename event-actions to event_push_actions as per PR request
2016-01-04 14:05:37 +00:00
42ad49f5b7
still very WIP, but now sends unread_notifications_count in the room object on sync (only actually corrrect in a full sync: hardcoded to 0 in incremental syncs).
2015-12-16 18:42:09 +00:00
aa667ee396
Save event actions to the db
2015-12-10 17:51:15 +00:00
d57c5cda71
Bump schema version.
...
As we released version 26 in v0.11.1
2015-12-04 15:28:39 +00:00
b8680b82c3
Merge pull request #414 from matrix-org/erikj/if_not_exists
...
Older versions of SQLite don't like IF NOT EXISTS in virtual tables
2015-12-03 13:21:09 +00:00
e515b48929
Just replace the table definition with the one from full_schema 16
2015-12-02 17:23:52 +00:00
8810eb8c39
Fix schema delta 15 on postgres in the very unlikley event that anyone upgrades to 15...
2015-12-02 17:19:11 +00:00
b2def42bfd
Older versions of SQLite don't like IF NOT EXISTS in virtual tables
2015-12-02 13:29:14 +00:00
95f30ecd1f
Add API for setting account_data globaly or on a per room basis
2015-12-01 18:41:32 +00:00
4dcaa42b6d
Allow paginating search ordered by recents
2015-11-30 17:45:31 +00:00
7dfa455508
Remove size specifier for database column
...
Postgres doesn't support them like this.
We don't have a bool type in common between postgres and sqlite.
2015-11-23 18:35:25 +00:00
1cfda3d2d8
Merge branch 'develop' into daniel/forgetrooms
2015-11-19 16:53:13 +00:00
d153f482dd
Rename the database table
2015-11-18 15:33:02 +00:00
ba26eb3d5d
Allow users to forget rooms
2015-11-17 17:17:30 -05:00
e21cef9bb5
Merge pull request #359 from matrix-org/markjh/incremental_indexing
...
Incremental background updates for db indexes
2015-11-11 17:19:51 +00:00
e1627388d1
Fix param style to work on both sqlite and postgres
2015-11-11 17:14:56 +00:00
940a161192
Fix the background update
2015-11-11 13:59:40 +00:00
cf437900e0
Return world_readable and guest_can_join in /publicRooms
2015-11-10 17:10:27 +00:00
90b503216c
Use a background task to update databases to use the full text search
2015-11-10 16:20:13 +00:00
c6a01f2ed0
Add storage module for tracking background updates.
...
The progress for each background update is stored as a JSON blob in the
database. Each background update is broken up into separate batches.
The batch size is automatically tuned to try avoid blocking single
threaded databases for too long.
2015-11-09 17:26:27 +00:00
f2c4ee41b9
Remove accidentally added ID column
2015-11-06 14:27:49 +00:00
ca2f90742d
Open up /events to anonymous users for room events only
...
Squash-merge of PR #345 from daniel/anonymousevents
2015-11-05 14:32:26 +00:00
6797fcd9ab
Merge pull request #335 from matrix-org/markjh/room_tags
...
Add APIs for adding and removing tags from rooms
2015-11-03 16:45:53 +00:00
ddd8566f41
Store room tag content and return the content in the m.tag event
2015-11-02 15:11:31 +00:00
621e84d9a0
Add missing column
2015-10-30 16:25:53 +00:00
892e70ec84
Add APIs for adding and removing tags from rooms
2015-10-28 16:06:57 +00:00
4cf633d5e9
Pull out sender when computing search results
2015-10-23 15:41:36 +01:00
0c36098c1f
Implement rank function for SQLite FTS
2015-10-23 13:23:48 +01:00
f142898f52
PEP8
2015-10-22 11:18:01 +01:00
4d25bc6c92
Move FTS to delta 25
2015-10-22 11:12:28 +01:00
22a8c91448
Split up run_upgrade
2015-10-16 11:19:44 +01:00
88971fd034
Merge branch 'erikj/store_engine' into erikj/search
2015-10-13 14:03:30 +01:00
f9340ea0d5
Merge branch 'erikj/store_engine' into erikj/search
2015-10-13 13:29:02 +01:00
1a40afa756
Add sqlite schema
2015-10-13 10:36:25 +01:00
61561b9df7
Keep FTS indexes up to date. Only search through rooms currently joined
2015-10-12 10:49:53 +01:00
c85c912562
Add basic full text search impl.
2015-10-09 15:48:31 +01:00
7213588083
Implement configurable stats reporting
...
SYN-287
This requires that HS owners either opt in or out of stats reporting.
When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
dffc9c4ae0
Drop unused index
2015-09-14 14:41:37 +01:00
037481a033
Remove autoincrement since we incrementing the ID in the storage layer
2015-08-24 17:48:57 +01:00
e8cf77fa49
Merge branch 'develop' into refresh
...
Conflicts:
synapse/rest/client/v1/login.py
2015-08-20 16:25:40 +01:00
Erik Johnston