Commit Graph

1775 Commits (0d81e267699fb1182ededbf4642435e60c497e65)

Author SHA1 Message Date
Erik Johnston 80ad710217 Remove other bit of deduplication 2016-08-03 13:25:59 +01:00
Richard van der Hoff 4fec5e57be Default device_display_name to null
It turns out that it's more useful to return a null device display name (and
let clients decide how to handle it: eg, falling back to device_id) than using
a constant string like "unknown device".
2016-08-03 11:53:00 +01:00
Erik Johnston a8a32d2714 Ensure we only persist an event once at a time 2016-08-03 11:23:39 +01:00
Erik Johnston 5aa024e501 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader 2016-07-29 11:24:56 +01:00
Mark Haines 8dad08a950 Fix SQL to supply arguments in the same order 2016-07-29 09:57:13 +01:00
Mark Haines 0a7d3cd00f Create separate methods for getting messages to push
for the email and http pushers rather than trying to make a single
method that will work with their conflicting requirements.

The http pusher needs to get the messages in ascending stream order, and
doesn't want to miss a message.

The email pusher needs to get the messages in descending timestamp order,
and doesn't mind if it misses messages.
2016-07-28 20:24:24 +01:00
Kegan Dougal 370135ad0b Comment get_unread_push_actions_for_user_in_range function 2016-07-28 16:47:37 +01:00
Erik Johnston 6ede23ff1b Add more key storage funcs into slave store 2016-07-28 15:41:26 +01:00
Richard van der Hoff 40e539683c Merge pull request #956 from matrix-org/rav/check_device_id_on_key_upload
Make the device id on e2e key upload optional
2016-07-27 18:16:57 +01:00
Richard van der Hoff 26cb0efa88 SQL syntax fix 2016-07-27 12:30:22 +01:00
Richard van der Hoff d47115ff8b Delete e2e keys on device delete 2016-07-27 12:24:52 +01:00
Richard van der Hoff 2e3d90d67c Make the device id on e2e key upload optional
We should now be able to get our device_id from the access_token, so the
device_id on the upload request is optional. Where it is supplied, we should
check that it matches.

For active access_tokens without an associated device_id, we ought to register
the device in the devices table.

Also update the table on upgrade so that all of the existing e2e keys are
associated with real devices.
2016-07-26 23:38:12 +01:00
Mark Haines 87ffd21b29 Fix a couple of bugs in the transaction and keyring code 2016-07-26 19:19:08 +01:00
Richard van der Hoff 33d7776473 Fix typo 2016-07-26 13:32:15 +01:00
Richard van der Hoff 9ad8d9b17c Merge branch 'develop' into rav/delete_refreshtoken_on_delete_device 2016-07-26 13:29:46 +01:00
Mark Haines 9c4cf83259 Merge pull request #948 from matrix-org/markjh/auth_fixes
Don't add rejections to the state_group, persist all rejections
2016-07-26 13:22:57 +01:00
Richard van der Hoff 8e02494166 Delete refresh tokens when deleting devices 2016-07-26 11:10:37 +01:00
Mark Haines a6f06ce3e2 Fix how push_actions are redacted. 2016-07-26 11:05:39 +01:00
Mark Haines efeb6176c1 Don't add rejected events if we've seen them befrore. Add some comments to explain what the code is doing mechanically 2016-07-26 10:49:52 +01:00
Richard van der Hoff 012b4c1913 Implement updating devices
You can update the displayname of devices now.
2016-07-26 07:35:48 +01:00
Richard van der Hoff 436bffd15f Implement deleting devices 2016-07-26 07:35:48 +01:00
Mark Haines 1b3c3e6d68 Only update the events and event_json tables for rejected events 2016-07-25 18:44:30 +01:00
Mark Haines 8f7f4cb92b Don't add the events to forward extremities if the event is rejected 2016-07-25 17:13:37 +01:00
Mark Haines 2623cec874 Don't add rejections to the state_group, persist all rejections 2016-07-25 16:12:16 +01:00
Mark Haines 955ef1f06c fix: defer.returnValue takes one argument 2016-07-25 16:04:45 +01:00
Richard van der Hoff 2ee4c9ee02 background updates: fix assert again 2016-07-25 16:01:46 +01:00
Richard van der Hoff 9dbd903f41 background updates: Fix assertion to do something 2016-07-25 14:05:23 +01:00
Richard van der Hoff 465117d7ca Fix background_update tests
A bit of a cleanup for background_updates, and make sure that the real
background updates have run before we start the unit tests, so that they don't
interfere with the tests.
2016-07-25 12:10:42 +01:00
Richard van der Hoff 363786845b PEP8 2016-07-22 13:21:07 +01:00
Richard van der Hoff ec5717caf5 Create index on user_ips in the background
user_ips is kinda big, so really we want to add the index in the background
once we're running. Replace the schema delta with one which will do that.

I've done this in a way that's reasonably easy to reuse as there a few other
indexes I need, and I don't suppose they will be the last.
2016-07-22 13:16:39 +01:00
Erik Johnston aede7248ab Split out a FederationReader process 2016-07-21 17:37:44 +01:00
Richard van der Hoff c445f5fec7 storage/client_ips: remove some dead code 2016-07-21 11:58:47 +01:00
Richard van der Hoff 7314bf4682 Merge branch 'develop' into rav/get_devices_api
(pick up PR #938 in the hope of fixing the UTs)
2016-07-20 17:40:00 +01:00
Richard van der Hoff bc8f265f0a GET /devices endpoint
implement a GET /devices endpoint which lists all of the user's devices.

It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
Richard van der Hoff ec041b335e Record device_id in client_ips
Record the device_id when we add a client ip; it's somewhat redundant as we
could get it via the access_token, but it will make querying rather easier.
2016-07-20 16:41:03 +01:00
Richard van der Hoff 053e83dafb More doc-comments
Fix some more comments on some things
2016-07-20 16:40:28 +01:00
David Baker e967bc86e7 Merge pull request #932 from matrix-org/rav/register_refactor
Further registration refactoring
2016-07-20 11:03:33 +01:00
Erik Johnston 1e2a7f18a1 Merge pull request #922 from matrix-org/erikj/file_api2
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
Erik Johnston f91faf09b3 Comment 2016-07-20 10:18:09 +01:00
Richard van der Hoff 40cbffb2d2 Further registration refactoring
* `RegistrationHandler.appservice_register` no longer issues an access token:
  instead it is left for the caller to do it. (There are two of these, one in
  `synapse/rest/client/v1/register.py`, which now simply calls
  `AuthHandler.issue_access_token`, and the other in
  `synapse/rest/client/v2_alpha/register.py`, which is covered below).

* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
  access_tokens into `_create_registration_details`. This means that the normal
  flow no longer needs to call `AuthHandler.issue_access_token`; the
  shared-secret flow can tell `RegistrationHandler.register` not to generate a
  token; and the appservice flow continues to work despite the above change.
2016-07-19 18:46:19 +01:00
Richard van der Hoff 022b9176fe schema fix
device_id should be text, not bigint.
2016-07-19 11:44:05 +01:00
Richard van der Hoff f863a52cea Add device_id support to /login
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.

Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.

Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
Erik Johnston f52565de50 Fix /purge_history bug
This was caused by trying to insert duplicate backward extremeties
2016-07-15 14:23:15 +01:00
Erik Johnston 978fa53cc2 Pull out min stream_ordering from ex_outlier_stream 2016-07-15 10:22:30 +01:00
Erik Johnston eec9609e96 event_backwards_extremeties may not be empty 2016-07-15 10:22:09 +01:00
Erik Johnston d554ca5e1d Add support for filters in paginate_room_events 2016-07-14 15:59:04 +01:00
Erik Johnston b64aa6d687 Add sender and contains_url field to events table 2016-07-14 15:35:43 +01:00
Mark Haines dfde67a6fe Add a comment explaining allow_none 2016-07-08 15:57:06 +01:00
Mark Haines 10c843fcfb Ensure that the guest user is in the database when upgrading accounts 2016-07-08 15:15:55 +01:00
Erik Johnston 58930da52b Merge branch 'master' of github.com:matrix-org/synapse into develop 2016-07-08 14:11:37 +01:00
Erik Johnston b92e7955be Comment 2016-07-07 11:42:15 +01:00
Erik Johnston c98e1479bd Return 400 rather than 500 2016-07-07 11:41:07 +01:00
Erik Johnston 67f2c901ea Add rest servlet. Fix SQL. 2016-07-06 15:56:59 +01:00
Erik Johnston a17e7caeb7 Merge branch 'erikj/shared_secret' into erikj/test2 2016-07-06 14:46:31 +01:00
Erik Johnston be3548f7e1 Remove spurious txn 2016-07-05 17:46:51 +01:00
Erik Johnston 4adf93e0f7 Fix for postgres 2016-07-05 17:34:25 +01:00
Erik Johnston 651faee698 Add an admin option to shared secret registration 2016-07-05 17:30:22 +01:00
Erik Johnston 8f8798bc0d Add ReadWriteLock for pagination and history prune 2016-07-05 15:30:25 +01:00
Mark Haines dd2ccee27d Fix typo 2016-07-05 14:06:07 +01:00
Mark Haines b6b0132ac7 Make get_events_around more efficient on sqlite3 2016-07-05 13:55:18 +01:00
Mark Haines d44d11d864 Use true/false for boolean parameter inclusive to avoid potential for sqli, and possibly make the code clearer 2016-07-05 10:39:13 +01:00
Erik Johnston 2d21d43c34 Add purge_history API 2016-07-05 10:28:51 +01:00
Mark Haines 0fb76c71ac Use different SQL for postgres and sqlite3 for when using multicolumn indexes 2016-07-04 19:44:55 +01:00
Erik Johnston a67bf0b074 Add storage function to purge history for a room 2016-07-04 16:02:50 +01:00
Mark Haines f18d7546c6 Use a query that postgresql optimises better for get_events_around 2016-07-04 15:48:25 +01:00
Erik Johnston f328d95cef Feature: Add deactivate account admin API
Allows server admins to "deactivate" accounts, which:

- Revokes all access tokens
- Removes all threepids
- Removes password

The API is a POST to `/admin/deactivate/<user_id>`
2016-06-30 15:40:58 +01:00
Mark Haines 6783534a0f Merge pull request #886 from matrix-org/markjh/async_commit
Optionally make committing to postgres asynchronous.
2016-06-29 15:21:58 +01:00
Erik Johnston a70688445d Implement purge_media_cache admin API 2016-06-29 14:57:59 +01:00
Erik Johnston 314b146b2e Track approximate last access time for remote media 2016-06-29 11:41:20 +01:00
David Baker aa3a4944d5 more pep8 2016-06-24 11:45:23 +01:00
David Baker 870c45913e Use similar naming we use in email notifs for push
Fixes https://github.com/vector-im/vector-web/issues/1654
2016-06-24 11:41:11 +01:00
David Baker b5fb7458d5 Actually we need to order these properly
otherwise we'll end up returning the wrong 20
2016-06-22 18:07:14 +01:00
David Baker f73fdb04a6 Style 2016-06-22 17:51:40 +01:00
David Baker 3a4120e49a Put most recent 20 messages in notif
Fixes https://github.com/vector-im/vector-web/issues/1648
2016-06-22 17:47:18 +01:00
Mark Haines d5fb561709 Optionally make committing to postgres asynchronous.
Useful when running tests when you don't care whether the server
will lose data that it claims that it has committed.
2016-06-20 17:53:38 +01:00
Mark Haines 41e4b2efea Add the create_profile method back since the tests use it 2016-06-17 19:20:47 +01:00
Mark Haines 9f1800fba8 Remove registered_users from the distributor.
The only place that was observed was to set the profile. I've made it
so that the profile is set within store.register in the same transaction
that creates the user.

This required some slight changes to the registration code for upgrading
guest users, since it previously relied on the distributor swallowing errors
if the profile already existed.
2016-06-17 19:14:16 +01:00
Erik Johnston 1a815fb04f Don't hit DB for noop replications queries 2016-06-08 11:33:30 +01:00
Mark Haines 0b2158719c Remove dead code.
Loading push rules now happens in the datastore, so we can remove
the methods that loaded them outside the datastore.

The ``waiting_for_join_list`` in federation handler is populated by
anything, so can be removed.

The ``_get_members_events_txn`` method isn't called from anywhere
so can be removed.
2016-06-07 15:07:11 +01:00
Erik Johnston 84379062f9 Fix AS retries, but with correct ordering 2016-06-07 10:24:50 +01:00
Erik Johnston 310197bab5 Fix AS retries 2016-06-07 09:34:50 +01:00
Erik Johnston e4134c5e13 Merge pull request #841 from matrix-org/erikj/event_counter
Add metric counter for number of persisted events
2016-06-06 14:17:40 +01:00
Erik Johnston 7aa778fba9 Add metric counter for number of persisted events 2016-06-06 11:58:09 +01:00
Erik Johnston 70aee0717c Add events to cache when we persist them 2016-06-06 11:34:53 +01:00
Erik Johnston cffe46408f Don't rely on options when inserting event into cache 2016-06-03 18:25:21 +01:00
Erik Johnston 10ea3f46ba Change the way we cache events 2016-06-03 17:57:50 +01:00
Erik Johnston 05e01f21d7 Remove event fetching from DB threads 2016-06-03 17:22:13 +01:00
Mark Haines eef541a291 Move insert_client_ip to a separate class 2016-06-03 14:42:35 +01:00
Erik Johnston d06f2a229e Merge pull request #828 from matrix-org/erikj/joined_hosts_for_room
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:50:30 +01:00
Erik Johnston 722472b48c Merge pull request #825 from matrix-org/erikj/cache_push_rules
Load push rules in storage layer so that they get cached
2016-06-03 11:44:32 +01:00
Mark Haines b09f348530 Merge pull request #824 from matrix-org/markjh/slaved_presence_store
Add a slaved store for presence
2016-06-03 11:26:33 +01:00
Erik Johnston ccb56fc24b Make get_joined_hosts_for_room use get_users_in_room 2016-06-03 11:20:23 +01:00
Erik Johnston 6a0afa582a Load push rules in storage layer, so that they get cached 2016-06-03 11:10:00 +01:00
Mark Haines 3ae915b27e Add a slaved store for presence 2016-06-03 11:05:53 +01:00
Erik Johnston 9c26b390a2 Only get local users 2016-06-03 11:04:31 +01:00
Erik Johnston 065e739d6e Merge pull request #811 from matrix-org/erikj/state_users_in_room
Use state to calculate get_users_in_room
2016-06-03 10:58:27 +01:00
Erik Johnston 696d7c5937 Merge pull request #809 from matrix-org/erikj/cache_receipts_in_room
Add get_users_with_read_receipts_in_room cache
2016-06-03 10:58:24 +01:00
Mark Haines 80f34d7b57 Fix setting the _clock in SQLBaseStore 2016-06-02 15:23:56 +01:00
Erik Johnston a04a2d043c Merge pull request #807 from matrix-org/erikj/push_rules_cache
Ensure we always return boolean in push rules
2016-06-01 18:07:48 +01:00
Erik Johnston 0f06b496d1 Merge pull request #806 from matrix-org/erikj/hash_cache
Cache get_event_reference_hashes
2016-06-01 18:07:42 +01:00
Erik Johnston c8285564a3 Use state to calculate get_users_in_room 2016-06-01 15:25:25 +01:00
David Baker d60eed0710 Limit number of notifications in an email notification 2016-06-01 11:45:43 +01:00
Erik Johnston 43db0d9f6a Add get_users_with_read_receipts_in_room cache 2016-06-01 10:54:32 +01:00
Erik Johnston dea9f20f8c Force boolean 2016-05-31 17:24:30 +01:00
Mark Haines c8c5bf950a Fix synapse/storage/schema/delta/30/as_users.py 2016-05-31 17:10:40 +01:00
Erik Johnston c9ca285d33 Merge pull request #805 from matrix-org/erikj/push_rules_cache
Fix GET /push_rules
2016-05-31 16:42:21 +01:00
Erik Johnston 1d4ee854e2 Fix typo 2016-05-31 15:45:53 +01:00
Erik Johnston cca0093fa9 Change fix 2016-05-31 15:44:08 +01:00
Erik Johnston aefd2d1cbc Cache get_event_reference_hashes 2016-05-31 15:32:32 +01:00
Erik Johnston 10de8c2631 Merge pull request #804 from matrix-org/erikj/push_rules_cache
Add caches to bulk_get_push_rules*
2016-05-31 15:04:40 +01:00
Erik Johnston e5b0bbcd33 Add caches to bulk_get_push_rules* 2016-05-31 13:46:58 +01:00
Erik Johnston 85b992f621 Fix to allow start with postgres 2016-05-27 10:44:44 +01:00
David Baker 37b7e84620 Include the ts the notif was received at 2016-05-24 11:33:32 +01:00
David Baker a24bc5b2dc Add GET /notifications API 2016-05-23 18:33:51 +01:00
David Baker 31b5395ab6 Remove debug logging 2016-05-23 16:32:01 +01:00
David Baker c2da3406fc Oops, missing comma 2016-05-20 18:03:31 +01:00
David Baker ccffb0965d Remove stale line 2016-05-20 17:59:10 +01:00
David Baker 18d68bfee4 Handle empty events table 2016-05-20 17:58:09 +01:00
David Baker d4503e25ed Make deleting push actions more efficient
There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.
2016-05-20 17:56:10 +01:00
David Baker 149fa411e2 Only delete push actions after 30 days 2016-05-20 15:25:12 +01:00
Erik Johnston 42368ea8db Add desc to get_presence_for_users 2016-05-18 11:38:10 +01:00
Mark Haines 6a30a0bfd3 Move the functions for parsing app service config 2016-05-17 11:28:58 +01:00
Mark Haines 816df9f267 get_room_members is unused now 2016-05-16 19:51:43 +01:00
Mark Haines 1a3a2002ff Spell "domain" correctly
s/domian/domain/g
2016-05-16 19:17:23 +01:00
Mark Haines b7381d5338 Allow receipts for events we haven't seen in the db 2016-05-13 15:46:41 +01:00
Erik Johnston ba5c616ff4 Merge pull request #778 from matrix-org/erikj/add_pusher
Fixup add_pusher
2016-05-13 14:43:23 +01:00
Erik Johnston 0c11c1be88 Spelling 2016-05-13 14:42:25 +01:00
Erik Johnston e00e8f2166 Merge pull request #769 from matrix-org/erikj/push_actions_delete
Delete old pushers
2016-05-13 14:41:36 +01:00
Erik Johnston fd8e921b6e Merge pull request #779 from matrix-org/erikj/receipts
Use tree cache for get_linearized_receipts_for_room
2016-05-13 14:41:21 +01:00
Erik Johnston c9aff0736c Remove topics table 2016-05-13 14:40:38 +01:00
Mark Haines 9295fa30a8 Annotate the removed indicies with why they were removed. 2016-05-13 14:16:57 +01:00
Erik Johnston 5e50058473 Remove unused indices
This includes removing both unused indices and indices that are subsets
of other indices.
2016-05-13 13:28:07 +01:00
Erik Johnston 6da7f39d95 Use tree cache for get_linearized_receipts_for_room 2016-05-13 11:41:23 +01:00
Erik Johnston 13d37c3c56 Fixup add_pusher 2016-05-13 11:25:02 +01:00
Matrix 3b1930e8ec unbreak schema 2016-05-10 16:42:37 +01:00
Matthew Hodgson fe97b81c09 Merge pull request #759 from matrix-org/dbkr/email_notifs
Send email notifications for missed messages
2016-05-10 16:30:05 +02:00
David Baker 997db04648 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-05-10 14:40:19 +02:00
Erik Johnston c9eb6dfc1b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user 2016-05-09 13:21:06 +01:00
Erik Johnston 3f84da139c Merge pull request #773 from matrix-org/erikj/get_domian_from_id
Add and use get_domain_from_id
2016-05-09 13:21:00 +01:00
Erik Johnston def64d6ef3 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user 2016-05-09 13:05:09 +01:00
Erik Johnston 8715731559 Merge pull request #772 from matrix-org/erikj/get_user_cache
Add cache to get_user_by_id
2016-05-09 11:12:11 +01:00
Erik Johnston 08dfa8eee2 Add and use get_domian_from_id 2016-05-09 10:36:03 +01:00
Erik Johnston 4ea762c1a2 Add cache to get_user_by_id 2016-05-09 10:08:21 +01:00
Erik Johnston 012cb5416c Merge branch 'develop' of github.com:matrix-org/synapse into erikj/push_actions_delete 2016-05-06 15:59:20 +01:00
Erik Johnston fcb2c3f0db Remove unused import 2016-05-06 15:47:40 +01:00
Erik Johnston fd85b167ec Pull loop one level up 2016-05-06 15:38:42 +01:00
Erik Johnston b6e0be701e Queue events for persistence 2016-05-06 14:31:44 +01:00
Erik Johnston d13459636f Pull prev txn from in memory 2016-05-06 11:30:55 +01:00
Erik Johnston 1d275dba69 Don't needlessly enter transaction 2016-05-06 11:25:58 +01:00
Erik Johnston 56b5e83e36 Reduce database inserts when sending transactions 2016-05-06 11:20:18 +01:00
Mark Haines 9c272da05f Add an openidish mechanism for proving to third parties that you own a given user_id 2016-05-05 13:42:44 +01:00
Erik Johnston 1f0f5ffa1e Add bulk fetch storage API 2016-05-05 10:03:15 +01:00
Erik Johnston fee1118a20 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user 2016-05-04 19:08:27 +01:00
Erik Johnston fcd1eb642d Add primary key 2016-05-04 16:51:51 +01:00
Erik Johnston 8e6a163f27 Add timestamp and auto incrementing ID 2016-05-04 15:19:12 +01:00
David Baker 80be396464 Correct SQL statement for postgres
In standard sql, join binds tighter than comma, so we were joining on the wrong table. Postgres follows the standard (apparently).
2016-05-04 13:19:59 +01:00
Erik Johnston 984d4a2c0f Add /report endpoint 2016-05-04 11:28:10 +01:00
Erik Johnston a438a6d2bc Implement basic ignore user 2016-05-04 10:16:46 +01:00
Erik Johnston 183f23f10d Delete old pushers 2016-05-03 14:22:33 +01:00
David Baker 35b7b8e4bc Remove unused function 2016-04-29 20:10:34 +01:00
David Baker b0a1036d93 Use explicit join 2016-04-29 19:28:56 +01:00
David Baker 8f99cd5996 Oops, actually specify the user id 2016-04-29 19:27:03 +01:00
David Baker 60f44c098d Remove unnecessary if 2016-04-29 19:17:10 +01:00
David Baker 50ad8005e4 Put spaces at start of line 2016-04-29 19:16:15 +01:00
David Baker acded821c4 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-04-29 10:05:20 +01:00
Mark Haines 871357d539 Check that somethign has happend before running the selects 2016-04-27 11:54:13 +01:00
Erik Johnston 183cacac90 Simplify query and handle finishing correctly 2016-04-22 10:01:57 +01:00
Erik Johnston ae571810f2 Order NULLs first 2016-04-21 18:14:18 +01:00
Erik Johnston 3ddbb1687c Fix query 2016-04-21 18:02:36 +01:00
Erik Johnston 8fae3d7b1e Use special UPDATE syntax 2016-04-21 18:01:49 +01:00
Erik Johnston b57dcb4b51 Typo 2016-04-21 17:49:00 +01:00
Erik Johnston 26db18bc90 Need to do _background_update_progress_txn in actual transaction 2016-04-21 17:45:56 +01:00
Erik Johnston e395eb1108 Update progress when creating index 2016-04-21 17:39:24 +01:00
Erik Johnston 3b0fa77f50 Fix SQL statement 2016-04-21 17:37:42 +01:00
Erik Johnston 129e403487 Create index must be on a conn 2016-04-21 17:35:51 +01:00
Erik Johnston 51bb339ab2 Create index concurrently 2016-04-21 17:16:11 +01:00
Erik Johnston b743c1237e Add missing run_upgrade 2016-04-21 17:12:04 +01:00
Erik Johnston c877f0f034 Optimise event_search in postgres 2016-04-21 16:56:14 +01:00
David Baker f63bd4ff47 Send a rather basic email notif
Also pep8 fixes
2016-04-20 13:02:01 +01:00
Erik Johnston 61c7edfd34 Add cache to _get_state_groups_from_groups 2016-04-19 17:22:03 +01:00
David Baker 07d765209d First bits of emailpusher
Mostly logic of when to send an email
2016-04-19 14:24:36 +01:00
Erik Johnston d213d69fe3 Add desc arg 2016-04-14 11:36:23 +01:00
Erik Johnston 96bcfb29c7 Add index 2016-04-14 11:26:33 +01:00
Erik Johnston a2546b9082 Fix query for get_unread_push_actions_for_user_in_range 2016-04-14 11:08:31 +01:00
David Baker 767fc0b739 pep8 2016-04-13 14:23:27 +01:00
David Baker 54d08c8868 Only return unread notifications
Make get_unread_push_actions_for_user_in_range only return unread event actions, being more true to its name. Done in two separate sql queries to get actions after a read receipt and those in a room wiht no receipt at all. SQL queries by Erik.
2016-04-13 14:16:45 +01:00
David Baker d33d623f0d Merge pull request #716 from matrix-org/dbkr/get_pushers
Add get endpoint for pushers
2016-04-12 14:40:37 +01:00
David Baker 7984ffdc6a Unneccessarywhitespaceisunnecessary 2016-04-12 13:55:57 +01:00
David Baker c1267d04c5 Oops, forgot the desc. 2016-04-12 13:55:32 +01:00
David Baker 44891b4a0a Tidy up get_pusher functions
Decodes pushers rows on the main thread rather than the db thread and uses _simple_select_list. Also do the same to the function I copied and factor out the duplication into a helper function.
2016-04-12 13:47:17 +01:00
David Baker 7b39bcdaae Mis-named function 2016-04-12 13:35:08 +01:00
Erik Johnston 762ada1e07 Add back backfilled parameter that was removed 2016-04-12 11:58:04 +01:00
Erik Johnston cccf86dd05 Check if we've already backfilled events 2016-04-12 11:19:32 +01:00
David Baker 8a76094965 Add get endpoint for pushers
As per https://github.com/matrix-org/matrix-doc/pull/308
2016-04-11 18:00:03 +01:00
David Baker 2547dffccc Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions
Change pushers to use the event_actions table
2016-04-11 12:58:55 +01:00
Matthew Hodgson 4bd3d25218 Merge pull request #688 from matrix-org/matthew/preview_urls
URL previewing support
2016-04-11 10:40:29 +01:00
Matthew Hodgson 6ff7a79308 move local_media_repository_url_cache.sql to schema v31 2016-04-08 19:09:02 +01:00
Matthew Hodgson 2460d904bd fix error checking for new SQL 2016-04-08 19:04:29 +01:00
Matthew Hodgson fb83f6a1fc fix SQL based on PR feedback 2016-04-08 18:55:38 +01:00
David Baker ed3979df5f Fix invite pushes
* If the event is an invite event, add the invitee to list of user we run push rules for (if they have a pusher etc)
 * Move invite_for_me to be higher prio than member events otherwise member events matches them
 * Spell override right
2016-04-08 15:29:59 +01:00
Mark Haines 7e2f971c08 Remove some unused functions (#711)
* Remove some unused functions

* get_room_events_stream is only used in tests

* is_exclusive_room might actually be something we want
2016-04-08 14:01:56 +01:00
David Baker d9f38561c8 Literally a dictionary 2016-04-07 17:45:01 +01:00
David Baker 4836864f56 generate id in the main thread 2016-04-07 17:38:48 +01:00
David Baker 3fb35cbd6f Oops, inequality fail 2016-04-07 17:33:37 +01:00
Matthew Hodgson d6e7333ae4 Merge branch 'develop' into matthew/preview_urls 2016-04-07 17:26:44 +01:00
Mark Haines ceb599e789 Add tests for redactions 2016-04-07 16:52:07 +01:00
David Baker 05d044aac3 pep8 2016-04-07 16:45:38 +01:00
David Baker 2d5c693fd3 Fix port script for changes merged from develop 2016-04-07 16:43:54 +01:00
David Baker 9c99ab4572 Merge remote-tracking branch 'origin/develop' into dbkr/pushers_use_event_actions 2016-04-07 16:35:22 +01:00
David Baker 92e3071623 Send badge count pushes.
Also fix bugs with retrying.
2016-04-07 15:39:53 +01:00
David Baker 0fd1cd2400 pep8 2016-04-06 16:50:47 +01:00
Mark Haines e815763b7f Merge pull request #697 from matrix-org/markjh/slaveI
Add a slaved events store class
2016-04-06 16:19:25 +01:00
David Baker 7e2c89a37f Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do:
* Make badges work again
 * Remove old, unused code
2016-04-06 15:42:15 +01:00
Erik Johnston b713934b2e Merge pull request #698 from matrix-org/erikj/port_script_fix
Don't require config to create database
2016-04-06 14:32:45 +01:00
Mark Haines 75fb9ac1be Add a slaved events store class
Add a test to check that get_room_names_and_aliases does the same
thing on both the master and on the slave data store.
2016-04-06 14:18:35 +01:00
Erik Johnston 8aab9d87fa Don't require config to create database 2016-04-06 14:15:45 +01:00
Mark Haines 7d11f825aa Merge pull request #694 from matrix-org/markjh/caches
Move _get_cache_dict into the SQLBaseStore
2016-04-06 13:21:25 +01:00
Mark Haines 87f2dec8d4 Make the cache objects be per instance rather than being global 2016-04-06 13:08:05 +01:00
Mark Haines a1e0d316ea Move _get_cache_dict into the SQLBaseStore 2016-04-06 13:05:19 +01:00
Erik Johnston b29f98377d Merge pull request #691 from matrix-org/erikj/member
Fix stuck invites
2016-04-05 12:44:39 +01:00
Erik Johnston df727f2126 Fix stuck invites
If rejecting a remote invite fails with an error response don't fail
the entire request; instead mark the invite as locally rejected.

This fixes the bug where users can get stuck invites which they can
neither accept nor reject.
2016-04-05 11:13:24 +01:00
Erik Johnston 7a77f8b6d5 Merge pull request #690 from matrix-org/erikj/member
Store invites in a separate table.
2016-04-05 09:12:27 +01:00
Erik Johnston 0c53d750e7 Docs and indents 2016-04-04 18:02:48 +01:00
Erik Johnston 92ab45a330 Add upgrade path, rename table 2016-04-04 17:07:43 +01:00
Erik Johnston 3d76b7cb2b Store invites in a separate table. 2016-04-04 16:30:15 +01:00
Erik Johnston bf14883a04 Merge pull request #689 from matrix-org/erikj/member
Do checks for memberships before creating events
2016-04-04 11:56:40 +01:00
Matthew Hodgson 9f7dc2bef7 Merge branch 'develop' into matthew/preview_urls 2016-04-04 00:38:21 +01:00
Matthew Hodgson 8b98a7e8c3 pep8 2016-04-03 12:56:29 +01:00
Matthew Hodgson 7426c86eb8 add a persistent cache of URL lookups, and fix up the in-memory one to work 2016-04-03 00:31:57 +01:00
Erik Johnston d76d89323c Use computed prev event ids 2016-04-01 17:39:32 +01:00
Mark Haines 89e6839a48 Merge pull request #686 from matrix-org/markjh/doc_strings
Use google style doc strings.
2016-04-01 16:20:09 +01:00
Mark Haines 2a37467fa1 Use google style doc strings.
pycharm supports them so there is no need to use the other format.

Might as well convert the existing strings to reduce the risk of
people accidentally cargo culting the wrong doc string format.
2016-04-01 16:12:07 +01:00
Mark Haines f2b916534b Merge pull request #684 from matrix-org/markjh/backfill_id_gen
Use a stream id generator for backfilled ids
2016-04-01 15:13:14 +01:00
Mark Haines 9bc5b4c663 Assert that the step != 0 2016-04-01 15:08:20 +01:00
Mark Haines 35b5c4ba1b use google style doc strings 2016-04-01 15:07:01 +01:00
Mark Haines a2866e2e6a Rename direction to step, apply checks consistently 2016-04-01 13:50:54 +01:00
Mark Haines e36bfbab38 Use a stream id generator for backfilled ids 2016-04-01 13:29:05 +01:00
Mark Haines 7753fc6570 Fix the invalidation of the names and aliases cache 2016-04-01 10:34:51 +01:00
Mark Haines 5d06929169 Move the check for backfilled outside the for loop 2016-03-31 15:09:09 +01:00
Mark Haines 76503f95ed Remove the is_new_state argument to persist event.
Move the checks for whether an event is new state inside persist
event itself.

This was harder than expected because there wasn't enough information
passed to persist event to correctly handle invites from remote servers
for new rooms.
2016-03-31 15:00:42 +01:00
Mark Haines 62e395f0e3 Merge pull request #676 from matrix-org/markjh/replicate_stateIII
Add replication streams for ex outliers and current state resets
2016-03-31 11:20:57 +01:00
Mark Haines 2ec5426035 Use a namedtuple rather than tuple unpacking 2016-03-31 10:33:02 +01:00
Mark Haines 1fbb094c6f Add replication streams for ex outliers and current state resets 2016-03-30 17:19:56 +01:00
Mark Haines 8b8052909f return the state_group for backfill 2016-03-30 16:20:07 +01:00
Mark Haines 61407986b4 Add a entry to current_state_resets table when the current state is reset 2016-03-30 16:18:46 +01:00
Mark Haines 31a9eceda5 Add a replication stream for state groups 2016-03-30 16:01:58 +01:00
Mark Haines 1e25f62ee6 Use a stream id generator to assign state group ids 2016-03-30 12:55:02 +01:00
Erik Johnston 2f0180b09e Don't bother interning keys that are already interned 2016-03-23 16:29:46 +00:00
Erik Johnston acdfef7b14 Intern all the things 2016-03-23 16:25:54 +00:00
Erik Johnston 75daede92f String intern 2016-03-23 14:53:53 +00:00