Commit Graph

2297 Commits (012e8e142a4ca7d87e1ffd66cce44b23bf943e9c)

Author SHA1 Message Date
Erik Johnston bfbf1e1f1a Up cache size of get_global_account_data_by_type_for_user 2017-11-13 09:52:11 +00:00
Erik Johnston 2dce6b15c3 Fix typo 2017-11-09 15:56:16 +00:00
Richard van der Hoff 618c7b816a
Merge pull request #2656 from matrix-org/rav/fix_deactivate
Fix 'NoneType' not iterable in /deactivate
2017-11-09 15:20:35 +00:00
Richard van der Hoff 889102315e Fix 'NoneType' not iterable in /deactivate
make sure we actually return a value from user_delete_access_tokens
2017-11-09 15:15:33 +00:00
Erik Johnston e8814410ef Have an explicit API to update room config 2017-11-08 16:13:27 +00:00
Erik Johnston 94ff2cda73
Revert "Modify group room association API to allow modification of is_public" 2017-11-08 15:43:34 +00:00
Erik Johnston 167eb01d83
Merge pull request #2637 from spantaleev/avoid-noop-media-deletes
Avoid no-op media deletes
2017-11-08 11:53:27 +00:00
Richard van der Hoff 02a9a93bde
Merge pull request #2649 from matrix-org/rav/fix_delta_on_state_res
Fix bug in state group storage
2017-11-08 09:22:13 +00:00
Matthew Hodgson 228ccf1fe3
Merge pull request #2643 from matrix-org/matthew/user_dir_typos
Fix various embarrassing typos around user_directory and add some doc.
2017-11-07 17:31:11 +00:00
Richard van der Hoff f5cf3638e9 move _state_group_cache to statestore
this is internal to statestore, so let's keep it there.
2017-11-07 16:43:00 +00:00
Luke Barnard 38b265cb51 Remember to pick is_admin out of the db 2017-11-07 11:24:04 +00:00
Matthew Hodgson 631fa4a1b7 create new indexes before dropping old ones to keep safetynet in place 2017-11-07 10:41:55 +00:00
Matthew Hodgson 4ad883398f s/users_in_pubic_room/users_in_public_rooms/g 2017-11-04 19:39:40 +00:00
Matthew Hodgson d802e8ca6a s/users_in_pubic_room/users_in_public_rooms/g 2017-11-04 19:38:13 +00:00
Matthew Hodgson a100700630 fix copyright.... 2017-11-04 19:35:49 +00:00
Matthew Hodgson b6b075fd49 s/popualte/populate/ 2017-11-04 19:35:33 +00:00
Slavi Pantaleev 805196fbeb Avoid no-op media deletes
If there are no media entries to delete,
avoid creating transactions, prepared statements
and unnecessary log entries.

Signed-off-by: Slavi Pantaleev <slavi@devture.com>
2017-11-04 09:50:15 +02:00
Richard van der Hoff bc8a5c0330 Notify auth providers on logout
Provide a hook by which auth providers can be notified of logouts.
2017-11-01 16:51:51 +00:00
Matthew Hodgson 275826f234
Merge pull request #2617 from matrix-org/matthew/auto-displayname
automatically set default displayname on register
2017-11-01 16:21:16 +00:00
David Baker fbbacb284e
Merge pull request #2613 from matrix-org/rav/kill_refresh_tokens
Remove the last vestiges of refresh_tokens
2017-11-01 15:57:35 +00:00
Matthew Hodgson 9f7a555b4e switch to setting default displayname in the storage layer
to avoid clobbering guest user displaynames on registration
2017-11-01 15:51:30 +00:00
Luke Barnard 99354b430e
Merge pull request #2612 from matrix-org/luke/groups-room-relationship-is-public
Modify group room association API to allow modification of is_public
2017-11-01 11:08:36 +00:00
Luke Barnard 318a249c8b Leave `is_public` as required argument of update_room_group_association 2017-11-01 09:36:01 +00:00
Richard van der Hoff 356bcafc44 Remove the last vestiges of refresh_tokens 2017-10-31 20:35:58 +00:00
Luke Barnard 20fe347906 Modify group room association API to allow modification of is_public
also includes renamings to make things more consistent.
2017-10-31 17:04:28 +00:00
Richard van der Hoff 1650eb5847 DB schema interface for password auth providers
Provide an interface by which password auth providers can register db schema
files to be run at startup
2017-10-31 14:01:53 +00:00
Erik Johnston 4ab8abbc2b Merge branch 'erikj/attestation_local_fix' of github.com:matrix-org/synapse into develop 2017-10-27 15:07:08 +01:00
Luke Barnard 5451cc7792 Request is_public from database 2017-10-27 11:27:43 +01:00
Luke Barnard 6362298fa5 Create groups with is_public = True 2017-10-27 11:04:20 +01:00
Erik Johnston 82d8c1bacb Fixup 2017-10-27 10:30:21 +01:00
Erik Johnston 195abfe7a5 Remove incorrect attestations 2017-10-27 09:58:13 +01:00
Luke Barnard 007cd48af6 Recreate groups table instead of adding column
Adding a column with non-constant default not possible in sqlite3
2017-10-26 17:55:22 +01:00
Luke Barnard 713e60b9b6 Awful hack to get default true 2017-10-26 17:38:14 +01:00
Luke Barnard cfa4e658e0 Bump schema version to 46 2017-10-26 17:23:49 +01:00
Luke Barnard 9b2feef9eb Add is_public to groups table to allow for private groups
Prevent group API access to non-members for private groups

Also make all the group code paths consistent with `requester_user_id` always being the User ID of the requesting user.
2017-10-26 16:51:32 +01:00
Richard van der Hoff eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Richard van der Hoff 2e9f5ea31a Fix logcontext handling for persist_events
* don't use preserve_context_over_deferred, which is known broken.

* remove a redundant preserve_fn.

* add/improve some comments
2017-10-17 10:59:30 +01:00
Luke Barnard 2c5972f87f Implement GET /groups/$groupId/invited_users 2017-10-16 15:31:11 +01:00
Erik Johnston 818b08d0e4 peeeeeeeeep8888888888888888888888888888 2017-10-11 15:54:00 +01:00
Erik Johnston ea18996f54 Fix group stream replication
The stream update functions expect the storage function to return a list
of tuples.
2017-10-11 15:44:39 +01:00
Erik Johnston c2c47550f9 Fix schema delta versions 2017-10-11 13:23:15 +01:00
Erik Johnston 535cc49f27 Merge pull request #2466 from matrix-org/erikj/groups_merged
Initial Group Implementation
2017-10-11 13:20:07 +01:00
Erik Johnston e8496efe84 Fix up comment 2017-10-09 15:17:34 +01:00
Erik Johnston 11d62f43c9 Invalidate cache 2017-10-03 14:12:28 +01:00
Erik Johnston e4ab96021e Update comments 2017-10-03 14:10:41 +01:00
Erik Johnston 30848c0fcd Ignore incoming events for rooms that we have left
When synapse receives an event for a room its not in over federation, it
double checks with the remote server to see if it is in fact in the
room. This is done so that if the server has forgotten about the room
(usually as a result of the database being dropped) it can recover from
it.

However, in the presence of state resets in large rooms, this can cause
a lot of work for servers that have legitimately left. As a hacky
solution that supports both cases we drop incoming events for rooms that
we have explicitly left.

This means that we no longer support the case of servers having
forgotten that they've rejoined a room, but that is sufficiently rare
that we're not going to support it for now.
2017-10-03 11:18:21 +01:00
David Baker 27955056e0 Merge branch 'develop' into erikj/groups_merged 2017-10-02 16:20:41 +01:00
Erik Johnston 768f00dedb Up the limits on number of url cache entries to delete at one time 2017-09-28 14:27:27 +01:00
Erik Johnston 4dc07e93a8 Add old indices 2017-09-28 14:10:33 +01:00
Erik Johnston 93247a424a Only pull out local media that were for url cache 2017-09-28 13:48:14 +01:00
Erik Johnston 7a44c01d89 Fix typo 2017-09-28 12:46:04 +01:00
Erik Johnston ae79764fe5 Change expires column to expires_ts 2017-09-28 12:37:53 +01:00
Erik Johnston 77f1d24de3 More brackets 2017-09-28 12:23:15 +01:00
Erik Johnston 9ccb4226ba Delete expired url cache data 2017-09-28 12:18:06 +01:00
Erik Johnston 17b8e2bd02 Add remove room API 2017-09-26 15:52:41 +01:00
Erik Johnston a8e2a3df32 Add unique index to group_rooms table 2017-09-26 15:39:21 +01:00
Erik Johnston e1dec2f1a7 Remove user from group summary when the leave the group 2017-09-21 16:09:57 +01:00
Erik Johnston 197d82dc07 Correctly return next token 2017-09-20 11:12:11 +01:00
Erik Johnston 069ae2df12 Fix initial sync 2017-09-20 10:52:12 +01:00
Richard van der Hoff e76d1135dd Invalidate signing key cache when we gat an update
This might make the cache slightly more efficient.
2017-09-20 01:32:42 +01:00
Erik Johnston 4a9b1cf253 Add user profiles to summary from group server 2017-08-25 16:23:58 +01:00
Erik Johnston 27ebc5c8f2 Add remote profile cache 2017-08-25 11:25:47 +01:00
Erik Johnston 97c544f91f Add _simple_update 2017-08-25 11:11:37 +01:00
Erik Johnston 2800983f3e Merge pull request #2410 from matrix-org/erikj/groups_publicise
Add ability to publicise group membership
2017-08-21 16:51:56 +01:00
Erik Johnston 8b50fe5330 Use BOOLEAN rather than TEXT type 2017-08-21 16:37:29 +01:00
Erik Johnston 175a01f56c Groups: Fix mising json.load in initial sync 2017-08-21 14:45:56 +01:00
Erik Johnston ef8e578677 Add bulk group publicised lookup API 2017-08-09 13:36:22 +01:00
Erik Johnston b880ff190a Allow update group publicity 2017-08-08 14:19:41 +01:00
Erik Johnston 05e21285aa Store whether the user wants to publicise their membership of a group 2017-08-08 13:01:46 +01:00
Erik Johnston 966a70f1fa Update comment 2017-07-24 17:49:39 +01:00
Erik Johnston 629cdfb124 Use join rather than joined, etc. 2017-07-24 14:54:05 +01:00
Erik Johnston ed666d3969 Fix all the typos 2017-07-24 14:05:09 +01:00
Erik Johnston b76ef6ccb8 Include users membership in group in summary API 2017-07-24 13:55:39 +01:00
Erik Johnston 851aeae7c7 Check users/rooms are in group before adding to summary 2017-07-24 13:40:56 +01:00
Erik Johnston 96917d5552 Merge pull request #2378 from matrix-org/erikj/group_sync_support
Add groups to sync stream
2017-07-21 11:05:39 +01:00
Erik Johnston 0401604222 Merge pull request #2377 from matrix-org/erikj/group_profile_update
Add update group profile API
2017-07-20 17:53:39 +01:00
Erik Johnston 2cc998fed8 Fix replication. And notify 2017-07-20 17:13:18 +01:00
Erik Johnston c544188ee3 Add groups to sync stream 2017-07-20 16:36:42 +01:00
Erik Johnston 8209b5f033 Fix a storage desc 2017-07-20 16:22:22 +01:00
Erik Johnston 57826d645b Fix typo 2017-07-20 13:15:22 +01:00
Erik Johnston 6f443a74cf Add update group profile API 2017-07-20 09:46:33 +01:00
Erik Johnston 94ecd871a0 Fix typos 2017-07-18 16:38:54 +01:00
Erik Johnston 3e703eb04e Comment 2017-07-18 10:17:25 +01:00
Erik Johnston 508460f240 Remove sync stuff 2017-07-18 09:55:46 +01:00
Erik Johnston 4540730111 Remove unused tables 2017-07-18 09:38:15 +01:00
Erik Johnston e96ee95a7e Remove sync stuff 2017-07-18 09:38:08 +01:00
Erik Johnston 2f9eafdd36 Add local group server support 2017-07-17 12:03:49 +01:00
Erik Johnston 85fda57208 Add DEFAULT_ROLE_ID 2017-07-14 14:03:54 +01:00
Erik Johnston 4b203bdba5 Correctly increment orders 2017-07-14 14:02:31 +01:00
Erik Johnston 3b0470dba5 Remove unused functions 2017-07-13 13:54:01 +01:00
Erik Johnston 8575e3160f Comments 2017-07-13 13:52:41 +01:00
Erik Johnston 26451a09eb Comments 2017-07-12 14:47:18 +01:00
Erik Johnston a62406aaa5 Add group summary APIs 2017-07-12 11:36:15 +01:00
Erik Johnston 6d586dc05c Comment 2017-07-12 09:58:37 +01:00
Erik Johnston 410b4e14a1 Move comment 2017-07-11 15:44:18 +01:00
Erik Johnston e52c391cd4 Rename column to attestation_json 2017-07-11 14:25:46 +01:00
Erik Johnston 0aac30d53b Comments 2017-07-11 14:23:50 +01:00
Erik Johnston 83936293eb Comments 2017-07-11 11:42:25 +01:00
Erik Johnston b8ca494ee9 Initial group server implementation 2017-07-10 15:44:15 +01:00
Erik Johnston f502b0dea1 Perf: Don't filter events for push
We know the users are joined and we can explicitly check for if they are
ignoring the user, so lets do that.
2017-07-07 14:04:40 +01:00
Erik Johnston ed9a7f5436 Merge pull request #2309 from matrix-org/erikj/user_ip_repl
Fix up user_ip replication commands
2017-07-06 14:33:14 +01:00
Erik Johnston b5e8d529e6 Define CACHE_SIZE_FACTOR once 2017-07-04 09:56:44 +01:00
Erik Johnston 6ff14ddd2e Make into list 2017-06-29 15:47:37 +01:00
Erik Johnston 5946aa0877 Prefill forward extrems and event to state groups 2017-06-29 15:38:48 +01:00
Erik Johnston 8c23221666 Fix up 2017-06-27 15:53:45 +01:00
Erik Johnston a0a561ae85 Fix up client ips to read from pending data 2017-06-27 14:46:12 +01:00
Erik Johnston ed3d0170d9 Batch upsert user ips 2017-06-27 13:37:04 +01:00
Erik Johnston 8abdd7b553 Fix up indices for users_who_share_rooms 2017-06-26 14:01:30 +01:00
Erik Johnston 7fe8ed1787 Store URL cache preview downloads seperately
This makes it easier to clear old media out at a later date
2017-06-23 11:14:11 +01:00
Erik Johnston e0004aa28a Add desc 2017-06-22 10:03:48 +01:00
Erik Johnston b668112320 Merge pull request #2296 from matrix-org/erikj/dont_appserver_shar
Don't work out users who share room with appservice users
2017-06-21 14:50:24 +01:00
Erik Johnston dae9a00a28 Initialise exclusive_user_regex 2017-06-21 14:19:33 +01:00
Krombel 4202fba82a Merge branch 'develop' into avoid_duplicate_filters 2017-06-21 14:48:21 +02:00
Krombel 812c030e87 replaced json.dumps with encode_canonical_json 2017-06-21 14:48:12 +02:00
Erik Johnston 385dcb7c60 Handle thumbnail urls 2017-06-19 17:48:28 +01:00
Erik Johnston b8b936a6ea Add API to quarantine media 2017-06-19 17:39:21 +01:00
Erik Johnston d7fe6b356c Add shutdown room API 2017-06-19 12:37:27 +01:00
Erik Johnston dfeca6cf40 Merge pull request #2286 from matrix-org/erikj/split_out_user_dir
Split out user directory to a separate process
2017-06-16 13:01:19 +01:00
Erik Johnston 6aa5bc8635 Initial worker impl 2017-06-16 11:47:11 +01:00
Erik Johnston d8f47d2efa Merge pull request #2280 from matrix-org/erikj/share_room_user_dir
Include users who you share a room with in user directory
2017-06-16 11:06:00 +01:00
Erik Johnston 052c5d19d5 Merge pull request #2281 from matrix-org/erikj/phone_home_stats
Fix phone home stats
2017-06-15 12:46:23 +01:00
Erik Johnston a9d6fa8b2b Include users who share room with requester in user directory 2017-06-15 10:17:21 +01:00
Erik Johnston 4564b05483 Implement updating users who share rooms on the fly 2017-06-15 10:17:17 +01:00
Erik Johnston 72613bc379 Implement initial population of users who share rooms table 2017-06-15 09:59:04 +01:00
Erik Johnston ebcd55d641 Add DB schema for tracking users who share rooms 2017-06-15 09:45:48 +01:00
Erik Johnston 4b461a6931 Add some more stats 2017-06-15 09:39:39 +01:00
Erik Johnston 617304b2cf Fix phone home stats 2017-06-14 19:47:15 +01:00
Erik Johnston d9fd937e39 Fix user directory insertion due to missing room_id 2017-06-13 11:50:24 +01:00
Erik Johnston 505e7e8b9d Fix up sql 2017-06-13 11:19:18 +01:00
Erik Johnston 6fd7e6db3d Fix user dir to not assume existence of user 2017-06-13 11:11:26 +01:00
Erik Johnston fdca6e36ee Merge pull request #2274 from matrix-org/erikj/cache_is_host_joined
Add cache for is_host_joined
2017-06-13 10:58:53 +01:00
Erik Johnston b58e24cc3c Tweak the ranking of PG user dir search 2017-06-13 10:16:31 +01:00
Erik Johnston d53fe399eb Add cache for is_host_joined 2017-06-13 09:56:18 +01:00
Erik Johnston a837765e8c Merge pull request #2266 from matrix-org/erikj/host_in_room
Change is_host_joined to use current_state table
2017-06-12 09:49:51 +01:00
Erik Johnston 8060974344 Fix replication 2017-06-09 16:40:52 +01:00
Erik Johnston b0d975e216 Comments 2017-06-09 16:25:42 +01:00
Erik Johnston e54d7d536e Cache state deltas 2017-06-09 16:24:00 +01:00
Erik Johnston 935e588799 Tweak SQL 2017-06-09 13:01:23 +01:00
Erik Johnston 0185b75381 Change is_host_joined to use current_state table
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
2017-06-09 10:52:26 +01:00
Erik Johnston ea11ee09f3 Ensure we don't use unpersisted state group as prev group 2017-06-08 11:59:57 +01:00
Erik Johnston c62c480dc6 Merge pull request #2259 from matrix-org/erikj/fix_state_woes
Fix bug where state_group tables got corrupted
2017-06-07 17:51:25 +01:00
Erik Johnston 197bd126f0 Fix bug where state_group tables got corrupted
This is due to the fact that we prefilled caches using txn.call_after,
which always gets called including on error.

We fix this by making txn.call_after only fire when a transaction
completes successfully, which is what we want most of the time anyway.
2017-06-07 17:39:36 +01:00
Erik Johnston a053ff3979 Merge pull request #2248 from matrix-org/erikj/state_fixup
Faster cache for get_joined_hosts
2017-06-07 14:01:06 +01:00
Erik Johnston 64ed74c01e When pruning, delete from device_lists_outbound_last_success 2017-06-07 11:20:47 +01:00
Erik Johnston 1a81a1898e Keep pruning background task 2017-06-07 11:16:56 +01:00
Erik Johnston 6ba21bf2b8 Comments 2017-06-07 11:08:36 +01:00
Erik Johnston 09e4bc0501 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup 2017-06-07 11:05:23 +01:00
Erik Johnston 6e2a7ee1bc Remove spurious log lines 2017-06-07 11:05:17 +01:00
Erik Johnston 65f0513a33 Split up device_lists_outbound_pokes table for faster updates. 2017-06-07 11:02:38 +01:00
Erik Johnston 6f83c4537c Increase size of IP cache 2017-06-07 10:18:44 +01:00