Commit Graph

7665 Commits (25b0ba30b1ffab9cb799bd8fc331581b7ff6f7aa)

Author SHA1 Message Date
Erik Johnston fecb45e0c3 Remove last usage of ujson 2018-03-26 13:32:29 +01:00
Erik Johnston 44cd6e1358 PEP8 2018-03-26 12:06:48 +01:00
Erik Johnston 8d6dc106d1 Don't use _cursor_to_dict in find_next_generated_user_id_localpart 2018-03-26 12:02:44 +01:00
Erik Johnston a052aa42e7 Linearize calls to _generate_user_id 2018-03-26 12:02:20 +01:00
Matthew Hodgson 8cbbfaefc1 404 correctly on missing paths via NoResource
fixes https://github.com/matrix-org/synapse/issues/2043 and https://github.com/matrix-org/synapse/issues/2029
2018-03-23 10:32:50 +00:00
Erik Johnston 84b5cc69f5
Merge pull request #3006 from matrix-org/erikj/state_iter
Use .iter* to avoid copies in StateHandler
2018-03-22 11:51:13 +00:00
Erik Johnston fde8e8f09f Fix s/iteriterms/itervalues 2018-03-22 11:42:16 +00:00
Erik Johnston eb9fc021e3 Merge branch 'release-v0.27.0' of github.com:matrix-org/synapse into develop 2018-03-22 10:19:53 +00:00
Erik Johnston 1c41b05c8c Add Cache-Control headers to all JSON APIs
It is especially important that sync requests don't get cached, as if a
sync returns the same token given then the client will call sync with
the same parameters again. If the previous response was cached it will
get reused, resulting in the client tight looping making the same
request and never making any progress.

In general, clients will expect to get up to date data when requesting
APIs, and so its safer to do a blanket no cache policy than only
whitelisting APIs that we know will break things if they get cached.
2018-03-21 17:46:26 +00:00
Neil Johnson e66fbcbb02 fix merge conflicts 2018-03-20 14:25:31 +00:00
Erik Johnston 9aa5a0af51 Explicitly use simplejson 2018-03-20 09:58:13 +00:00
Erik Johnston 610accbb7f Fix replication after switch to simplejson
Turns out that simplejson serialises namedtuple's as dictionaries rather
than tuples by default.
2018-03-19 16:12:48 +00:00
Neil Johnson c384705ee8
Update __init__.py
bump version
2018-03-19 15:11:58 +00:00
Erik Johnston 3f961e638a
Merge pull request #3005 from matrix-org/erikj/fix_cache_size
Fix bug where state cache used lots of memory
2018-03-19 11:44:26 +00:00
Erik Johnston fa72803490 Merge branch 'master' of github.com:matrix-org/synapse into develop 2018-03-19 11:41:01 +00:00
Erik Johnston 9a0d783c11 Add comments 2018-03-19 11:35:53 +00:00
Matthew Hodgson 38f952b9bc spell out not to massively increase bcrypt rounds 2018-03-19 09:27:36 +00:00
Erik Johnston a8ce159be4 Replace some ujson with simplejson to make it work 2018-03-16 00:27:09 +00:00
Erik Johnston 5b631ff41a Remove wrong comment 2018-03-16 00:07:08 +00:00
Erik Johnston ba48755d56 Bump version and changelog 2018-03-15 23:57:26 +00:00
Erik Johnston 926ba76e23 Replace ujson with simplejson 2018-03-15 23:43:31 +00:00
Erik Johnston 9cf519769b Use .iter* to avoid copies in StateHandler 2018-03-15 17:50:26 +00:00
Erik Johnston 7c7706f42b Fix bug where state cache used lots of memory
The state cache bases its size on the sum of the size of entries. The
size of the entry is calculated once on insertion, so it is important
that the size of entries does not change.

The DictionaryCache modified the entries size, which caused the state
cache to incorrectly think it was smaller than it actually was.
2018-03-15 15:46:54 +00:00
NotAFile 2cc9f76bc3 replace old style error catching with 'as' keyword
This is both easier to read and compatible with python3 (not that that
matters)

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-03-15 16:11:17 +01:00
Erik Johnston ddb00efc1d Bump version number 2018-03-15 14:41:30 +00:00
Krombel 91ea0202e6 move handling of auto_join_rooms to RegisterHandler
Currently the handling of auto_join_rooms only works when a user
registers itself via public register api. Registrations via
registration_shared_secret and ModuleApi do not work

This auto_joins the users in the registration handler which enables
the auto join feature for all 3 registration paths.

This is related to issue #2725

Signed-Off-by: Matthias Kesler <krombel@krombel.de>
2018-03-14 16:45:37 +01:00
Erik Johnston 4f28018c83 Register membership/state servlets in event_creator 2018-03-14 14:30:06 +00:00
Erik Johnston 57db62e554
Merge pull request #2992 from matrix-org/erikj/implement_member_workre
Implement RoomMemberWorkerHandler
2018-03-14 14:29:33 +00:00
Erik Johnston 0011ede3b0 Fix imports 2018-03-14 14:19:23 +00:00
Erik Johnston 62ad701326 s/join/joined/ in notify_user_membership_change 2018-03-14 14:17:43 +00:00
Erik Johnston 3f0f06cb31 Split RoomMemberWorkerHandler to separate file 2018-03-14 11:41:45 +00:00
Erik Johnston 3e839e0548
Merge pull request #2989 from matrix-org/erikj/profile_cache_master
Only update remote profile cache on master
2018-03-14 09:42:27 +00:00
Erik Johnston ebd0127999
Merge pull request #2988 from matrix-org/erikj/split_profile_store
Split up ProfileStore
2018-03-14 09:41:06 +00:00
Erik Johnston cfe75a9fb6
Merge pull request #2991 from matrix-org/erikj/fixup_rm
_remote_join and co take a requester
2018-03-14 09:40:57 +00:00
Erik Johnston f51565e023
Merge pull request #2993 from matrix-org/erikj/is_blocked
Add is_blocked to worker store
2018-03-14 09:39:18 +00:00
Matthew Hodgson d144ed6ffb
fix bug #2926 (loading all state for a given type from the DB if the state_key is None) (#2990)
Fixes a regression that had crept in where the caching layer upholds requests for loading state which is filtered by type (but not by state_key), but the DB layer itself would interpret a missing state_key as a request to filter by null state_key rather than returning all state_keys.
2018-03-13 22:36:04 +00:00
Erik Johnston a08726fc42 Add is_blocked to worker store 2018-03-13 18:28:44 +00:00
Erik Johnston b27320b550 Implement RoomMemberWorkerHandler 2018-03-13 18:26:00 +00:00
Erik Johnston 350331d466 _remote_join and co take a requester 2018-03-13 17:50:39 +00:00
Erik Johnston 1a69c6d590
Merge pull request #2987 from matrix-org/erikj/split_room_member_handler
Split RoomMemberHandler into base and master class
2018-03-13 17:40:00 +00:00
Erik Johnston df8ff682a7 Only update remote profile cache on master 2018-03-13 17:38:21 +00:00
Erik Johnston 3518d0ea8f Split up ProfileStore 2018-03-13 17:36:50 +00:00
Erik Johnston d45a114824 Raise, don't return, exception 2018-03-13 17:24:34 +00:00
Erik Johnston 6dbebef141 Add missing param to docstrings 2018-03-13 17:15:32 +00:00
Erik Johnston 16adb11cc0 Correct import order 2018-03-13 16:57:07 +00:00
Erik Johnston 82f16faa78 Move user_*_room distributor stuff to master class
I added yields when calling user_left_room, but they shouldn't matter on
the master process as they always return None anyway.
2018-03-13 16:38:15 +00:00
Erik Johnston b78717b87b Split RoomMemberHandler into base and master class
The intention here is to split the class into the bits that can be done
on workers and the bits that have to be done on the master.

In future there will also be a class that can be run on the worker,
which will delegate work to the master when necessary.
2018-03-13 16:37:41 +00:00
Erik Johnston 95cb401ae0
Merge pull request #2978 from matrix-org/erikj/refactor_replication_layer
Remove ReplicationLayer and user Client/Server directly
2018-03-13 15:45:08 +00:00
Erik Johnston 5d8476d8ff
Merge pull request #2981 from matrix-org/erikj/factor_remote_leave
Factor out _remote_reject_invite in RoomMember
2018-03-13 15:44:56 +00:00
Jonas Platte 47ce527f45 Add room_id to the response of `rooms/{roomId}/join`
Fixes #2349
2018-03-13 14:48:12 +01:00
Erik Johnston 56e709857c
Merge pull request #2979 from matrix-org/erikj/no_handlers
Don't build handlers on workers unnecessarily
2018-03-13 13:46:38 +00:00
Erik Johnston cb9f8e527c s/replication_client/federation_client/ 2018-03-13 13:26:52 +00:00
Erik Johnston cea462e285 s/replication_server/federation_server 2018-03-13 13:22:21 +00:00
Erik Johnston bf8e97bd3c Merge branch 'develop' of github.com:matrix-org/synapse into erikj/factor_remote_leave 2018-03-13 13:17:08 +00:00
Erik Johnston ea3442c15c Add docstring 2018-03-13 13:16:21 +00:00
Erik Johnston 16469a4f15
Merge pull request #2980 from matrix-org/erikj/rm_priv
Make RoomMemberHandler functions private that can be
2018-03-13 13:11:11 +00:00
Erik Johnston c82111a55f
Merge pull request #2982 from matrix-org/erikj/fix_extra_users
extra_users is actually a list of UserIDs
2018-03-13 13:11:04 +00:00
Erik Johnston da87791975
Merge pull request #2983 from matrix-org/erikj/rename_register_3pid
Refactor get_or_register_3pid_guest
2018-03-13 13:10:52 +00:00
Erik Johnston 99e9b4f26c
Merge pull request #2984 from matrix-org/erikj/fix_rest_regeix
RoomMembershipRestServlet doesn't handle /forget
2018-03-13 13:10:44 +00:00
Erik Johnston f5160d4a3e RoomMembershipRestServlet doesn't handle /forget
Due to the order we register the REST handlers `/forget` was handled by
the correct handler.
2018-03-13 12:12:55 +00:00
Erik Johnston 8b3573a8b2 Refactor get_or_register_3pid_guest 2018-03-13 12:08:58 +00:00
Richard van der Hoff 299fd740c7
Merge pull request #2975 from matrix-org/rav/measure_persist_events
Add Measure block for persist_events
2018-03-13 12:06:25 +00:00
Erik Johnston 9a2d9b4789
Merge pull request #2977 from matrix-org/erikj/replication_move_props
Move property setting from ReplicationLayer to base classes
2018-03-13 11:45:25 +00:00
Erik Johnston f43b6d6d9b Fix docstring types 2018-03-13 11:29:35 +00:00
Erik Johnston 0f942f68c1 Factor out _remote_reject_invite in RoomMember 2018-03-13 11:22:45 +00:00
Erik Johnston d0fcc48f9d extra_users is actually a list of UserIDs 2018-03-13 11:20:06 +00:00
Erik Johnston 31becf4ac3 Make functions private that can be 2018-03-13 11:15:16 +00:00
Erik Johnston d023ecb810 Don't build handlers on workers unnecessarily 2018-03-13 11:08:10 +00:00
Erik Johnston ea7b3c4b1b Remove unused ReplicationLayer 2018-03-13 11:00:04 +00:00
Erik Johnston 265b993b8a Split replication layer into two 2018-03-13 10:55:47 +00:00
Erik Johnston e05bf34117 Move property setting from ReplicationLayer to FederationBase 2018-03-13 10:51:30 +00:00
Erik Johnston c3f79c9da5 Split out edu/query registration to a separate class 2018-03-13 10:24:27 +00:00
Richard van der Hoff 889a2a853a Add Measure block for persist_events
This seems like a useful thing to measure.
2018-03-13 10:01:42 +00:00
Richard van der Hoff e48c7aac4d Add transactional API to history purge
Make the purge request return quickly, and allow scripts to poll for updates.
2018-03-12 16:22:55 +00:00
Richard van der Hoff 1708412f56 Return an error when doing two purges on a room
Queuing up purges doesn't sound like a good thing.
2018-03-12 16:22:54 +00:00
Richard van der Hoff b984dd0b73
Merge pull request #2961 from matrix-org/rav/run_in_background
Factor run_in_background out from preserve_fn
2018-03-12 16:19:13 +00:00
Richard van der Hoff ba1d08bc4b
Merge pull request #2965 from matrix-org/rav/request_logging
Add a metric which increments when a request is received
2018-03-12 09:45:33 +00:00
Richard van der Hoff 58dd148c4f Add some docstrings to help figure this out 2018-03-09 18:05:41 +00:00
Richard van der Hoff 88541f9009 Add a metric which increments when a request is received
It's useful to know when there are peaks in incoming requests - which isn't
quite the same as there being peaks in outgoing responses, due to the time
taken to handle requests.
2018-03-09 16:30:26 +00:00
Richard van der Hoff dbe80a286b refactor JsonResource
rephrase the OPTIONS and unrecognised request handling so that they look
similar to the common flow.
2018-03-09 16:22:16 +00:00
Richard van der Hoff 20f40348d4 Factor run_in_background out from preserve_fn
It annoys me that we create temporary function objects when there's really no
need for it. Let's factor the gubbins out of preserve_fn and start using it.
2018-03-08 11:50:11 +00:00
Erik Johnston a56d54dcb7 Fix up log message 2018-03-07 11:55:31 +00:00
Erik Johnston 02a1296ad6 Fix typo 2018-03-07 11:55:31 +00:00
Erik Johnston 8cb44da4aa Fix race in sync when joining room
The race happens when the user joins a room at the same time as doing a
sync. We fetch the current token and then get the rooms the user is in.
If the join happens after the current token, but before we get the rooms
we end up sending down a partial room entry in the sync.

This is fixed by looking at the stream ordering of the membership
returned by get_rooms_for_user, and handling the case when that stream
ordering is after the current token.
2018-03-07 11:55:31 +00:00
Richard van der Hoff 8ffaacbee3
Merge pull request #2949 from krombel/use_bcrypt_checkpw
use bcrypt.checkpw
2018-03-06 11:56:06 +00:00
Richard van der Hoff b2932107bb
Merge pull request #2946 from matrix-org/rav/timestamp_to_purge
Implement purge_history by timestamp
2018-03-06 11:20:23 +00:00
Erik Johnston 7aed50a038
Merge pull request #2948 from matrix-org/erikj/kill_as_sync
Remove ability for AS users to call /events and /sync
2018-03-06 11:10:09 +00:00
Erik Johnston b6c4b851f1
Merge pull request #2947 from matrix-org/erikj/split_directory_store
Split Directory store
2018-03-05 18:17:32 +00:00
Krombel ed9b5eced4 use bcrypt.checkpw
in bcrypt 3.1.0 checkpw got introduced (already 2 years ago)
This makes use of that with enhancements which might get introduced
by that

Signed-Off-by: Matthias Kesler <krombel@krombel.de>
2018-03-05 18:02:59 +01:00
Erik Johnston d4ffe61d4f Remove ability for AS users to call /events and /sync
This functionality has been deprecated for a while as well as being
broken for a while. Instead of fixing it lets just remove it entirely.

See: https://github.com/matrix-org/matrix-doc/issues/1144
2018-03-05 15:44:46 +00:00
Erik Johnston 69ce365b79 Fix cache invalidation on deletion 2018-03-05 15:29:03 +00:00
Erik Johnston 2e223163ff Split Directory store 2018-03-05 15:11:30 +00:00
Richard van der Hoff f8bfcd7e0d Provide a means to pass a timestamp to purge_history 2018-03-05 14:37:23 +00:00
Richard van der Hoff d032785aa7
Merge pull request #2943 from matrix-org/rav/fix_find_first_stream_ordering_after_ts
Test and fix find_first_stream_ordering_after_ts
2018-03-05 12:26:14 +00:00
Richard van der Hoff c818fcab11 Test and fix find_first_stream_ordering_after_ts
It seemed to suffer from a bunch of off-by-one errors.
2018-03-05 12:04:02 +00:00
Richard van der Hoff 06a14876e5 Add find_first_stream_ordering_after_ts
Expose this as a public function which can be called outside a txn
2018-03-05 11:53:39 +00:00
Erik Johnston 42174946f8
Merge pull request #2934 from matrix-org/erikj/cache_fix
Fix bug with delayed cache invalidation stream
2018-03-05 11:33:17 +00:00
Erik Johnston f394f5574d
Merge pull request #2929 from matrix-org/erikj/split_regististration_store
Split registration store
2018-03-05 11:33:07 +00:00
dklug af7ed8e1ef Return 401 for invalid access_token on logout
Signed-off-by: Duncan Klug <dklug@ucmerced.edu>
2018-03-02 22:01:27 -08:00
Erik Johnston efb79820b4 Fix bug with delayed cache invalidation stream
We poked the notifier before updated the current token for the cache
invalidation stream. This mean that sometimes the update wouldn't be
sent until the next time a cache was invalidated.
2018-03-02 14:45:15 +00:00
Erik Johnston fafa3e7114 Split registration store 2018-03-02 13:48:27 +00:00
Erik Johnston d960d23830 Add missing yield during 3pid signature checks 2018-03-02 11:03:18 +00:00
Erik Johnston 1a6c7cdf54
Merge pull request #2928 from matrix-org/erikj/read_marker_caches
Fix typo in getting replication account data processing
2018-03-01 17:56:14 +00:00
Erik Johnston 89b7232ff8 Fix typo in getting replication account data processing 2018-03-01 17:50:30 +00:00
Erik Johnston 1773df0632
Merge pull request #2925 from matrix-org/erikj/split_sig_fed
Split out SignatureStore and EventFederationStore
2018-03-01 17:32:58 +00:00
Erik Johnston 65cf454fd1 Remove unused DataStore 2018-03-01 17:27:53 +00:00
Erik Johnston 9e08a93a7b
Merge pull request #2927 from matrix-org/erikj/read_marker_caches
Improve caching for read_marker API
2018-03-01 17:12:34 +00:00
Erik Johnston 4b44f05f19 Fewer lies are better 2018-03-01 17:08:17 +00:00
Erik Johnston a83c514d1f Improve caching for read_marker API
We add a new storage function to get a paritcular type of room account
data. This allows us to prefill the cache when updating that acount
data.
2018-03-01 17:08:17 +00:00
Erik Johnston 33bebb63f3 Add some caches to help read marker API 2018-03-01 17:08:17 +00:00
Erik Johnston 483e8104db
Merge pull request #2926 from matrix-org/erikj/member_handler_move
Move RoomMemberHandler out of Handlers
2018-03-01 17:01:25 +00:00
Erik Johnston 2ad4d5b5bb Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_sig_fed 2018-03-01 16:59:39 +00:00
Erik Johnston 529c026ac1 Move back to hs.is_mine 2018-03-01 16:49:12 +00:00
Erik Johnston 7c371834cc Stub out broken function only used for cache 2018-03-01 16:44:13 +00:00
Erik Johnston 64346be26d Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_stream_store 2018-03-01 16:26:42 +00:00
Erik Johnston 22518e2833
Merge pull request #2923 from matrix-org/erikj/stream_ago_worker
Calculate stream_ordering_month_ago correctly on workers
2018-03-01 16:23:54 +00:00
Erik Johnston 884b26ae41 Remove unused variables 2018-03-01 16:23:48 +00:00
Erik Johnston 1b2af11650 Document abstract class and method better 2018-03-01 16:20:57 +00:00
Erik Johnston 872ff95ed4 Default stream_ordering_*_ago to None 2018-03-01 16:00:05 +00:00
Erik Johnston 22004b524e Fix comment typo 2018-03-01 15:59:40 +00:00
Erik Johnston 4bc4236faf
Merge pull request #2922 from matrix-org/erikj/split_room_store
Split up RoomStore
2018-03-01 15:55:01 +00:00
Richard van der Hoff 2324124a72
Merge pull request #2921 from matrix-org/rav/unyielding_make_deferred_yieldable
Rewrite make_deferred_yieldable avoiding inlineCallbacks
2018-03-01 15:39:10 +00:00
Erik Johnston f793bc3877 Split out stream store 2018-03-01 15:13:08 +00:00
Erik Johnston 784f036306 Move RoomMemberHandler out of Handlers 2018-03-01 14:36:50 +00:00
Erik Johnston 6411f725be Calculate stream_ordering_month_ago correctly on workers 2018-03-01 14:20:53 +00:00
Erik Johnston a9a2d66cdd Split out SignatureStore and EventFederationStore 2018-03-01 14:17:53 +00:00
Erik Johnston 0c8ba5dd1c Split up RoomStore 2018-03-01 14:01:19 +00:00
Richard van der Hoff 3a75de923b Rewrite make_deferred_yieldable avoiding inlineCallbacks
... because (a) it's actually simpler (b) it might be marginally more
performant?
2018-03-01 12:40:05 +00:00
Erik Johnston 126b9bf96f Log in the correct places 2018-03-01 12:05:33 +00:00
Erik Johnston 157298f986 Don't do preserve_fn for every request 2018-03-01 11:59:45 +00:00
Erik Johnston 89f90d808a Add some logging 2018-03-01 11:59:16 +00:00
Erik Johnston 8ded8ba2c7 Make repl send_event idempotent and retry on timeouts
If we treated timeouts as failures on the worker we would attempt to
clean up e.g. push actions while the master might still process the
event.
2018-03-01 11:20:34 +00:00
Erik Johnston f381d63813 Check event auth on the worker 2018-03-01 10:18:37 +00:00
Erik Johnston 6b8604239f Correctly send ratelimit and extra_users params 2018-03-01 10:08:39 +00:00
Erik Johnston f756f961ea Fixup comments 2018-03-01 10:05:27 +00:00
Erik Johnston 28e973ac11 Calculate push actions on worker 2018-02-28 18:02:30 +00:00
Erik Johnston 493e25d554 Move storage functions for push calculations
This will allow push actions for an event to be calculated on workers.
2018-02-27 13:58:16 +00:00
Erik Johnston 3594dbc6dc
Merge pull request #2904 from matrix-org/erikj/receipt_cache_invalidation
Fix missing invalidations for receipt storage
2018-02-27 11:34:26 +00:00
Erik Johnston 2311189ee4
Merge pull request #2903 from matrix-org/erikj/split_roommember_store
Split out RoomMemberStore
2018-02-27 11:32:10 +00:00
Erik Johnston c57607874c
Merge pull request #2901 from matrix-org/erikj/split_as_stores
Split AS stores
2018-02-27 10:07:07 +00:00
Erik Johnston 8956f0147a Add comment 2018-02-27 10:06:51 +00:00
Erik Johnston e5b4a208ce
Merge pull request #2892 from matrix-org/erikj/batch_inserts_push_actions
Batch inserts into event_push_actions_staging
2018-02-26 14:45:40 +00:00
Erik Johnston 45b5fe9122 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/handle_unpersisted_events_push 2018-02-26 13:49:24 +00:00
Erik Johnston d62ce972f8 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_roommember_store 2018-02-23 11:46:24 +00:00
Erik Johnston 6ae9a3d2a6 Update copyright 2018-02-23 11:44:49 +00:00
Erik Johnston a90c60912f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_event_push_actions 2018-02-23 11:26:31 +00:00
Erik Johnston 50e8657867
Merge pull request #2902 from matrix-org/erikj/split_events_store
Split out get_events and co into a worker store
2018-02-23 11:23:52 +00:00
Erik Johnston 1cf9e071dd
Merge pull request #2899 from matrix-org/erikj/split_pushers
Split PusherStore
2018-02-23 11:23:35 +00:00
Erik Johnston d0957753bf
Merge pull request #2898 from matrix-org/erikj/split_push_rules_store
Split PushRulesStore
2018-02-23 11:23:23 +00:00
Erik Johnston 199dba6c15
Merge pull request #2897 from matrix-org/erikj/split_account_data
Split AccountDataStore and TagStore
2018-02-23 11:23:11 +00:00
Erik Johnston 70349872c2 Update copyright 2018-02-23 11:14:35 +00:00
Erik Johnston eba93b05bf Split EventsWorkerStore into separate file 2018-02-23 11:01:21 +00:00
Erik Johnston bf8a36e080 Update copyright 2018-02-23 10:52:10 +00:00
Erik Johnston 5d0f665848 Remove redundant clock 2018-02-23 10:49:58 +00:00
Erik Johnston 3bd760628b _event_persist_queue shouldn't be in worker store 2018-02-23 10:49:18 +00:00
Erik Johnston eb9b5eec81 Update copyright 2018-02-23 10:42:39 +00:00
Erik Johnston c2ecfcc3a4 Update copyright 2018-02-23 10:41:34 +00:00
Erik Johnston 7e6cf89dc2 Update copyright 2018-02-23 10:39:19 +00:00
Erik Johnston 26d37f7a63 Update copyright 2018-02-23 10:33:55 +00:00
Erik Johnston bb73f55fc6 Use absolute imports 2018-02-23 10:31:16 +00:00
Erik Johnston faeb369f15 Fix missing invalidations for receipt storage 2018-02-21 15:19:54 +00:00
Erik Johnston 3dec9c66b3 Split out RoomMemberStore 2018-02-21 12:07:26 +00:00
Erik Johnston 46244b2759 Split AS stores 2018-02-21 11:49:34 +00:00
Erik Johnston 27b094f382 Split out get_events and co into a worker store 2018-02-21 11:41:48 +00:00
Erik Johnston 573712da6b Update comments 2018-02-21 11:29:49 +00:00
Erik Johnston c96d547f4d Actually use new param 2018-02-21 11:03:42 +00:00
Erik Johnston d15d237b0d Split out EventPushActionWorkerStore 2018-02-21 11:01:13 +00:00
Erik Johnston 27939cbb0e
Merge pull request #2893 from matrix-org/erikj/delete_from_staging_fed
Delete from push_actions_staging in federation too
2018-02-21 11:00:06 +00:00
Erik Johnston 6f72765371 Split PusherStore 2018-02-21 10:54:21 +00:00
Erik Johnston cbaad969f9 Split PushRulesStore 2018-02-21 10:43:31 +00:00
Erik Johnston ca9b9d9703 Split AccountDataStore and TagStore 2018-02-21 10:15:04 +00:00
Erik Johnston 8fbb4d0d19 Raise exception in abstract method 2018-02-20 17:59:23 +00:00
Erik Johnston 95e4cffd85 Fix comment 2018-02-20 17:58:40 +00:00
Erik Johnston e316bbb4c0 Use abstract base class to access stream IDs 2018-02-20 17:43:57 +00:00
Erik Johnston f5ac4dc2d4 Split ReceiptsStore 2018-02-20 16:28:28 +00:00
Erik Johnston 24087bffa9 Ensure all push actions are deleted from staging 2018-02-20 12:34:31 +00:00
Erik Johnston ad0ccf15ea Refactor _set_push_actions_for_event_and_users_txn to use events_and_contexts 2018-02-20 12:34:28 +00:00
Erik Johnston d874d4f2d7 Delete from push_actions_staging in federation too 2018-02-20 11:37:52 +00:00
Erik Johnston 6ff8c87484 Batch inserts into event_push_actions_staging 2018-02-20 11:33:07 +00:00
Erik Johnston 324c3e9399
Merge pull request #2868 from matrix-org/erikj/refactor_media_storage
Make store_file use store_into_file
2018-02-20 11:31:24 +00:00
Pascal Bach 3acd616979 Update pynacl dependency to 1.2.1 or higher
Signed-off-by: Pascal Bach <pascal.bach@nextrem.ch>
2018-02-19 10:45:22 +01:00
Richard van der Hoff d1a3325f99 (Really) fix tablescan of event_push_actions on purge
commit 278d21b5 added new code to avoid the tablescan, but didn't remove the
old :/
2018-02-16 14:02:31 +00:00
Erik Johnston 6af025d3c4 Fix typo of double is_highlight 2018-02-16 11:35:31 +00:00
Erik Johnston 012e8e142a Comments 2018-02-16 11:35:01 +00:00
Erik Johnston b96278d6fe Ensure that we delete staging push actions on errors 2018-02-15 15:47:06 +00:00
Erik Johnston 4810f7effd Remove context.push_actions 2018-02-15 15:47:06 +00:00
Erik Johnston c714c61853 Update event_push_actions table from staging table 2018-02-15 15:47:06 +00:00
Erik Johnston acac21248c Store push actions in staging area 2018-02-15 15:47:04 +00:00
Erik Johnston 106906a65e Don't serialize current state over replication 2018-02-15 13:53:18 +00:00
Erik Johnston 5fb347fc41
Merge pull request #2872 from matrix-org/erikj/event_worker_dont_log
Don't log errors propogated from send_event
2018-02-15 12:31:49 +00:00
Erik Johnston fd1601c596 Fix state group storage bug in workers
We needed to move `_count_state_group_hops_txn` to the
StateGroupWorkerStore.
2018-02-15 11:04:32 +00:00
Erik Johnston ef344b10e5 Don't log errors propogated from send_event 2018-02-15 11:03:49 +00:00
Richard van der Hoff b8d821aa68
Merge pull request #2867 from matrix-org/rav/rework_purge
purge_history cleanups
2018-02-15 09:49:07 +00:00
Erik Johnston 92c52df702 Make store_file use store_into_file 2018-02-14 17:55:18 +00:00
Richard van der Hoff d28ec43e15
Merge pull request #2769 from matrix-org/matthew/hit_the_gin
switch back from GIST to GIN indexes
2018-02-14 16:59:03 +00:00
Richard van der Hoff 39bf47319f purge_history: fix sqlite syntax error
apparently sqlite insists on indexes being named
2018-02-14 16:42:19 +00:00
Richard van der Hoff ac27f6a35e purge_history: handle sqlite asshattery
apparently creating a temporary table commits the transaction. because that's a
useful thing.
2018-02-14 16:41:12 +00:00
Richard van der Hoff 5978dccff0 remove overzealous exception handling 2018-02-14 15:54:09 +00:00
Richard van der Hoff 278d21b5e4 purge_history: fix index use
event_push_actions doesn't have an index on event_id, so we need to specify
room_id.
2018-02-14 15:44:51 +00:00
Richard van der Hoff 5fcbf1e07c Rework event purge to use a temporary table
... which should speed things up by reducing the amount of data being shuffled
across the connection
2018-02-14 11:02:22 +00:00