Commit Graph

1924 Commits (4ad613f6be7211edf4b39d7b031bc7d2947d8297)

Author SHA1 Message Date
Erik Johnston 4ad613f6be Merge branch 'develop' of github.com:matrix-org/synapse into erikj/e2e_one_time_upsert 2017-03-29 10:57:19 +01:00
Erik Johnston ac6bc55512 Correctly look up key 2017-03-29 10:56:26 +01:00
Erik Johnston d28db583da Merge pull request #2063 from matrix-org/erikj/device_list_batch
Batch sending of device list pokes
2017-03-28 11:35:41 +01:00
Erik Johnston 58a35366be The algorithm is part of the key id 2017-03-28 11:34:37 +01:00
Erik Johnston bac9bf1b12 Typo 2017-03-27 18:02:17 +01:00
Erik Johnston d82c42837f Short circuit if all new events have same state group 2017-03-27 18:00:47 +01:00
Erik Johnston 2a28b79e04 Batch sending of device list pokes 2017-03-24 14:44:49 +00:00
Erik Johnston 281553afe6 Merge pull request #2062 from matrix-org/erikj/presence_replication
Use presence replication stream to invalidate cache
2017-03-24 13:57:45 +00:00
Erik Johnston 987f4945b4 Actually call invalidate 2017-03-24 13:28:20 +00:00
Erik Johnston 09f79aaad0 Use presence replication stream to invalidate cache
Instead of using the cache invalidation replication stream to invalidate
the _get_presence_cache, we can instead rely on the presence replication
stream. This reduces the amount of replication traffic considerably.
2017-03-24 13:21:08 +00:00
Erik Johnston 7fc1f1e2b6 Cache hosts in room 2017-03-24 11:46:24 +00:00
Erik Johnston d58b1ffe94 Replace some calls to cursor_to_dict
cursor_to_dict can be surprisinglh expensive for large result sets, so lets
only call it when we need to.
2017-03-24 11:07:02 +00:00
Erik Johnston e71940aa64 Use iter(items|values) 2017-03-24 10:57:02 +00:00
Erik Johnston 00957d1aa4 User Cursor.__iter__ instead of fetchall
This prevents unnecessary construction of lists
2017-03-23 17:53:49 +00:00
Erik Johnston 6ebe2d23b1 Raise a more helpful exception 2017-03-23 13:48:30 +00:00
Erik Johnston e64655c25d Don't user upsert to persist new one time keys
Instead we no-op duplicate one time key uploads, an error if the key_id
already exists but encodes a different key.
2017-03-23 13:17:00 +00:00
Richard van der Hoff 64778693be fix up some key verif docstrings 2017-03-21 13:27:50 +00:00
Erik Johnston 37a187bfab Merge pull request #2033 from matrix-org/erikj/repl_speed
Don't send the full event json over replication
2017-03-21 13:11:15 +00:00
Richard van der Hoff eddce9d74a Merge pull request #2027 from matrix-org/rav/logcontext_leaks
A few fixes to logcontext things
2017-03-20 11:53:36 +00:00
Richard van der Hoff 994d7ae7c5 Remove broken use of clock.call_later
background_updates was using `call_later` in a way that leaked the logcontext
into the reactor.

We could have rewritten it to do it properly, but given that we weren't using
the fancier facilities provided by `call_later`, we might as well just use
`async.sleep`, which does the logcontext stuff properly.
2017-03-18 00:01:37 +00:00
Erik Johnston 61f471f779 Don't send the full event json over replication 2017-03-17 15:50:01 +00:00
Richard van der Hoff 0c01f829ae Avoid resetting state on rejected events
When we get a rejected event, give it the same state_group as its prev_event,
rather than no state_group at all.

This should fix https://github.com/matrix-org/synapse/issues/1935.
2017-03-17 15:06:08 +00:00
Richard van der Hoff 5068fb16a5 Refactoring and cleanups
A few non-functional changes:

* A bunch of docstrings to document types
* Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more
  readable.
* Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid
  mind-bending conditional.
* Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid
  mind-bending conditional.
2017-03-17 15:06:07 +00:00
Erik Johnston da146657c9 Comments 2017-03-16 13:04:07 +00:00
Erik Johnston 6957bfdca6 Don't recreate so many sets 2017-03-16 11:54:26 +00:00
Richard van der Hoff 29ed09e80a Fix assertion to stop transaction queue getting wedged
... and update some docstrings to correctly reflect the types being used.

get_new_device_msgs_for_remote can return a long under some circumstances,
which was being stored in last_device_list_stream_id_by_dest, and was then
upsetting things on the next loop.
2017-03-15 12:16:55 +00:00
Erik Johnston bb256ac96f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_list_speed 2017-03-14 11:35:05 +00:00
Erik Johnston cc7a294e2e Fix current_state_events table to not lie
If we try and persist two state events that have the same ancestor we
calculate the wrong current state when persisting those events.
2017-03-14 10:57:43 +00:00
Luke Barnard bbeeb97f75 Implement _simple_delete_many_txn, use it to delete devices
(But this doesn't implement the same for deleting access tokens or e2e keys.

Also respond to code review.
2017-03-13 17:53:23 +00:00
Erik Johnston 79926e016e Assume rooms likely haven't changed 2017-03-13 09:50:10 +00:00
Erik Johnston 8ffbe43ba1 Get current state by using current_state_events table 2017-03-10 17:39:35 +00:00
Erik Johnston 64d62e41b8 Noop repated delete device inbox calls from /sync 2017-03-10 10:36:43 +00:00
Richard van der Hoff 6ad71cc29d Remove spurious SQL logging (#1972)
looks like the upsert function was accidentally sending sql logging to the
general logger. We already log the sql in `txn.execute`.
2017-03-08 18:00:44 +00:00
Erik Johnston ac5491f563 Select distinct devices from DB
Otherwise we might pull out tonnes of duplicate user_ids and this can
make synapse sad.
2017-03-06 11:10:14 +00:00
Erik Johnston da52d3af31 Fix up 2017-03-03 15:29:13 +00:00
Erik Johnston d766343668 Add index to device_lists_stream 2017-03-01 15:56:30 +00:00
Erik Johnston 3365117151 Clobber old device list stream entries 2017-03-01 10:21:30 +00:00
Erik Johnston d638a7484b Merge pull request #1959 from matrix-org/erikj/intern_once
Intern table column names once
2017-02-28 15:15:16 +00:00
Erik Johnston b84907bdbb Intern table column names once 2017-02-28 14:38:16 +00:00
Erik Johnston e4919b9329 Add stream_id index to device_lists_outbound_pokes
As this is used for replication streaming
2017-02-28 11:19:06 +00:00
Erik Johnston 8a12b6f1eb Fix up txn name 2017-02-28 10:15:50 +00:00
Erik Johnston a41dce8f8a Remove needless check 2017-02-27 18:54:43 +00:00
Erik Johnston c0d6045776 It should be all 2017-02-27 18:45:24 +00:00
Erik Johnston 49f4bc4709 Don't fetch current state in common case
Currently we fetch the list of current state events whenever we send
something in a room. This is overkill for the common case of persisting
a simple chain of non-state events, so lets handle that case specially.
2017-02-27 18:33:41 +00:00
Erik Johnston fc5d876dba Merge pull request #1954 from matrix-org/erikj/cache_device2
Cache get_user_devices_from_cache
2017-02-27 16:36:40 +00:00
Erik Johnston f58dbb02a6 Cache get_user_devices_from_cache 2017-02-27 16:22:12 +00:00
Erik Johnston 1a4f8022e6 Strip newlines from SQL queries 2017-02-23 11:15:31 +00:00
Erik Johnston b2d20e94fa Remove lock from rotate notifs 2017-02-22 14:24:02 +00:00
Erik Johnston b7442c3e2b Store looping call 2017-02-21 13:59:25 +00:00
Erik Johnston 699be7d1be Fix up notif rotation 2017-02-18 14:42:39 +00:00