MatrixSynapse/synapse/storage/databases/main
Erik Johnston 5c9e39e619
Track device list updates per room. (#12321)
This is a first step in dealing with #7721.

The idea is basically that rather than calculating the full set of users a device list update needs to be sent to up front, we instead simply record the rooms the user was in at the time of the change. This will allow a few things:

1. we can defer calculating the set of remote servers that need to be poked about the change; and
2. during `/sync` and `/keys/changes` we can avoid also avoid calculating users who share rooms with other users, and instead just look at the rooms that have changed.

However, care needs to be taken to correctly handle server downgrades. As such this PR writes to both `device_lists_changes_in_room` and the `device_lists_outbound_pokes` table synchronously. In a future release we can then bump the database schema compat version to `69` and then we can assume that the new `device_lists_changes_in_room` exists and is handled.

There is a temporary option to disable writing to `device_lists_outbound_pokes` synchronously, allowing us to test the new code path does work (and by implication upgrading to a future release and downgrading to this one will work correctly).

Note: Ideally we'd do the calculation of room to servers on a worker (e.g. the background worker), but currently only master can write to the `device_list_outbound_pokes` table.
2022-04-04 15:25:20 +01:00
..
__init__.py Track device list updates per room. (#12321) 2022-04-04 15:25:20 +01:00
account_data.py Use the ignored_users table to test event visibility & sync. (#12225) 2022-03-15 14:06:05 -04:00
appservice.py Use a sequence to generate AS transaction IDs, drop `last_txn` AS state (#12209) 2022-04-01 13:33:25 +00:00
cache.py Add cache for `get_membership_from_event_ids` (#12272) 2022-03-25 14:58:56 +00:00
censor_events.py Type hint the constructors of the data store classes (#11555) 2021-12-13 17:05:00 +00:00
client_ips.py Move `update_client_ip` background job from the main process to the background worker. (#12251) 2022-04-01 13:08:55 +01:00
deviceinbox.py Reduce to-device queries for /sync. (#12163) 2022-03-04 17:57:27 +00:00
devices.py Track device list updates per room. (#12321) 2022-04-04 15:25:20 +01:00
directory.py Replace uses of simple_insert_many with simple_insert_many_values. (#11742) 2022-01-13 19:44:18 -05:00
e2e_room_keys.py Replace uses of simple_insert_many with simple_insert_many_values. (#11742) 2022-01-13 19:44:18 -05:00
end_to_end_keys.py Add support for MSC3202: sending one-time key counts and fallback key usage states to Application Services. (#11617) 2022-02-24 17:55:45 +00:00
event_federation.py Exclude outliers in `on_backfill_request` (#12314) 2022-03-28 19:20:14 +01:00
event_push_actions.py Replace uses of simple_insert_many with simple_insert_many_values. (#11742) 2022-01-13 19:44:18 -05:00
events.py Add cache for `get_membership_from_event_ids` (#12272) 2022-03-25 14:58:56 +00:00
events_bg_updates.py Replace uses of simple_insert_many with simple_insert_many_values. (#11742) 2022-01-13 19:44:18 -05:00
events_forward_extremities.py Attempt to annotate events_forward_extremities (#11314) 2021-11-12 15:58:17 +00:00
events_worker.py Optimise `_get_state_after_missing_prev_event`: use `/state` (#12040) 2022-04-01 12:53:42 +01:00
filtering.py Improve type hints in storage classes. (#11652) 2021-12-29 13:04:28 +00:00
group_server.py Add some type hints to datastore (#12248) 2022-03-18 15:24:18 +00:00
keys.py
lock.py Type hint the constructors of the data store classes (#11555) 2021-12-13 17:05:00 +00:00
media_repository.py Add some type hints to datastore. (#12255) 2022-03-28 14:11:14 -04:00
metrics.py Remove redundant `COALESCE()`s around `COUNT()`s in database queries (#11570) 2021-12-14 12:34:30 +00:00
monthly_active_users.py Move `update_client_ip` background job from the main process to the background worker. (#12251) 2022-04-01 13:08:55 +01:00
openid.py Add type hints to some storage classes (#11307) 2021-11-11 08:47:31 -05:00
presence.py Add type hints to `synapse/storage/databases/main` (#11984) 2022-02-21 16:03:06 +00:00
profile.py Database storage profile passes mypy (#11342) 2021-11-15 12:59:33 +00:00
purge_events.py Add type hints to `synapse/storage/databases/main` (#11984) 2022-02-21 16:03:06 +00:00
push_rule.py Revert experimental push rules from #7997. (#11884) 2022-02-02 09:49:31 -05:00
pusher.py Replace uses of simple_insert_many with simple_insert_many_values. (#11742) 2022-01-13 19:44:18 -05:00
receipts.py Add some type hints to datastore. (#12255) 2022-03-28 14:11:14 -04:00
registration.py Move `update_client_ip` background job from the main process to the background worker. (#12251) 2022-04-01 13:08:55 +01:00
rejections.py
relations.py Remove an unnecessary class from the relations code. (#12338) 2022-03-31 07:13:49 -04:00
room.py Add some type hints to datastore. (#12255) 2022-03-28 14:11:14 -04:00
room_batch.py Correct type hint for room_batch.py (#11310) 2021-11-11 16:49:28 +00:00
roommember.py Add a configuration to exclude rooms from sync response (#12310) 2022-03-30 09:43:04 +00:00
search.py Add some type hints to datastore. (#12255) 2022-03-28 14:11:14 -04:00
session.py Run `pyupgrade --py37-plus --keep-percent-format` on Synapse (#11685) 2022-01-05 09:53:05 -08:00
signatures.py Stop reading from `event_reference_hashes` (#11794) 2022-01-21 09:18:10 +00:00
state.py Raise an exception when getting state at an outlier (#12191) 2022-04-01 13:01:49 +01:00
state_deltas.py Improve log messages for stream ids (#11536) 2021-12-08 14:15:14 +00:00
stats.py Add some type hints to datastore. (#12255) 2022-03-28 14:11:14 -04:00
stream.py Add a configuration to exclude rooms from sync response (#12310) 2022-03-30 09:43:04 +00:00
tags.py Add type hints to `synapse/storage/databases/main/account_data.py` (#11546) 2021-12-13 16:28:10 +00:00
transactions.py Add admin API to get a list of federated rooms (#11658) 2022-01-25 16:11:40 +00:00
ui_auth.py Use auto_attribs/native type hints for attrs classes. (#11692) 2022-01-13 13:49:28 +00:00
user_directory.py Add some type hints to datastore. (#12255) 2022-03-28 14:11:14 -04:00
user_erasure_store.py Annotations for user_erasure_store (#11313) 2021-11-11 19:22:19 +00:00