MatrixSynapse/synapse
Eric Eastwood df8b91ed2b
Limit and filter the number of backfill points to get from the database (#13879)
There is no need to grab thousands of backfill points when we only need 5 to make the `/backfill` request with. We need to grab a few extra in case the first few aren't visible in the history.

Previously, we grabbed thousands of backfill points from the database, then sorted and filtered them in the app. Fetching the 4.6k backfill points for `#matrix:matrix.org` from the database takes ~50ms - ~570ms so it's not like this saves a lot of time 🤷. But it might save us more time now that `get_backfill_points_in_room`/`get_insertion_event_backward_extremities_in_room` are more complicated after https://github.com/matrix-org/synapse/pull/13635 

This PR moves the filtering and limiting to the SQL query so we just have less data to work with in the first place.

Part of https://github.com/matrix-org/synapse/issues/13356
2022-09-28 15:26:16 -05:00
..
_scripts Support enabling/disabling pushers (from MSC3881) (#13799) 2022-09-21 14:39:01 +00:00
api Faster Remote Room Joins: tell remote homeservers that we are unable to authorise them if they query a room which has partial state on our server. (#13823) 2022-09-23 11:47:16 +01:00
app Handle the case of remote users leaving a partial join room for device lists (#13885) 2022-09-27 13:01:08 +01:00
appservice Add prometheus counters for content types other than events (#13175) 2022-07-13 15:18:20 +01:00
config Accept & store thread IDs for receipts (implement MSC3771). (#13782) 2022-09-23 14:33:28 +00:00
crypto Instrument `_check_sigs_and_hash_and_fetch` to trace time spent in child concurrent calls (#13588) 2022-08-23 21:53:37 -05:00
events fix: Push notifications for invite over federation (#13719) 2022-09-28 12:31:53 +00:00
federation Prioritize outbound to-device over device list updates (#13922) 2022-09-27 15:17:41 +01:00
handlers Limit and filter the number of backfill points to get from the database (#13879) 2022-09-28 15:26:16 -05:00
http Fix mypy errors with latest canonicaljson (#13905) 2022-09-26 13:30:00 +01:00
logging Tag trace with instance name (#13761) 2022-09-09 11:31:37 -05:00
metrics Fix cache metrics not being updated when not using the legacy exposition module. (#13717) 2022-09-08 15:30:48 +01:00
module_api Add cache invalidation across workers to module API (#13667) 2022-09-21 15:32:01 +02:00
push fix: Push notifications for invite over federation (#13719) 2022-09-28 12:31:53 +00:00
replication Persist CreateRoom events to DB in a batch (#13800) 2022-09-28 10:11:48 +00:00
res Add viewport directive to HTML templates to optimise for mobile (#13493) 2022-08-11 14:04:20 +01:00
rest Support the stable dir parameter for /relations. (#13920) 2022-09-27 10:47:34 -04:00
server_notices Directly lookup local membership instead of getting all members in a room first (`get_users_in_room` mis-use) (#13608) 2022-08-24 14:13:12 -05:00
spam_checker_api Fix import in module_api module and docs on the new check_event_for_spam signature (#12918) 2022-05-31 12:04:53 +02:00
state Prepatory work for batching events to send (#13487) 2022-09-28 10:39:03 +01:00
static Add viewport directive to HTML templates to optimise for mobile (#13493) 2022-08-11 14:04:20 +01:00
storage Limit and filter the number of backfill points to get from the database (#13879) 2022-09-28 15:26:16 -05:00
streams Instrument `/messages` for understandable traces in Jaeger (#13368) 2022-08-03 10:57:38 -05:00
util Fix `have_seen_event` cache not being invalidated (#13863) 2022-09-27 15:55:43 -05:00
__init__.py Check if Rust lib needs rebuilding. (#13759) 2022-09-12 10:03:42 +00:00
event_auth.py Rename the `EventFormatVersions` enum values so that they line up with room version numbers. (#13706) 2022-09-07 11:08:20 +01:00
notifier.py Notifier: accept callbacks to fire on room joins (#13254) 2022-07-13 19:48:24 +01:00
py.typed Mark Module API error imports as re-exported and mark Synapse as containing type annotations (#11054) 2021-10-13 08:42:41 +01:00
server.py Fix error in `is_mine_id` when encountering a malformed ID (#13746) 2022-09-08 15:54:36 +01:00
types.py Accept & store thread IDs for receipts (implement MSC3771). (#13782) 2022-09-23 14:33:28 +00:00
visibility.py Add some miscellaneous comments around sync (#13474) 2022-08-10 12:58:20 +01:00