Commit Graph

18582 Commits (4a6623bc00f9047256daaec382e3386b8f83741c)

Author SHA1 Message Date
Damir Jelić 4a6623bc00 EventIndex: Rework the crawler cancellation. 2019-11-18 17:00:30 +01:00
Damir Jelić 8d7e7d0cc4 EventIndex: Remove the unused deleteEventIndex method.
We need to support the deletion of the event index even if it's not
currently initialized, therefore the deletion ended up in the
EventIndexPeg class.
2019-11-18 14:40:38 +01:00
Damir Jelić c48ccf9761 EventIndex: Remove some unnecessary checks if event indexing is supported. 2019-11-18 14:40:04 +01:00
Damir Jelić b983eaa3f9 EventIndex: Rename the file to be consistent with the class. 2019-11-18 14:36:08 +01:00
Damir Jelić 45e7aab41e EventIndexing: Rename our EventIndexer class. 2019-11-18 14:30:07 +01:00
Damir Jelić 579cbef7b0 EventIndexPeg: Rewrite the module documentation. 2019-11-18 14:29:03 +01:00
Damir Jelić 2bb331cdf0 Searching: Fix a typo. 2019-11-18 10:58:57 +01:00
Damir Jelić 0e3a0008df Searching: Remove the func suffix from our search functions. 2019-11-18 10:58:54 +01:00
Damir Jelić a4ad8151f8 Searching: Use the short form to build the search arguments object. 2019-11-18 10:58:53 +01:00
Damir Jelić b4a6123295 Searching: Move a comment to the correct place. 2019-11-18 10:58:50 +01:00
Damir Jelić 3b06c684d2 EventIndexing: Don't capitalize homeserver. 2019-11-18 10:58:47 +01:00
Damir Jelić 050e52ce46 EventIndexPeg: Treat both cases of unavailable platform support the same. 2019-11-18 10:58:45 +01:00
Damir Jelić ddb536e94a EventIndexPeg: Move a docstring to the correct place. 2019-11-18 10:58:44 +01:00
Damir Jelić 910c3ac08d BaseEventIndexManager: Fix some type annotations. 2019-11-18 10:58:42 +01:00
Damir Jelić 5149164010 MatrixChat: Revert the unnecessary changes in the MatrixChat class. 2019-11-18 10:58:40 +01:00
Damir Jelić 9fa8e8238a BaseEventIndexManager: Fix a typo. 2019-11-18 10:58:38 +01:00
Damir Jelić ab93745460 Fix the copyright headers from New Vector to The Matrix Foundation. 2019-11-18 10:58:36 +01:00
Damir Jelić 30d4dd36a7 BaseEventIndexManager: Remove the flow annotation. 2019-11-18 10:58:31 +01:00
Damir Jelić eb0b0a400f EventIndexPeg: Remove the now unused import of MatrixClientPeg. 2019-11-14 16:18:36 +01:00
Damir Jelić d82d4246e9 BaseEventIndexManager: Remove a return from a docstring. 2019-11-14 16:17:50 +01:00
Damir Jelić 7516f2724a EventIndexing: Rework the index initialization and deletion. 2019-11-14 16:13:22 +01:00
Damir Jelić 448c9a8290 EventIndexPeg: Add a missing return statement. 2019-11-14 16:01:14 +01:00
Damir Jelić 28d2e658a4 EventIndexing: Don't scope the event index per user. 2019-11-14 14:13:49 +01:00
Damir Jelić 9b32ec10b4 EventIndexing: Use the correct timeout value. 2019-11-13 16:47:21 +01:00
Damir Jelić d4b31cb7e0 EventIndexing: Move the max events per crawl constant into the class. 2019-11-13 16:35:26 +01:00
Damir Jelić 368a77ec3e EventIndexing: Fix a style issue. 2019-11-13 16:35:04 +01:00
Damir Jelić cc2ee53824 EventIndex: Add some more docs and fix some lint issues. 2019-11-13 16:21:26 +01:00
Damir Jelić c26df9d9ef EventIndexing: Fix a typo. 2019-11-13 15:57:12 +01:00
Damir Jelić bf558b46c3 EventIndexPeg: Clean up the event index initialization. 2019-11-13 15:39:39 +01:00
Damir Jelić c33f5ba0ca BaseEventIndexManager: Add a method to perform runtime checks for indexing support. 2019-11-13 15:39:06 +01:00
Damir Jelić ab7f34b45a EventIndexing: Don't mention Seshat in the logs. 2019-11-13 15:26:27 +01:00
Damir Jelić 1316e04776 EventIndexing: Check if there is a room when resetting the timeline. 2019-11-13 15:23:08 +01:00
Damir Jelić f453fea24a BasePlatform: Move the event indexing methods into a separate class. 2019-11-13 14:46:17 +01:00
Damir Jelić 80b28004e1 Searching: Define the room id in the const object. 2019-11-13 11:02:54 +01:00
Damir Jelić 54b352f69c MatrixChat: Fix the limited timeline checkpoint adding. 2019-11-13 10:37:20 +01:00
Damir Jelić 1df28c7526 Fix some lint errors. 2019-11-13 10:30:38 +01:00
Damir Jelić 1cc64f2426 Searching: Move the small helper functions out of the eventIndexSearch function. 2019-11-13 10:10:35 +01:00
Damir Jelić 008554463d Lifecycle: Move the event index deletion into the clear storage method. 2019-11-13 09:52:59 +01:00
Damir Jelić 3502454c61 LifeCycle: Stop the crawler and delete the index when whe log out. 2019-11-12 15:58:38 +01:00
Damir Jelić d69eb78b66 EventIndexing: Add a missing platform getting. 2019-11-12 15:41:14 +01:00
Damir Jelić ecbc47c548 EventIndexing: Rename the stop method. 2019-11-12 15:40:49 +01:00
Damir Jelić d911055f5d MatrixChat: Move the indexing limited room logic to a different event. 2019-11-12 15:39:54 +01:00
Damir Jelić e296fd05c0 RoomView: Move the search logic into a separate module. 2019-11-12 15:39:26 +01:00
Damir Jelić cfdcf45ac6 MatrixChat: Move the event indexing logic into separate modules. 2019-11-12 13:29:07 +01:00
Damir Jelić 2c5565e502 MatrixChat: Add some missing semicolons. 2019-11-11 15:08:00 +01:00
Damir Jelić 64061173e1 MatrixChat: Check if our state array is empty in the crawled messages response. 2019-10-18 16:33:07 +02:00
Damir Jelić 89f14e55a2 MatrixChat: Catch errors when fetching room messages in the crawler. 2019-10-18 16:32:43 +02:00
Damir Jelić 1b63886a6b MatrixChat: Add more detailed logging to the event crawler. 2019-10-18 16:31:39 +02:00
Damir Jelić 3f53691834 RoomView: Use platform specific search if our platform supports it.
This patch extends our search to include our platform specific event
index.

There are 3 search scenarios and are handled differently when platform
support for indexing is present:

    - Search a single non-encrypted room: Use the server-side search
        like before.
    - Search a single encrypted room: Search using our platform specific
        event index.
    - Search across all rooms: Search encrypted rooms using our local
        event index. Search non-encrypted rooms using the classic
        server-side search. Combine the results.

The combined search will result in having twice the amount of search
results since comparing the scores fairly wasn't deemed sensible.
2019-10-11 16:55:01 +02:00
Damir Jelić 4acec19d40 MatrixChat: Add new crawler checkpoints if there was a limited timeline.
A sync call may not have all events that happened since the last time
the client synced. In such a case the room is marked as limited and
events need to be fetched separately.

When such a sync call happens our event index will have a gap. To
close the gap checkpoints are added to start crawling our room again.
Unnecessary full re-crawls are prevented by checking if our current
/room/roomId/messages request contains only events that were already
present in our event index.
2019-10-11 16:55:01 +02:00