64e7e11853
Implement cache replication stream
2016-08-15 11:16:45 +01:00
5aa024e501
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader
2016-07-29 11:24:56 +01:00
0a7d3cd00f
Create separate methods for getting messages to push
...
for the email and http pushers rather than trying to make a single
method that will work with their conflicting requirements.
The http pusher needs to get the messages in ascending stream order, and
doesn't want to miss a message.
The email pusher needs to get the messages in descending timestamp order,
and doesn't mind if it misses messages.
2016-07-28 20:24:24 +01:00
ec8b217722
Add destination retry to slave store
2016-07-28 17:35:53 +01:00
76b89d0edb
Add slace storage functions for public room list
2016-07-28 17:03:40 +01:00
0fcbca531f
Add get_auth_chain to slave store
2016-07-28 16:36:28 +01:00
6ede23ff1b
Add more key storage funcs into slave store
2016-07-28 15:41:26 +01:00
aede7248ab
Split out a FederationReader process
2016-07-21 17:37:44 +01:00
46b7362304
pep8
2016-06-24 11:44:57 +01:00
870c45913e
Use similar naming we use in email notifs for push
...
Fixes https://github.com/vector-im/vector-web/issues/1654
2016-06-24 11:41:11 +01:00
05e01f21d7
Remove event fetching from DB threads
2016-06-03 17:22:13 +01:00
20eccd84d4
Merge pull request #827 from matrix-org/markjh/more_slaved_methods
...
Add methods to events, account data and receipt slaves
2016-06-03 11:46:21 +01:00
b09f348530
Merge pull request #824 from matrix-org/markjh/slaved_presence_store
...
Add a slaved store for presence
2016-06-03 11:26:33 +01:00
81cf449daa
Add methods to events, account data and receipt slaves
...
Adds the methods needed by /sync to the slaved events,
account data and receipt stores.
2016-06-03 11:19:27 +01:00
3ae915b27e
Add a slaved store for presence
2016-06-03 11:05:53 +01:00
f88d747f79
Add a comment explaining why the filter cache doesn't need exipiring
2016-06-03 11:03:10 +01:00
0eae075723
Add slaved stores for filters, tokens, and push rules
2016-06-03 10:58:03 +01:00
70599ce925
Allow external processes to mark a user as syncing. ( #812 )
...
* Add infrastructure to the presence handler to track sync requests in external processes
* Expire stale entries for dead external processes
* Add an http endpoint for making users as syncing
Add some docstrings and comments.
* Fixes
2016-06-02 15:20:15 +01:00
0cb441fedd
Move typing handler out of the Handlers object
2016-05-17 15:58:46 +01:00
3b86ecfa79
Move the presence handler out of the Handlers object
2016-05-16 18:56:37 +01:00
0466454b00
Assert that stream replicated stream positions are ints
2016-05-13 17:33:44 +01:00
1f71f386f6
Merge branch 'develop' into dbkr/email_notifs_on_pusher
2016-05-13 16:59:56 +01:00
206eb9fd94
Shift some of the state_group methods into the SlavedEventStore
2016-05-13 16:58:14 +01:00
3abab26458
Add a slaved datastore for account data
2016-05-13 15:34:06 +01:00
0e792e7903
Log the stream IDs in an order that makes sense
2016-05-13 11:54:44 +01:00
3547e66bc6
Make sure we advance our stream position
2016-05-13 11:53:00 +01:00
5f46be19a7
Pass through get_events to pusher too
2016-05-10 18:43:40 +02:00
d46b18a00f
Pass through _get_event_txn
2016-05-10 18:27:06 +02:00
8940281d1b
Don't warn
2016-05-05 15:10:03 +01:00
5d8a93a10e
Add some log information at returned replication streams
2016-05-05 10:29:21 +01:00
8d7ad44331
Report per request metrics for all of the things using request_handler
2016-04-28 10:57:49 +01:00
8a65666454
Fix backfill replication to advance the stream correctly
2016-04-27 15:38:43 +01:00
78741cf025
Merge pull request #743 from matrix-org/markjh/slave_pushers
...
Replicate the pushers
2016-04-21 17:21:29 +01:00
31719ad124
Merge pull request #744 from matrix-org/markjh/replication_remove_pusher
...
Add a replication endpoint for deleting pushers
2016-04-21 17:10:49 +01:00
cfe1ff4bdb
Add a replication endpoint for deleting pushers
2016-04-21 16:33:05 +01:00
d4823efad9
Replicate the pushers
2016-04-21 16:18:00 +01:00
c0d8e0eb63
Replicate push actions
2016-04-21 15:25:58 +01:00
f52dd35ac3
Merge pull request #738 from matrix-org/markjh/slaved_receipts
...
Add a slaved receipts store
2016-04-19 17:31:59 +01:00
5bbd424ee0
Add a slaved receipts store
2016-04-19 17:14:08 +01:00
e99365f601
Replicate get_invited_rooms_for_user
2016-04-19 15:22:14 +01:00
ceb599e789
Add tests for redactions
2016-04-07 16:52:07 +01:00
60ec9793fb
Add tests for get_latest_event_ids_in_room and get_current_state
2016-04-07 13:17:56 +01:00
6bfec56796
Test that room membership is replicated
2016-04-06 16:20:13 +01:00
75fb9ac1be
Add a slaved events store class
...
Add a test to check that get_room_names_and_aliases does the same
thing on both the master and on the slave data store.
2016-04-06 14:18:35 +01:00
1d4deff25a
Separate generating the replication response...
...
from doing the http request parsing to make it easier
to write unit tests for replication.
2016-04-05 11:23:57 +01:00
62e395f0e3
Merge pull request #676 from matrix-org/markjh/replicate_stateIII
...
Add replication streams for ex outliers and current state resets
2016-03-31 11:20:57 +01:00
2ec5426035
Use a namedtuple rather than tuple unpacking
2016-03-31 10:33:02 +01:00
9113316b0e
typo
2016-03-30 17:29:42 +01:00
1fbb094c6f
Add replication streams for ex outliers and current state resets
2016-03-30 17:19:56 +01:00
31a9eceda5
Add a replication stream for state groups
2016-03-30 16:01:58 +01:00
b6e8420aee
Add replication stream for pushers
2016-03-15 17:33:10 +00:00
ebcbb23226
s/stream_ordering/event_stream_ordering/ in push
2016-03-04 16:15:23 +00:00
2223204eba
Hook push rules up to the replication API
2016-03-02 17:26:20 +00:00
60a0f81c7a
Add a /replication API for extracting the updates that happened on
...
synapse
This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.
This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.
Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.
Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.
The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
2016-03-01 14:49:41 +00:00
Erik Johnston