6a6cbfcf1e
Track the max_stream_device_id in a separate table, since we delete from the inbox table
2016-09-09 11:48:23 +01:00
2a0159b8ae
Fix the stream change cache to work over replication
2016-09-07 15:58:00 +01:00
31a07d2335
Add stream change caches for device messages
2016-09-07 15:27:07 +01:00
5dc2a702cf
Make _state_groups_id_gen a normal IdGenerator
2016-08-30 16:55:11 +01:00
e993925279
Add store-and-forward direct-to-device messaging
2016-08-25 17:35:37 +01:00
d9664344ec
Rename table. Add docs.
2016-08-15 11:45:57 +01:00
64e7e11853
Implement cache replication stream
2016-08-15 11:16:45 +01:00
4d70d1f80e
Add some invalidations to a cache_stream
2016-08-15 11:15:17 +01:00
f863a52cea
Add device_id support to /login
...
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.
Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.
Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
978fa53cc2
Pull out min stream_ordering from ex_outlier_stream
2016-07-15 10:22:30 +01:00
eef541a291
Move insert_client_ip to a separate class
2016-06-03 14:42:35 +01:00
3ae915b27e
Add a slaved store for presence
2016-06-03 11:05:53 +01:00
85b992f621
Fix to allow start with postgres
2016-05-27 10:44:44 +01:00
d4503e25ed
Make deleting push actions more efficient
...
There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.
2016-05-20 17:56:10 +01:00
9c272da05f
Add an openidish mechanism for proving to third parties that you own a given user_id
2016-05-05 13:42:44 +01:00
8e6a163f27
Add timestamp and auto incrementing ID
2016-05-04 15:19:12 +01:00
a1e0d316ea
Move _get_cache_dict into the SQLBaseStore
2016-04-06 13:05:19 +01:00
df727f2126
Fix stuck invites
...
If rejecting a remote invite fails with an error response don't fail
the entire request; instead mark the invite as locally rejected.
This fixes the bug where users can get stuck invites which they can
neither accept nor reject.
2016-04-05 11:13:24 +01:00
a2866e2e6a
Rename direction to step, apply checks consistently
2016-04-01 13:50:54 +01:00
e36bfbab38
Use a stream id generator for backfilled ids
2016-04-01 13:29:05 +01:00
1e25f62ee6
Use a stream id generator to assign state group ids
2016-03-30 12:55:02 +01:00
b6e8420aee
Add replication stream for pushers
2016-03-15 17:33:10 +00:00
7076082ae6
Fix relative imports so they work in both py3 and py27
2016-03-08 11:45:50 +00:00
9848b54cac
Prefill from the correct stream
2016-03-04 16:20:22 +00:00
deda48068c
prefill the push rules stream change cache
2016-03-04 16:19:42 +00:00
1b4f4a936f
Hook up the push rules stream to account_data in /sync
2016-03-04 14:44:01 +00:00
a1cf9e3bf3
Add a stream for push rule updates
2016-03-01 18:16:37 +00:00
54172924c8
Load the current id in the IdGenerator constructor
...
Rather than loading them lazily. This allows us to remove all
the yield statements and spurious arguments for the get_next
methods.
It also allows us to replace all instances of get_next_txn with
get_next since get_next no longer needs to access the db.
2016-03-01 14:32:56 +00:00
e6c5e3f28a
Close cursor
2016-02-18 16:39:28 +00:00
112283e230
Prefix TS fields with _ts
2016-02-18 10:11:43 +00:00
e5999bfb1a
Initial cut
2016-02-17 15:40:50 +00:00
d83d004ccd
Fix flake8 warnings for new flake8
2016-02-02 17:18:50 +00:00
25c311eaf6
Cache get_room_changes_for_user
2016-01-29 16:52:48 +00:00
8da95b6f1b
Comment. Remove superfluous order by
2016-01-29 15:39:17 +00:00
b5dbced938
Don't prefill account data
2016-01-29 14:53:59 +00:00
3d60686c0c
Actually use cache
2016-01-29 14:49:11 +00:00
45488e0ffa
Max is not a function
2016-01-29 14:42:01 +00:00
f67d60496a
Convert param style
2016-01-29 14:41:16 +00:00
18579534ea
Prefill stream change caches
2016-01-29 14:37:59 +00:00
7ed2bbeb11
Clean up a bit. Add comment
2016-01-28 14:32:05 +00:00
87f9477b10
Add a Homeserver.setup method.
...
This is for setting up dependencies that require work on startup. This
is useful for the DataStore that wants to read a bunch from the database
before initiliazing.
2016-01-26 15:51:06 +00:00
c232780081
Merge pull request #456 from matrix-org/store_event_actions
...
Send unread notification counts
2016-01-08 14:47:15 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
c914d67cda
Rename event-actions to event_push_actions as per PR request
2016-01-04 14:05:37 +00:00
aa667ee396
Save event actions to the db
2015-12-10 17:51:15 +00:00
95f30ecd1f
Add API for setting account_data globaly or on a per room basis
2015-12-01 18:41:32 +00:00
892e70ec84
Add APIs for adding and removing tags from rooms
2015-10-28 16:06:57 +00:00
f9340ea0d5
Merge branch 'erikj/store_engine' into erikj/search
2015-10-13 13:29:02 +01:00
ec398af41c
Expose error more nicely
2015-10-13 11:43:43 +01:00
40b6a5aad1
Split out the schema preparation and update logic into its own module
2015-10-13 11:38:48 +01:00
c85c912562
Add basic full text search impl.
2015-10-09 15:48:31 +01:00
6d59ffe1ce
Add some docstrings
2015-09-22 13:47:40 +01:00
7213588083
Implement configurable stats reporting
...
SYN-287
This requires that HS owners either opt in or out of stats reporting.
When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
a0b181bd17
Remove completely unused concepts from codebase
...
Removes device_id and ClientInfo
device_id is never actually written, and the matrix.org DB has no
non-null entries for it. Right now, it's just cluttering up code.
This doesn't remove the columns from the database, because that's
fiddly.
2015-08-25 16:23:06 +01:00
cecbd636e9
/tokenrefresh POST endpoint
...
This allows refresh tokens to be exchanged for (access_token,
refresh_token).
It also starts issuing them on login, though no clients currently
interpret them.
2015-08-20 16:21:35 +01:00
8f4165628b
Add index receipts_linearized_room_stream
2015-08-17 14:43:54 +01:00
7dec0b2bee
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-07 19:28:39 +01:00
0db40d3e93
Don't complain about extra .pyc files we find while hunting for database schemas
2015-08-07 17:22:11 +01:00
b8e386db59
Change Cache to not use *args in its interface
2015-08-07 11:52:21 +01:00
5989637f37
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts
2015-07-13 13:50:57 +01:00
716e426933
Fix various typos
2015-07-07 10:55:31 +01:00
b5770f8947
Add store for client end to end keys
2015-07-06 18:46:47 +01:00
901f56fa63
Add tables for receipts
2015-06-30 15:29:47 +01:00
1e365e88bd
Bump schema version
2015-06-11 15:50:39 +01:00
63a7b3ad1e
Add script to (re)convert the pushers table to changing the unique key. Also give the python db upgrade scripts the database engine so they can convert parameter strings, and add *args **kwargs to the upgrade function so we can add more args in future and previous scripts will ignore them.
2015-06-04 16:16:01 +01:00
4f1d984e56
Add index on events
2015-05-13 17:22:26 +01:00
84e1cacea4
Bump schema version
2015-05-08 15:58:14 +01:00
657298cebd
Don't lock user_ips table for upsert.
2015-05-01 10:46:48 +01:00
970a9b9d2b
We can't use REPLACE when upgrading databases now we have postgres
2015-04-29 13:55:44 +01:00
e26a3d8d9e
bump database schema version
2015-04-29 13:32:32 +01:00
327ca883ec
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-28 13:39:42 +01:00
8a5be236e0
pep8
2015-04-27 11:49:18 +01:00
b02e1006b9
Run database check before daemonizing, at the cost of database hygiene.
2015-04-27 11:46:00 +01:00
5a95cd4442
Rename user_ips.user -> user_id
2015-04-14 13:54:09 +01:00
58d8339966
Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS.
2015-04-14 13:53:20 +01:00
ce797ad373
Bump schema version
2015-04-10 11:22:30 +01:00
4fe95094d1
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:05:39 +01:00
0bfa78b39b
PEP8
2015-04-07 12:16:05 +01:00
6bc9edd8b2
Fix prepare_sqlite3_database's convert_param_style
2015-04-07 12:13:58 +01:00
05a35d62b6
Bump database version
2015-04-07 12:10:15 +01:00
9236136f3a
Make work in both Maria and SQLite. Fix tests
2015-04-01 14:12:33 +01:00
4edcbcee3b
Merge branch 'develop' into application-services-txn-reliability
...
Conflicts:
synapse/storage/__init__.py
2015-03-26 10:07:59 +00:00
0e8f5095c7
Fix unicode database support
2015-03-25 17:15:20 +00:00
9115421ace
Use _simple_upsert
2015-03-24 16:17:39 +00:00
a63b4f7101
Remember the 'last seen' time for a given user/IP/device combination and only bother INSERTing another if it's stale
2015-03-20 18:25:49 +00:00
f6583796fe
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-03-20 16:31:48 +00:00
dc0c989ef4
Give sensible names for '_simple_...' transactions
2015-03-20 15:59:18 +00:00
87db64b839
Rearrange storage modules
2015-03-20 14:11:38 +00:00
cb8162d3d1
Rearrange storage modules
2015-03-20 13:52:56 +00:00
d7a0496f3e
Convert storage layer to be mysql compatible
2015-03-19 15:59:48 +00:00
835e01fc70
Minor PR comment tweaks.
2015-03-16 10:16:59 +00:00
f9232c7917
Merge branch 'develop' into application-services-txn-reliability
...
Conflicts:
synapse/storage/appservice.py
2015-03-16 10:09:15 +00:00
59a5f012cc
Also give _execute() a description
2015-03-11 17:19:17 +00:00
099e4b88d8
Add a description to storage layer's _execute_and_decode()
2015-03-11 17:08:57 +00:00
8916acbc13
These aren't defined for redacted events so don't crash
2015-03-10 11:21:37 +00:00
21fd84dcb8
Use seconds; start gluing in the AS scheduler into the AS handler.
2015-03-09 17:01:19 +00:00
01c099d9ef
Add appservice txns sql schema
2015-03-06 17:16:47 +00:00
d56c01fff4
Note that we don't specify execution order
2015-03-04 15:10:05 +00:00
17d319a20d
s/schema_deltas/applied_schema_deltas/
2015-03-04 15:06:22 +00:00
5681264faa
s/%r/%s/
2015-03-04 14:21:53 +00:00
f701197227
Add example directory structures in doc
2015-03-04 14:20:14 +00:00
16dd87d848
Don't assume db conn is a Context Manager.
...
Twisted adbapi wrapped connections aren't context managers.
2015-03-04 14:03:41 +00:00
5eefd1f618
Add unique constraint on schema_version.lock schema. Use conflict clause in sql.
2015-03-04 13:52:18 +00:00
640e53935d
Use context manager with db conn to correctly commit and rollback
2015-03-04 13:43:17 +00:00
c3530c3fb3
More docs. Rename 'schema/current' to 'schema/full_schemas'
2015-03-04 13:34:11 +00:00
811355ccd0
Add some docs and remove unused variables
2015-03-04 13:11:01 +00:00
82b34e813d
SYN-67: Finish up implementing new database schema management
2015-03-04 12:04:19 +00:00
8d33adfbbb
SYN-67: Begin changing the way we handle schema versioning
2015-03-02 18:23:55 +00:00
9f03553f48
Add missing comma
2015-03-02 16:38:40 +00:00
b41dc68773
We purposefully don't have a version 14 delta script.
2015-03-02 16:36:19 +00:00
c3c01641d2
Run deltas and bump user_version in upgrade script
2015-03-02 13:38:57 +00:00
2c29ed3e84
Use absolute path when loading delta sql files
2015-02-17 17:22:24 +00:00
1a989c436c
Bump schema version
2015-02-17 15:45:55 +00:00
0d872f5aa6
Merge pull request #50 from matrix-org/application-services
...
Application Services
2015-02-13 15:06:14 +00:00
183b3d4e47
Prepare the database whenever a connection is opened from the db_pool so that in-memory databases will work
2015-02-13 14:38:24 +00:00
42bc56dad3
Merge branch 'develop' of github.com:matrix-org/synapse into use-simplejson
2015-02-11 17:01:38 +00:00
7eef84a95b
pyflakes
2015-02-11 16:52:22 +00:00
f2fdcb7c4b
Merge branch 'develop' into application-services
2015-02-11 16:43:26 +00:00
ddb816cf60
Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict
2015-02-11 15:44:28 +00:00
ddc25cf4e2
Invalidate the cache for an event if it is redacted
2015-02-11 15:23:28 +00:00
f5a70e0d2e
Add a cache for get_event
2015-02-11 15:01:15 +00:00
4ebbaf0d43
Blunty replace json with simplejson
2015-02-11 14:23:10 +00:00
951690e54d
Merge branch 'develop' into application-services
2015-02-05 14:28:03 +00:00
03d415a6a2
Brief comment on why we do some things on every call to persist_event and not others
2015-02-04 10:40:59 +00:00
c0462dbf15
Rearrange persist_event so that do all the queries that need to be done before returning early if we have already persisted that event.
2015-02-04 10:16:51 +00:00
02be8da5e1
Add doc to get_event
2015-02-03 17:34:07 +00:00
e7ca813dd4
Try to ensure we don't persist an event we have already persisted. In persist_event check if we already have the event, if so then update instead of replacing so that we don't cause a bump of the stream_ordering.
2015-02-03 10:39:41 +00:00
1a2de0c5fe
Implement txns for AS (un)registration.
2015-02-02 17:39:41 +00:00
941f59101b
Don't fail an entire request if one of the returned events fails a signature check. If an event does fail a signature check, look in the local database and request it from the originator.
2015-02-02 16:56:01 +00:00
a006d168c5
Actually merge into develop.
2015-02-02 16:05:34 +00:00
c059c9fea5
Merge branch 'develop' into application-services
...
Conflicts:
synapse/handlers/__init__.py
synapse/storage/__init__.py
2015-02-02 15:57:59 +00:00
9ec9d6f2cb
Merge pull request #42 from matrix-org/replication_split
...
Replication split
2015-01-30 15:14:10 +00:00
91015ad008
Remove merge conflict
2015-01-30 14:58:54 +00:00
4f7fe63b6d
Remember to add schema file to list
2015-01-30 14:57:53 +00:00
fdd2ac495a
Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
...
Conflicts:
synapse/storage/__init__.py
2015-01-30 14:57:33 +00:00
471c47441d
Merge pull request #37 from matrix-org/client_v2_filter
...
Client v2 filter
2015-01-30 14:56:08 +00:00
322a047502
Add room member count condition and default rule to make a noise on rooms of only 2 people.
2015-01-30 14:46:45 +00:00
84b78c3b5f
Merge branch 'rejections_storage' of github.com:matrix-org/synapse into replication_split
2015-01-30 14:17:47 +00:00
e0b7c521cb
Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
...
Conflicts:
synapse/storage/__init__.py
synapse/storage/schema/delta/v12.sql
2015-01-30 14:08:28 +00:00
9d8f798a3f
Merge changes from develop
2015-01-29 14:55:27 +00:00
60b143a52e
Move pushers delta to v12 and bump schema version
2015-01-28 15:48:28 +00:00
42876969b9
Add basic application_services SQL, and hook up parts of the appservice store to read from it.
2015-01-28 11:59:38 +00:00
8398f19bce
Created schema delta
2015-01-27 19:00:09 +00:00
06cc147012
Initial stab at real SQL storage implementation of user filter definitions
2015-01-27 18:46:03 +00:00
54e513b4e6
Move storage of user filters into real datastore layer; now have to mock it out in the REST-level tests
2015-01-27 17:48:13 +00:00
51449e0665
Add appservice handler and store. Glue together rest > handler > store.
2015-01-27 15:50:28 +00:00
69a75b7ebe
Add brackets to make get room name / alias work
2015-01-26 16:52:47 +00:00
7b88619241
Split up replication_layer module into client, server and transaction queue
2015-01-26 10:45:24 +00:00
3b9cc882a5
Add storage method have_events
2015-01-23 15:42:52 +00:00
673773b217
oops, this is not its own schema file
2015-01-22 18:27:07 +00:00
Mark Haines