Commit Graph

323 Commits (4262aba17b643bc82c5cce92298dac0a27b2727c)

Author SHA1 Message Date
Mark Haines e36bfbab38 Use a stream id generator for backfilled ids 2016-04-01 13:29:05 +01:00
Mark Haines 1e25f62ee6 Use a stream id generator to assign state group ids 2016-03-30 12:55:02 +01:00
Mark Haines b6e8420aee Add replication stream for pushers 2016-03-15 17:33:10 +00:00
Mark Haines 7076082ae6 Fix relative imports so they work in both py3 and py27 2016-03-08 11:45:50 +00:00
Mark Haines 9848b54cac Prefill from the correct stream 2016-03-04 16:20:22 +00:00
Mark Haines deda48068c prefill the push rules stream change cache 2016-03-04 16:19:42 +00:00
Mark Haines 1b4f4a936f Hook up the push rules stream to account_data in /sync 2016-03-04 14:44:01 +00:00
Mark Haines a1cf9e3bf3 Add a stream for push rule updates 2016-03-01 18:16:37 +00:00
Mark Haines 54172924c8 Load the current id in the IdGenerator constructor
Rather than loading them lazily. This allows us to remove all
the yield statements and spurious arguments for the get_next
methods.

It also allows us to replace all instances of get_next_txn with
get_next since get_next no longer needs to access the db.
2016-03-01 14:32:56 +00:00
Erik Johnston e6c5e3f28a Close cursor 2016-02-18 16:39:28 +00:00
Erik Johnston 112283e230 Prefix TS fields with _ts 2016-02-18 10:11:43 +00:00
Erik Johnston e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall d83d004ccd Fix flake8 warnings for new flake8 2016-02-02 17:18:50 +00:00
Erik Johnston 25c311eaf6 Cache get_room_changes_for_user 2016-01-29 16:52:48 +00:00
Erik Johnston 8da95b6f1b Comment. Remove superfluous order by 2016-01-29 15:39:17 +00:00
Erik Johnston b5dbced938 Don't prefill account data 2016-01-29 14:53:59 +00:00
Erik Johnston 3d60686c0c Actually use cache 2016-01-29 14:49:11 +00:00
Erik Johnston 45488e0ffa Max is not a function 2016-01-29 14:42:01 +00:00
Erik Johnston f67d60496a Convert param style 2016-01-29 14:41:16 +00:00
Erik Johnston 18579534ea Prefill stream change caches 2016-01-29 14:37:59 +00:00
Erik Johnston 7ed2bbeb11 Clean up a bit. Add comment 2016-01-28 14:32:05 +00:00
Erik Johnston 87f9477b10 Add a Homeserver.setup method.
This is for setting up dependencies that require work on startup. This
is useful for the DataStore that wants to read a bunch from the database
before initiliazing.
2016-01-26 15:51:06 +00:00
David Baker c232780081 Merge pull request #456 from matrix-org/store_event_actions
Send unread notification counts
2016-01-08 14:47:15 +00:00
Matthew Hodgson 6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
David Baker c914d67cda Rename event-actions to event_push_actions as per PR request 2016-01-04 14:05:37 +00:00
David Baker aa667ee396 Save event actions to the db 2015-12-10 17:51:15 +00:00
Mark Haines 95f30ecd1f Add API for setting account_data globaly or on a per room basis 2015-12-01 18:41:32 +00:00
Mark Haines 892e70ec84 Add APIs for adding and removing tags from rooms 2015-10-28 16:06:57 +00:00
Erik Johnston f9340ea0d5 Merge branch 'erikj/store_engine' into erikj/search 2015-10-13 13:29:02 +01:00
Erik Johnston ec398af41c Expose error more nicely 2015-10-13 11:43:43 +01:00
Erik Johnston 40b6a5aad1 Split out the schema preparation and update logic into its own module 2015-10-13 11:38:48 +01:00
Erik Johnston c85c912562 Add basic full text search impl. 2015-10-09 15:48:31 +01:00
Daniel Wagner-Hall 6d59ffe1ce Add some docstrings 2015-09-22 13:47:40 +01:00
Daniel Wagner-Hall 7213588083 Implement configurable stats reporting
SYN-287

This requires that HS owners either opt in or out of stats reporting.

When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
Daniel Wagner-Hall a0b181bd17 Remove completely unused concepts from codebase
Removes device_id and ClientInfo

device_id is never actually written, and the matrix.org DB has no
non-null entries for it. Right now, it's just cluttering up code.

This doesn't remove the columns from the database, because that's
fiddly.
2015-08-25 16:23:06 +01:00
Daniel Wagner-Hall cecbd636e9 /tokenrefresh POST endpoint
This allows refresh tokens to be exchanged for (access_token,
refresh_token).

It also starts issuing them on login, though no clients currently
interpret them.
2015-08-20 16:21:35 +01:00
Erik Johnston 8f4165628b Add index receipts_linearized_room_stream 2015-08-17 14:43:54 +01:00
Erik Johnston 7dec0b2bee Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache 2015-08-07 19:28:39 +01:00
Paul "LeoNerd" Evans 0db40d3e93 Don't complain about extra .pyc files we find while hunting for database schemas 2015-08-07 17:22:11 +01:00
Erik Johnston b8e386db59 Change Cache to not use *args in its interface 2015-08-07 11:52:21 +01:00
Erik Johnston 5989637f37 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts 2015-07-13 13:50:57 +01:00
Erik Johnston 716e426933 Fix various typos 2015-07-07 10:55:31 +01:00
Mark Haines b5770f8947 Add store for client end to end keys 2015-07-06 18:46:47 +01:00
Erik Johnston 901f56fa63 Add tables for receipts 2015-06-30 15:29:47 +01:00
Erik Johnston 1e365e88bd Bump schema version 2015-06-11 15:50:39 +01:00
David Baker 63a7b3ad1e Add script to (re)convert the pushers table to changing the unique key. Also give the python db upgrade scripts the database engine so they can convert parameter strings, and add *args **kwargs to the upgrade function so we can add more args in future and previous scripts will ignore them. 2015-06-04 16:16:01 +01:00
Erik Johnston 4f1d984e56 Add index on events 2015-05-13 17:22:26 +01:00
Mark Haines 84e1cacea4 Bump schema version 2015-05-08 15:58:14 +01:00
Erik Johnston 657298cebd Don't lock user_ips table for upsert. 2015-05-01 10:46:48 +01:00
Erik Johnston 970a9b9d2b We can't use REPLACE when upgrading databases now we have postgres 2015-04-29 13:55:44 +01:00
Mark Haines e26a3d8d9e bump database schema version 2015-04-29 13:32:32 +01:00
Erik Johnston 327ca883ec Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-28 13:39:42 +01:00
David Baker 8a5be236e0 pep8 2015-04-27 11:49:18 +01:00
David Baker b02e1006b9 Run database check before daemonizing, at the cost of database hygiene. 2015-04-27 11:46:00 +01:00
Erik Johnston 5a95cd4442 Rename user_ips.user -> user_id 2015-04-14 13:54:09 +01:00
Erik Johnston 58d8339966 Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS. 2015-04-14 13:53:20 +01:00
Erik Johnston ce797ad373 Bump schema version 2015-04-10 11:22:30 +01:00
Erik Johnston 4fe95094d1 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:05:39 +01:00
Erik Johnston 0bfa78b39b PEP8 2015-04-07 12:16:05 +01:00
Erik Johnston 6bc9edd8b2 Fix prepare_sqlite3_database's convert_param_style 2015-04-07 12:13:58 +01:00
Erik Johnston 05a35d62b6 Bump database version 2015-04-07 12:10:15 +01:00
Erik Johnston 9236136f3a Make work in both Maria and SQLite. Fix tests 2015-04-01 14:12:33 +01:00
Kegan Dougal 4edcbcee3b Merge branch 'develop' into application-services-txn-reliability
Conflicts:
	synapse/storage/__init__.py
2015-03-26 10:07:59 +00:00
Erik Johnston 0e8f5095c7 Fix unicode database support 2015-03-25 17:15:20 +00:00
Erik Johnston 9115421ace Use _simple_upsert 2015-03-24 16:17:39 +00:00
Paul "LeoNerd" Evans a63b4f7101 Remember the 'last seen' time for a given user/IP/device combination and only bother INSERTing another if it's stale 2015-03-20 18:25:49 +00:00
Erik Johnston f6583796fe Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-03-20 16:31:48 +00:00
Erik Johnston dc0c989ef4 Give sensible names for '_simple_...' transactions 2015-03-20 15:59:18 +00:00
Erik Johnston 87db64b839 Rearrange storage modules 2015-03-20 14:11:38 +00:00
Erik Johnston cb8162d3d1 Rearrange storage modules 2015-03-20 13:52:56 +00:00
Erik Johnston d7a0496f3e Convert storage layer to be mysql compatible 2015-03-19 15:59:48 +00:00
Kegan Dougal 835e01fc70 Minor PR comment tweaks. 2015-03-16 10:16:59 +00:00
Kegan Dougal f9232c7917 Merge branch 'develop' into application-services-txn-reliability
Conflicts:
	synapse/storage/appservice.py
2015-03-16 10:09:15 +00:00
Paul "LeoNerd" Evans 59a5f012cc Also give _execute() a description 2015-03-11 17:19:17 +00:00
Paul "LeoNerd" Evans 099e4b88d8 Add a description to storage layer's _execute_and_decode() 2015-03-11 17:08:57 +00:00
David Baker 8916acbc13 These aren't defined for redacted events so don't crash 2015-03-10 11:21:37 +00:00
Kegan Dougal 21fd84dcb8 Use seconds; start gluing in the AS scheduler into the AS handler. 2015-03-09 17:01:19 +00:00
Kegan Dougal 01c099d9ef Add appservice txns sql schema 2015-03-06 17:16:47 +00:00
Erik Johnston d56c01fff4 Note that we don't specify execution order 2015-03-04 15:10:05 +00:00
Erik Johnston 17d319a20d s/schema_deltas/applied_schema_deltas/ 2015-03-04 15:06:22 +00:00
Erik Johnston 5681264faa s/%r/%s/ 2015-03-04 14:21:53 +00:00
Erik Johnston f701197227 Add example directory structures in doc 2015-03-04 14:20:14 +00:00
Erik Johnston 16dd87d848 Don't assume db conn is a Context Manager.
Twisted adbapi wrapped connections aren't context managers.
2015-03-04 14:03:41 +00:00
Erik Johnston 5eefd1f618 Add unique constraint on schema_version.lock schema. Use conflict clause in sql. 2015-03-04 13:52:18 +00:00
Erik Johnston 640e53935d Use context manager with db conn to correctly commit and rollback 2015-03-04 13:43:17 +00:00
Erik Johnston c3530c3fb3 More docs. Rename 'schema/current' to 'schema/full_schemas' 2015-03-04 13:34:11 +00:00
Erik Johnston 811355ccd0 Add some docs and remove unused variables 2015-03-04 13:11:01 +00:00
Erik Johnston 82b34e813d SYN-67: Finish up implementing new database schema management 2015-03-04 12:04:19 +00:00
Erik Johnston 8d33adfbbb SYN-67: Begin changing the way we handle schema versioning 2015-03-02 18:23:55 +00:00
Erik Johnston 9f03553f48 Add missing comma 2015-03-02 16:38:40 +00:00
Erik Johnston b41dc68773 We purposefully don't have a version 14 delta script. 2015-03-02 16:36:19 +00:00
Kegan Dougal c3c01641d2 Run deltas and bump user_version in upgrade script 2015-03-02 13:38:57 +00:00
Erik Johnston 2c29ed3e84 Use absolute path when loading delta sql files 2015-02-17 17:22:24 +00:00
Erik Johnston 1a989c436c Bump schema version 2015-02-17 15:45:55 +00:00
Mark Haines 0d872f5aa6 Merge pull request #50 from matrix-org/application-services
Application Services
2015-02-13 15:06:14 +00:00
Mark Haines 183b3d4e47 Prepare the database whenever a connection is opened from the db_pool so that in-memory databases will work 2015-02-13 14:38:24 +00:00
Erik Johnston 42bc56dad3 Merge branch 'develop' of github.com:matrix-org/synapse into use-simplejson 2015-02-11 17:01:38 +00:00
Erik Johnston 7eef84a95b pyflakes 2015-02-11 16:52:22 +00:00
Kegan Dougal f2fdcb7c4b Merge branch 'develop' into application-services 2015-02-11 16:43:26 +00:00
Erik Johnston ddb816cf60 Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict 2015-02-11 15:44:28 +00:00