Commit Graph

1101 Commits (fc27ca90066b49d3dc1ee785ba50b1f6745e230c)

Author SHA1 Message Date
Erik Johnston d2d61a8288 Fix deadlock in id_generators. No idea why this was an actual deadlock. 2015-04-29 19:15:23 +01:00
Erik Johnston 3864b3a8e6 Actually return rows 2015-04-29 18:07:36 +01:00
Erik Johnston 0618978238 Typo, args wrong way round 2015-04-29 18:04:35 +01:00
Erik Johnston 09177f4f2e Decode buffers in same thread 2015-04-29 18:03:42 +01:00
Erik Johnston 472be88674 We store pusher data as bytes 2015-04-29 17:43:46 +01:00
David Baker 12d381bd5d Decode the data json in the storage layer (was moved but this part was missed) 2015-04-29 17:13:51 +01:00
David Baker f8c30faf25 Oops, update the contraint too 2015-04-29 16:58:42 +01:00
David Baker 61cd5d9045 Be more postgressive 2015-04-29 16:57:14 +01:00
David Baker fb95035a65 Be postgressive 2015-04-29 16:53:41 +01:00
David Baker 4669def000 Oops, forgot the schema delta file 2015-04-29 16:50:16 +01:00
Erik Johnston 0337eaf321 txn.execute doesn't return cursors 2015-04-29 16:43:39 +01:00
Erik Johnston 884fb88e28 txn.execute doesn't return cursors 2015-04-29 16:35:20 +01:00
Erik Johnston d76c058eea Fix invalid SQL to work in postgres land 2015-04-29 16:30:25 +01:00
Erik Johnston c8ed9bd278 pushers table requires an access_token 2015-04-29 15:33:27 +01:00
Erik Johnston 970a9b9d2b We can't use REPLACE when upgrading databases now we have postgres 2015-04-29 13:55:44 +01:00
Erik Johnston 64991b0c8b Merge pull request #129 from matrix-org/key_distribution
Key distribution v2
2015-04-29 13:34:38 +01:00
Mark Haines e26a3d8d9e bump database schema version 2015-04-29 13:32:32 +01:00
Mark Haines a9549fdce3 Use bytea rather than BLOB 2015-04-29 13:16:09 +01:00
Mark Haines 4ad8b45155 Merge branch 'develop' into key_distribution
Conflicts:
	synapse/config/homeserver.py
2015-04-29 13:15:14 +01:00
Erik Johnston cd0864121b Make postgres database error slightly more helpful 2015-04-29 12:12:25 +01:00
Erik Johnston 4932a7e2d9 Use __all__ instead of assert to stop pyflakes from warning about unused import in __init__ 2015-04-29 12:12:25 +01:00
Erik Johnston a08bf11138 Appease PEP8 2015-04-29 11:44:48 +01:00
Erik Johnston 204132a998 Check that postgres database has correct charset set 2015-04-29 11:42:28 +01:00
Erik Johnston 327ca883ec Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-28 13:39:42 +01:00
Erik Johnston 18f8247701 Use TEXT instead of VARCHAR(n), since PostgreSQL treats them the same except for a limit 2015-04-28 12:41:33 +01:00
Mark Haines 9182f87664 Merge pull request #126 from matrix-org/csauth
Client / Server Auth Refactor
2015-04-28 11:00:27 +01:00
Mark Haines 55e1bc8920 And don't bump the schema version unnecessarily 2015-04-28 10:54:15 +01:00
Mark Haines b96c133034 Add server_keys.sql to the current delta rather than creating a new delta 2015-04-28 10:50:00 +01:00
Erik Johnston 8558e1ec73 Make get_max_token into inlineCallbacks so that the lock works. 2015-04-27 15:19:44 +01:00
Erik Johnston 6f8e2d517e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-27 14:41:40 +01:00
Erik Johnston 2732be83d9 Shuffle operations so that locking upsert happens last in the txn. This ensures the lock is held for the least amount of time possible. 2015-04-27 13:22:30 +01:00
Erik Johnston e4c4664d73 Handle the fact that postgres databases can be restarted from under us 2015-04-27 12:40:49 +01:00
David Baker 8a5be236e0 pep8 2015-04-27 11:49:18 +01:00
David Baker b02e1006b9 Run database check before daemonizing, at the cost of database hygiene. 2015-04-27 11:46:00 +01:00
David Baker f8152f2708 rename db method to be more informative 2015-04-27 10:16:26 +01:00
David Baker 2f475bd5d5 pep8 2015-04-24 18:15:07 +01:00
David Baker a7b51f4539 Check users in our table aren't on a different domain to the one we're configured with to try & fix SYN-266 2015-04-24 18:11:21 +01:00
Mark Haines eede182df7 Merge branch 'develop' into key_distribution 2015-04-24 10:35:49 +01:00
Mark Haines 4e2f8b8722 Copyright notices 2015-04-24 10:35:29 +01:00
Mark Haines f30d47c876 Implement remote key lookup api 2015-04-22 14:21:08 +01:00
David Baker 91c8f828e1 pep8 2015-04-17 19:56:04 +01:00
David Baker 8db6832db8 Password reset, finally. 2015-04-17 19:53:47 +01:00
David Baker 117f35ac4a Add endpoint to get threepids from server 2015-04-17 17:20:18 +01:00
David Baker 4eea5cf6c2 pep8 2015-04-17 16:46:45 +01:00
David Baker f96ab9d18d make add3pid servlet work 2015-04-17 16:44:49 +01:00
Erik Johnston e3417bbbe0 Revert needless change to storage.profile 2015-04-17 16:13:14 +01:00
David Baker cb03fafdf1 Merge branch 'develop' into csauth 2015-04-17 13:51:10 +01:00
Erik Johnston 0ec346d942 Add unique index to room_aliases, remove duplicates on upgrade. Convert some columns back to TEXT from bytea 2015-04-17 11:10:20 +01:00
Erik Johnston e1c0970c11 PEP8 2015-04-16 11:18:45 +01:00
Erik Johnston b8092fbc82 Go back to storing JSON in TEXT 2015-04-16 11:17:52 +01:00
Erik Johnston bc9e69e160 Move encoding and decoding of JSON into storage layer 2015-04-16 11:01:09 +01:00
Erik Johnston ffad75bd62 Remove mysql/maria support 2015-04-15 17:00:50 +01:00
Mark Haines a429515bdd Add methods for storing and retrieving the raw key json 2015-04-15 16:58:35 +01:00
Erik Johnston cf04cedf21 Change full_schemas/11 to work with postgres 2015-04-15 16:53:47 +01:00
Erik Johnston 5b31afcbd1 Remove debug logging 2015-04-15 16:27:04 +01:00
Erik Johnston 6e91f14d09 Add missing yield 2015-04-15 16:25:07 +01:00
Erik Johnston ed26e4012b pushers table requires a unique id. 2015-04-15 16:24:14 +01:00
Erik Johnston a5c72780e6 Don't pass in removed flag 2015-04-15 15:13:22 +01:00
Erik Johnston 4af32a2817 Postgres does not allow you to continue using a cursor after a DB exception has been raised, so move _simple_insert or_ignore flag out of transaction 2015-04-15 14:51:21 +01:00
Erik Johnston 25d2914fba Turn off persistance of sent_transactions.response_json 2015-04-15 14:09:35 +01:00
Erik Johnston cce5d057d3 Add index on events (topological_ordering, stream_ordering) to help with ORDER BY clauses 2015-04-15 11:43:25 +01:00
Erik Johnston a971fa9d58 Use try..finally in contextlib.contextmanager 2015-04-15 10:25:43 +01:00
Erik Johnston ded4128965 Use True for True rather than 1 2015-04-15 10:24:24 +01:00
Erik Johnston f9e12f79ca Add missing yield in storage func 2015-04-15 10:24:07 +01:00
Erik Johnston c756dfeb14 Correctly identify deadlocks 2015-04-15 10:23:42 +01:00
Mark Haines 32e14d8181 Return a sha256 fingerprint rather than the entire tls certificate 2015-04-14 19:10:09 +01:00
Erik Johnston 127fad17dd Add postgres database engine 2015-04-14 14:50:29 +01:00
Erik Johnston 5a95cd4442 Rename user_ips.user -> user_id 2015-04-14 13:54:09 +01:00
Erik Johnston 58d8339966 Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS. 2015-04-14 13:53:20 +01:00
Erik Johnston 3c741682e5 Correctly increment the _next_id initially 2015-04-14 09:54:44 +01:00
Erik Johnston 86fc9b617c For backwards compat, make state_groups.id have a type of int, not varchar 2015-04-13 17:03:49 +01:00
Erik Johnston 93937b2b31 Remove duplicate rows 2015-04-10 18:47:03 +01:00
Erik Johnston c5365dee56 Use case sensitive collations 2015-04-10 18:46:33 +01:00
Erik Johnston 4103b1c470 DROP indexes before recreating them 2015-04-10 18:46:09 +01:00
Erik Johnston 4d5b098626 Use LONGBLOB and TEXT for arbitary length rows 2015-04-10 18:45:09 +01:00
Erik Johnston 7ed2ec3061 Handle the fact that in sqlite binary data might be stored as unicode or bytes 2015-04-10 13:41:54 +01:00
Erik Johnston ce797ad373 Bump schema version 2015-04-10 11:22:30 +01:00
Erik Johnston 7e863c51e6 Use unsigned bigint 2015-04-10 11:22:04 +01:00
Erik Johnston 0f12772e32 SQLite wants INTEGER and not BIGINT for primary keys 2015-04-10 11:16:09 +01:00
Erik Johnston d5d4281647 Update full_schemas/16 to match delta files. Add delta/16 scripts 2015-04-10 10:59:46 +01:00
Erik Johnston cda4a6f93f Revert non-trivial changes to upgrade scripts 2015-04-10 10:19:50 +01:00
Erik Johnston e2722f58ee Fix schema again 2015-04-10 10:16:29 +01:00
Erik Johnston a1665c5094 Revert non-trivial schema changes and move them to a new schema version. 2015-04-10 10:05:44 +01:00
Erik Johnston 2ded344620 Remove unused import 2015-04-09 13:46:06 +01:00
Erik Johnston 9707acfc40 Remove spurious spaces 2015-04-09 13:45:20 +01:00
Erik Johnston 8bf285e082 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-09 13:13:26 +01:00
Erik Johnston 8ad0f4912e
Stream ordering and out of order insertions.
Handle the fact that events can be persisted out of order, and so to get
the "current max" stream token becomes non trivial - as we need to make
sure that *all* stream tokens less than the current max have also
successfully been persisted.
2015-04-09 11:41:36 +01:00
Erik Johnston 279a547a8b Use generic db exceptions rather than sqlite3 specific ones 2015-04-08 16:53:48 +01:00
Mark Haines 83f5125d52 Merge pull request #114 from matrix-org/improve_get_event_cache
Improve get event cache
2015-04-08 16:50:27 +01:00
Erik Johnston 3887350e47 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-08 14:11:07 +01:00
Erik Johnston c1b34af441 Move database timer logging to seperate logger 2015-04-08 13:31:19 +01:00
Erik Johnston 9a05795619 Retry transaction, not SQL query 2015-04-08 13:11:28 +01:00
Erik Johnston 24d8134ac1 Fix maria engine to correctly recognize deadlocks 2015-04-08 13:10:54 +01:00
Erik Johnston 4fe95094d1 Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-04-07 18:05:39 +01:00
Erik Johnston 49d6aa1394 Retry on deadlock 2015-04-07 15:28:37 +01:00
Erik Johnston 0bfa78b39b PEP8 2015-04-07 12:16:05 +01:00
Erik Johnston 6bc9edd8b2 Fix prepare_sqlite3_database's convert_param_style 2015-04-07 12:13:58 +01:00
Erik Johnston 05a35d62b6 Bump database version 2015-04-07 12:10:15 +01:00
Erik Johnston 8574bf62dc Add index to presence table 2015-04-07 12:09:36 +01:00
Erik Johnston 0af5f5efaf Don't use multiple UNIQUE constraints; it will cause deadlocks 2015-04-07 12:08:35 +01:00
Erik Johnston c8d3f6486d Implement or_ignore flag on inserts 2015-04-07 12:06:22 +01:00
Erik Johnston 304111afd0 Don't use AUTOINCREMENT, use an in memory version 2015-04-07 12:05:36 +01:00
Erik Johnston 779f7b0f44 Fix unicode support 2015-04-02 10:06:22 +01:00
Erik Johnston 9236136f3a Make work in both Maria and SQLite. Fix tests 2015-04-01 14:12:33 +01:00
Kegan Dougal 5e88a09a42 Add same user_id char checks as registration. 2015-03-31 14:00:25 +01:00
Kegan Dougal cf1fa59f4b Use a sender localpart instead of a user ID.
Form the user ID at runtime instead, This gives less room for error in AS
config files since they cannot specify the domain of another HS.
2015-03-31 13:48:03 +01:00
Kegan Dougal c217504949 Edit SQL schema to use string IDs not ints. Use token as ID. Update tests. 2015-03-31 12:07:56 +01:00
Kegan Dougal b59aa74556 Fix tests and missing returns on deferreds. 2015-03-31 11:35:45 +01:00
Kegan Dougal d33ae65efc Remove more reg/unreg methods. Read config not database for cache. 2015-03-31 11:00:00 +01:00
Paul "LeoNerd" Evans 3e420aebd8 Revert "Add another @cached wrapper, this time on get_presence_state()"
This reverts commit ff1fa0fbf8.
2015-03-27 16:16:58 +00:00
Paul "LeoNerd" Evans ff1fa0fbf8 Add another @cached wrapper, this time on get_presence_state() 2015-03-27 15:57:16 +00:00
David Baker a32e876ef4 Delete pushers when changing password 2015-03-26 13:40:16 +00:00
Paul "LeoNerd" Evans a198894bf7 Appease pep8 2015-03-26 11:53:58 +00:00
Kegan Dougal 32206dde3f Fixes from PR comments 2015-03-26 10:11:52 +00:00
Kegan Dougal 4edcbcee3b Merge branch 'develop' into application-services-txn-reliability
Conflicts:
	synapse/storage/__init__.py
2015-03-26 10:07:59 +00:00
Paul "LeoNerd" Evans 953e40f9dc Implement the main getEvent cache using Cache() instead of a custom application of LruCache; also unify its two-level structure into just one 2015-03-25 19:12:16 +00:00
David Baker c1a256cc4c Allow multiple pushers for a single app ID & pushkey, honouring the 'append' flag in the API. 2015-03-25 19:06:22 +00:00
Paul "LeoNerd" Evans f173d40a32 Use FrozenEvent's reject_reason to decide whether to return it; don't include allow_rejected in the main getEvents cache key 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans 1b988b051b Store the rejected reason in (Frozen)Event structs 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans 033a517feb Indirect invalidations of _get_event_cache via a helper method to keep all uses of the cache lexically within one .py file 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans 9ba6487b3f Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict() 2015-03-25 19:05:34 +00:00
Erik Johnston 0e8f5095c7 Fix unicode database support 2015-03-25 17:15:20 +00:00
David Baker c7023f2155 1) Pushers are now associated with an access token
2) Change places where we mean unauthenticated to 401, not 403, in C/S v2: hack so it stays as 403 in v1 because web client relies on it.
2015-03-24 17:24:15 +00:00
Erik Johnston 0ba393924a Escape non printing ascii character 2015-03-24 16:31:52 +00:00
Erik Johnston f488293d96 Don't reinsert into event_edges 2015-03-24 16:20:26 +00:00
Erik Johnston 1aa44939fc Fix bugs in transactions storage 2015-03-24 16:20:05 +00:00
Erik Johnston 5a447098dd Don't use room hosts table 2015-03-24 16:19:24 +00:00
Erik Johnston 9e98f1022a Don't order by rowid 2015-03-24 16:19:01 +00:00
Erik Johnston 9115421ace Use _simple_upsert 2015-03-24 16:17:39 +00:00
David Baker d19e79ecc9 Make deleting other access tokens when you change your password actually work 2015-03-24 15:33:48 +00:00
Erik Johnston 6e7131f02f Remove uses of REPLACE and ON CONFLICT IGNORE to make the SQL more portable. 2015-03-23 15:38:56 +00:00
Erik Johnston 9a7f496298 Sanitize RoomMemberStore 2015-03-23 15:29:04 +00:00
David Baker d98660a60d Implement password changing (finally) along with a start on making client/server auth more general. 2015-03-23 14:20:28 +00:00
Erik Johnston d5272b1d2c Use 'update or insert' rather than on 'conflict replace' 2015-03-23 14:02:34 +00:00
Erik Johnston 278149f533 Sanitize TransactionStore 2015-03-23 13:43:21 +00:00
Paul "LeoNerd" Evans 72d8406409 Put a cache on get_aliases_for_room 2015-03-20 19:21:13 +00:00
Paul "LeoNerd" Evans a63b4f7101 Remember the 'last seen' time for a given user/IP/device combination and only bother INSERTing another if it's stale 2015-03-20 18:25:49 +00:00
Paul "LeoNerd" Evans 0f86312c4c Pull out the cache logic from the @cached wrapper into its own class we can reuse 2015-03-20 18:25:42 +00:00
Paul "LeoNerd" Evans b1022ed8b5 func(*EXPR) is valid Python syntax, really... 2015-03-20 17:47:45 +00:00
Erik Johnston f6583796fe Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-03-20 16:31:48 +00:00
Erik Johnston 80cd08c190 PEP8 2015-03-20 16:03:25 +00:00
Erik Johnston 9517f4da4d Merge branch 'develop' of github.com:matrix-org/synapse into store_rearrangement 2015-03-20 16:02:47 +00:00
Erik Johnston dc0c989ef4 Give sensible names for '_simple_...' transactions 2015-03-20 15:59:18 +00:00
Paul "LeoNerd" Evans ceb61daa70 Add the tiniest of tiny one-element caches to get_room_events_max_id() as it's read every time someone hits eventstream 2015-03-20 15:44:06 +00:00
Erik Johnston fce0114005 Start removing Tables 2015-03-20 15:05:44 +00:00
Erik Johnston 7e282a53a5 Tidy up _simple_... methods 2015-03-20 15:05:10 +00:00
Paul "LeoNerd" Evans 91cb46191d Allow @cached-wrapped functions to have more or fewer than 1 argument; assert on the total count of them though 2015-03-20 14:59:45 +00:00
Erik Johnston 87db64b839 Rearrange storage modules 2015-03-20 14:11:38 +00:00
Erik Johnston cb8162d3d1 Rearrange storage modules 2015-03-20 13:52:56 +00:00
Erik Johnston d288d273e1 Generate transaction id in code 2015-03-20 10:57:44 +00:00
Erik Johnston d4f50f3ae5 decode_result takes an iterable 2015-03-20 10:57:26 +00:00
Erik Johnston 0d0610870d Fix up schemas some more 2015-03-20 10:55:31 +00:00
Erik Johnston d7a0496f3e Convert storage layer to be mysql compatible 2015-03-19 15:59:48 +00:00
Erik Johnston 58ed393235 Remove redundant key 2015-03-19 15:12:05 +00:00
Erik Johnston fae059cc18 Fix up schemas to work with mariadb 2015-03-19 13:42:39 +00:00
Erik Johnston bb24609158 Clean out event_forward_extremities table when the server rejoins the room 2015-03-18 11:19:47 +00:00
Erik Johnston 89036579ed Update schema to work with mariadb 2015-03-18 11:18:49 +00:00
Paul "LeoNerd" Evans 93978c5e2b @cached() annotate get_user_by_token() - achieves a minor DB performance improvement 2015-03-17 17:24:51 +00:00
Paul "LeoNerd" Evans 1489521ee5 Be polite and ensure we use @functools.wraps() when creating a function decorator 2015-03-17 17:19:22 +00:00
Paul "LeoNerd" Evans be170b1426 Add a metric for the scheduling latency of SQL queries 2015-03-16 17:21:59 +00:00
Erik Johnston cd2539ab2a Merge pull request #110 from matrix-org/fix_ban
Fix ban
2015-03-16 15:36:52 +00:00
Kegan Dougal f0d6f724a2 Set the service ID as soon as it is known. 2015-03-16 15:24:32 +00:00
Erik Johnston f1d2b94e0b Copy dict of context.current_state before changing it. 2015-03-16 15:13:05 +00:00
Kegan Dougal 835e01fc70 Minor PR comment tweaks. 2015-03-16 10:16:59 +00:00
Kegan Dougal f9232c7917 Merge branch 'develop' into application-services-txn-reliability
Conflicts:
	synapse/storage/appservice.py
2015-03-16 10:09:15 +00:00
Paul "LeoNerd" Evans e75fa8bbbf Bugfix to sql_txn_timer increment - add only the per-TXN duration, not the total time ever spent since boot 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 63cb7ece62 Rename the timer metrics exported by synapse.storage to append _time, so the meaning of ':total' is clearer 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans f1fbe3e09f Rename TimerMetric to DistributionMetric; as it could count more than just time 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 642f725fd7 Pretend the 'getEvent' cache is just another cache in the set of all the others for metric 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans b0cf867319 Use _ instead of . as a metric namespacing separator, for Prometheus 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans f9478e475b Rename Metrics' "keys" to "labels" 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans fa319a5786 Add TimerMetrics to shadow the PerformanceCounters in synapse.storage; with the view to eventually replacing them entirely 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 25187ab674 Collect per-SQL-verb timer stats on query execution time 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans e1a7e3564f Delete a couple of TODO markers of monitoring stats now done 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 8664599af7 Rename CacheCounterMetric to just CacheMetric; add a CallbackMetric component to give the size of the cache 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans b0cdf097f4 Sprinkle some CacheCounterMetrics around the synapse.storage layer 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans a7a28f85ae Appease pep8 2015-03-11 17:32:43 +00:00
Paul "LeoNerd" Evans 59a5f012cc Also give _execute() a description 2015-03-11 17:19:17 +00:00
Paul "LeoNerd" Evans 099e4b88d8 Add a description to storage layer's _execute_and_decode() 2015-03-11 17:08:57 +00:00
David Baker 04f8478aaa Add the master push rule for the break-my-push button. Allow server default rules to be disabled by default. 2015-03-10 17:26:25 +00:00
David Baker 8916acbc13 These aren't defined for redacted events so don't crash 2015-03-10 11:21:37 +00:00
Kegan Dougal db1fbc6c6f Fix remaining scheduler bugs. Add more informative logging. 2015-03-10 10:04:20 +00:00
Kegan Dougal 7e0bba555c Remove unused import 2015-03-09 17:48:37 +00:00
Kegan Dougal 04c9751f24 Bug fixes whilst putting it all together 2015-03-09 17:45:41 +00:00
Kegan Dougal b98cd03193 Use event IDs instead of dumping event content in the txns table. 2015-03-09 17:25:20 +00:00
Kegan Dougal 21fd84dcb8 Use seconds; start gluing in the AS scheduler into the AS handler. 2015-03-09 17:01:19 +00:00
Kegan Dougal 0a60bbf4fa Finish appservice txn storage impl and tests. 2015-03-09 15:53:03 +00:00
Kegan Dougal 1ead1caa18 Implement create_appservice_txn with tests. 2015-03-09 13:54:20 +00:00
Kegan Dougal 1c2dcf762a Partially implement txn store methods with tests. 2015-03-09 13:10:31 +00:00
Kegan Dougal 406d32f8b5 Start implementing ApplicationServiceTransactionStore 2015-03-06 17:35:14 +00:00
Kegan Dougal 34ce2ca62f Merge branch 'develop' into application-services-txn-reliability 2015-03-06 17:28:49 +00:00
Kegan Dougal 4a6afa6abf Assign the AS ID from the database; replace old placeholder txn id. 2015-03-06 17:27:55 +00:00
Kegan Dougal 01c099d9ef Add appservice txns sql schema 2015-03-06 17:16:47 +00:00
Kegan Dougal 2602ddc379 Apply clarity and docstrings 2015-03-06 16:16:14 +00:00
Kegan Dougal 0354659f9d Finish synapse.appservice.scheduler implementation.
With tests to assert behaviour. Not hooked up yet. Stub datastore methods
not implemented yet.
2015-03-06 16:09:05 +00:00
Kegan Dougal f260cb72cd Flesh out more stub functions. 2015-03-06 15:12:24 +00:00
Kegan Dougal 141ec04d19 Add stub ApplicationServiceTransactionStore. Bootstrap Recoverers. Fill in stub Transaction functions. 2015-03-06 14:53:35 +00:00
David Baker cf66ddc1b4 Schema change as delta in v14 2015-03-06 14:11:49 +00:00
David Baker ed72fc3a50 Merge branch 'develop' into pushrules2
Conflicts:
	synapse/storage/schema/pusher.sql
2015-03-04 15:24:21 +00:00
Erik Johnston d56c01fff4 Note that we don't specify execution order 2015-03-04 15:10:05 +00:00
Erik Johnston 17d319a20d s/schema_deltas/applied_schema_deltas/ 2015-03-04 15:06:22 +00:00
David Baker 92b3dc3219 Merge branch 'develop' into pushrules2 2015-03-04 14:56:41 +00:00
Erik Johnston 5681264faa s/%r/%s/ 2015-03-04 14:21:53 +00:00
Erik Johnston f701197227 Add example directory structures in doc 2015-03-04 14:20:14 +00:00
David Baker 2a45f3d448 Use if not results rather than len, as per feedback. 2015-03-04 14:17:59 +00:00
Erik Johnston 16dd87d848 Don't assume db conn is a Context Manager.
Twisted adbapi wrapped connections aren't context managers.
2015-03-04 14:03:41 +00:00
Erik Johnston 5eefd1f618 Add unique constraint on schema_version.lock schema. Use conflict clause in sql. 2015-03-04 13:52:18 +00:00
Erik Johnston b4c38738f4 Change to use logger in db upgrade script 2015-03-04 13:43:35 +00:00
Erik Johnston 640e53935d Use context manager with db conn to correctly commit and rollback 2015-03-04 13:43:17 +00:00
Erik Johnston 8c8354e85a Actually add full_schemas dir 2015-03-04 13:34:38 +00:00
Erik Johnston c3530c3fb3 More docs. Rename 'schema/current' to 'schema/full_schemas' 2015-03-04 13:34:11 +00:00
Erik Johnston 811355ccd0 Add some docs and remove unused variables 2015-03-04 13:11:01 +00:00
Erik Johnston 82b34e813d SYN-67: Finish up implementing new database schema management 2015-03-04 12:04:19 +00:00
Erik Johnston 8d33adfbbb SYN-67: Begin changing the way we handle schema versioning 2015-03-02 18:23:55 +00:00
David Baker 6fab7bd2c1 s/user_name/user/ as per mjark's comment 2015-03-02 18:17:19 +00:00
Erik Johnston 9f03553f48 Add missing comma 2015-03-02 16:38:40 +00:00
Erik Johnston b41dc68773 We purposefully don't have a version 14 delta script. 2015-03-02 16:36:19 +00:00
David Baker 20436cdf75 Blank lines 2015-03-02 15:58:12 +00:00
Kegsay 8ad024ea80 Merge pull request #93 from matrix-org/application-services-exclusive
Application services exclusive flag support
2015-03-02 14:56:32 +00:00
Erik Johnston 0a036944bd Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu 2015-03-02 13:53:30 +00:00
Kegan Dougal c3c01641d2 Run deltas and bump user_version in upgrade script 2015-03-02 13:38:57 +00:00
Kegan Dougal cb97ea3ec2 PEP8 2015-03-02 11:23:46 +00:00
Kegan Dougal 377ae369c1 Wrap all of get_app_service_rooms in a txn. 2015-03-02 11:20:51 +00:00
Kegan Dougal b216b36892 JOIN state_events rather than parsing unrecognized_keys to pull out member state_keys 2015-03-02 10:41:35 +00:00
Kegan Dougal 3d73383d18 Modify _simple_select_list to allow an empty WHERE clause. Use it for get_all_rooms and get_all_users. 2015-03-02 10:16:24 +00:00
Kegan Dougal ebc4830666 PR tweaks: set earlier on and use 'as json' for compat 2015-03-02 09:53:00 +00:00
Kegan Dougal 16b90764ad Convert expected format for AS regex to include exclusivity.
Previously you just specified the regex as a string, now it expects a JSON
object with a 'regex' key and an 'exclusive' boolean, as per spec.
2015-02-27 10:44:32 +00:00
Kegan Dougal 806a6c886a PEP8 2015-02-27 09:48:57 +00:00
David Baker 1959088156 Add API for getting/setting enabled-ness of push rules. 2015-02-26 18:07:44 +00:00
Kegan Dougal f0995436e7 Check for membership invite events correctly. 2015-02-26 17:21:17 +00:00
Kegan Dougal dcec7175dc Finish impl to get new events for AS. ASes should now be able to poll /events 2015-02-26 16:23:01 +00:00
Kegan Dougal 92478e96d6 Finish impl to extract all room IDs an AS may be interested in when polling the event stream. 2015-02-26 14:35:28 +00:00
David Baker 944003021b whitespace 2015-02-26 13:43:05 +00:00
David Baker 94fa334b01 Add enable/disable overlay for push rules (REST API not yet hooked up) 2015-02-25 19:17:07 +00:00
Kegan Dougal 29267cf9d7 PEP8 and pyflakes 2015-02-25 17:42:28 +00:00
Kegan Dougal 978ce87c86 Comment unused variables. 2015-02-25 17:37:48 +00:00
Kegan Dougal 2c79c4dc7f Fix alias query. 2015-02-25 17:37:14 +00:00
Kegan Dougal 2b8ca84296 Add support for extracting matching room_ids and room_aliases for a given AS. 2015-02-25 17:15:25 +00:00
Kegan Dougal 2d20466f9a Add stub functions and work out execution flow to implement AS event stream polling. 2015-02-25 15:00:59 +00:00
Paul "LeoNerd" Evans 9640510de2 Use OrderedDict for @cached backing store, so we can evict the oldest key unbiased 2015-02-23 18:41:58 +00:00
Paul "LeoNerd" Evans f53fcbce97 Use cache.pop() instead of a separate membership test + del [] 2015-02-23 18:30:45 +00:00
Mark Haines 27080698e7 Fix code style warning 2015-02-23 18:19:13 +00:00
Mark Haines 74048bdd41 Remove unused import 2015-02-23 18:17:43 +00:00
Paul "LeoNerd" Evans a09e59a698 Pull the _get_event_cache.setdefault() call out of the try block, as it doesn't need to be there and is confusing 2015-02-23 16:55:57 +00:00
Paul "LeoNerd" Evans 044d813ef7 Use the @cached decorator to implement the destination_retry_timings cache 2015-02-23 16:04:40 +00:00
Paul "LeoNerd" Evans 357fba2c24 RoomMemberStore no longer needs a _user_rooms_cache member 2015-02-23 15:57:41 +00:00
Paul "LeoNerd" Evans e76d485e29 Allow @cached-wrapped functions to have a prefill method for setting entries 2015-02-23 15:41:54 +00:00
Paul "LeoNerd" Evans 4631b737fd Squash out the now-redundant ApplicationServicesCache object class 2015-02-23 14:38:44 +00:00
Erik Johnston 42b972bccd Revert get_auth_chain changes 2015-02-23 14:35:23 +00:00
Paul "LeoNerd" Evans 55022d6ca5 Remove a TODO note 2015-02-19 18:38:09 +00:00
Paul "LeoNerd" Evans ebc3db295b Take named arguments to @cached() decorator, add a 'max_entries' limit 2015-02-19 18:36:02 +00:00
Paul "LeoNerd" Evans 077d200342 Move @cached decorator out into synapse.storage._base; add minimal docs 2015-02-19 17:29:39 +00:00
Erik Johnston 0ac2a79faa Initial stab at implementing a batched get_missing_pdus request 2015-02-19 17:24:14 +00:00
Paul "LeoNerd" Evans 61959928bb Pull out the 'get_rooms_for_user' cache logic into a reüsable @cached decorator 2015-02-19 14:58:07 +00:00
Erik Johnston 2c29ed3e84 Use absolute path when loading delta sql files 2015-02-17 17:22:24 +00:00
Erik Johnston 1a989c436c Bump schema version 2015-02-17 15:45:55 +00:00
Erik Johnston fa6c93bd26 Merge branch 'consumeErrors' of github.com:matrix-org/synapse into develop 2015-02-17 15:18:17 +00:00
Erik Johnston 72a4de2ce6 Use consumeErrors=True on all DeferredLists.
This is so that the DeferredLists actually consume the error instead of
propogating down the non-existent errback chain. This should reduce the
number of unhandled errors we are seeing.
2015-02-17 10:07:01 +00:00
David Baker 64def4f953 Merge branch 'hotfixes-0.7.0e' into develop 2015-02-13 16:18:34 +00:00
David Baker 8d5cce62ab Update pushers by app id and pushkey, not user id and pushkey 2015-02-13 16:16:16 +00:00
Mark Haines 0d872f5aa6 Merge pull request #50 from matrix-org/application-services
Application Services
2015-02-13 15:06:14 +00:00
Mark Haines 183b3d4e47 Prepare the database whenever a connection is opened from the db_pool so that in-memory databases will work 2015-02-13 14:38:24 +00:00
Erik Johnston 6b186a57ba Merge branch 'fix' of github.com:matrix-org/synapse into hotfixes-v0.7.0a 2015-02-12 14:51:46 +00:00
Erik Johnston 717687e1fc Get an auth query one at a time 2015-02-12 14:39:31 +00:00
Erik Johnston 83d41f25d8 Set database schema version in delta 2015-02-12 10:05:47 +00:00
Erik Johnston 8bbdf32849 Convert get_rooms to use runInteraction so the transacion has a more helpful description 2015-02-11 18:56:13 +00:00
Erik Johnston 42bc56dad3 Merge branch 'develop' of github.com:matrix-org/synapse into use-simplejson 2015-02-11 17:01:38 +00:00
Erik Johnston 7eef84a95b pyflakes 2015-02-11 16:52:22 +00:00
Kegan Dougal f2fdcb7c4b Merge branch 'develop' into application-services 2015-02-11 16:43:26 +00:00
Paul "LeoNerd" Evans b164e0896c Merge branch 'bugs/SYN-264' into develop 2015-02-11 16:23:30 +00:00
Paul "LeoNerd" Evans 7f47ba7383 Added another TODO note 2015-02-11 16:18:21 +00:00
Paul "LeoNerd" Evans 45b56609ae Cache the result of a get_rooms_for_user query, to make user_rooms_intersect() much lighter in the read-common case 2015-02-11 16:04:08 +00:00
Paul "LeoNerd" Evans 7be0f6594e First step of making user_rooms_intersect() faster - implement in intersection logic in Python code terms of a DB query that is cacheable per user 2015-02-11 15:53:56 +00:00
Erik Johnston ddb816cf60 Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict 2015-02-11 15:44:28 +00:00
Mark Haines ddc25cf4e2 Invalidate the cache for an event if it is redacted 2015-02-11 15:23:28 +00:00
Mark Haines aff892ce79 Fix formatting 2015-02-11 15:02:35 +00:00
Mark Haines f5a70e0d2e Add a cache for get_event 2015-02-11 15:01:15 +00:00
Erik Johnston 4ebbaf0d43 Blunty replace json with simplejson 2015-02-11 14:23:10 +00:00
Kegan Dougal fd40d992ad PEP8-ify 2015-02-11 10:41:33 +00:00
Kegan Dougal 9978c5c103 Merge branch 'develop' into application-services 2015-02-11 10:03:24 +00:00
Mark Haines b085fac735 Code-style fixes 2015-02-10 16:30:48 +00:00
Mark Haines fda4422bc9 Fix pyflakes 2015-02-10 14:54:07 +00:00
Mark Haines d7c7efb691 Add performance counters for different stages of loading events 2015-02-10 14:50:53 +00:00
Mark Haines 0c4536da8f Use the transaction 'desc' rather than 'name', increment the txn_ids in
txn names
2015-02-09 18:06:31 +00:00
Mark Haines 347b497db0 Formatting 2015-02-09 17:57:09 +00:00
Mark Haines 3a5ad7dbd5 Performance counters for database transaction names 2015-02-09 17:55:56 +00:00
Kegan Dougal 53557fc532 Merge branch 'develop' into application-services 2015-02-09 15:20:56 +00:00
Erik Johnston c4ee4ce93e Fix typo 2015-02-09 15:00:37 +00:00
Erik Johnston 66fde49f07 Log database time every 10s and log as percentage 2015-02-09 14:45:15 +00:00
Erik Johnston 75656712e3 Time how long we're spending on the database thread 2015-02-09 14:22:52 +00:00
Kegan Dougal 5a7dd05818 Modify auth.get_user_by_req for authing appservices directly.
Add logic to map the appservice token to the autogenned appservice user ID.
Add unit tests for all forms of get_user_by_req (user/appservice,
valid/bad/missing tokens)
2015-02-09 14:14:15 +00:00
Kegan Dougal ac3183caaa Register a user account for the AS when the AS registers. Add 'sender' column to AS table. 2015-02-09 12:03:37 +00:00
Erik Johnston 0cd66885e3 Move delta/v13.sql to delta/v12.sql 2015-02-06 14:38:04 +00:00
Kegan Dougal a3c6010718 Add delta sql file. 2015-02-05 16:48:57 +00:00
Kegan Dougal fc8bcc809d Merge branch 'develop' into application-services 2015-02-05 15:32:45 +00:00
David Baker aaf50bf6f3 Give server default rules the 'default' attribute and fix various brokenness. 2015-02-05 15:11:38 +00:00
Kegan Dougal 951690e54d Merge branch 'develop' into application-services 2015-02-05 14:28:03 +00:00
Erik Johnston f08bd95880 Merge pull request #47 from matrix-org/signature_failures
Federation fixes.
2015-02-05 14:03:00 +00:00
Kegan Dougal 27091f146a Add hs_token column and generate a different token f.e application service. 2015-02-05 10:08:12 +00:00
Kegan Dougal 89f2e8fbdf Fix bug in store defer. Add more unit tests. 2015-02-04 15:21:03 +00:00
Kegan Dougal 17753f0c20 Add stub ApplicationServiceApi and glue it with the handler. 2015-02-04 11:19:18 +00:00
Erik Johnston 03d415a6a2 Brief comment on why we do some things on every call to persist_event and not others 2015-02-04 10:40:59 +00:00
Erik Johnston c0462dbf15 Rearrange persist_event so that do all the queries that need to be done before returning early if we have already persisted that event. 2015-02-04 10:16:51 +00:00
Erik Johnston 02be8da5e1 Add doc to get_event 2015-02-03 17:34:07 +00:00
David Baker dc7bb70f22 s/instance_handle/profile_tag/ 2015-02-03 16:51:07 +00:00
Kegan Dougal 94a5db9f4d Add appservice package and move ApplicationService into it. 2015-02-03 14:44:16 +00:00
Kegan Dougal f2c039bfb9 Implement restricted namespace checks. Begin fleshing out the main hook for notifying application services. 2015-02-03 13:29:27 +00:00
Kegan Dougal a060b47b13 Add namespace constants. Add restrict_to option to limit namespace checks. 2015-02-03 13:17:28 +00:00
Kegan Dougal 3bd2841fdb Everyone loves SQL typos 2015-02-03 11:37:52 +00:00
Kegan Dougal 197f3ea4ba Implement regex checks for app services.
Expose handler.get_services_for_event which manages the checks for all
services.
2015-02-03 11:26:33 +00:00
Erik Johnston e7ca813dd4 Try to ensure we don't persist an event we have already persisted. In persist_event check if we already have the event, if so then update instead of replacing so that we don't cause a bump of the stream_ordering. 2015-02-03 10:39:41 +00:00
Kegan Dougal 9ff349a3cb Add defers in the right places. 2015-02-02 17:42:49 +00:00
Kegan Dougal 1a2de0c5fe Implement txns for AS (un)registration. 2015-02-02 17:39:41 +00:00
Erik Johnston 941f59101b Don't fail an entire request if one of the returned events fails a signature check. If an event does fail a signature check, look in the local database and request it from the originator. 2015-02-02 16:56:01 +00:00
Kegan Dougal a006d168c5 Actually merge into develop. 2015-02-02 16:05:34 +00:00
Kegan Dougal c059c9fea5 Merge branch 'develop' into application-services
Conflicts:
	synapse/handlers/__init__.py
	synapse/storage/__init__.py
2015-02-02 15:57:59 +00:00
Erik Johnston 7d897f5bfc Merge pull request #43 from matrix-org/rejections
Rejections
2015-01-30 16:11:14 +00:00
Mark Haines 6dc92d3427 Merge pull request #41 from matrix-org/client_v2_sync
Client v2 sync
2015-01-30 15:54:25 +00:00
Erik Johnston 1bd540ef79 Merge branch 'develop' of github.com:matrix-org/synapse into rejections
Conflicts:
	synapse/storage/schema/im.sql
2015-01-30 15:16:38 +00:00
Mark Haines 9ec9d6f2cb Merge pull request #42 from matrix-org/replication_split
Replication split
2015-01-30 15:14:10 +00:00
Erik Johnston 91015ad008 Remove merge conflict 2015-01-30 14:58:54 +00:00
Erik Johnston 4f7fe63b6d Remember to add schema file to list 2015-01-30 14:57:53 +00:00
Erik Johnston fdd2ac495a Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
Conflicts:
	synapse/storage/__init__.py
2015-01-30 14:57:33 +00:00
Mark Haines 8bc3066e0b Merge branch 'client_v2_filter' into client_v2_sync 2015-01-30 14:57:04 +00:00
Erik Johnston 471c47441d Merge pull request #37 from matrix-org/client_v2_filter
Client v2 filter
2015-01-30 14:56:08 +00:00
Mark Haines e97f756a05 Use 'in' to test if the key exists, remove unused _filters_for_user 2015-01-30 14:54:06 +00:00
Erik Johnston 2f4cb04f45 Be more specific in naming columns in selects. 2015-01-30 14:48:11 +00:00
Erik Johnston 472cf532b7 Put CREATE rejections into seperate .sql 2015-01-30 14:48:03 +00:00
David Baker 322a047502 Add room member count condition and default rule to make a noise on rooms of only 2 people. 2015-01-30 14:46:45 +00:00
Erik Johnston c515d37797 Merge branch 'replication_split' of github.com:matrix-org/synapse into rejections
Conflicts:
	synapse/storage/schema/delta/v12.sql
2015-01-30 14:19:49 +00:00
Erik Johnston 84b78c3b5f Merge branch 'rejections_storage' of github.com:matrix-org/synapse into replication_split 2015-01-30 14:17:47 +00:00
Erik Johnston e0b7c521cb Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
Conflicts:
	synapse/storage/__init__.py
	synapse/storage/schema/delta/v12.sql
2015-01-30 14:08:28 +00:00
Mark Haines 8498d348d8 Fix token formatting 2015-01-30 11:42:09 +00:00
Mark Haines 22dd1cde2d Filter the recent events before applying the limit when doing an incremental sync with a gap 2015-01-30 11:32:35 +00:00
Erik Johnston 2ebf795c0a Merge branch 'develop' of github.com:matrix-org/synapse into rejections
Conflicts:
	synapse/storage/__init__.py
	synapse/storage/schema/delta/v12.sql
2015-01-30 11:10:37 +00:00
Erik Johnston 78015948a7 Initial implementation of auth conflict resolution 2015-01-29 16:52:33 +00:00
Mark Haines 8e571cbed8 Merge branch 'client_v2_filter' into client_v2_sync 2015-01-29 16:18:59 +00:00
Mark Haines 295322048d Merge branch 'develop' into client_v2_filter 2015-01-29 16:18:34 +00:00
Mark Haines acb68a39e0 Code style fixes. 2015-01-29 16:12:40 +00:00
Mark Haines 396a67a09a Merge branch 'client_v2_filter' into client_v2_sync
Conflicts:
	synapse/rest/client/v2_alpha/__init__.py
2015-01-29 14:58:00 +00:00
Mark Haines 9d8f798a3f Merge changes from develop 2015-01-29 14:55:27 +00:00
Mark Haines e4f50fa0aa Move bump schema delta 2015-01-29 14:53:18 +00:00
Mark Haines e016f4043b Use get_room_events_stream to get changes to the rooms if the number of changes is small 2015-01-29 14:40:28 +00:00
Kegan Dougal 3773759c0f Also edit the filter column on the delta SQL 2015-01-29 09:15:33 +00:00
Mark Haines b0b80074e0 SYN-252: Supply the stream and topological parts in the correct order to the constructor 2015-01-29 01:48:48 +00:00
David Baker d5bdf3c0c7 Allow the push rule delete method to take more specifiers. 2015-01-28 18:06:04 +00:00
Kegan Dougal 11634017f4 s/definition/filter_json/ since definition is now used to mean a component of the filter, rather than the complete json 2015-01-28 17:42:19 +00:00
Mark Haines 9c61556504 Merge branch 'develop' into client_v2_sync 2015-01-28 17:29:30 +00:00
Mark Haines 26c8fff19e Merge pull request #36 from matrix-org/device_id_from_access_token
Extract the device id and token id from the access token when autheniticating users
2015-01-28 17:19:28 +00:00
Mark Haines 388581e087 Extract the id token of the token when authing users, include the token and device_id in the internal meta data for the event along with the transaction id when sending events 2015-01-28 16:58:23 +00:00
Kegan Dougal c23e3db544 Add filter JSON sanity checks. 2015-01-28 16:45:18 +00:00
Erik Johnston 0ef5bfd6a9 Start implementing auth conflict res 2015-01-28 16:16:53 +00:00
David Baker 60b143a52e Move pushers delta to v12 and bump schema version 2015-01-28 15:48:28 +00:00
David Baker e78dd33292 Use %s instead of + 2015-01-28 14:52:58 +00:00
David Baker 93aac9bb7b Newline 2015-01-28 14:51:01 +00:00
David Baker 445ad9941e Redundant parens 2015-01-28 14:49:59 +00:00
David Baker 6d485dd1c7 unnecessary newlines 2015-01-28 14:48:42 +00:00
David Baker fb0928097a More magic commas (including the place I copied it from...) 2015-01-28 14:48:07 +00:00
David Baker 0cbb6b0f52 Google doc style 2015-01-28 14:44:41 +00:00
David Baker c291a4d522 Merge branch 'develop' into pushers
Conflicts:
	synapse/handlers/events.py
	synapse/server.py
2015-01-28 12:51:05 +00:00
Kegan Dougal 42876969b9 Add basic application_services SQL, and hook up parts of the appservice store to read from it. 2015-01-28 11:59:38 +00:00
Kegan Dougal b46fa8603e Remove unused import 2015-01-28 09:17:48 +00:00
Mark Haines e020574d65 Fix Formatting 2015-01-27 20:19:36 +00:00
Paul "LeoNerd" Evans 8398f19bce Created schema delta 2015-01-27 19:00:09 +00:00
Paul "LeoNerd" Evans 06cc147012 Initial stab at real SQL storage implementation of user filter definitions 2015-01-27 18:46:03 +00:00
Paul "LeoNerd" Evans 54e513b4e6 Move storage of user filters into real datastore layer; now have to mock it out in the REST-level tests 2015-01-27 17:48:13 +00:00
Kegan Dougal fbeaeb8689 Log when ASes are registered/unregistered. 2015-01-27 17:34:40 +00:00
Kegan Dougal ec3719b583 Use ApplicationService when registering. 2015-01-27 17:15:06 +00:00
Kegan Dougal 92171f9dd1 Add stub methods, TODOs and docstrings for application services. 2015-01-27 16:53:59 +00:00
Mark Haines a56008842b Start implementing incremental initial sync 2015-01-27 16:24:22 +00:00
Kegan Dougal 7331d34839 Add AS specific classes with docstrings. 2015-01-27 16:23:46 +00:00
Kegan Dougal 51449e0665 Add appservice handler and store. Glue together rest > handler > store. 2015-01-27 15:50:28 +00:00
David Baker 69a75b7ebe Add brackets to make get room name / alias work 2015-01-26 16:52:47 +00:00
Erik Johnston 7b88619241 Split up replication_layer module into client, server and transaction queue 2015-01-26 10:45:24 +00:00
David Baker 5f84ba8ea1 Add API to delete push rules. 2015-01-23 17:49:37 +00:00
Erik Johnston 9b1e552b51 Merge branch 'develop' of github.com:matrix-org/synapse into rejections 2015-01-23 15:51:48 +00:00
Erik Johnston 3b9cc882a5 Add storage method have_events 2015-01-23 15:42:52 +00:00
Mark Haines 5759bec43c Replace hs.parse_userid with UserID.from_string 2015-01-23 11:47:15 +00:00
David Baker bcd48b9636 Fix adding rules without before/after & add the rule that we couldn't find to the error 2015-01-23 10:28:25 +00:00
David Baker 8a850573c9 As yet fairly untested GET API for push rules 2015-01-22 19:32:17 +00:00
David Baker 673773b217 oops, this is not its own schema file 2015-01-22 18:27:07 +00:00
David Baker 7ecb49ef25 Insufficient newlines 2015-01-22 17:53:30 +00:00
David Baker 5c6189ea3e Merge branch 'develop' into pushers
Conflicts:
	synapse/rest/__init__.py
2015-01-22 17:46:16 +00:00
David Baker ede491b4e0 Oops: second part of commit dc938606 2015-01-22 17:38:53 +00:00
David Baker dc93860619 Add rest API & store for creating push rules
Also make unrecognised request error look more like synapse errors
because it makes it easier to throw them from within rest classes.
2015-01-22 17:37:12 +00:00
Erik Johnston b1b85753d7 Add support for storing rejected events in EventContext and data stores 2015-01-22 15:50:17 +00:00
David Baker afb714f7be add instance_handles to pushers so we have a way to refer to them even if the push token changes. 2015-01-20 11:49:48 +00:00
David Baker 2d2953cf5f Require device language when adding a pusher.
Because this seems like it might be useful to do sooner rather
than later.
2015-01-16 11:24:10 +00:00
David Baker 2ca2dbc821 Send room name and first alias in notification poke. 2015-01-15 16:56:18 +00:00
David Baker 2cb30767fa Honour the 'rejected' return from push gateways
Add a timestamp to push tokens so we know the last time they we
got them from the device. Send it to the push gateways so it can
determine whether its failure is more recent than the token.
Stop and remove pushers that have been rejected.
2015-01-13 19:48:37 +00:00
Mark Haines 3891597eb3 Remove unused functions 2015-01-13 15:57:26 +00:00
Mark Haines fda63064fc get_room_events isn't called anywhere 2015-01-13 14:43:26 +00:00
Mark Haines 895fcb377e Fix stream token ordering 2015-01-13 14:38:53 +00:00
David Baker c06a9063e1 Merge branch 'develop' into pushers 2015-01-13 13:15:51 +00:00
Kegan Dougal c43d898119 SYN-178: Fix off by one. 2015-01-12 17:38:40 +00:00
Erik Johnston 36a2a877e2 Use time.time() instead of time.clock() 2015-01-06 16:34:41 +00:00
Erik Johnston fd9a8db7ea Only fetch the columns we need. 2015-01-06 15:59:31 +00:00
Erik Johnston a01416cf21 Add delta and bump DB version 2015-01-06 15:42:18 +00:00
Erik Johnston f6da237c35 Add index on transaction_id to sent_transcations 2015-01-06 15:40:38 +00:00
Erik Johnston 9bd07bed23 Actually time that function 2015-01-06 15:28:56 +00:00
Erik Johnston 03a501456c Time how long calls to _get_destination_retry_timings take 2015-01-06 15:22:28 +00:00
Erik Johnston 52b2c6c9c7 Don't include None's in _get_events_txn 2015-01-06 14:56:57 +00:00
Erik Johnston 8a12df8cf3 Merge branch 'erikj-perf' of github.com:matrix-org/synapse into develop 2015-01-06 14:45:57 +00:00
Erik Johnston 96707ed718 Name 'user_rooms_intersect' transaction 2015-01-06 14:44:27 +00:00
Erik Johnston 76ec154e95 We don't need the full events for get_rooms_for_user_where_membership_is 2015-01-06 14:37:00 +00:00
Mark Haines 5e23a19204 Merge pull request #28 from matrix-org/erikj-perf
Database performance improvements.
2015-01-06 13:33:40 +00:00
Mark Haines adb04b1e57 Update copyright notices 2015-01-06 13:21:39 +00:00
Erik Johnston af1c7c7808 PEP8 2015-01-06 13:13:17 +00:00
Erik Johnston 12819d5082 Remove debug lines 2015-01-06 13:12:30 +00:00
Erik Johnston 52d8519008 Don't do batching when getting events. 2015-01-06 13:10:27 +00:00
Erik Johnston 98933e3db6 Only fetch prev_content when a client is streaming/paginating. Use transactions for event streams. 2015-01-06 13:03:23 +00:00
Erik Johnston 3e26720e05 Temporarily turn off 'redacted_because' and 'prev_content' keys 2015-01-06 11:26:58 +00:00
Erik Johnston f4ea78e9e2 More debug logging 2015-01-06 11:24:18 +00:00
Erik Johnston 753126b8cc Add some debug logging 2015-01-06 11:18:12 +00:00
Erik Johnston d7e8ea67b3 Reformat 2015-01-06 11:18:02 +00:00
Erik Johnston f0128f9600 Add RoomMemberStore.get_users_in_room, so that we can get the list of joined users without having to retrieve the full events 2015-01-06 10:55:43 +00:00
Erik Johnston d61109f578 Merge branch 'hotfixes-v0.6.0' of github.com:matrix-org/synapse into erikj-perf 2014-12-19 16:37:08 +00:00
Mark Haines 9c71d945d6 Look for name, topic in the event content rather than the event itself when persisting room name and topic events 2014-12-19 15:16:48 +00:00
David Baker b56730bb6e Merge branch 'develop' into pushers
Conflicts:
	synapse/api/errors.py
	synapse/server.py
	synapse/storage/__init__.py
2014-12-18 15:15:22 +00:00
David Baker afa953a293 schema version is now 10 2014-12-18 15:11:06 +00:00
David Baker 0a6664493a Merge branch 'master' into pushers 2014-12-18 15:06:11 +00:00
David Baker 4c7ad50f6e Thank you, pyflakes 2014-12-18 14:55:04 +00:00
David Baker 173264b656 ...and bump SCHEMA_VERSION 2014-12-18 14:53:10 +00:00
David Baker fc7c5e9cd7 Rename the pusher SQL delta to v9 which the next free one 2014-12-18 14:51:29 +00:00
David Baker 9728c305a3 after a few rethinks, a working implementation of pushers. 2014-12-18 14:49:22 +00:00
Erik Johnston f3788e3c78 Test some ideas that might help performance a bit 2014-12-17 23:37:08 +00:00
Erik Johnston dec5b62339 Use _get_events_txn instead of _parse_events_txn 2014-12-16 19:16:41 +00:00
Erik Johnston 2215faa361 Merge branch 'hotfixes-v0.5.4a' of github.com:matrix-org/synapse into release-v0.6.0 2014-12-16 19:11:13 +00:00
Erik Johnston 96779d2490 Fix bug where we did not send the full auth chain to people that joined over federation 2014-12-16 18:57:36 +00:00
Erik Johnston f76269392b Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.6.0
Conflicts:
	synapse/state.py
2014-12-16 18:35:46 +00:00
Erik Johnston 52f99243ab Use is_outlier() so that we don't get AttributeError 2014-12-16 18:33:50 +00:00
Erik Johnston 5b39cfff69 Don't assume an event exists 2014-12-16 18:25:24 +00:00
Mark Haines 56db465047 Merge branch 'release-v0.6.0' into develop 2014-12-16 17:29:49 +00:00
Mark Haines 627e4f01d2 Remove send_message since nothing was calling it. Remove Snapshot because only send_message was using it 2014-12-16 16:07:41 +00:00
Erik Johnston 42b725ce52 Fix upgrade script to run all the missing deltas. 2014-12-16 15:13:34 +00:00
Erik Johnston b3c793e362 Do run all deltas up to missing delta 10 2014-12-16 14:44:53 +00:00
Erik Johnston 2e44714214 Make failure to run appropraite upgrade scripts more helpful. 2014-12-16 14:20:32 +00:00
Erik Johnston ef5a141050 Bump database version 2014-12-16 13:57:47 +00:00
Erik Johnston 35f4f6b070 Update upgrade script 2014-12-16 13:27:53 +00:00
Erik Johnston 882dc8dcab Persist internal_metadata 2014-12-16 13:17:09 +00:00
Erik Johnston 3c77d13aa5 Kill off synapse.api.events.* 2014-12-16 11:29:05 +00:00
Erik Johnston f280929a12 Use frozenutils 2014-12-15 17:31:36 +00:00
Erik Johnston c8dd3314d6 Fix bug where we ignored event_edge_hashes table 2014-12-15 13:55:22 +00:00
Erik Johnston c39beb5559 Store json as UTF-8 and not bytes 2014-12-12 14:53:37 +00:00
Erik Johnston 75085bb4d1 Pyflakes 2014-12-12 14:34:34 +00:00
Erik Johnston cde840a82c Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor
Conflicts:
	setup.py
2014-12-11 17:48:48 +00:00
Erik Johnston 0b04369238 Fix public room joining by making sure replaces_state never points to itself. 2014-12-11 15:56:06 +00:00
Erik Johnston 9191292b0f Fix prev_content 2014-12-11 15:16:55 +00:00
Erik Johnston 8cdebce470 Fix redactions. Fix 'age' key 2014-12-11 13:25:19 +00:00
Erik Johnston 02e4c18171 Remove dead code 2014-12-10 18:00:36 +00:00