Commit Graph

2370 Commits (3f0f06cb31d0293e75943e804b3e1ed8fe58ebed)

Author SHA1 Message Date
Kegan Dougal 92478e96d6 Finish impl to extract all room IDs an AS may be interested in when polling the event stream. 2015-02-26 14:35:28 +00:00
David Baker 944003021b whitespace 2015-02-26 13:43:05 +00:00
David Baker 94fa334b01 Add enable/disable overlay for push rules (REST API not yet hooked up) 2015-02-25 19:17:07 +00:00
Kegan Dougal 29267cf9d7 PEP8 and pyflakes 2015-02-25 17:42:28 +00:00
Kegan Dougal 978ce87c86 Comment unused variables. 2015-02-25 17:37:48 +00:00
Kegan Dougal 2c79c4dc7f Fix alias query. 2015-02-25 17:37:14 +00:00
Kegan Dougal 2b8ca84296 Add support for extracting matching room_ids and room_aliases for a given AS. 2015-02-25 17:15:25 +00:00
Kegan Dougal 2d20466f9a Add stub functions and work out execution flow to implement AS event stream polling. 2015-02-25 15:00:59 +00:00
Paul "LeoNerd" Evans 9640510de2 Use OrderedDict for @cached backing store, so we can evict the oldest key unbiased 2015-02-23 18:41:58 +00:00
Paul "LeoNerd" Evans f53fcbce97 Use cache.pop() instead of a separate membership test + del [] 2015-02-23 18:30:45 +00:00
Mark Haines 27080698e7 Fix code style warning 2015-02-23 18:19:13 +00:00
Mark Haines 74048bdd41 Remove unused import 2015-02-23 18:17:43 +00:00
Paul "LeoNerd" Evans a09e59a698 Pull the _get_event_cache.setdefault() call out of the try block, as it doesn't need to be there and is confusing 2015-02-23 16:55:57 +00:00
Paul "LeoNerd" Evans 044d813ef7 Use the @cached decorator to implement the destination_retry_timings cache 2015-02-23 16:04:40 +00:00
Paul "LeoNerd" Evans 357fba2c24 RoomMemberStore no longer needs a _user_rooms_cache member 2015-02-23 15:57:41 +00:00
Paul "LeoNerd" Evans e76d485e29 Allow @cached-wrapped functions to have a prefill method for setting entries 2015-02-23 15:41:54 +00:00
Paul "LeoNerd" Evans 4631b737fd Squash out the now-redundant ApplicationServicesCache object class 2015-02-23 14:38:44 +00:00
Erik Johnston 42b972bccd Revert get_auth_chain changes 2015-02-23 14:35:23 +00:00
Paul "LeoNerd" Evans 55022d6ca5 Remove a TODO note 2015-02-19 18:38:09 +00:00
Paul "LeoNerd" Evans ebc3db295b Take named arguments to @cached() decorator, add a 'max_entries' limit 2015-02-19 18:36:02 +00:00
Paul "LeoNerd" Evans 077d200342 Move @cached decorator out into synapse.storage._base; add minimal docs 2015-02-19 17:29:39 +00:00
Erik Johnston 0ac2a79faa Initial stab at implementing a batched get_missing_pdus request 2015-02-19 17:24:14 +00:00
Paul "LeoNerd" Evans 61959928bb Pull out the 'get_rooms_for_user' cache logic into a reüsable @cached decorator 2015-02-19 14:58:07 +00:00
Erik Johnston 2c29ed3e84 Use absolute path when loading delta sql files 2015-02-17 17:22:24 +00:00
Erik Johnston 1a989c436c Bump schema version 2015-02-17 15:45:55 +00:00
Erik Johnston fa6c93bd26 Merge branch 'consumeErrors' of github.com:matrix-org/synapse into develop 2015-02-17 15:18:17 +00:00
Erik Johnston 72a4de2ce6 Use consumeErrors=True on all DeferredLists.
This is so that the DeferredLists actually consume the error instead of
propogating down the non-existent errback chain. This should reduce the
number of unhandled errors we are seeing.
2015-02-17 10:07:01 +00:00
David Baker 64def4f953 Merge branch 'hotfixes-0.7.0e' into develop 2015-02-13 16:18:34 +00:00
David Baker 8d5cce62ab Update pushers by app id and pushkey, not user id and pushkey 2015-02-13 16:16:16 +00:00
Mark Haines 0d872f5aa6 Merge pull request #50 from matrix-org/application-services
Application Services
2015-02-13 15:06:14 +00:00
Mark Haines 183b3d4e47 Prepare the database whenever a connection is opened from the db_pool so that in-memory databases will work 2015-02-13 14:38:24 +00:00
Erik Johnston 6b186a57ba Merge branch 'fix' of github.com:matrix-org/synapse into hotfixes-v0.7.0a 2015-02-12 14:51:46 +00:00
Erik Johnston 717687e1fc Get an auth query one at a time 2015-02-12 14:39:31 +00:00
Erik Johnston 83d41f25d8 Set database schema version in delta 2015-02-12 10:05:47 +00:00
Erik Johnston 8bbdf32849 Convert get_rooms to use runInteraction so the transacion has a more helpful description 2015-02-11 18:56:13 +00:00
Erik Johnston 42bc56dad3 Merge branch 'develop' of github.com:matrix-org/synapse into use-simplejson 2015-02-11 17:01:38 +00:00
Erik Johnston 7eef84a95b pyflakes 2015-02-11 16:52:22 +00:00
Kegan Dougal f2fdcb7c4b Merge branch 'develop' into application-services 2015-02-11 16:43:26 +00:00
Paul "LeoNerd" Evans b164e0896c Merge branch 'bugs/SYN-264' into develop 2015-02-11 16:23:30 +00:00
Paul "LeoNerd" Evans 7f47ba7383 Added another TODO note 2015-02-11 16:18:21 +00:00
Paul "LeoNerd" Evans 45b56609ae Cache the result of a get_rooms_for_user query, to make user_rooms_intersect() much lighter in the read-common case 2015-02-11 16:04:08 +00:00
Paul "LeoNerd" Evans 7be0f6594e First step of making user_rooms_intersect() faster - implement in intersection logic in Python code terms of a DB query that is cacheable per user 2015-02-11 15:53:56 +00:00
Erik Johnston ddb816cf60 Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict 2015-02-11 15:44:28 +00:00
Mark Haines ddc25cf4e2 Invalidate the cache for an event if it is redacted 2015-02-11 15:23:28 +00:00
Mark Haines aff892ce79 Fix formatting 2015-02-11 15:02:35 +00:00
Mark Haines f5a70e0d2e Add a cache for get_event 2015-02-11 15:01:15 +00:00
Erik Johnston 4ebbaf0d43 Blunty replace json with simplejson 2015-02-11 14:23:10 +00:00
Kegan Dougal fd40d992ad PEP8-ify 2015-02-11 10:41:33 +00:00
Kegan Dougal 9978c5c103 Merge branch 'develop' into application-services 2015-02-11 10:03:24 +00:00
Mark Haines b085fac735 Code-style fixes 2015-02-10 16:30:48 +00:00
Mark Haines fda4422bc9 Fix pyflakes 2015-02-10 14:54:07 +00:00
Mark Haines d7c7efb691 Add performance counters for different stages of loading events 2015-02-10 14:50:53 +00:00
Mark Haines 0c4536da8f Use the transaction 'desc' rather than 'name', increment the txn_ids in
txn names
2015-02-09 18:06:31 +00:00
Mark Haines 347b497db0 Formatting 2015-02-09 17:57:09 +00:00
Mark Haines 3a5ad7dbd5 Performance counters for database transaction names 2015-02-09 17:55:56 +00:00
Kegan Dougal 53557fc532 Merge branch 'develop' into application-services 2015-02-09 15:20:56 +00:00
Erik Johnston c4ee4ce93e Fix typo 2015-02-09 15:00:37 +00:00
Erik Johnston 66fde49f07 Log database time every 10s and log as percentage 2015-02-09 14:45:15 +00:00
Erik Johnston 75656712e3 Time how long we're spending on the database thread 2015-02-09 14:22:52 +00:00
Kegan Dougal 5a7dd05818 Modify auth.get_user_by_req for authing appservices directly.
Add logic to map the appservice token to the autogenned appservice user ID.
Add unit tests for all forms of get_user_by_req (user/appservice,
valid/bad/missing tokens)
2015-02-09 14:14:15 +00:00
Kegan Dougal ac3183caaa Register a user account for the AS when the AS registers. Add 'sender' column to AS table. 2015-02-09 12:03:37 +00:00
Erik Johnston 0cd66885e3 Move delta/v13.sql to delta/v12.sql 2015-02-06 14:38:04 +00:00
Kegan Dougal a3c6010718 Add delta sql file. 2015-02-05 16:48:57 +00:00
Kegan Dougal fc8bcc809d Merge branch 'develop' into application-services 2015-02-05 15:32:45 +00:00
David Baker aaf50bf6f3 Give server default rules the 'default' attribute and fix various brokenness. 2015-02-05 15:11:38 +00:00
Kegan Dougal 951690e54d Merge branch 'develop' into application-services 2015-02-05 14:28:03 +00:00
Erik Johnston f08bd95880 Merge pull request #47 from matrix-org/signature_failures
Federation fixes.
2015-02-05 14:03:00 +00:00
Kegan Dougal 27091f146a Add hs_token column and generate a different token f.e application service. 2015-02-05 10:08:12 +00:00
Kegan Dougal 89f2e8fbdf Fix bug in store defer. Add more unit tests. 2015-02-04 15:21:03 +00:00
Kegan Dougal 17753f0c20 Add stub ApplicationServiceApi and glue it with the handler. 2015-02-04 11:19:18 +00:00
Erik Johnston 03d415a6a2 Brief comment on why we do some things on every call to persist_event and not others 2015-02-04 10:40:59 +00:00
Erik Johnston c0462dbf15 Rearrange persist_event so that do all the queries that need to be done before returning early if we have already persisted that event. 2015-02-04 10:16:51 +00:00
Erik Johnston 02be8da5e1 Add doc to get_event 2015-02-03 17:34:07 +00:00
David Baker dc7bb70f22 s/instance_handle/profile_tag/ 2015-02-03 16:51:07 +00:00
Kegan Dougal 94a5db9f4d Add appservice package and move ApplicationService into it. 2015-02-03 14:44:16 +00:00
Kegan Dougal f2c039bfb9 Implement restricted namespace checks. Begin fleshing out the main hook for notifying application services. 2015-02-03 13:29:27 +00:00
Kegan Dougal a060b47b13 Add namespace constants. Add restrict_to option to limit namespace checks. 2015-02-03 13:17:28 +00:00
Kegan Dougal 3bd2841fdb Everyone loves SQL typos 2015-02-03 11:37:52 +00:00
Kegan Dougal 197f3ea4ba Implement regex checks for app services.
Expose handler.get_services_for_event which manages the checks for all
services.
2015-02-03 11:26:33 +00:00
Erik Johnston e7ca813dd4 Try to ensure we don't persist an event we have already persisted. In persist_event check if we already have the event, if so then update instead of replacing so that we don't cause a bump of the stream_ordering. 2015-02-03 10:39:41 +00:00
Kegan Dougal 9ff349a3cb Add defers in the right places. 2015-02-02 17:42:49 +00:00
Kegan Dougal 1a2de0c5fe Implement txns for AS (un)registration. 2015-02-02 17:39:41 +00:00
Erik Johnston 941f59101b Don't fail an entire request if one of the returned events fails a signature check. If an event does fail a signature check, look in the local database and request it from the originator. 2015-02-02 16:56:01 +00:00
Kegan Dougal a006d168c5 Actually merge into develop. 2015-02-02 16:05:34 +00:00
Kegan Dougal c059c9fea5 Merge branch 'develop' into application-services
Conflicts:
	synapse/handlers/__init__.py
	synapse/storage/__init__.py
2015-02-02 15:57:59 +00:00
Erik Johnston 7d897f5bfc Merge pull request #43 from matrix-org/rejections
Rejections
2015-01-30 16:11:14 +00:00
Mark Haines 6dc92d3427 Merge pull request #41 from matrix-org/client_v2_sync
Client v2 sync
2015-01-30 15:54:25 +00:00
Erik Johnston 1bd540ef79 Merge branch 'develop' of github.com:matrix-org/synapse into rejections
Conflicts:
	synapse/storage/schema/im.sql
2015-01-30 15:16:38 +00:00
Mark Haines 9ec9d6f2cb Merge pull request #42 from matrix-org/replication_split
Replication split
2015-01-30 15:14:10 +00:00
Erik Johnston 91015ad008 Remove merge conflict 2015-01-30 14:58:54 +00:00
Erik Johnston 4f7fe63b6d Remember to add schema file to list 2015-01-30 14:57:53 +00:00
Erik Johnston fdd2ac495a Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
Conflicts:
	synapse/storage/__init__.py
2015-01-30 14:57:33 +00:00
Mark Haines 8bc3066e0b Merge branch 'client_v2_filter' into client_v2_sync 2015-01-30 14:57:04 +00:00
Erik Johnston 471c47441d Merge pull request #37 from matrix-org/client_v2_filter
Client v2 filter
2015-01-30 14:56:08 +00:00
Mark Haines e97f756a05 Use 'in' to test if the key exists, remove unused _filters_for_user 2015-01-30 14:54:06 +00:00
Erik Johnston 2f4cb04f45 Be more specific in naming columns in selects. 2015-01-30 14:48:11 +00:00
Erik Johnston 472cf532b7 Put CREATE rejections into seperate .sql 2015-01-30 14:48:03 +00:00
David Baker 322a047502 Add room member count condition and default rule to make a noise on rooms of only 2 people. 2015-01-30 14:46:45 +00:00
Erik Johnston c515d37797 Merge branch 'replication_split' of github.com:matrix-org/synapse into rejections
Conflicts:
	synapse/storage/schema/delta/v12.sql
2015-01-30 14:19:49 +00:00
Erik Johnston 84b78c3b5f Merge branch 'rejections_storage' of github.com:matrix-org/synapse into replication_split 2015-01-30 14:17:47 +00:00
Erik Johnston e0b7c521cb Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
Conflicts:
	synapse/storage/__init__.py
	synapse/storage/schema/delta/v12.sql
2015-01-30 14:08:28 +00:00
Mark Haines 8498d348d8 Fix token formatting 2015-01-30 11:42:09 +00:00
Mark Haines 22dd1cde2d Filter the recent events before applying the limit when doing an incremental sync with a gap 2015-01-30 11:32:35 +00:00
Erik Johnston 2ebf795c0a Merge branch 'develop' of github.com:matrix-org/synapse into rejections
Conflicts:
	synapse/storage/__init__.py
	synapse/storage/schema/delta/v12.sql
2015-01-30 11:10:37 +00:00
Erik Johnston 78015948a7 Initial implementation of auth conflict resolution 2015-01-29 16:52:33 +00:00
Mark Haines 8e571cbed8 Merge branch 'client_v2_filter' into client_v2_sync 2015-01-29 16:18:59 +00:00
Mark Haines 295322048d Merge branch 'develop' into client_v2_filter 2015-01-29 16:18:34 +00:00
Mark Haines acb68a39e0 Code style fixes. 2015-01-29 16:12:40 +00:00
Mark Haines 396a67a09a Merge branch 'client_v2_filter' into client_v2_sync
Conflicts:
	synapse/rest/client/v2_alpha/__init__.py
2015-01-29 14:58:00 +00:00
Mark Haines 9d8f798a3f Merge changes from develop 2015-01-29 14:55:27 +00:00
Mark Haines e4f50fa0aa Move bump schema delta 2015-01-29 14:53:18 +00:00
Mark Haines e016f4043b Use get_room_events_stream to get changes to the rooms if the number of changes is small 2015-01-29 14:40:28 +00:00
Kegan Dougal 3773759c0f Also edit the filter column on the delta SQL 2015-01-29 09:15:33 +00:00
Mark Haines b0b80074e0 SYN-252: Supply the stream and topological parts in the correct order to the constructor 2015-01-29 01:48:48 +00:00
David Baker d5bdf3c0c7 Allow the push rule delete method to take more specifiers. 2015-01-28 18:06:04 +00:00
Kegan Dougal 11634017f4 s/definition/filter_json/ since definition is now used to mean a component of the filter, rather than the complete json 2015-01-28 17:42:19 +00:00
Mark Haines 9c61556504 Merge branch 'develop' into client_v2_sync 2015-01-28 17:29:30 +00:00
Mark Haines 26c8fff19e Merge pull request #36 from matrix-org/device_id_from_access_token
Extract the device id and token id from the access token when autheniticating users
2015-01-28 17:19:28 +00:00
Mark Haines 388581e087 Extract the id token of the token when authing users, include the token and device_id in the internal meta data for the event along with the transaction id when sending events 2015-01-28 16:58:23 +00:00
Kegan Dougal c23e3db544 Add filter JSON sanity checks. 2015-01-28 16:45:18 +00:00
Erik Johnston 0ef5bfd6a9 Start implementing auth conflict res 2015-01-28 16:16:53 +00:00
David Baker 60b143a52e Move pushers delta to v12 and bump schema version 2015-01-28 15:48:28 +00:00
David Baker e78dd33292 Use %s instead of + 2015-01-28 14:52:58 +00:00
David Baker 93aac9bb7b Newline 2015-01-28 14:51:01 +00:00
David Baker 445ad9941e Redundant parens 2015-01-28 14:49:59 +00:00
David Baker 6d485dd1c7 unnecessary newlines 2015-01-28 14:48:42 +00:00
David Baker fb0928097a More magic commas (including the place I copied it from...) 2015-01-28 14:48:07 +00:00
David Baker 0cbb6b0f52 Google doc style 2015-01-28 14:44:41 +00:00
David Baker c291a4d522 Merge branch 'develop' into pushers
Conflicts:
	synapse/handlers/events.py
	synapse/server.py
2015-01-28 12:51:05 +00:00
Kegan Dougal 42876969b9 Add basic application_services SQL, and hook up parts of the appservice store to read from it. 2015-01-28 11:59:38 +00:00
Kegan Dougal b46fa8603e Remove unused import 2015-01-28 09:17:48 +00:00
Mark Haines e020574d65 Fix Formatting 2015-01-27 20:19:36 +00:00
Paul "LeoNerd" Evans 8398f19bce Created schema delta 2015-01-27 19:00:09 +00:00
Paul "LeoNerd" Evans 06cc147012 Initial stab at real SQL storage implementation of user filter definitions 2015-01-27 18:46:03 +00:00
Paul "LeoNerd" Evans 54e513b4e6 Move storage of user filters into real datastore layer; now have to mock it out in the REST-level tests 2015-01-27 17:48:13 +00:00
Kegan Dougal fbeaeb8689 Log when ASes are registered/unregistered. 2015-01-27 17:34:40 +00:00
Kegan Dougal ec3719b583 Use ApplicationService when registering. 2015-01-27 17:15:06 +00:00
Kegan Dougal 92171f9dd1 Add stub methods, TODOs and docstrings for application services. 2015-01-27 16:53:59 +00:00
Mark Haines a56008842b Start implementing incremental initial sync 2015-01-27 16:24:22 +00:00
Kegan Dougal 7331d34839 Add AS specific classes with docstrings. 2015-01-27 16:23:46 +00:00
Kegan Dougal 51449e0665 Add appservice handler and store. Glue together rest > handler > store. 2015-01-27 15:50:28 +00:00
David Baker 69a75b7ebe Add brackets to make get room name / alias work 2015-01-26 16:52:47 +00:00
Erik Johnston 7b88619241 Split up replication_layer module into client, server and transaction queue 2015-01-26 10:45:24 +00:00
David Baker 5f84ba8ea1 Add API to delete push rules. 2015-01-23 17:49:37 +00:00
Erik Johnston 9b1e552b51 Merge branch 'develop' of github.com:matrix-org/synapse into rejections 2015-01-23 15:51:48 +00:00
Erik Johnston 3b9cc882a5 Add storage method have_events 2015-01-23 15:42:52 +00:00
Mark Haines 5759bec43c Replace hs.parse_userid with UserID.from_string 2015-01-23 11:47:15 +00:00
David Baker bcd48b9636 Fix adding rules without before/after & add the rule that we couldn't find to the error 2015-01-23 10:28:25 +00:00
David Baker 8a850573c9 As yet fairly untested GET API for push rules 2015-01-22 19:32:17 +00:00
David Baker 673773b217 oops, this is not its own schema file 2015-01-22 18:27:07 +00:00
David Baker 7ecb49ef25 Insufficient newlines 2015-01-22 17:53:30 +00:00
David Baker 5c6189ea3e Merge branch 'develop' into pushers
Conflicts:
	synapse/rest/__init__.py
2015-01-22 17:46:16 +00:00
David Baker ede491b4e0 Oops: second part of commit dc938606 2015-01-22 17:38:53 +00:00
David Baker dc93860619 Add rest API & store for creating push rules
Also make unrecognised request error look more like synapse errors
because it makes it easier to throw them from within rest classes.
2015-01-22 17:37:12 +00:00
Erik Johnston b1b85753d7 Add support for storing rejected events in EventContext and data stores 2015-01-22 15:50:17 +00:00
David Baker afb714f7be add instance_handles to pushers so we have a way to refer to them even if the push token changes. 2015-01-20 11:49:48 +00:00
David Baker 2d2953cf5f Require device language when adding a pusher.
Because this seems like it might be useful to do sooner rather
than later.
2015-01-16 11:24:10 +00:00
David Baker 2ca2dbc821 Send room name and first alias in notification poke. 2015-01-15 16:56:18 +00:00
David Baker 2cb30767fa Honour the 'rejected' return from push gateways
Add a timestamp to push tokens so we know the last time they we
got them from the device. Send it to the push gateways so it can
determine whether its failure is more recent than the token.
Stop and remove pushers that have been rejected.
2015-01-13 19:48:37 +00:00
Mark Haines 3891597eb3 Remove unused functions 2015-01-13 15:57:26 +00:00
Mark Haines fda63064fc get_room_events isn't called anywhere 2015-01-13 14:43:26 +00:00
Mark Haines 895fcb377e Fix stream token ordering 2015-01-13 14:38:53 +00:00
David Baker c06a9063e1 Merge branch 'develop' into pushers 2015-01-13 13:15:51 +00:00
Kegan Dougal c43d898119 SYN-178: Fix off by one. 2015-01-12 17:38:40 +00:00
Erik Johnston 36a2a877e2 Use time.time() instead of time.clock() 2015-01-06 16:34:41 +00:00
Erik Johnston fd9a8db7ea Only fetch the columns we need. 2015-01-06 15:59:31 +00:00
Erik Johnston a01416cf21 Add delta and bump DB version 2015-01-06 15:42:18 +00:00
Erik Johnston f6da237c35 Add index on transaction_id to sent_transcations 2015-01-06 15:40:38 +00:00
Erik Johnston 9bd07bed23 Actually time that function 2015-01-06 15:28:56 +00:00
Erik Johnston 03a501456c Time how long calls to _get_destination_retry_timings take 2015-01-06 15:22:28 +00:00
Erik Johnston 52b2c6c9c7 Don't include None's in _get_events_txn 2015-01-06 14:56:57 +00:00
Erik Johnston 8a12df8cf3 Merge branch 'erikj-perf' of github.com:matrix-org/synapse into develop 2015-01-06 14:45:57 +00:00
Erik Johnston 96707ed718 Name 'user_rooms_intersect' transaction 2015-01-06 14:44:27 +00:00
Erik Johnston 76ec154e95 We don't need the full events for get_rooms_for_user_where_membership_is 2015-01-06 14:37:00 +00:00
Mark Haines 5e23a19204 Merge pull request #28 from matrix-org/erikj-perf
Database performance improvements.
2015-01-06 13:33:40 +00:00
Mark Haines adb04b1e57 Update copyright notices 2015-01-06 13:21:39 +00:00
Erik Johnston af1c7c7808 PEP8 2015-01-06 13:13:17 +00:00
Erik Johnston 12819d5082 Remove debug lines 2015-01-06 13:12:30 +00:00
Erik Johnston 52d8519008 Don't do batching when getting events. 2015-01-06 13:10:27 +00:00
Erik Johnston 98933e3db6 Only fetch prev_content when a client is streaming/paginating. Use transactions for event streams. 2015-01-06 13:03:23 +00:00
Erik Johnston 3e26720e05 Temporarily turn off 'redacted_because' and 'prev_content' keys 2015-01-06 11:26:58 +00:00
Erik Johnston f4ea78e9e2 More debug logging 2015-01-06 11:24:18 +00:00
Erik Johnston 753126b8cc Add some debug logging 2015-01-06 11:18:12 +00:00
Erik Johnston d7e8ea67b3 Reformat 2015-01-06 11:18:02 +00:00
Erik Johnston f0128f9600 Add RoomMemberStore.get_users_in_room, so that we can get the list of joined users without having to retrieve the full events 2015-01-06 10:55:43 +00:00
Erik Johnston d61109f578 Merge branch 'hotfixes-v0.6.0' of github.com:matrix-org/synapse into erikj-perf 2014-12-19 16:37:08 +00:00
Mark Haines 9c71d945d6 Look for name, topic in the event content rather than the event itself when persisting room name and topic events 2014-12-19 15:16:48 +00:00
David Baker b56730bb6e Merge branch 'develop' into pushers
Conflicts:
	synapse/api/errors.py
	synapse/server.py
	synapse/storage/__init__.py
2014-12-18 15:15:22 +00:00
David Baker afa953a293 schema version is now 10 2014-12-18 15:11:06 +00:00
David Baker 0a6664493a Merge branch 'master' into pushers 2014-12-18 15:06:11 +00:00
David Baker 4c7ad50f6e Thank you, pyflakes 2014-12-18 14:55:04 +00:00
David Baker 173264b656 ...and bump SCHEMA_VERSION 2014-12-18 14:53:10 +00:00
David Baker fc7c5e9cd7 Rename the pusher SQL delta to v9 which the next free one 2014-12-18 14:51:29 +00:00
David Baker 9728c305a3 after a few rethinks, a working implementation of pushers. 2014-12-18 14:49:22 +00:00
Erik Johnston f3788e3c78 Test some ideas that might help performance a bit 2014-12-17 23:37:08 +00:00
Erik Johnston dec5b62339 Use _get_events_txn instead of _parse_events_txn 2014-12-16 19:16:41 +00:00
Erik Johnston 2215faa361 Merge branch 'hotfixes-v0.5.4a' of github.com:matrix-org/synapse into release-v0.6.0 2014-12-16 19:11:13 +00:00
Erik Johnston 96779d2490 Fix bug where we did not send the full auth chain to people that joined over federation 2014-12-16 18:57:36 +00:00
Erik Johnston f76269392b Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.6.0
Conflicts:
	synapse/state.py
2014-12-16 18:35:46 +00:00
Erik Johnston 52f99243ab Use is_outlier() so that we don't get AttributeError 2014-12-16 18:33:50 +00:00
Erik Johnston 5b39cfff69 Don't assume an event exists 2014-12-16 18:25:24 +00:00
Mark Haines 56db465047 Merge branch 'release-v0.6.0' into develop 2014-12-16 17:29:49 +00:00
Mark Haines 627e4f01d2 Remove send_message since nothing was calling it. Remove Snapshot because only send_message was using it 2014-12-16 16:07:41 +00:00
Erik Johnston 42b725ce52 Fix upgrade script to run all the missing deltas. 2014-12-16 15:13:34 +00:00
Erik Johnston b3c793e362 Do run all deltas up to missing delta 10 2014-12-16 14:44:53 +00:00
Erik Johnston 2e44714214 Make failure to run appropraite upgrade scripts more helpful. 2014-12-16 14:20:32 +00:00
Erik Johnston ef5a141050 Bump database version 2014-12-16 13:57:47 +00:00
Erik Johnston 35f4f6b070 Update upgrade script 2014-12-16 13:27:53 +00:00
Erik Johnston 882dc8dcab Persist internal_metadata 2014-12-16 13:17:09 +00:00
Erik Johnston 3c77d13aa5 Kill off synapse.api.events.* 2014-12-16 11:29:05 +00:00
Erik Johnston f280929a12 Use frozenutils 2014-12-15 17:31:36 +00:00
Erik Johnston c8dd3314d6 Fix bug where we ignored event_edge_hashes table 2014-12-15 13:55:22 +00:00
Erik Johnston c39beb5559 Store json as UTF-8 and not bytes 2014-12-12 14:53:37 +00:00
Erik Johnston 75085bb4d1 Pyflakes 2014-12-12 14:34:34 +00:00
Erik Johnston cde840a82c Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor
Conflicts:
	setup.py
2014-12-11 17:48:48 +00:00
Erik Johnston 0b04369238 Fix public room joining by making sure replaces_state never points to itself. 2014-12-11 15:56:06 +00:00
Erik Johnston 9191292b0f Fix prev_content 2014-12-11 15:16:55 +00:00
Erik Johnston 8cdebce470 Fix redactions. Fix 'age' key 2014-12-11 13:25:19 +00:00
Erik Johnston 02e4c18171 Remove dead code 2014-12-10 18:00:36 +00:00
Mark Haines 4f37c0ea9d Merge branch 'develop' into media_repository 2014-12-10 16:55:06 +00:00
Mark Haines 61fc37e467 Merge branch 'develop' into media_repository 2014-12-10 16:14:17 +00:00
Erik Johnston 1d2a0040cf Fix bug where we clobbered old state group values 2014-12-10 15:55:03 +00:00
Mark Haines e5275d856e Get the code actually working 2014-12-10 15:46:18 +00:00
Mark Haines cc84d3ea78 Thumbnail uploaded and cached images 2014-12-10 15:40:52 +00:00
Erik Johnston 02db7eb209 Fix bug when uploading state with empty state_key 2014-12-10 14:02:48 +00:00
Erik Johnston aae8a37e63 Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor 2014-12-10 13:18:40 +00:00
Erik Johnston 95aa903ffa Try and figure out how and why signatures are being changed. 2014-12-10 11:37:47 +00:00
Erik Johnston b8d30899b1 Code style. 2014-12-10 10:16:09 +00:00
Matthew Hodgson 2b1acb7671 squidge to 79 columns as per pep8 2014-12-10 00:03:55 +00:00
Erik Johnston aa3f66cf7f Change the way we implement get_events to be less sucky 2014-12-09 13:35:26 +00:00
Matthew Hodgson 8529fba02d fix a million stupid bugs and make it actually work 2014-12-08 19:34:51 +00:00
Matthew Hodgson 0d3fa1ac6e add a write-through cache on the retry schedule 2014-12-08 17:48:57 +00:00
Erik Johnston ba3d1e2fc0 Remove unused import 2014-12-08 12:01:25 +00:00
Erik Johnston d044121168 Various typos and bug fixes. 2014-12-08 09:08:26 +00:00
Matthew Hodgson 5cd43d4b9f fix stupid syntax thinkos 2014-12-07 23:44:16 +00:00
Matthew Hodgson aed62a3583 track replication destination health, and perform exponential back-off when sending transactions. does *not* yet retry transactions, but drops them on the floor if waiting for a server to recover. 2014-12-07 02:26:07 +00:00
Mark Haines a953be097f Add a method field to thumbnail storage 2014-12-05 16:31:56 +00:00
Erik Johnston 6630e1b579 Start making more things use EventContext rather than event.* 2014-12-05 16:20:48 +00:00
Erik Johnston c31dba86ec Convert rest and handlers to use new event structure 2014-12-04 15:50:01 +00:00
Mark Haines c01fd5573c Implement download support for media_repository 2014-12-04 14:22:31 +00:00
Erik Johnston 5d7c9ab789 Begin converting things to use the new Event structure 2014-12-04 11:27:59 +00:00
David Baker 88af58d41d Update to app_id / app_instance_id (partially) and mangle to be PEP8 compliant. 2014-12-03 13:37:02 +00:00
Erik Johnston 6941a19715 Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor 2014-12-03 11:56:49 +00:00
Paul "LeoNerd" Evans 10eb8f070c Workaround for non-uniqueness of room member events in the database confusing HAVING COUNT() test of room membership intersection (with thanks to Tom Molesworth) 2014-12-02 21:36:53 +00:00
Mark Haines 2f804a7072 Fix pyflakes and pep8 warnings 2014-12-02 19:55:18 +00:00
Mark Haines 5da65085d1 Get uploads working with new media repo 2014-12-02 19:51:47 +00:00
Mark Haines 279c48c8b4 Write the upload portion of version 1 of the media repository 2014-12-02 17:13:14 +00:00
David Baker 7642d95d5e Merge branch 'develop' into pushers 2014-12-02 13:50:05 +00:00
Erik Johnston ec2b5d8c28 Store full JSON of events in db 2014-12-01 16:22:07 +00:00
Erik Johnston 1505055334 Don't return outliers when we get recent events for rooms. 2014-11-27 16:38:50 +00:00
Erik Johnston 4e2ffe79a4 Don't delete the entire current_state_events table 2014-11-26 15:17:08 +00:00
Erik Johnston cb76945688 Add update delta for schema change 2014-11-26 11:17:19 +00:00
Erik Johnston 87538711b6 Update schema to support multiple signatures 2014-11-26 11:14:30 +00:00
Erik Johnston 3598c11c8d Correctly handle the case where we get an event for an unknown room, which turns out we are actually in 2014-11-26 10:41:08 +00:00
Erik Johnston a46e5ef621 SYN-163: Add an order by rowid to selects.
This should fix the bug where the edges of the graph get returned in a
different order than they were inserted in, and so no get_event no
longer returned the exact same JSON as was inserted. This meant that
signature checks failed.
2014-11-24 10:56:36 +00:00
David Baker eb6aedf92c More work on pushers. Attempt to do HTTP pokes. Not sure if the actual HTTP pokes work or not yet but the retry semantics are pretty good. 2014-11-21 12:21:00 +00:00
David Baker 23465a30b6 Merge branch 'develop' into pushers 2014-11-20 18:17:46 +00:00
Mark Haines db9ce032a4 Fix pep8 codestyle warnings 2014-11-20 17:26:36 +00:00
Mark Haines dfdda2c871 Use module loggers rather than the root logger. Exceptions caused by bad clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str' 2014-11-20 17:10:37 +00:00
Mark Haines 32090aee16 Add a few missing yields, Move deferred lists inside PreserveLoggingContext because they don't interact well with the logging contexts 2014-11-20 16:24:00 +00:00
David Baker 74c3879760 Start creating a module to do generic notifications (just prints them to stdout currently!) 2014-11-19 18:20:59 +00:00
Erik Johnston 5b46ce579b Bump version, changelog and upgrade.rst 2014-11-19 18:00:57 +00:00
Mark Haines 1731af3f29 SYN-104: When going backwards the end token should be before the last event 2014-11-18 16:45:06 +00:00
Mark Haines ae9c2ab165 SYN-149: Send join event immediately after the room create event 2014-11-18 15:29:48 +00:00
Mark Haines a5b88c489e Split out sending the room alias events from creating the alias so that we can do them in the right point when creating a room 2014-11-18 15:03:13 +00:00
Mark Haines cb4b6c844a Merge PDUs and Events into one object 2014-11-14 21:25:02 +00:00
Mark Haines 8c2b5ea7c4 Fix PDU and event signatures 2014-11-14 19:11:04 +00:00
Mark Haines e903c941cb Merge branch 'develop' into request_logging
Conflicts:
	setup.py
	synapse/storage/_base.py
	synapse/util/async.py
2014-11-14 11:16:50 +00:00
Erik Johnston f04b3d5042 Store all signatures on events rather than just dropping them 2014-11-12 17:02:34 +00:00
Erik Johnston e715741abc Update some of the docs in event_federation 2014-11-12 16:20:30 +00:00
Erik Johnston e24d5cb97d Document StateStore and use transactions 2014-11-12 14:33:48 +00:00
Erik Johnston 58c0ef90c9 Add indices to state group tables 2014-11-12 14:33:48 +00:00
Erik Johnston 6fea478d2e Fix bugs with invites/joins across federatiom.
Both in terms of auth and not trying to fetch missing PDUs for invites,
joins etc.
2014-11-12 11:24:11 +00:00
Erik Johnston 5ff0bfb81d Fix bug where we /always/ created a new state group 2014-11-11 14:16:41 +00:00
Erik Johnston a8e565eca8 Add an EventValidator. Fix bugs in auth ++ storage 2014-11-10 18:25:42 +00:00
Erik Johnston cdc1b5d629 Fix regression where we did not return redacted events. 2014-11-10 15:21:30 +00:00
Erik Johnston 5d439b127b PEP8 2014-11-10 13:46:44 +00:00
Erik Johnston 6447db063a Fix backfill to work. Add auth to backfill request 2014-11-10 11:59:51 +00:00
Erik Johnston 65f846ade0 Notify users about invites. 2014-11-10 11:15:02 +00:00
Erik Johnston 6cb6cb9e69 Tidy up some of the unused sql tables 2014-11-10 10:31:00 +00:00
Erik Johnston 1c06806f90 Finish redaction algorithm. 2014-11-10 10:21:32 +00:00
Erik Johnston 02c3b1c9e2 Add '/event_auth/' federation api 2014-11-07 15:35:53 +00:00
Erik Johnston 16a0815fac Fix bug in _get_auth_chain_txn 2014-11-07 11:21:20 +00:00
Erik Johnston 49948d72f3 Fix joining over federation 2014-11-07 10:53:38 +00:00
Erik Johnston 8b0e96474b Implement method to get auth_chain from a given event_id 2014-11-07 10:53:38 +00:00
Erik Johnston bf6b72eb55 Start implementing auth chains 2014-11-07 10:53:38 +00:00
Erik Johnston 351c64e99e Amalgamate all power levels.
Remove concept of reqired power levels, something similiar can be done
using the new power level event.
2014-11-06 16:59:13 +00:00
Erik Johnston 4317c8e583 Implement new replace_state and changed prev_state
`prev_state` is now a list of previous state ids, similiar to
prev_events. `replace_state` now points to what we think was replaced.
2014-11-06 15:10:55 +00:00
Erik Johnston cc44ecc62f Get correct prev_events 2014-11-05 13:23:35 +00:00
Erik Johnston aa76bf39ab Remove unused imports 2014-11-04 14:14:02 +00:00
Erik Johnston d59aa6af25 For now, don't store txn -> pdu mappings. 2014-11-03 11:35:19 +00:00
Erik Johnston f139c02e95 Formatting 2014-11-03 11:34:49 +00:00
Erik Johnston ecabff7eb4 Sign evnets 2014-10-31 17:08:52 +00:00
Erik Johnston 80b2710e6f Remove unused signature storage methods 2014-10-31 17:08:36 +00:00
Erik Johnston fb3a01fa3a Remove unused sql file. 2014-10-31 16:04:27 +00:00
Erik Johnston d30d79b5be Make prev_event signing work again. 2014-10-31 15:35:39 +00:00
Erik Johnston 2f39dc19a2 Remove more references to dead PDU tables 2014-10-31 14:27:14 +00:00
Erik Johnston bfa36a72b9 Remove PDU tables. 2014-10-31 14:00:32 +00:00
Erik Johnston 21fe249d62 Actually don't store any PDUs 2014-10-31 10:47:34 +00:00
Erik Johnston 841df4da71 Don't store any PDUs 2014-10-31 09:59:59 +00:00
Erik Johnston f2de2d644a Move the impl of backfill to use events. 2014-10-31 09:59:02 +00:00
Erik Johnston ef9c4476a0 Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization 2014-10-30 11:18:28 +00:00
Mark Haines 7c06399512 Merge branch 'develop' into request_logging
Conflicts:
	synapse/config/logger.py
2014-10-30 11:13:58 +00:00
Mark Haines 7d709542ca Fix pep8 warnings 2014-10-30 11:10:17 +00:00
Erik Johnston aa80900a8e Fix SQL so that accepts we may want to persist events twice. 2014-10-30 10:11:06 +00:00
Mark Haines b29517bd01 Add a request-id to each log line 2014-10-30 01:21:33 +00:00
Erik Johnston e7858b6d7e Start filling out and using new events tables 2014-10-29 16:59:24 +00:00
Erik Johnston a10c2ec88d Don't reference PDU when persisting event 2014-10-28 17:15:32 +00:00
Erik Johnston 2d1dfb3b34 Begin implementing all the PDU storage stuff in Events land 2014-10-28 16:42:35 +00:00
Erik Johnston da1dda3e1d Add transaction level logging and timing information. Add a _simple_delete method 2014-10-28 11:18:04 +00:00
Erik Johnston 967ce43b59 Clean up LoggingTransaction 2014-10-28 10:53:11 +00:00
Erik Johnston 8e358ef35a Add timer to LoggingTransaction 2014-10-28 10:34:05 +00:00
Erik Johnston ad9226eeec Merge branch 'event_signing' of github.com:matrix-org/synapse into federation_authorization
Conflicts:
	synapse/storage/__init__.py
2014-10-27 11:58:32 +00:00
Mark Haines 5e2236f9ff fix pyflakes warnings 2014-10-27 11:19:15 +00:00
Mark Haines acb2d171e8 Merge branch 'develop' into event_signing 2014-10-27 11:14:11 +00:00
Mark Haines d56e389a95 Fix pyflakes warnings 2014-10-27 10:33:17 +00:00
Erik Johnston bb4a20174c Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization
Conflicts:
	synapse/federation/transport.py
	synapse/handlers/message.py
2014-10-27 10:20:44 +00:00
Mark Haines db2e350e29 Wrap preparing the database in a transaction. Otherwise it will take many seconds to complete because sqlite will create a transaction per statement 2014-10-24 19:04:26 +01:00
Erik Johnston 5662be894e Bump database version number. 2014-10-17 20:26:18 +01:00
Erik Johnston 5ffe5ab43f Use state groups to get current state. Make join dance actually work. 2014-10-17 18:56:42 +01:00
Mark Haines dc3c2823ac Merge branch 'develop' into event_signing
Conflicts:
	synapse/federation/replication.py
2014-10-17 17:33:58 +01:00
Mark Haines 82c5820767 keep 'origin_server_ts' as 'ts' in the database to avoid needlessly updating schema 2014-10-17 17:31:48 +01:00
Mark Haines f5cf7ac25b SPEC-7: Rename 'ts' to 'origin_server_ts' 2014-10-17 17:12:25 +01:00
Mark Haines c8f996e29f Hash the same content covered by the signature when referencing previous PDUs rather than reusing the PDU content hashes 2014-10-17 11:40:35 +01:00
Mark Haines bb04447c44 Include hashes of previous pdus when referencing them 2014-10-16 23:25:12 +01:00
Mark Haines 66104da10c Sign outgoing PDUs. 2014-10-16 00:09:48 +01:00
Mark Haines 1c445f88f6 persist hashes and origin signatures for PDUs 2014-10-15 17:09:04 +01:00
Erik Johnston e7bc1291a0 Begin making auth use event.old_state_events 2014-10-15 16:06:59 +01:00
Erik Johnston 80472ac198 Add missing package storate.state 2014-10-15 10:04:55 +01:00
Erik Johnston 5fefc12d1e Begin implementing state groups. 2014-10-14 16:59:51 +01:00
Mark Haines 3dac27a8a9 Storage for pdu signatures 2014-10-14 14:58:31 +01:00
Mark Haines 07639c79d9 Respond with more helpful error messages for unsigned requests 2014-10-13 16:39:15 +01:00
Mark Haines 9605593d11 Merge branch 'develop' into server2server_signing
Conflicts:
	synapse/storage/__init__.py
	tests/rest/test_presence.py
2014-09-30 17:55:06 +01:00
Mark Haines b95a178584 SYN-75 Verify signatures on server to server transactions 2014-09-30 15:15:10 +01:00
Erik Johnston fbf6320614 pyflakes cleanup 2014-09-30 12:38:38 +01:00
Erik Johnston e06adc6d7e SYN-2: Allow server admins to delete room aliases 2014-09-30 11:31:42 +01:00
Erik Johnston 7151615260 Update docstring 2014-09-29 15:35:54 +01:00
Erik Johnston 1550ab9e2f SYN-48: Delete dead code 2014-09-29 15:04:47 +01:00
Erik Johnston 3ccb17ce59 SYN-48: Implement WHOIS rest servlet 2014-09-29 14:59:52 +01:00
Erik Johnston c65306f877 Add auth check to test if a user is an admin or not. 2014-09-29 13:35:38 +01:00
Erik Johnston f7d80930f2 SYN-48: Track User-Agents as well as IPs for client devices. 2014-09-29 13:35:15 +01:00
Erik Johnston 0fdf308874 Track the IP users connect with. Add an admin column to users table. 2014-09-26 16:36:24 +01:00
Erik Johnston 37bfe44046 Merge branch 'deletions' of github.com:matrix-org/synapse into develop 2014-09-25 17:02:53 +01:00
Erik Johnston c818aa13eb Add LIMIT to scalar subquery 2014-09-25 15:51:21 +01:00
Erik Johnston ba87eb6753 Fix bug where we tried to insert state events with null state key 2014-09-25 14:45:27 +01:00
Erik Johnston 70899d3ab2 Rename deletions to redactions 2014-09-24 15:27:59 +01:00
Erik Johnston 7d9a84a445 Make deleting deletes not undelete 2014-09-24 14:18:08 +01:00
Erik Johnston 1e6c5b205c Fix bug where we didn't correctly pull out the event_id of the deletion 2014-09-24 13:29:20 +01:00
Erik Johnston 4354590a69 Add v4 deltas to current sql. 2014-09-24 11:06:41 +01:00
Erik Johnston efea61dc50 Rename 'pruned' to 'pruned_because' 2014-09-23 17:40:58 +01:00
Erik Johnston bc250a6afa SYN-12: Implement auth for deletion by adding a 'delete_level' on the ops levels event
SYN-12 # comment Auth has been added.
2014-09-23 17:36:24 +01:00
Erik Johnston b99f6eb904 Make sure we don't persist the 'pruned' key 2014-09-23 15:29:27 +01:00
Erik Johnston 78af6bbb98 Add m.room.deletion. If an event is deleted it will be returned to clients 'pruned', i.e. all client specified keys will be removed. 2014-09-23 15:28:32 +01:00
Erik Johnston 231afe464a Add a deletions table 2014-09-22 13:42:52 +01:00
Erik Johnston 28bcd01e8d SYN-47: Fix bug where we still returned events for rooms we had left.
SYN-47 #resolve
2014-09-19 14:45:21 +01:00
Paul "LeoNerd" Evans 4571cf7baa Merge branch 'develop' into test-sqlite-memory 2014-09-17 18:27:47 +01:00
Erik Johnston 59516a8bb1 Correctly handle receiving 'missing' Pdus from federation, rather than just discarding them. 2014-09-15 16:40:44 +01:00
Paul "LeoNerd" Evans b0406b9ead Merge remote-tracking branch 'origin/develop' into test-sqlite-memory 2014-09-15 14:15:10 +01:00
Erik Johnston 5bd9369a62 Correctly handle the 'age' key in events and pdus 2014-09-15 13:26:11 +01:00
Erik Johnston 14975ce5bc Fix bug where we relied on the current_state_events being updated when we are handling type specific persistence 2014-09-12 17:57:02 +01:00
Erik Johnston 667e747ed1 Fix bug where we no longer stored user_id on Pdus 2014-09-12 17:56:21 +01:00
Paul "LeoNerd" Evans 1c51c8ab7d Merge remote-tracking branch 'origin/develop' into test-sqlite-memory
Conflicts:
	synapse/storage/pdu.py
2014-09-12 17:20:06 +01:00
Erik Johnston 39e3fc69e5 Make the state resolution use actual power levels rather than taking them from a Pdu key. 2014-09-12 17:11:09 +01:00
Erik Johnston b42fe05c51 Fix bug where we incorrectly removed a remote host from the list of hosts in a room when any user from that host left that room even if they weren't the last user from that host in that room 2014-09-12 17:11:09 +01:00
Erik Johnston ca1ae7cf9b Fix bug where we didn't return a tuple when expected. 2014-09-12 17:11:09 +01:00
Paul "LeoNerd" Evans aa525e4a63 More accurate docs / clearer paramter names in RoomMemberStore 2014-09-12 16:43:49 +01:00
Paul "LeoNerd" Evans a87eac4308 Revert recent changes to RoomMemberStore 2014-09-12 15:51:51 +01:00
Paul "LeoNerd" Evans a840ff8f3f Now don't need the other logger.debug() call in _execute 2014-09-12 14:38:27 +01:00
Paul "LeoNerd" Evans 1c20249884 Logging of all SQL queries via the 'synapse.storage.SQL' logger 2014-09-12 14:37:55 +01:00
Paul "LeoNerd" Evans e53d77b501 Add a .runInteraction() method on SQLBaseStore itself to wrap the .db_pool 2014-09-12 14:28:07 +01:00
Paul "LeoNerd" Evans 249e8f2277 Add a better _store_room_member_txn() method that takes separated fields instead of an event object; also add FIXME comment about a big bug in the logic 2014-09-11 18:52:35 +01:00
Paul "LeoNerd" Evans aaf9ab68c6 Rename _store_room_member_txn to _store_room_member_from_event_txn so we can create another, more sensible function of that name 2014-09-11 18:44:04 +01:00
Paul "LeoNerd" Evans 55397f6347 prepare_database() on db_conn, not plain name, so we can pass in the connection from outside 2014-09-10 16:23:58 +01:00
Paul "LeoNerd" Evans ce55a8cc4b Move database preparing code out of homserver.py into storage where it belongs 2014-09-10 15:42:15 +01:00
Erik Johnston a75f8686ba Fix bug where we used an unbound local variable if we ended up rolling back the persist_event transaction 2014-09-09 16:27:59 +01:00
Erik Johnston e062f2dfa8 Apparently we can't do txn.rollback(), so raise and catch an exception instead. 2014-09-08 22:37:19 +01:00
Erik Johnston 83ce57302d Fix bug in state handling where we incorrectly identified a missing pdu. Update tests to catch this case. 2014-09-08 19:50:59 +01:00
Erik Johnston c0577ea87a Rollback if we try and insert duplicate events 2014-09-08 18:34:18 +01:00
Erik Johnston 781ff713ba When getting a state event also include the previous content 2014-09-06 02:23:36 +01:00
Erik Johnston 9dd4570b68 Generate m.room.aliases event when the HS creates a room alias 2014-09-05 21:35:56 +01:00
Paul "LeoNerd" Evans d703e712f7 Add support to _simple_insert() to do INSERT OR REPLACE 2014-09-03 18:18:41 +01:00
Matthew Hodgson 8a7c1d6a00 fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch. 2014-09-03 17:31:57 +01:00
Paul "LeoNerd" Evans 756e171ad0 Store SQL DDL deltas as well; attempt to upgrade the database on startup if it's too old 2014-09-03 14:14:39 +01:00
Paul "LeoNerd" Evans da3e04df8b Rename the 'do_users_share_a_room' to something slightly less verb-sounding 2014-09-03 10:40:48 +01:00
Mark Haines c6eafdfbaf Add copyright notices and fix pyflakes errors 2014-09-03 09:43:11 +01:00
Erik Johnston 4b8244fbf8 Fix bug where we didn't correctly store the ops power levels event. 2014-09-02 12:12:14 +01:00
Erik Johnston f452899fe2 Merge branch 'develop' of github.com:matrix-org/synapse into room_config 2014-09-01 19:57:28 +01:00
Erik Johnston b8ab9f1c0a Add all the necessary checks to make banning work. 2014-09-01 18:24:56 +01:00
Erik Johnston db7109c43b Add beginnings of ban support. 2014-09-01 16:15:34 +01:00
Mark Haines 9ea1de432d Fix homeserver config parsing 2014-09-01 15:51:15 +01:00
Erik Johnston 865469f233 Implement power level lists, default power levels and send_evnet_level/add_state_level events. 2014-09-01 14:39:09 +01:00
Mark Haines 3eb45eba0e Merge branch 'develop' into server2server_tls 2014-08-31 16:08:20 +01:00
Erik Johnston 1118f02689 Start adding storage for new events. 2014-08-29 15:18:30 +01:00
Erik Johnston ee06023573 Get the equalities right. 2014-08-29 13:28:06 +01:00
Mark Haines d7ae9b90a0 Add store for server certificates and keys 2014-08-28 18:19:47 +01:00
Paul "LeoNerd" Evans 2c7c12bc6e Initial room event stream token must be s0, not s1, or everyone will miss the very first room event 2014-08-28 17:39:34 +01:00
Mark Haines 2aeaa7b77c Merge branch 'develop' into storage_transactions
Conflicts:
	synapse/handlers/room.py
	synapse/storage/stream.py
2014-08-27 17:15:58 +01:00
Mark Haines f54b70520a Return the store_id from persist_event 2014-08-27 17:03:45 +01:00
Mark Haines b30358f439 add _get_room_member, fix datastore methods 2014-08-27 16:51:54 +01:00
Mark Haines a03c7f27a8 Fill out prev_events before calling persist_event 2014-08-27 14:32:19 +01:00
Erik Johnston 47519cd8c2 Merge branch 'develop' of github.com:matrix-org/synapse into stream_refactor
Conflicts:
	synapse/handlers/events.py
	synapse/rest/events.py
	synapse/rest/room.py
2014-08-27 14:13:06 +01:00
Mark Haines 474dcecb11 Remove unused populate_previous_pdus 2014-08-27 11:34:31 +01:00
Mark Haines bf05218c4b Merge branch 'develop' into storage_transactions 2014-08-27 11:19:37 +01:00
Kegan Dougal c65885e166 Added support for GET /events/$eventid with auth checks. 2014-08-27 10:33:01 +01:00
Mark Haines d2798de660 Fold federation/handler into handlers/federation 2014-08-26 19:49:42 +01:00
Erik Johnston 93cff1668c Merge branch 'develop' of github.com:matrix-org/synapse into stream_refactor 2014-08-26 18:57:55 +01:00
Erik Johnston 3a2a5b959c WIP: Completely change how event streaming and pagination work. This reflects the change in the underlying storage model. 2014-08-26 18:57:46 +01:00
Mark Haines 64e2a5d58e Move pdu and event persistence into a single persist_event function 2014-08-26 18:01:36 +01:00
Kegan Dougal 5a3df1d029 Feedback: Removed FeedbackRestServlet. Modified keys on FeedbackEvent. Expanded the feedback constants to fully explain what type of feedback they are. 2014-08-26 17:49:46 +01:00
Mark Haines 4b63b06cad Merge branch 'develop' into storage_transactions
Conflicts:
	synapse/api/auth.py
	synapse/handlers/room.py
	synapse/storage/__init__.py
2014-08-26 16:07:05 +01:00
Mark Haines ac21dfff6d Fix pyflakes errors 2014-08-26 16:02:59 +01:00
Mark Haines 32347bfcc9 fix a few pyflakes errors 2014-08-26 16:01:29 +01:00
Mark Haines 4b2ad549d5 Move the event storage into a single transaction 2014-08-26 14:36:03 +01:00
Erik Johnston a664ec20e0 Add a do_users_share_a_room method and use that in the presence handler. 2014-08-26 13:38:26 +01:00
Kegan Dougal 47c3a089c5 Merge branch 'develop' of github.com:matrix-org/synapse into client_server_url_rename 2014-08-26 09:26:33 +01:00
Kegan Dougal cab3095803 Removed member list servlet: now using generic state paths. 2014-08-26 09:26:07 +01:00
Erik Johnston be6abdff19 Order 'get_recent_events_for_room' correctly. 2014-08-26 09:22:58 +01:00
Erik Johnston 9d86c8c7a6 Add a unique constraint on the room hosts table 2014-08-24 11:29:29 +01:00
Erik Johnston 537ecd4e99 Turn off spammy logging 2014-08-22 18:12:38 +01:00
Erik Johnston 9f514915af Add indices to schema 2014-08-22 18:03:39 +01:00
Mark Haines 1379dcae6f Take a snapshot of the state of the room before performing updates 2014-08-22 17:00:10 +01:00
Erik Johnston 808f663ed1 Don't return state event outlier's when paginating. 2014-08-22 13:06:07 +01:00
Erik Johnston e7ee0b9fc1 Change IM sync api to also return the current presence list. 2014-08-21 16:40:21 +01:00
Erik Johnston 063e1b22e6 Stop internal keys from getting into SynapseEvents 2014-08-21 15:06:00 +01:00
Erik Johnston ebd3c41ede Make event stream storage return all membership events about the user, regardless of if they were in the room or not. 2014-08-20 17:09:44 +01:00
Erik Johnston e8244c23ba Give the event_id of the failed event 2014-08-20 15:53:07 +01:00
Erik Johnston d4fb1c8a92 Only hit get_room_events_stream if we have a valid user_id 2014-08-19 17:18:19 +01:00
Erik Johnston ae493c9418 Fix token to correct format 2014-08-19 16:45:55 +01:00
Erik Johnston eea2dc7dde Remove debug logging from token parsing funcs. 2014-08-19 16:40:38 +01:00
Erik Johnston d94765999d Add comment about what strorage.stream does 2014-08-19 16:40:25 +01:00
Erik Johnston 840771190f Fix bug where we sometimes set min_token to None. 2014-08-19 14:32:47 +01:00
Erik Johnston 75b6d982a0 Add a 'backfill room' button 2014-08-19 14:20:03 +01:00
Erik Johnston 598a1d8ff9 Change the way pagination works to support out of order events. 2014-08-19 14:19:48 +01:00
Erik Johnston 1422a22970 Fix typos in SQL and where we still had rowid's (which no longer exist) 2014-08-18 16:25:18 +01:00
Erik Johnston 4eb8f84aa8 Make snapshot_all_rooms return results in the correct form, including start and end tokens. 2014-08-18 16:20:21 +01:00
Erik Johnston 709a92cee8 SQL doesn't allow AUTOINCREMENT on non PRIMARY KEY columns. 2014-08-18 16:00:46 +01:00
Erik Johnston fc26275bb3 Add two different columns for ordering the events table, one which can be used for pagination and one which can be as tokens for notifying clients. Also add a 'processed' field which is currently always set to True 2014-08-18 15:50:41 +01:00
Erik Johnston 1c2caacd67 Merge branch 'master' of github.com:matrix-org/synapse into sql_refactor 2014-08-18 11:14:57 +01:00
Matthew Hodgson 0f9b633af7 retcols is mandatory. i have no idea how this could ever have worked? 2014-08-16 01:46:35 +01:00
Erik Johnston 0e938b1ff7 Rename method name to not clash with other ones in storage. 2014-08-15 16:47:48 +01:00
Erik Johnston 506711749f We no longer need to special case room config events. 2014-08-15 16:45:16 +01:00
Erik Johnston d260a42ca2 PEP8 cleanups 2014-08-15 16:17:36 +01:00
Erik Johnston 8fa3cc37f9 Comment. 2014-08-15 16:11:25 +01:00
Erik Johnston cd2967d271 Fix bug when generating a key when get_room_events_stream returned zero rows 2014-08-15 16:05:46 +01:00
Erik Johnston 86be66c34e Actually use MAX_STREAM_SIZE constant. 2014-08-15 16:04:54 +01:00
Erik Johnston 8d1f763209 Fix pagination to work with new db schema 2014-08-15 15:53:06 +01:00
Paul "LeoNerd" Evans 3c532314ec Fix imsync's SELECT query to only find the rooms I'm actually joined in, not every room I have ever joined 2014-08-15 15:44:53 +01:00
Erik Johnston 01f089d9fb Correctly return new token when returning events. Serialize events correctly. 2014-08-15 15:28:54 +01:00
Erik Johnston 114984a236 Start chagning the events stream to work with the new DB schema 2014-08-15 13:58:28 +01:00
Erik Johnston d72f897f07 Merge branch 'master' of github.com:matrix-org/synapse into sql_refactor
Conflicts:
	synapse/storage/stream.py
2014-08-15 11:50:14 +01:00
Erik Johnston 5002efa31b Reimplement the get public rooms api to work with new DB schema 2014-08-15 10:26:35 +01:00
Erik Johnston 2c46bb6208 Fix up typos and correct sql queries 2014-08-14 18:40:50 +01:00
Erik Johnston 7e681ad778 Update StreamStore 2014-08-14 18:01:39 +01:00
Erik Johnston 661c711765 Start fixing places that use the data store. 2014-08-14 17:34:37 +01:00
Erik Johnston 78b501eba6 Fix typo 2014-08-14 17:09:28 +01:00
Erik Johnston 2529f2bc01 Rename _execute_query 2014-08-14 16:58:51 +01:00
Paul "LeoNerd" Evans 93a8be7bef We really don't need debug logging of all the SQL statements we execute; we're quite happy these all work now 2014-08-14 16:15:57 +01:00
Kegan Dougal fb93e14e53 Be more helpful when failing to register/login, stating why (communication error, user in user, wrong credentials, etc). Make the HS send M_USER_IN_USE. 2014-08-14 16:03:04 +01:00
Erik Johnston 937c175029 Fix up RoomMemberStore to work with the new schema. 2014-08-14 16:02:10 +01:00
Erik Johnston 6d6a1c3454 Actually encode dicts as json in the DB 2014-08-14 14:30:25 +01:00
Erik Johnston 10294b6082 Merge branch 'master' of github.com:matrix-org/synapse into sql_refactor
Conflicts:
	synapse/storage/_base.py
2014-08-14 10:01:04 +01:00
Paul "LeoNerd" Evans d05aa651f8 An initial hack at storing presence state-change mtimes in database and presenting age durations to clients/federation events 2014-08-13 19:19:15 +01:00
Paul "LeoNerd" Evans baf04be5cf Set datastore's .hs field in SQLBaseStore rather than in the toplevel DataStore mixed-in result class 2014-08-13 19:19:15 +01:00
Matthew Hodgson 59dfbaba3b when we're talking about backfilling data in federation, call it backfilling - not pagination. 2014-08-13 18:17:09 +01:00
Erik Johnston beaf4384d9 Make feedback table also store sender. 2014-08-13 18:03:41 +01:00
Erik Johnston 336987bb8d Initial stab at refactoring the SQL tables, including rejigging some of the storage layer. 2014-08-13 18:03:41 +01:00
Erik Johnston 3dfa84bec8 Convert im schema to a 'one' table structure 2014-08-13 18:03:41 +01:00
Kegan Dougal 9b3702c924 Return the room_alias when GETing public rooms. 2014-08-13 11:57:49 +01:00
Matthew Hodgson f98e6380f1 add in whitespace after copyright statements to improve legibility 2014-08-13 03:14:34 +01:00
matrix.org 4f475c7697 Reference Matrix Home Server 2014-08-12 15:10:52 +01:00