OpenCloud
/
MatrixSynapse
mirror of https://github.com/matrix-org/synapse
1
0
Fork 0
Code Issues Releases Wiki Activity
3,296 Commits
842 Branches
671 Tags
284 MiB
Commit Graph

1912 Commits (0ec346d942450ac4a29c74d115428fd856af94b2)

Author SHA1 Message Date
Paul "LeoNerd" Evans ff1fa0fbf8 Add another @cached wrapper, this time on get_presence_state() 2015-03-27 15:57:16 +00:00
Paul "LeoNerd" Evans a198894bf7 Appease pep8 2015-03-26 11:53:58 +00:00
Kegan Dougal 32206dde3f Fixes from PR comments 2015-03-26 10:11:52 +00:00
Kegan Dougal 4edcbcee3b Merge branch 'develop' into application-services-txn-reliability 
Conflicts:
	synapse/storage/__init__.py
 2015-03-26 10:07:59 +00:00
Paul "LeoNerd" Evans 953e40f9dc Implement the main getEvent cache using Cache() instead of a custom application of LruCache; also unify its two-level structure into just one 2015-03-25 19:12:16 +00:00
Paul "LeoNerd" Evans f173d40a32 Use FrozenEvent's reject_reason to decide whether to return it; don't include allow_rejected in the main getEvents cache key 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans 1b988b051b Store the rejected reason in (Frozen)Event structs 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans 033a517feb Indirect invalidations of _get_event_cache via a helper method to keep all uses of the cache lexically within one .py file 2015-03-25 19:06:05 +00:00
Paul "LeoNerd" Evans 9ba6487b3f Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict() 2015-03-25 19:05:34 +00:00
Paul "LeoNerd" Evans d6b3ea75d4 Implement the 'key in dict' test for LruCache() 2015-03-25 19:04:59 +00:00
Erik Johnston 0e8f5095c7 Fix unicode database support 2015-03-25 17:15:20 +00:00
Erik Johnston 0ba393924a Escape non printing ascii character 2015-03-24 16:31:52 +00:00
Erik Johnston f488293d96 Don't reinsert into event_edges 2015-03-24 16:20:26 +00:00
Erik Johnston 1aa44939fc Fix bugs in transactions storage 2015-03-24 16:20:05 +00:00
Erik Johnston 5a447098dd Don't use room hosts table 2015-03-24 16:19:24 +00:00
Erik Johnston 9e98f1022a Don't order by rowid 2015-03-24 16:19:01 +00:00
Erik Johnston 9115421ace Use _simple_upsert 2015-03-24 16:17:39 +00:00
Paul "LeoNerd" Evans ed008e85a8 Reduce activity timer granularity to avoid too many quick updates (SYN-247) 2015-03-23 17:25:50 +00:00
Erik Johnston 6e7131f02f Remove uses of REPLACE and ON CONFLICT IGNORE to make the SQL more portable. 2015-03-23 15:38:56 +00:00
Erik Johnston 9a7f496298 Sanitize RoomMemberStore 2015-03-23 15:29:04 +00:00
Erik Johnston d5272b1d2c Use 'update or insert' rather than on 'conflict replace' 2015-03-23 14:02:34 +00:00
Erik Johnston 278149f533 Sanitize TransactionStore 2015-03-23 13:43:21 +00:00
Paul "LeoNerd" Evans 72d8406409 Put a cache on get_aliases_for_room 2015-03-20 19:21:13 +00:00
Paul "LeoNerd" Evans a63b4f7101 Remember the 'last seen' time for a given user/IP/device combination and only bother INSERTing another if it's stale 2015-03-20 18:25:49 +00:00
Paul "LeoNerd" Evans 0f86312c4c Pull out the cache logic from the @cached wrapper into its own class we can reuse 2015-03-20 18:25:42 +00:00
Paul "LeoNerd" Evans b1022ed8b5 func(*EXPR) is valid Python syntax, really... 2015-03-20 17:47:45 +00:00
Erik Johnston f6583796fe Merge branch 'develop' of github.com:matrix-org/synapse into mysql 2015-03-20 16:31:48 +00:00
Erik Johnston 80cd08c190 PEP8 2015-03-20 16:03:25 +00:00
Erik Johnston 9517f4da4d Merge branch 'develop' of github.com:matrix-org/synapse into store_rearrangement 2015-03-20 16:02:47 +00:00
Erik Johnston dc0c989ef4 Give sensible names for '_simple_...' transactions 2015-03-20 15:59:18 +00:00
Paul "LeoNerd" Evans ceb61daa70 Add the tiniest of tiny one-element caches to get_room_events_max_id() as it's read every time someone hits eventstream 2015-03-20 15:44:06 +00:00
Erik Johnston fce0114005 Start removing Tables 2015-03-20 15:05:44 +00:00
Erik Johnston 7e282a53a5 Tidy up _simple_... methods 2015-03-20 15:05:10 +00:00
Paul "LeoNerd" Evans 91cb46191d Allow @cached-wrapped functions to have more or fewer than 1 argument; assert on the total count of them though 2015-03-20 14:59:45 +00:00
Erik Johnston 87db64b839 Rearrange storage modules 2015-03-20 14:11:38 +00:00
Erik Johnston cb8162d3d1 Rearrange storage modules 2015-03-20 13:52:56 +00:00
Erik Johnston d288d273e1 Generate transaction id in code 2015-03-20 10:57:44 +00:00
Erik Johnston d4f50f3ae5 decode_result takes an iterable 2015-03-20 10:57:26 +00:00
Erik Johnston 455579ca90 Make database selection configurable 2015-03-20 10:55:55 +00:00
Erik Johnston 0d0610870d Fix up schemas some more 2015-03-20 10:55:31 +00:00
Erik Johnston 56f2d31676 Bump version 2015-03-19 17:48:33 +00:00
Erik Johnston c178e4e6ca Add missing servlet to list 2015-03-19 17:48:21 +00:00
Erik Johnston d7a0496f3e Convert storage layer to be mysql compatible 2015-03-19 15:59:48 +00:00
Erik Johnston 58ed393235 Remove redundant key 2015-03-19 15:12:05 +00:00
Erik Johnston fae059cc18 Fix up schemas to work with mariadb 2015-03-19 13:42:39 +00:00
Erik Johnston 0dcb145c7e Bump version 2015-03-19 11:26:03 +00:00
Erik Johnston 6cc046302f Bump version 2015-03-18 11:41:00 +00:00
Erik Johnston ed4d44d833 Merge pull request #109 from matrix-org/default_registration 
Disable registration by default. Add script to register new users.
 2015-03-18 11:38:52 +00:00
Erik Johnston f88db7ac0b Factor out user id validation checks 2015-03-18 11:34:18 +00:00
Erik Johnston 57976f646f Do more validation of incoming request 2015-03-18 11:30:04 +00:00
Erik Johnston bb24609158 Clean out event_forward_extremities table when the server rejoins the room 2015-03-18 11:19:47 +00:00
Erik Johnston 89036579ed Update schema to work with mariadb 2015-03-18 11:18:49 +00:00
Paul "LeoNerd" Evans 93978c5e2b @cached() annotate get_user_by_token() - achieves a minor DB performance improvement 2015-03-17 17:24:51 +00:00
Paul "LeoNerd" Evans 1489521ee5 Be polite and ensure we use @functools.wraps() when creating a function decorator 2015-03-17 17:19:22 +00:00
David Baker 6d33f97703 pep8 2015-03-17 11:53:55 +00:00
David Baker 7564dac8cb Wire up the webclient option 
It existed but was hardcoded to True.
Give it an underscore for consistency.
Also don't pull in syweb unless we're actually using the web client.
 2015-03-17 12:45:37 +01:00
Paul "LeoNerd" Evans 3f7a31d366 Add a DistributionMetric to HTTP request/response processing time in the server 2015-03-16 18:31:29 +00:00
Paul "LeoNerd" Evans be170b1426 Add a metric for the scheduling latency of SQL queries 2015-03-16 17:21:59 +00:00
Erik Johnston cd2539ab2a Merge pull request #110 from matrix-org/fix_ban 
Fix ban
 2015-03-16 15:36:52 +00:00
Kegan Dougal f0d6f724a2 Set the service ID as soon as it is known. 2015-03-16 15:24:32 +00:00
Erik Johnston f1d2b94e0b Copy dict of context.current_state before changing it. 2015-03-16 15:13:05 +00:00
Erik Johnston 857810d2dd Revert incorrect changes to where we send events 2015-03-16 15:12:47 +00:00
Kegan Dougal d04fa1f712 Implement ServiceQueuer with tests. 2015-03-16 14:03:16 +00:00
Erik Johnston c2c9471cba Don't block waiting on waking up all the listeners when sending an event. 2015-03-16 13:16:37 +00:00
Kegan Dougal 6279285b2a Replace EventGrouper for ServiceQueuer to move to push-based txns. Fix tests and add stub tests for ServiceQueuer. 2015-03-16 13:15:40 +00:00
Erik Johnston 8bad40701b Comment. 2015-03-16 13:13:07 +00:00
Erik Johnston 250e143084 Use 403 instead of 400 2015-03-16 13:11:42 +00:00
Erik Johnston b2e6ee5b43 Remove concept of context.auth_events, instead use context.current_state 2015-03-16 13:06:23 +00:00
Kegan Dougal c9c444f562 Wrap polling/retry blocks in try/excepts to avoid sending to other ASes breaking permanently should an error occur. 2015-03-16 10:38:02 +00:00
Kegan Dougal 835e01fc70 Minor PR comment tweaks. 2015-03-16 10:16:59 +00:00
Kegan Dougal f9232c7917 Merge branch 'develop' into application-services-txn-reliability 
Conflicts:
	synapse/storage/appservice.py
 2015-03-16 10:09:15 +00:00
Erik Johnston 758d114cbc Send all membership events to the remote homeserver 2015-03-16 00:27:59 +00:00
Erik Johnston ea8590cf66 Make context.auth_events grap auth events from current state. Otherwise auth is wrong. 2015-03-16 00:18:08 +00:00
Erik Johnston ab8229479b Respect ban membership 2015-03-16 00:17:25 +00:00
Matthew Hodgson c8032aec17 actually uphold the bind_host parameter. in theory should make ipv6 binds work like bind_host: 'fe80::1%lo0' 2015-03-14 00:12:20 +00:00
Paul Evans e731d30d90 Merge pull request #108 from matrix-org/metrics 
Metrics
 2015-03-13 17:31:10 +00:00
Erik Johnston 98a3825614 Allow enabling of registration with --disable-registration false 2015-03-13 16:49:18 +00:00
Erik Johnston 9266cb0a22 PEP8 2015-03-13 15:26:00 +00:00
Erik Johnston dea236e4fa Add missing commas 2015-03-13 15:24:03 +00:00
Erik Johnston 69135f59aa Implement registering with shared secret. 2015-03-13 15:23:37 +00:00
Erik Johnston 58367a9da2 Disable registration by default 2015-03-13 12:59:45 +00:00
Erik Johnston 58247c8b4b Also bump dependency link version 2015-03-13 11:39:57 +00:00
Matthew Hodgson f55bd3f94b bump dep on syweb 0.6.5 2015-03-12 18:56:53 +00:00
Paul "LeoNerd" Evans e90002ca1d Merge remote-tracking branch 'origin/develop' into metrics 2015-03-12 16:55:25 +00:00
David Baker bbb010a30f More sacrifices to the pep8 gods. 2015-03-12 16:53:12 +00:00
Paul "LeoNerd" Evans 05a056a409 Appease pyflakes 2015-03-12 16:45:05 +00:00
Paul "LeoNerd" Evans 0eb7e6b9a8 Delete unused import of NOT_READY_YET 2015-03-12 16:39:52 +00:00
Paul "LeoNerd" Evans 128cf2daf7 Appease pep8 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans b98b4c135d Option to serve metrics from their own localhost-only TCP port instead of muxed on the main listener 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans a2cdd11d4a Fold the slightly-odd bind_port/secure_port/etc.. logic into SynapseHomeServer.start_listening() 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans e0214a263b Build MetricsResource as a specific HomeServer dependency 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans e75fa8bbbf Bugfix to sql_txn_timer increment - add only the per-TXN duration, not the total time ever spent since boot 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans c782e893ec Neater metrics from TransactionQueue 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 89ac1fa8ba Add a counter to track total number of events served by the notifier 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 2e4f0b2bd7 Replace the @metrics.counted annotations in federation with specifically-written counters and distributions 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans c1cdd7954d Add an .inc_by() method to CounterMetric; implement DistributionMetric a neater way 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 63cb7ece62 Rename the timer metrics exported by synapse.storage to append _time, so the meaning of ':total' is clearer 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 493e3fa0ca Don't forbid '_' in metric basenames any more, to allow things like foo_time 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans f1fbe3e09f Rename TimerMetric to DistributionMetric; as it could count more than just time 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 642f725fd7 Pretend the 'getEvent' cache is just another cache in the set of all the others for metric 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans cbc0406be8 Export CacheMetric as hits+total, rather than hits+misses, as it's easier to derive hit ratio from that 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 1748605c5d Count incoming HTTP requests per servlet that responds 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 4d661ec0f3 Remember to emit final linefeed from /metrics page, or Prometheus gets upset 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 0e847540c3 Prometheus needs "escaped" label values 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 22b37b75db Kill unused CounterMetric.fetch() method 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans b0cf867319 Use _ instead of . as a metric namespacing separator, for Prometheus 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 0b96bb793e Have all @metrics.counted use a single metric name vectored on the method name, rather than a brand new scalar counter per counted method 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans b3a0179d64 Bugfix to rendering output of vectored TimerMetrics 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans f9478e475b Rename Metrics' "keys" to "labels" 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 399689dcc7 Provide some process resource usage metrics 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans fa319a5786 Add TimerMetrics to shadow the PerformanceCounters in synapse.storage; with the view to eventually replacing them entirely 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 6d146e15df Put some gauge metrics on the number of notifier listeners, and notified-on objects (users, rooms, appservices) 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 25187ab674 Collect per-SQL-verb timer stats on query execution time 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans f52acf3b12 Neater register_* methods on overall Metrics container 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans a99d6edc05 Neater implementation of metric render methods by pulling out 'render' as a base method that calls self.render_item 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 72625f2f4d Initial hack at a TimerMetric; for storing counts + duration accumulators 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans e1a7e3564f Delete a couple of TODO markers of monitoring stats now done 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 094803cf82 Put vector gauges on transaction queue pending PDU and EDU dicts 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans e9c4b0d178 Ensure that /_synapse/metrics response is UTF-8 encoded 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 23ab0c68c2 Implement vector CallbackMetrics 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 849300bc73 Neater introspection methods on BaseMetric so that subclasses don't need to touch self.keys directly 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 8664599af7 Rename CacheCounterMetric to just CacheMetric; add a CallbackMetric component to give the size of the cache 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans e02cc249da Ensure that exceptions while rendering individual metrics don't stop others from being rendered anyway - especially useful for CallbackMetric 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 59c448f074 Add a scalar gauge metric on the size of the presence user cachemap 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans d8caa5454d Initial attempt at a scalar callback-based metric to give instantaneous snapshot gauges 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans b0cdf097f4 Sprinkle some CacheCounterMetrics around the synapse.storage layer 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans ce8b5769f7 Create the concept of a cachecounter metric; generating two counters specific to caches 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 7d72e44eb9 Add vector counters to HTTP clients and servers; count the requests by method and responses by method and response code 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans c53ec53d80 Pull out all uses of the underlying HTTP user agent .request() method into a single wrapper function, to make adding metrics easier 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 9470412316 Initial attempt at sprinkling some @metrics.counted decorations around the federation code 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans a594087f06 Have the MetricsResource actually render metric counters 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 74bc42cfdd An initial implementation of a 'metrics' instance, similar to a 'logger' for keeping counter stats on method calls 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 120b689284 Delete pointless (and unreachable) __init__ method from FederationClient 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans e7420a3bef Initial tiny attempt at (vectorable) counter metrics 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans e07fc62833 A trivial 'hello world'-style resource on /_synapse/metrics, with optional commandline flag 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 5b6e11d560 Commandline option to enable metrics system 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans 211c14c391 No need to explicitly pass 'web_client' in to create_resource_tree as it can be found via config 2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans ad5701f50f Expose 'config' as a real HomeServer dependency key 2015-03-12 16:24:50 +00:00
David Baker c92fdf88a3 Log the matching push rule. 2015-03-11 22:17:31 +00:00
Paul "LeoNerd" Evans a7a28f85ae Appease pep8 2015-03-11 17:32:43 +00:00
Paul "LeoNerd" Evans 59a5f012cc Also give _execute() a description 2015-03-11 17:19:17 +00:00
Paul "LeoNerd" Evans 099e4b88d8 Add a description to storage layer's _execute_and_decode() 2015-03-11 17:08:57 +00:00
David Baker cdb2e045ee Again, underscore, not hyphen 2015-03-11 14:22:35 +00:00
David Baker 465354ffde 'false' is not False 2015-03-11 11:24:50 +00:00
David Baker 83b1e7fb3c PEP8 blank lines 2015-03-11 10:01:17 +00:00
David Baker 04f8478aaa Add the master push rule for the break-my-push button. Allow server default rules to be disabled by default. 2015-03-10 17:26:25 +00:00
David Baker 8916acbc13 These aren't defined for redacted events so don't crash 2015-03-10 11:21:37 +00:00
Erik Johnston abaf47bbb6 Fix bug in logging. 2015-03-10 10:28:29 +00:00
Erik Johnston 045afd6b61 in_thread takes no arguments 2015-03-10 10:19:03 +00:00
Erik Johnston 98b867f7b7 Fix bug in logging. 2015-03-10 10:16:09 +00:00