Commit Graph

322 Commits (6ba732fefe732f92b0266b17cb6e45388bbe002a)

Author SHA1 Message Date
Erik Johnston 4e1cebd56f Make synchrotron accept /events 2016-08-12 15:31:44 +01:00
Will Hunt 2510db3e76 Don't change status_msg on /sync 2016-08-10 12:59:59 +01:00
Erik Johnston 216a05b3e3 .values() returns list of sets 2016-06-06 16:00:09 +01:00
Erik Johnston 96dc600579 Fix typos 2016-06-06 15:44:41 +01:00
Erik Johnston 377eb480ca Fire after 30s not 8h 2016-06-06 15:14:21 +01:00
Erik Johnston 4ce84a1acd Change metric style 2016-06-03 13:49:16 +01:00
Erik Johnston a7ff5a1770 Presence metrics. Change def of small delta 2016-06-03 13:40:55 +01:00
Mark Haines 70599ce925 Allow external processes to mark a user as syncing. (#812)
* Add infrastructure to the presence handler to track sync requests in external processes

* Expire stale entries for dead external processes

* Add an http endpoint for making users as syncing

Add some docstrings and comments.

* Fixes
2016-06-02 15:20:15 +01:00
Mark Haines b153f5b150 Merge pull request #787 from matrix-org/markjh/liberate_presence_handler
Move the presence handler out of the Handlers object
2016-05-17 10:09:43 +01:00
Mark Haines 1a3a2002ff Spell "domain" correctly
s/domian/domain/g
2016-05-16 19:17:23 +01:00
Mark Haines e168abbcff Don't inherit PresenceHandler from BaseHandler, remove references to self.hs from presence handler 2016-05-16 19:08:40 +01:00
Mark Haines 3b86ecfa79 Move the presence handler out of the Handlers object 2016-05-16 18:56:37 +01:00
Erik Johnston 08dfa8eee2 Add and use get_domian_from_id 2016-05-09 10:36:03 +01:00
Erik Johnston 56b5e83e36 Reduce database inserts when sending transactions 2016-05-06 11:20:18 +01:00
Mark Haines 7a38612620 Remove another unused function from presence 2016-03-17 11:54:19 +00:00
Mark Haines 673c96ce97 Remove dead code left over from presence changes 2016-03-17 11:03:47 +00:00
Erik Johnston 2a78dac60d Merge pull request #612 from matrix-org/erikj/cache_size
Add environment variable SYNAPSE_CACHE_FACTOR, default it to 0.1
2016-03-02 14:36:10 +00:00
Mark Haines 60a0f81c7a Add a /replication API for extracting the updates that happened on
synapse

This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.

This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.

Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.

Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.

The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
2016-03-01 14:49:41 +00:00
Erik Johnston 910fc0f28f Add enviroment variable SYNAPSE_CACHE_FACTOR, default it to 0.1 2016-03-01 12:56:39 +00:00
Erik Johnston b5f77eb12a Check presence token interval is less than 100, rather than the token itself 2016-02-23 15:47:37 +00:00
Erik Johnston e3e0ac6ec7 Merge pull request #602 from matrix-org/erikj/presence
Change the way we figure out presence updates for small deltas
2016-02-23 15:18:41 +00:00
Erik Johnston 13f86c3489 Handle get_all_entities_changed returning None 2016-02-23 15:05:37 +00:00
Erik Johnston 6e0209112b Add comments 2016-02-23 14:57:45 +00:00
Erik Johnston c77dae7a1a Change the way we figure out presence updates for small deltas 2016-02-23 14:54:40 +00:00
Erik Johnston a7b2ce32f7 Merge pull request #600 from matrix-org/erikj/presence
Measure PresenceEventSource.get_new_events
2016-02-23 14:54:26 +00:00
Erik Johnston 02e928cf9b Don't include defer.returnValue in Measure block 2016-02-23 14:52:16 +00:00
Erik Johnston 56a94ccd9e Measure PresenceEventSource.get_new_events 2016-02-23 14:52:16 +00:00
Erik Johnston 138c405974 Pick up currently_active across federation 2016-02-23 10:40:11 +00:00
Erik Johnston 3dbaeef58c Correctly filter states 2016-02-19 12:27:35 +00:00
Erik Johnston 42ac5f0c1a Only send presence updates to remote hosts if user is ours 2016-02-19 12:19:56 +00:00
Erik Johnston 24d9f2c140 Add Measures to presence 2016-02-19 11:50:48 +00:00
Erik Johnston 4a95eb0a12 Add presence metric 2016-02-19 11:32:04 +00:00
Erik Johnston be799453aa Remove spurious extra metrics 2016-02-19 11:29:33 +00:00
Erik Johnston 929cb0ed7d Don't set currently_active for remote presence 2016-02-19 10:58:27 +00:00
Erik Johnston 5f4eca3816 More metrics 2016-02-19 10:21:41 +00:00
Erik Johnston 5614b4dafb Add presence metrics 2016-02-19 09:50:54 +00:00
Erik Johnston 48b652bcbe Remove invalid arg. 2016-02-18 14:57:09 +00:00
Erik Johnston b4796a62ee Add unit test 2016-02-18 11:52:33 +00:00
Erik Johnston fe95f2217c Add stuff pulled from the DB to the cache 2016-02-18 10:26:24 +00:00
Erik Johnston 8351538873 PEP8 2016-02-18 10:12:12 +00:00
Erik Johnston 112283e230 Prefix TS fields with _ts 2016-02-18 10:11:43 +00:00
Erik Johnston b31ec214a5 Remove status_msg when going offline. Don't offline -> online if you send a message 2016-02-18 09:54:08 +00:00
Erik Johnston 114b929f8b Check presence state is a valid one 2016-02-18 09:16:32 +00:00
Erik Johnston ddca9c56fc Move if statement 2016-02-18 09:11:53 +00:00
Erik Johnston 58371fa263 Comment 2016-02-18 09:09:50 +00:00
Erik Johnston c229c87398 Remove spurious comment 2016-02-17 15:48:29 +00:00
Erik Johnston e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Erik Johnston 2c1fbea531 Fix up logcontexts 2016-02-08 14:26:45 +00:00
Daniel Wagner-Hall d83d004ccd Fix flake8 warnings for new flake8 2016-02-02 17:18:50 +00:00
Matthew Hodgson 6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
Mark Haines a9526831a4 Wrap calls to distributor.fire in appropriately named functions so that static analysis can work out want is calling what 2015-12-01 20:53:04 +00:00
Mark Haines 31069ecf6a Rename presence_handler.send_invite to presence_handler.send_presence_invite to distinguish it from normal invites 2015-12-01 15:59:45 +00:00
Daniel Wagner-Hall 38d82edf0e Allow guest users to join and message rooms 2015-11-10 16:57:13 +00:00
Daniel Wagner-Hall ca2f90742d Open up /events to anonymous users for room events only
Squash-merge of PR #345 from daniel/anonymousevents
2015-11-05 14:32:26 +00:00
Mark Haines 987803781e Fix some races in the synapse presence handler caused by not yielding on deferreds 2015-10-09 19:59:50 +01:00
Erik Johnston 0d4abf7777 Typo 2015-08-18 11:19:08 +01:00
Erik Johnston e55291ce5e None check 2015-08-18 11:17:37 +01:00
Erik Johnston 83eb627b5a More helpful variable names 2015-08-18 10:33:11 +01:00
Erik Johnston 776ee6d92b Doc strings 2015-08-18 10:30:07 +01:00
Erik Johnston f72ed6c6a3 Remove debug try/catch 2015-08-18 10:29:49 +01:00
Erik Johnston 1a9510bb84 Implement a batched presence_handler.get_state and use it 2015-08-17 10:40:23 +01:00
Erik Johnston bd1236c0ee Consolidate duplicate code in notifier 2015-07-02 11:46:05 +01:00
Erik Johnston 0460406298 Don't do unecessary db ops in presence.get_state 2015-06-16 16:59:38 +01:00
Mark Haines 1a9a9abcc7 Add a cache for getting the presence list for a user 2015-05-22 16:11:17 +01:00
Mark Haines 106a3051b8 Remove spurious TODO comment 2015-05-22 15:53:03 +01:00
Mark Haines 7ae8afb7ef Removed unused 'is_visible' method 2015-05-20 14:48:11 +01:00
Mark Haines e01b825cc9 Clean up the presence_list checking logic a bit 2015-05-20 13:21:59 +01:00
Mark Haines e4c65b338d Speed up the get_pagination_rows as well 2015-05-18 18:21:06 +01:00
Mark Haines 591c4bf223 Cache the most recent serial for each room 2015-05-18 16:21:51 +01:00
Mark Haines e1150cac4b Move updating the serial and state of the presence cache into a single function 2015-05-18 15:46:37 +01:00
Mark Haines 880fb46de0 Merge branch 'notifier_performance' into markjh/presence_performance 2015-05-18 14:33:58 +01:00
Mark Haines 0b0033c40b Merge branch 'develop' into notifier_performance 2015-05-18 13:50:01 +01:00
Mark Haines 755def8083 Add more doc string, reduce C+P boilerplate for getting room list 2015-05-18 13:46:47 +01:00
Mark Haines 415b158ce2 More whitespace 2015-05-15 11:09:47 +01:00
Mark Haines 0a4330cd5d Add some missed argument types, cleanup the whitespace a bit 2015-05-14 17:48:12 +01:00
Mark Haines 47ec693e29 More doc-strings 2015-05-14 17:07:02 +01:00
Mark Haines c5d1b4986b Remove unused arguments and doc PresenceHandler.push_update_to_clients 2015-05-14 14:59:31 +01:00
Mark Haines 899d4675dd Merge branch 'notifier_unify' into notifier_performance 2015-05-14 11:36:44 +01:00
Mark Haines 63878c0379 Don't bother checking for updates if the stream token hasn't advanced for a user 2015-05-13 13:42:21 +01:00
Mark Haines d7b3ac46f8 Revert "Improvement to performance of presence event stream handling" 2015-05-12 15:44:21 +01:00
Mark Haines c167cbc9fd Merge pull request #147 from matrix-org/presence-performance
Improvement to performance of presence event stream handling
2015-05-12 15:24:54 +01:00
Erik Johnston 2236ef6c92 Fix up leak. Add warnings. 2015-05-08 19:53:34 +01:00
Erik Johnston 476899295f Change the way we do logging contexts so that they survive divergences 2015-05-08 16:32:18 +01:00
Paul "LeoNerd" Evans 45543028bb Use the presence cachemap ordering to early-abort the iteration loop 2015-05-07 22:40:10 +01:00
Paul "LeoNerd" Evans f683b5de47 Store presence cachemap in an ordered dict, so that the newer serials will be at the end 2015-05-07 21:27:53 +01:00
Erik Johnston a6e62cf6d0 Fix off by one in presence token handling 2015-04-29 17:37:11 +01:00
Erik Johnston 1783c7ca92 Ensure we never miss any presence updates 2015-04-28 17:24:24 +01:00
Erik Johnston 0126ef7f3c Fix typo 2015-04-28 17:23:53 +01:00
Erik Johnston d98edb548a Ensure the serial returned by presence is always an integer 2015-04-28 17:20:32 +01:00
Paul "LeoNerd" Evans e1e5e53127 Remove users from the remote_offline_serials list (and clean up empty elements) when they go online again 2015-04-23 19:01:37 +01:00
Paul "LeoNerd" Evans b3bda8a75f Don't let the remote offline serial list grow arbitrarily large 2015-04-23 18:40:47 +01:00
Paul "LeoNerd" Evans 8a785c3006 Store a list of the presence serial number at which remote users went offline, so that when we delete them from the cachemap, we can still synthesize OFFLINE events for them (SYN-261) 2015-04-23 18:40:19 +01:00
Paul "LeoNerd" Evans 191f7f09ce Generate presence event-stream JSON structures directly 2015-04-23 18:27:25 +01:00
Paul "LeoNerd" Evans ed008e85a8 Reduce activity timer granularity to avoid too many quick updates (SYN-247) 2015-03-23 17:25:50 +00:00
Paul "LeoNerd" Evans 128cf2daf7 Appease pep8 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans b0cf867319 Use _ instead of . as a metric namespacing separator, for Prometheus 2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans 59c448f074 Add a scalar gauge metric on the size of the presence user cachemap 2015-03-12 16:24:50 +00:00
Erik Johnston 8e28db5cc9 Change room handlers get_rooms_for_user to get_joined_rooms_for_user. This uses the a storage api that is cached. 2015-03-09 16:43:09 +00:00
Erik Johnston 72a4de2ce6 Use consumeErrors=True on all DeferredLists.
This is so that the DeferredLists actually consume the error instead of
propogating down the non-existent errback chain. This should reduce the
number of unhandled errors we are seeing.
2015-02-17 10:07:01 +00:00
Mark Haines 84a769cdb7 Fix code-style 2015-02-10 17:58:36 +00:00
Mark Haines b085fac735 Code-style fixes 2015-02-10 16:30:48 +00:00
David Baker c291a4d522 Merge branch 'develop' into pushers
Conflicts:
	synapse/handlers/events.py
	synapse/server.py
2015-01-28 12:51:05 +00:00
David Baker 273b12729b Reset badge count to zero when last active time is bumped 2015-01-28 11:55:49 +00:00
Mark Haines 5759bec43c Replace hs.parse_userid with UserID.from_string 2015-01-23 11:47:15 +00:00
Mark Haines adb04b1e57 Update copyright notices 2015-01-06 13:21:39 +00:00
Erik Johnston c5c32266d8 Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor 2014-12-04 15:58:24 +00:00
Paul "LeoNerd" Evans f5d2514fc0 @log_function on PresenceStream's get_new_events_for_user() 2014-12-03 19:48:14 +00:00
Paul "LeoNerd" Evans 52f1d3c886 Store any incoming presence push in the local cache anyway, even if there's no interested observers (yet *hint*) (SYN-115) 2014-12-03 19:06:24 +00:00
Erik Johnston 6941a19715 Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor 2014-12-03 11:56:49 +00:00
Paul "LeoNerd" Evans 274d137b00 Extra verbosity on logging of received presence info 2014-12-02 20:51:28 +00:00
Erik Johnston 9d53228158 Change DomainSpecificString so that it doesn't use a HomeServer object 2014-12-02 10:42:28 +00:00
Mark Haines 32090aee16 Add a few missing yields, Move deferred lists inside PreserveLoggingContext because they don't interact well with the logging contexts 2014-11-20 16:24:00 +00:00
Paul "LeoNerd" Evans 03dc63f6c8 Initialise UserPresenceCache instances to always contain a 'presence' key 2014-11-19 17:31:46 +00:00
Paul "LeoNerd" Evans 759db7d7d5 Added ability to .get_state() from the PresenceHandler by returning a complete m.presence event 2014-11-18 15:25:55 +00:00
Paul "LeoNerd" Evans 4c18e08036 Don't expect all _user_cachemap entries to definitely contain a "last_active" key 2014-11-18 15:10:11 +00:00
Mark Haines 7d709542ca Fix pep8 warnings 2014-10-30 11:10:17 +00:00
Paul "LeoNerd" Evans b1ee6fd7ed Fix an off-by-one bug in presence event stream pagination; this might be responsible for any number of bug reports 2014-10-29 16:16:01 +00:00
Paul "LeoNerd" Evans d6bcffa929 Construct a source-specific 'SourcePaginationConfig' to pass into get_pagination_rows; meaning each source doesn't have to care about its own name any more 2014-10-29 16:16:01 +00:00
Paul "LeoNerd" Evans c5a25f610a Remove redundant (and incorrect) presence pagination fetching code 2014-10-29 16:16:01 +00:00
Erik Johnston de55ba218f Fix race in presence handler where we evicted things from cache while handling a key therein 2014-09-06 17:38:11 +01:00
Matthew Hodgson 8a7c1d6a00 fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch. 2014-09-03 17:31:57 +01:00
Paul "LeoNerd" Evans cda31fb755 Kill the state
... key from all the Presence messages
2014-09-03 15:37:10 +01:00
Paul "LeoNerd" Evans dada11dc5f Bugfix for back-pagination of presence 2014-09-03 15:28:03 +01:00
Paul "LeoNerd" Evans da31b96b55 Implement presence state visibilty limiting when polling eventsource for stream 2014-09-03 13:46:52 +01:00
Kegan Dougal c54d8df504 Update API docs to use 'presence' key not 'state'. Fixed error messages when setting presence. 2014-09-03 11:45:30 +01:00
Paul "LeoNerd" Evans da3e04df8b Rename the 'do_users_share_a_room' to something slightly less verb-sounding 2014-09-03 10:40:48 +01:00
Paul "LeoNerd" Evans ddf3ca7ab3 Neater is_presence_visible() code 2014-09-03 10:40:21 +01:00
Paul "LeoNerd" Evans d9d6fbb085 Safer handling of incoming presence EDUs that may or maynot contain 'presence' or 'state' 2014-09-02 11:17:43 +01:00
Paul "LeoNerd" Evans 7b56a7a3cb 'continue' to go to the next item in a for loop, not 'break' 2014-09-02 11:15:46 +01:00
Paul "LeoNerd" Evans 270d302834 Reënable presence visibility permission checking now it's not used on redundant code paths 2014-09-01 22:15:02 +01:00
Paul "LeoNerd" Evans 32fdf8efd6 No need to test is_presence_visible in _start_polling_local because we know it will be by when it's called 2014-09-01 22:15:02 +01:00
Paul "LeoNerd" Evans 1c6ab2d759 Bump a user's presence last_active time every time they send a message to a room 2014-09-01 16:16:35 +01:00
Paul "LeoNerd" Evans 468d94c920 Rename API-visible 'mtime' presence field to 'last_active'; slightly different semantics 2014-09-01 15:39:50 +01:00
Paul "LeoNerd" Evans 02f4e3b3ff Rename 'state' presence key to the much more obvious 'presence'; maintain a legacy 'state' copy for now 2014-09-01 14:45:41 +01:00
Paul "LeoNerd" Evans 93407cf7cf Bugfixes on presence pushes on user joining:
* No need to inform clients of status of remote users; as that will
   arrive in due course anyway. We don't -have- the state currently, so
   we'd only send an unknown message

 * Remember to bump the presence serial for the event source, so the
   notifiers will wake up and report it
2014-08-29 19:53:33 +01:00
Paul "LeoNerd" Evans a8e8d1d06c Renamed get_current_token_part to get_current_key 2014-08-29 19:15:23 +01:00
Paul "LeoNerd" Evans eec67a675f Have EventSource's get_new_events_for_user() API work only on keys within that source, not overall eventstream tokens 2014-08-29 19:13:55 +01:00
Paul "LeoNerd" Evans 20d0db6cfb Move the *EventSource classes into the handlers they relate to, so it's easier to find the code 2014-08-29 17:10:28 +01:00
Erik Johnston 5dd38d579b Fix a couple of bugs in presence handler related to pushing updatesto the correct user. Fix presence tests. 2014-08-29 12:08:33 +01:00
Erik Johnston 8113eb7c79 Turn of trace_function logging 2014-08-28 18:45:00 +01:00
Erik Johnston aaf4fd98ee Only poll remote users if they are in our presence list, rather than in a common room 2014-08-28 18:43:03 +01:00
Erik Johnston 1b7686329e Don't query the rooms members table so much by using the new notifier api that allows you to specify room_ids to notify. 2014-08-28 17:43:15 +01:00
Erik Johnston 54d0a75573 Merge branch 'develop' of github.com:matrix-org/synapse into presence_logging
Conflicts:
	synapse/handlers/presence.py
2014-08-28 16:52:46 +01:00
Paul "LeoNerd" Evans efc5f3440d Only send presence "poll"/"unpoll" EDUs when changing from/to zero remotes 2014-08-28 16:43:55 +01:00
Paul "LeoNerd" Evans c46c806126 Re-enable presence, un-skip presence tests 2014-08-28 16:00:14 +01:00
Erik Johnston eb3094ed31 And more logging. 2014-08-28 15:58:38 +01:00
Erik Johnston e0ba81344c Add more logging. Up the event stream timer to 10s 2014-08-28 15:30:42 +01:00
Erik Johnston b8b52ca09d Add logging to try and figure out what is going on with the presence stuff 2014-08-28 14:58:51 +01:00
Erik Johnston 7917ff1271 Turn off presence again. 2014-08-27 16:09:48 +01:00
Erik Johnston bd16b93e8f Implement presence event source. Change the way the notifier indexes listeners 2014-08-27 14:03:27 +01:00
Erik Johnston 67c5f89244 Enable presence again. Fix up api to match old api. 2014-08-26 19:40:29 +01:00
Erik Johnston 3a2a5b959c WIP: Completely change how event streaming and pagination work. This reflects the change in the underlying storage model. 2014-08-26 18:57:46 +01:00
Mark Haines ac21dfff6d Fix pyflakes errors 2014-08-26 16:02:59 +01:00
Erik Johnston a664ec20e0 Add a do_users_share_a_room method and use that in the presence handler. 2014-08-26 13:38:26 +01:00
Erik Johnston cda4ff8519 Oops, we need to use defer.returnValue. 2014-08-22 18:23:02 +01:00
Erik Johnston 5b058a79cb Make is_presence_visible always return true as it was thrashing the database. 2014-08-22 18:21:21 +01:00
Erik Johnston 9f514915af Add indices to schema 2014-08-22 18:03:39 +01:00
Erik Johnston e2b861cc67 Merge branch 'master' of github.com:matrix-org/synapse into sql_refactor 2014-08-19 16:41:13 +01:00
Erik Johnston 5c00614aab PresenceStreamData was expecting *_key to be ints 2014-08-19 15:51:10 +01:00
Paul "LeoNerd" Evans 89cabba3e0 Don't delete OFFLINE users from the presence cache, so we can report on their offline mtime 2014-08-19 14:51:15 +01:00
Erik Johnston 347242a5c4 Merge branch 'master' of github.com:matrix-org/synapse into sql_refactor
Conflicts:
	tests/rest/test_presence.py
	tests/rest/test_rooms.py
	tests/utils.py
2014-08-19 14:48:19 +01:00
Erik Johnston 598a1d8ff9 Change the way pagination works to support out of order events. 2014-08-19 14:19:48 +01:00
Paul "LeoNerd" Evans 88f7482b92 Perform the 'REST'-level tests of Presence against the real Presence handler as well, mocking out the datastore beneath it 2014-08-19 11:16:23 +01:00
Paul "LeoNerd" Evans 83f031207e Implement and test presence dropping of remote users 2014-08-19 11:16:23 +01:00
Paul "LeoNerd" Evans faf25e3a83 Allow room presence visibility between users who share a room 2014-08-18 13:41:43 +01:00
Matthew Hodgson 98ed3d0222 improve comment 2014-08-16 01:52:03 +01:00
Matthew Hodgson fc846aa771 emergency bodges to turn off presence ACLs in order to make
matrix.org:8080 actually work
2014-08-16 01:47:17 +01:00
Paul "LeoNerd" Evans 9c41f635a9 When starting to poll for presence, also include members of all shared rooms 2014-08-15 17:12:07 +01:00
Paul "LeoNerd" Evans 0fa05ea331 Round Presence mtime and mtime_age to nearest msec; avoids floats for msec values over the wire 2014-08-14 14:16:01 +01:00
Paul "LeoNerd" Evans d05aa651f8 An initial hack at storing presence state-change mtimes in database and presenting age durations to clients/federation events 2014-08-13 19:19:15 +01:00
Matthew Hodgson f98e6380f1 add in whitespace after copyright statements to improve legibility 2014-08-13 03:14:34 +01:00
matrix.org 4f475c7697 Reference Matrix Home Server 2014-08-12 15:10:52 +01:00