77ac14b960
Pass around the reactor explicitly ( #3385 )
2018-06-22 09:37:10 +01:00
071206304d
cleanup pep8 errors
2018-05-22 16:54:22 -05:00
85ba83eb51
fixes
2018-05-22 16:28:23 -05:00
df9f72d9e5
replacing portions
2018-05-21 19:47:37 -05:00
6146332387
Merge remote-tracking branch 'origin/develop' into rav/deferred_timeout
2018-04-27 14:18:00 +01:00
9d2c1b8429
Backport deferred.addTimeout
...
Twisted 16.0 doesn't have addTimeout, so let's backport it.
2018-04-27 12:52:30 +01:00
9255a6cb17
Improve exception handling for background processes
...
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.
This is unsatisfactory for a number of reasons:
- logging on garbage collection is best-effort and may happen some time after
the error, if at all
- it can be hard to figure out where the error actually happened.
- it is logged as a scary CRITICAL error which (a) I always forget to grep for
and (b) it's not really CRITICAL if a background process we don't care about
fails.
So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
1ea904b9f0
Use deferred.addTimeout instead of time_bound_deferred
...
This doesn't feel like a wheel we need to reinvent.
2018-04-23 00:53:18 +01:00
f63ff73c7f
add __bool__ alias to __nonzero__ methods
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:40:47 +02:00
d4fb4f7c52
Clear logcontext before starting fed txn queue runner
...
These processes take a long time compared to the request, so there is lots of
"Entering|Restoring dead context" in the logs. Let's try to shut it up a bit.
2017-11-28 15:26:14 +00:00
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
ecdd2a3658
Don't start user_directory handling on workers
2017-06-07 12:02:53 +01:00
eeb2f9e546
Add user_directory to database
2017-05-31 11:51:01 +01:00
62b89daac6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server
2017-04-04 09:46:16 +01:00
65e1683680
Remove spurious PreserveLoggingContext
...
In `on_new_room_event`, remove `PreserveLoggingContext` - we can call its
subroutines with the logcontext set.
2017-04-03 15:42:38 +01:00
feb496056e
preserve_fn some deferred-returning things
...
In `Notifier._on_new_room_event`, `preserve_fn` around its subroutines which
return deferreds, so that it is safe to call it with an active logcontext.
2017-04-03 15:41:17 +01:00
773e1c6d68
Remove spurious @preserve_fn decorators
...
Remove `@preserve_fn` decorators on `on_new_room_event`,
`_notify_pending_new_room_events`, `_on_new_room_event`, `on_new_event`, and
`on_new_replication_data` - none of these functions return a deferred, and the
decorator does nothing unless the wrapped function returns a deferred, so the
decorator was a no-op.
2017-04-03 15:14:11 +01:00
1df7c28661
Use callbacks to notify tcp replication rather than deferreds
2017-03-31 15:42:51 +01:00
7984708a55
Add a simple hook to wait for replication traffic
2017-03-30 11:57:52 +01:00
6957bfdca6
Don't recreate so many sets
2017-03-16 11:54:26 +00:00
6c82de5100
Format presence events on the edges instead of reformatting them multiple times
2017-03-15 14:27:34 +00:00
872e75a3d5
Add some metrics on notifier
2017-03-15 10:56:51 +00:00
fd2eef49c8
Reduce spurious calls to generate sync
2017-03-14 16:33:45 +00:00
54a79c1d37
Make presence.get_new_events a bit faster
...
We do this by caching the set of users a user shares rooms with.
2017-02-02 13:07:18 +00:00
fbaf868f62
Correctly handle timeout errors
2016-12-09 16:30:29 +00:00
be14c24cea
Fix rare notifier bug where listeners dont timeout
...
There was a race condition that caused the notifier to 'miss' the
timeout notification, since there were no other checks for the timeout
this caused listeners to get stuck in a loop until something happened.
2016-12-09 15:43:18 +00:00
26072df6af
Ensure only main or federation_sender process can send federation traffic
2016-11-23 14:09:47 +00:00
f8ee66250a
Handle sending events and device messages over federation
2016-11-17 15:48:04 +00:00
c0238ecbed
Explicitly specify state_key for history_visibility fetching
2016-09-02 14:53:46 +01:00
9219139351
Preserve some logcontexts
2016-08-24 11:58:40 +01:00
afbf6b33fc
defer.returnValue must not be called within Measure
2016-08-19 18:23:44 +01:00
39242090e3
Add measure blocks to notifier
2016-08-19 18:23:44 +01:00
9da84a9a1e
Make AppserviceHandler stream events from database
...
This is for two reasons:
1. Suppresses duplicates correctly, as the notifier doesn't do any
duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
stream.
2016-08-18 11:54:41 +01:00
b9abf3e4e3
Remove dead appservice code
2016-08-17 11:48:23 +01:00
88625db05f
Notify users for events in rooms they join.
...
Change how the notifier updates the map from room_id to user streams on
receiving a join event. Make it update the map when it notifies for the
join event, rather than using the "user_joined_room" distributor signal
2016-06-07 11:33:36 +01:00
c626fc576a
Move the AS handler out of the Handlers object.
...
Access it directly from the homeserver itself. It already wasn't
inheriting from BaseHandler storing it on the Handlers object was
already somewhat dubious.
2016-05-31 13:53:48 +01:00
7e23476814
move filter_events_for_client out of base handler
2016-05-11 13:42:37 +01:00
424a7f48f8
Run filter_events_for_client
...
so we don't accidentally mail out events people shouldn't see
2016-04-27 17:50:49 +01:00
2a37467fa1
Use google style doc strings.
...
pycharm supports them so there is no need to use the other format.
Might as well convert the existing strings to reduce the risk of
people accidentally cargo culting the wrong doc string format.
2016-04-01 16:12:07 +01:00
12904932c4
Hook up adding a pusher to the notifier for replication.
2016-03-15 17:42:03 +00:00
ddf9e7b302
Hook up the push rules to the notifier
2016-03-03 14:57:45 +00:00
60a0f81c7a
Add a /replication API for extracting the updates that happened on
...
synapse
This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.
This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.
Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.
Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.
The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
2016-03-01 14:49:41 +00:00
2c1fbea531
Fix up logcontexts
2016-02-08 14:26:45 +00:00
d83d004ccd
Fix flake8 warnings for new flake8
2016-02-02 17:18:50 +00:00
d98a9f2583
Don't use before_token. Its wrong. Use actual limit.
2016-01-31 13:31:15 +00:00
13724569ec
Deal with None limit
2016-01-29 15:33:44 +00:00
4a6eb5eb45
Make /events always return a newer token, if one exists
2016-01-29 15:22:17 +00:00
0e07f2e15d
Only fetch events for rooms and receipts
2016-01-21 16:10:37 +00:00
91a222c66d
SYN-606: Peeking does not wake up /events
...
If a real user attempted to first peek into one room, and then another,
their room event stream would not be woken up for events in the later
room.
2016-01-21 13:22:26 +00:00
da417aa56d
Allow non-guests to peek on rooms using /events
2016-01-20 15:34:07 +00:00
2ef6de928d
Skip, rather than erroring, invalid guest requests
...
Erroring causes problems when people make illegal requests, because they
don't know what limit parameter they should pass.
This is definitely buggy. It leaks message counts for rooms people don't
have permission to see, via tokens. But apparently we already
consciously decided to allow that as a team, so this preserves that
behaviour.
2016-01-05 18:12:37 +00:00
f3948e001f
Missing yield on guest access auth check
...
Needs matrix-org/sytest#125 to land first
2015-12-23 14:10:06 +00:00
468a2ed4ec
Return non-room events from guest /events calls
2015-11-12 16:45:28 +00:00
ca2f90742d
Open up /events to anonymous users for room events only
...
Squash-merge of PR #345 from daniel/anonymousevents
2015-11-05 14:32:26 +00:00
33b3e04049
Merge branch 'develop' into daniel/removesomelies
...
Conflicts:
synapse/notifier.py
2015-11-04 16:01:00 +00:00
192241cf2a
Remove unused arguments and code
2015-11-02 15:10:59 +00:00
f40b0ed5e1
Inform the client of new room tags using v1 /events
2015-10-29 15:21:09 +00:00
f8f3d72e2b
Don't make pushers handle presence/typing events
2015-08-24 16:19:43 +01:00
53d1174aa9
Improve naming
2015-07-20 06:32:12 -07:00
bd1236c0ee
Consolidate duplicate code in notifier
2015-07-02 11:46:05 +01:00
ddf7979531
Add receipts_key to StreamToken
2015-07-02 11:45:44 +01:00
2f556e0c55
Fix typo
2015-06-19 16:22:53 +01:00
6f6ebd216d
PEP8
2015-06-18 17:00:32 +01:00
73513ececc
Documentation
2015-06-18 16:15:10 +01:00
1f24c2e589
Don't bother proxying lookups on _NotificationListener to underlying deferred
2015-06-18 16:09:53 +01:00
22049ea700
Refactor the notifier.wait_for_events code to be clearer. Add _NotifierUserStream.new_listener that accpets a token to avoid races.
2015-06-18 15:49:24 +01:00
050ebccf30
Fix notifier leak
2015-06-18 11:36:26 +01:00
554c63ca60
Iterate over the user_streams not the user_ids
2015-05-26 15:03:49 +01:00
88f1ea36ce
Oops, get_rooms_for_user returns a namedtuple, not a room_id
2015-05-21 15:23:58 +01:00
ad31fa3040
Don't bother sorting by the room_stream_ids, it shouldn't matter which order they are notified in
2015-05-18 14:04:58 +01:00
1e90715a3d
Make sure the notifier stream token goes forward when it is updated. Sort the pending events by the correct room_stream_id
2015-05-18 13:17:36 +01:00
0ad1c67234
Add some doc-strings to notifier
2015-05-14 14:35:07 +01:00
084c365c3a
Use the current token when timing out a notifier, make sure the user_id is a string in on_new_user_event
2015-05-14 12:03:26 +01:00
3edd2d5c93
Fix v2 sync, update the last_notified_ms only if there was an active listener
2015-05-14 11:25:30 +01:00
5e0c533672
Fix metric counter
2015-05-13 17:20:28 +01:00
f1b83d88a3
Discard unused NotifierUserStreams
2015-05-13 16:54:02 +01:00
9af432257d
Don't set a timer if there's already a result to return
2015-05-13 15:42:13 +01:00
df6db5c802
Don't bother checking for new events from a source if the stream token hasn't advanced for that source
2015-05-13 15:08:24 +01:00
63878c0379
Don't bother checking for updates if the stream token hasn't advanced for a user
2015-05-13 13:42:21 +01:00
cffe6057fb
Merge branch 'notifier_unify' into notifier_performance
...
Conflicts:
synapse/notifier.py
2015-05-12 16:37:50 +01:00
4429e4bf24
Merge branch 'develop' into notifier_unify
...
Conflicts:
synapse/notifier.py
2015-05-12 15:31:26 +01:00
78672a9fd5
Merge branch 'notifier_unify' into notifier_performance
2015-05-12 13:11:54 +01:00
2551b6645d
Update the end_token correctly, otherwise the token doesn't advance and the client gets duplicate events
2015-05-12 11:54:18 +01:00
5c75adff95
Add a NotifierUserStream to hold all the notification listeners for a user
2015-05-12 11:00:37 +01:00
e269c511f6
Don't bother passing the events to the notifier since it isn't using them
2015-05-11 15:01:51 +01:00
5e3b254dc8
Use wait_for_events to implement 'get_events'
2015-05-11 14:37:33 +01:00
4ac1941592
PEP8
2015-05-08 16:33:01 +01:00
476899295f
Change the way we do logging contexts so that they survive divergences
2015-05-08 16:32:18 +01:00
42c12c04f6
Remove some run_on_reactors
2015-05-01 14:41:25 +01:00
e19f794fee
Change from exception to warn
2015-04-15 15:12:57 +01:00
1280a47fc6
Add comment
2015-04-09 11:42:21 +01:00
23d285ad57
Unset the timer in the timeout callback so that we don't try to cancel it if it has been called
2015-04-09 11:41:50 +01:00
6f9dea7483
SYN-339: Cancel the notifier timeout when the notifier fires
2015-04-09 11:07:20 +01:00
5bc41fe9f8
Move comment into docstring
2015-04-08 14:01:22 +01:00
638be5a6b9
Factor out loops into '_discard_if_notified'
2015-04-08 13:58:32 +01:00
830d07db82
Also perform paranoia checks in 'on_new_user_event'
2015-04-08 13:40:20 +01:00
65f5e4e3e4
Add paranoia checks to make sure that we evict stale NotificationListeners when we are about to process them
2015-04-08 13:33:38 +01:00
07d4041709
Fix bug where we didn't inform the NotificataionListeners about new rooms they have been subscribed to. This meant that the listeners didn't clean themselves up fully from all the dicts
2015-04-08 13:33:38 +01:00
128cf2daf7
Appease pep8
2015-03-12 16:24:51 +00:00
89ac1fa8ba
Add a counter to track total number of events served by the notifier
2015-03-12 16:24:51 +00:00
Amber Brown