c7273c11bc
Give pushers their own background logcontext
...
Each pusher has its own loop which runs for as long as it has work to do. This
should run in its own background thread with its own logcontext, as other
similar loops elsewhere in the system do - which means that CPU usage is
consistently attributed to that loop, rather than to whatever request happened
to start the loop.
2018-10-22 16:12:11 +01:00
5110f4e425
move get_all_pushers call down
...
simplifies the interface to _start_pushers
2018-10-22 16:12:11 +01:00
04277d0ed8
Factor PusherPool._start_pusher out of _start_pushers
...
... and use it from start_pusher_by_id. This mostly simplifies
start_pusher_by_id.
2018-10-22 16:12:11 +01:00
3e8b02c939
Rename _refresh_pusher
...
This is public (or at least, called from outside the class), so ought to have a
better name.
2018-10-22 16:12:11 +01:00
cc325c7069
Calculate absolute path for email templates
2018-10-19 14:01:59 +01:00
1519572961
Ship the email templates as package_data
...
move the example email templates into the synapse package so that they can be
used as package data, which should mean that all of the packaging mechanisms
(pip, docker, debian, arch, etc) should now come with the example templates.
In order to grandfather in people who relied on the templates being in the old
place, check for that situation and fall back to using the defaults if the
templates directory does not exist.
2018-10-17 16:46:02 +01:00
7de1989ea2
fix link for case that config.email_riot_base_url is set
2018-09-13 22:43:50 +01:00
7c27c4d51c
merge ( #3576 )
2018-09-14 03:11:11 +10:00
66f7dc8c87
Fix logcontexts for running pushers
...
First of all, avoid resetting the logcontext before running the pushers, to fix
the "Starting db txn 'get_all_updated_receipts' from sentinel context" warning.
Instead, give them their own "background process" logcontexts.
2018-08-17 00:32:39 +01:00
b37c472419
Rename async to async_helpers because `async` is a keyword on Python 3.7 ( #3678 )
2018-08-10 23:50:21 +10:00
e42510ba63
Use new getters
2018-07-23 13:17:22 +01:00
49af402019
run isort
2018-07-09 16:09:20 +10:00
07cad26d65
Remove all global reactor imports & pass it around explicitly ( #3424 )
2018-06-25 14:08:28 +01:00
a61738b316
Remove run_on_reactor ( #3395 )
2018-06-14 18:27:37 +10:00
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy ( #3307 )
2018-05-31 19:03:47 +10:00
754826a830
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-28 18:57:23 +10:00
1f69693347
Merge pull request #3244 from NotAFile/py3-six-4
...
replace some iteritems with six
2018-05-24 13:04:07 -05:00
e987079037
fixes
2018-05-23 13:03:51 -05:00
53cc2cde1f
cleanup
2018-05-22 17:32:57 -05:00
85ba83eb51
fixes
2018-05-22 16:28:23 -05:00
fcc525b0b7
rest of the changes
2018-05-21 19:48:57 -05:00
df9f72d9e5
replacing portions
2018-05-21 19:47:37 -05:00
933bf2dd35
replace some iteritems with six
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:59:26 +02:00
d9fe2b2d9d
Replace some more comparisons with six
...
plus a bonus b"" string I missed last time
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:56:31 +02:00
950a32eb47
Merge pull request #3152 from NotAFile/py3-local-imports
...
make imports local
2018-04-30 01:28:13 +01:00
57b58e2174
make imports local
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-28 13:41:41 +02:00
fc149b4eeb
Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background
2018-04-27 14:31:23 +01:00
2a13af23bc
Use run_in_background in preference to preserve_fn
...
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
2018-04-27 12:55:51 +01:00
9c3da24561
Merge pull request #3138 from matrix-org/rav/catch_unhandled_exceptions
...
Improve exception handling for background processes
2018-04-27 11:47:49 +01:00
605defb9e4
Add missing consumeErrors
...
In general we want defer.gatherResults to consumeErrors, rather than having
exceptions hanging around and getting logged as CRITICAL unhandled errors.
2018-04-27 11:16:28 +01:00
9255a6cb17
Improve exception handling for background processes
...
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.
This is unsatisfactory for a number of reasons:
- logging on garbage collection is best-effort and may happen some time after
the error, if at all
- it can be hard to figure out where the error actually happened.
- it is logged as a scary CRITICAL error which (a) I always forget to grep for
and (b) it's not really CRITICAL if a background process we don't care about
fails.
So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
573712da6b
Update comments
2018-02-21 11:29:49 +00:00
6ff8c87484
Batch inserts into event_push_actions_staging
2018-02-20 11:33:07 +00:00
012e8e142a
Comments
2018-02-16 11:35:01 +00:00
4810f7effd
Remove context.push_actions
2018-02-15 15:47:06 +00:00
acac21248c
Store push actions in staging area
2018-02-15 15:47:04 +00:00
03dd745fe2
Better logging when pushes fail
2018-01-29 15:49:06 +00:00
e051abd20b
add appid/device_display_name to to pusher logging
2018-01-29 15:04:16 +00:00
4528dd2443
Fix logging and add user_id
2018-01-22 20:15:42 +00:00
93efd7eb04
logging and debug for http pusher
2018-01-22 18:14:10 +00:00
2c6d63922a
Remove pushers when deleting access tokens
...
Whenever an access token is invalidated, we should remove the associated
pushers.
2017-11-29 16:44:35 +00:00
1b83c09c03
Merge pull request #2675 from matrix-org/rav/remove_broken_logcontext_funcs
...
Remove preserve_context_over_{fn, deferred}
2017-11-15 11:13:53 +00:00
7e6fa29cb5
Remove preserve_context_over_{fn, deferred}
...
Both of these functions ae known to leak logcontexts. Replace the remaining
calls to them and kill them off.
2017-11-14 11:22:42 +00:00
2a98ba0ed3
Rename redact_content option to include_content
...
The redact_content option never worked because it read the wrong config
section. The PR introducing it
(https://github.com/matrix-org/synapse/pull/2301 ) had feedback suggesting the
name be changed to not re-use the term 'redact' but this wasn't
incorporated.
This reanmes the option to give it a less confusing name, and also
means that people who've set the redact_content option won't suddenly
see a behaviour change when upgrading synapse, but instead can set
include_content if they want to.
This PR also updates the wording of the config comment to clarify
that this has no effect on event_id_only push.
Includes https://github.com/matrix-org/synapse/pull/2422
2017-11-08 10:35:30 +00:00
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
9342bcfce0
Omit the *s for @room notifications
...
They're just redundant
2017-10-16 13:38:10 +01:00
81a5e0073c
pep8
2017-10-10 15:53:34 +01:00
ab1bc9bf5f
Don't KeyError if no power_levels event
2017-10-10 15:34:05 +01:00
0f1eb3e914
Use notification levels in power_levels
...
Rather than making the condition directly require a specific power
level. This way the level require to notify a room can be configured
per room.
2017-10-10 15:23:00 +01:00
c9f034b4ac
There was already a constant for this
...
also update copyright
2017-10-10 11:47:10 +01:00
a9f9d68631
More optimisation
2017-10-10 11:38:31 +01:00
707374d5dc
What year is it!? Who's the president!?
2017-10-10 11:21:41 +01:00
269af961e9
Make be faster
2017-10-05 13:27:12 +01:00
ed80c6b6cc
Add fastpath optimisation
2017-10-05 13:20:22 +01:00
e433393c4f
pep8
2017-10-05 13:08:02 +01:00
985ce80375
They're called rooms
2017-10-05 13:03:44 +01:00
b9b9714fd5
Get rule type right
2017-10-05 13:02:19 +01:00
fa969cfdde
Support for channel notifications
...
Add condition type to check the sender's power level and add a base
rule using it for @channel notifications.
2017-10-05 12:39:18 +01:00
0c8da8b519
Use better method for word boundary searching
...
From ebc95667b8
2017-10-05 11:57:43 +01:00
cbe3c3fdd4
pep8
2017-10-05 11:43:10 +01:00
6748f0a579
Fix notif kws that start/end with non-word chars
...
Only prepend / append word bounary characters if the search
expression starts or ends with a word character, otherwise they
don't work because there's no word bounary between whitespace and
a non-word char.
2017-10-05 11:33:30 +01:00
436ee0a2ea
Also include the room_id
...
as really it's part of the event ID
2017-09-18 15:58:38 +01:00
b393f5db51
Use .get - it's much shorter
2017-09-18 15:50:26 +01:00
a2562f9d74
Add support for event_id_only push format
...
Param in the data dict of a pusher that tells an HTTP pusher to
send just the event_id of the event it's notifying about and the
notification counts. For clients that want to go & fetch the body
of the event themselves anyway.
2017-09-18 15:39:39 +01:00
bfde076022
Increase cache hit ratio for push
...
We don't update the cache in all code paths, which causes subsequent
calls to miss the cache
2017-07-14 16:11:26 +01:00
8d26385d76
Add more metrics to push rule evaluation
2017-07-13 14:37:30 +01:00
f60218ec41
Push: Don't acquire lock unless necessary
2017-07-13 11:23:53 +01:00
f502b0dea1
Perf: Don't filter events for push
...
We know the users are joined and we can explicitly check for if they are
ignoring the user, so lets do that.
2017-07-07 14:04:40 +01:00
3d31b39297
Merge pull request #2332 from matrix-org/rav/fix_pushes
...
Fix caching error in the push evaluator
2017-07-05 11:10:53 +01:00
73cfe48031
Fix caching error in the push evaluator
...
Initialising `result` to `{}` in the parameters meant that every call to
_flatten_dict used the *same* target dictionary.
I'm hopeful this will fix https://github.com/matrix-org/synapse/issues/2270 ,
but I suspect it won't. (This code seems to have been here since forever,
unlike the bug, and I don't really think it explains the observed
behaviour). Still, it makes it hard to investigate the problem.
2017-07-05 00:28:43 +01:00
75eba3b07d
Fix TravisCI tests for PR #2301
2017-06-23 15:15:18 +02:00
1591eddaea
Add configuration parameter to allow redaction of content from push messages for google/apple devices
2017-06-23 13:01:04 +02:00
cca94272fa
Fix typo when getting app name
2017-06-06 11:50:07 +01:00
66b121b2fc
Fix wrong number of arguments
2017-06-06 11:46:38 +01:00
11c2a3655f
Only load jinja2 templates once
...
Instead of every time a new email pusher is created, as loading jinja2
templates is slow.
2017-05-22 17:48:58 +01:00
6489455bed
Comment
2017-05-22 16:22:04 +01:00
d668caa79c
Remove spurious log level guards
2017-05-22 16:21:06 +01:00
7fb80b5eae
Check if current event is a membership event
2017-05-22 15:02:12 +01:00
2d17b09a6d
Add debug logging
2017-05-22 15:01:36 +01:00
24c8f38784
Comment
2017-05-22 14:59:27 +01:00
25f03cf8e9
Use tuple unpacking
2017-05-22 14:58:22 +01:00
270e1c904a
Speed up calculating push rules
2017-05-19 16:51:05 +01:00
1c1c0257f4
Move invalidation cb to its own structure
2017-05-19 11:44:11 +01:00
cafe659f72
Store ActionGenerator in HomeServer
2017-05-19 10:09:56 +01:00
72ed8196b3
Don't push users who have left
2017-05-18 17:48:36 +01:00
760625acba
Make get_if_app_services_interested_in_user faster
2017-05-18 16:34:44 +01:00
c57789d138
Remove size of push get_rules cache
2017-05-18 16:17:23 +01:00
056ba9b795
Add comment
2017-05-18 11:45:56 +01:00
66d8ffabbd
Faster push rule calculation via push specific cache
...
We add a push rule specific cache that ensures that we can reuse
calculated push rules appropriately when a user join/leaves.
2017-05-17 16:55:40 +01:00
78f306a6f7
Revert "Speed up filtering of a single event in push"
...
This reverts commit 421fdf7460
2017-05-08 13:07:41 +01:00
421fdf7460
Speed up filtering of a single event in push
2017-04-28 09:52:36 +01:00
acb58bfb6a
fix up
2017-04-25 15:39:19 +01:00
f7181615f2
Don't specify default as dict
2017-04-25 15:22:59 +01:00
d9aa645f86
Reduce size of joined_user cache
...
The _get_joined_users_from_context cache stores a mapping from user_id
to avatar_url and display_name. Instead of storing those in a dict,
store them in a namedtuple as that uses much less memory.
We also try converting the string to ascii to further reduce the size.
2017-04-25 14:38:51 +01:00
5058292537
Support authenticated SMTP
...
Closes (SYN-714) #1385
Signed-off-by: Daniel Dent <matrixcontrib@contactdaniel.net>
2017-04-05 21:01:08 -07:00
9cee0ce7db
Merge pull request #2075 from matrix-org/erikj/cache_speed
...
Speed up cached function access
2017-03-31 10:10:56 +01:00
86780a8bc3
Don't convert to deferreds when not necessary
2017-03-30 14:14:36 +01:00
a3810136fe
Cache glob to regex at a higher level for push
2017-03-29 15:53:14 +01:00
6957bfdca6
Don't recreate so many sets
2017-03-16 11:54:26 +00:00
8ffbe43ba1
Get current state by using current_state_events table
2017-03-10 17:39:35 +00:00
Richard van der Hoff