47a9ba435d
Use match rather than search
2018-10-24 09:54:57 +01:00
e0b9d5f0af
Merge pull request #4075 from matrix-org/rav/fix_pusher_logcontexts
...
Clean up the way logcontexts and threads work in the pushers
2018-10-24 09:53:57 +01:00
dacbeb2e03
Comment
2018-10-24 09:47:49 +01:00
810715f79a
Rename resolve_events_with_factory
2018-10-24 09:44:22 +01:00
cb23aa4c42
Merge pull request #4063 from matrix-org/erikj/cleaup_alias_creation
...
Clean up room alias creation
2018-10-24 09:40:27 +01:00
c573794b22
Fix start_pushers vs _start_pushers confusion
2018-10-24 09:24:55 +01:00
e564306e31
sanity-check the is_processing flag
...
... and rename it, for even more sanity
2018-10-24 09:23:33 +01:00
a67d8ace9b
remove errant exception and style
2018-10-23 17:44:39 +01:00
43c3f0b02f
Merge pull request #3969 from turt2live/travis/fix-federated-group-requests
...
Handle HttpResponseException more safely for federated groups
2018-10-23 10:41:04 -06:00
3e704822be
Comments help
2018-10-23 10:25:31 -06:00
329d18b39c
remove white space
2018-10-23 15:27:20 +01:00
6105c6101f
fix race condiftion in calling initialise_reserved_users
2018-10-23 15:24:58 +01:00
b3f6dddad2
Give some more things logcontexts ( #4077 )
2018-10-23 14:29:17 +01:00
5c445114d3
Correctly account for cpu usage by background threads ( #4074 )
...
Wrap calls to deferToThread() in a thing which uses a child logcontext to
attribute CPU usage to the right request.
While we're in the area, remove the logcontext_tracer stuff, which is never
used, and afaik doesn't work.
Fixes #4064
2018-10-23 13:12:32 +01:00
026cd91ac8
Run PusherPool.start as a background process
...
We don't do anything with the result, so this is needed to give this code a
logcontext.
2018-10-22 16:12:11 +01:00
f749607c91
Make on_started synchronous too
...
This brings it into line with on_new_notifications and on_new_receipts. It
requires a little bit of hoop-jumping in EmailPusher to load the throttle
params before the first loop.
2018-10-22 16:12:11 +01:00
e7a16c6210
Remove redundant run_as_background_process() from pusherpool
...
`on_new_notifications` and `on_new_receipts` in `HttpPusher` and `EmailPusher`
now always return synchronously, so we can remove the `defer.gatherResults` on
their results, and the `run_as_background_process` wrappers can be removed too
because the PusherPool methods will now complete quickly enough.
2018-10-22 16:12:11 +01:00
c7273c11bc
Give pushers their own background logcontext
...
Each pusher has its own loop which runs for as long as it has work to do. This
should run in its own background thread with its own logcontext, as other
similar loops elsewhere in the system do - which means that CPU usage is
consistently attributed to that loop, rather than to whatever request happened
to start the loop.
2018-10-22 16:12:11 +01:00
5110f4e425
move get_all_pushers call down
...
simplifies the interface to _start_pushers
2018-10-22 16:12:11 +01:00
04277d0ed8
Factor PusherPool._start_pusher out of _start_pushers
...
... and use it from start_pusher_by_id. This mostly simplifies
start_pusher_by_id.
2018-10-22 16:12:11 +01:00
3e8b02c939
Rename _refresh_pusher
...
This is public (or at least, called from outside the class), so ought to have a
better name.
2018-10-22 16:12:11 +01:00
7aea00069c
Merge pull request #4076 from matrix-org/rav/fix_init_logcontexts
...
Run MAU queries as background processes
2018-10-22 14:46:59 +01:00
911db96658
Merge pull request #4073 from matrix-org/rav/require_psutil
...
Make psutil an explicit dependency
2018-10-22 12:33:21 +01:00
a6f421e812
Run MAU queries as background processes
...
Fixes #3820
2018-10-20 02:14:35 +01:00
e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) ( #4068 )
2018-10-20 11:16:55 +11:00
e5b52d0f94
Make psutil an explicit dependency
...
As of #4027 , we require psutil to be installed, so it should be in our
dependency list. We can also remove some of the conditional import code
introduced by #992 .
Fixes #4062 .
2018-10-19 21:51:15 +01:00
f62c597d14
Merge pull request #4069 from matrix-org/rav/fix_email_templates_4065
...
Calculate absolute path for email templates
2018-10-19 16:33:54 +01:00
67f7b9cb50
pep8
2018-10-19 16:06:59 +01:00
056f099126
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/purge_state_groups
2018-10-19 15:48:59 +01:00
47a9da28ca
Batch process handling state groups
2018-10-19 15:48:15 +01:00
cc325c7069
Calculate absolute path for email templates
2018-10-19 14:01:59 +01:00
e404ba9aac
Fix manhole on py3 (pt 2) ( #4067 )
2018-10-19 22:26:00 +11:00
b69216f768
Make the metrics less racy ( #4061 )
2018-10-19 21:45:45 +11:00
9fafdfa97d
Anchor returned regex to start and end of string
2018-10-19 10:22:45 +01:00
084046456e
Add config option to control alias creation
2018-10-19 10:22:45 +01:00
74e7617083
Clean up room alias creation
2018-10-19 10:11:56 +01:00
a36b0ec195
make a bytestring
2018-10-19 09:24:00 +11:00
c632bc8654
Merge branch 'master' into develop
2018-10-18 15:07:03 +01:00
926da4dda8
0.33.7
2018-10-18 14:57:32 +01:00
c7d0f34a3c
v0.33.7rc2
2018-10-17 17:40:19 +01:00
0fd2321629
Fix incorrect truncation in get_missing_events
...
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.
We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-17 17:35:26 +01:00
c8f2c19991
Put the warning blob at the top of the file
2018-10-17 16:56:22 +01:00
1519572961
Ship the email templates as package_data
...
move the example email templates into the synapse package so that they can be
used as package data, which should mean that all of the packaging mechanisms
(pip, docker, debian, arch, etc) should now come with the example templates.
In order to grandfather in people who relied on the templates being in the old
place, check for that situation and fall back to using the defaults if the
templates directory does not exist.
2018-10-17 16:46:02 +01:00
f6a0a02a62
Fix bug where we raised StopIteration in a generator
...
This made python 3.7 unhappy
2018-10-17 16:10:52 +01:00
d6a7797dd1
Fix roomlist since tokens on Python 3 ( #4046 )
...
Thanks @Half-Shot !!!
2018-10-17 23:04:55 +11:00
c6584f4b5f
clean up config error logic and imports
2018-10-17 11:36:41 +01:00
fc0f13dd03
Fix incorrect truncation in get_missing_events
...
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.
We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-16 21:10:04 +01:00
4a28d3d36f
Update event_auth table for rejected events
2018-10-16 16:28:42 +01:00
15133477ee
Fix up use of resolve_events_with_factory
2018-10-16 16:28:42 +01:00
6bd856caa2
User event.sender rather than alias event.user_id
2018-10-16 16:16:13 +01:00
e238013c44
Add v2 state res algorithm.
...
We hook this up to the vdh test room version.
2018-10-16 16:16:13 +01:00
b8a5b0097c
Various cleanups in the federation client code ( #4031 )
...
- Improve logging: log things in the right order, include destination and txids
in all log lines, don't log successful responses twice
- Fix the docstring on TransportLayerClient.send_transaction
- Don't use treq.request, which is overcomplicated for our purposes: just use a
twisted.web.client.Agent.
- simplify the logic for setting up the bodyProducer
- fix bytes/str confusions
2018-10-16 10:44:49 +01:00
f726f2dc6c
version bump
2018-10-15 22:21:45 +11:00
03c11032c3
Merge pull request #4019 from matrix-org/dbkr/e2e_backups
...
E2E backups
2018-10-15 10:19:43 +01:00
06bc8d2fe5
synapse/app: frontend_proxy.py: actually make workers work on py3
2018-10-14 20:08:39 +03:00
fb216a22db
synapse/visibility.py: fix SyntaxError on py3.7
2018-10-14 20:08:17 +03:00
1ccafb0c5e
no need to join room if creator
2018-10-13 21:14:21 +01:00
164f8e4843
isort
2018-10-12 15:11:59 -06:00
7bb651de6a
More sane handling of group errors and pep8
2018-10-12 14:53:30 -06:00
e3586f7c06
Merge branch 'develop' into travis/fix-federated-group-requests
2018-10-12 14:49:58 -06:00
a2bfb778c8
improve auto room join logic, comments and tests
2018-10-12 18:17:36 +01:00
a45f2c3a00
missed one
2018-10-12 14:33:55 +01:00
381d2cfdf0
Make workers work on Py3 ( #4027 )
2018-10-13 00:14:08 +11:00
67a1e315cc
Fix up comments
2018-10-12 13:49:48 +01:00
8c0ff0287a
Linting soothes the savage PEP8 monster
2018-10-12 13:47:43 +01:00
306361b31b
Misc PR feedback bits
2018-10-12 11:48:56 +01:00
bddfad253a
Don't mangle exceptions
2018-10-12 11:48:02 +01:00
86ef9760a7
Split /room_keys/version into 2 servlets
2018-10-12 11:35:08 +01:00
83e72bb2f0
PR feedback pt. 1
2018-10-12 11:26:18 +01:00
8ddd0f273c
Comments on get_all_new_events_stream
...
just some docstrings to clarify the behaviour here
2018-10-12 09:55:41 +01:00
7e561b5c1a
Add description to counter metric
2018-10-10 11:41:15 +01:00
3cbe8331e6
Track number of non-empty sync responses instead
2018-10-10 11:23:17 +01:00
395276b405
Append _total to metric and fix up spelling
2018-10-10 09:24:39 +01:00
b8d9e108be
Fix mergefail
2018-10-09 18:04:21 +01:00
bdc27d6716
Add metric to count lazy member sync requests
2018-10-09 14:17:52 +01:00
d3464ce708
isort
2018-10-09 10:33:59 +01:00
9eb1a79100
Merge pull request #4008 from matrix-org/erikj/log_looping_exceptions
...
Log looping call exceptions
2018-10-09 10:12:10 +01:00
dc045ef202
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
2018-10-09 10:05:02 +01:00
f4a4dbcad1
Apparently this blank line is Very Important
2018-10-09 09:47:04 +01:00
0c905ee015
be python3 compatible
2018-10-09 09:39:13 +01:00
6982320572
Remove unnecessary extra function call layer
2018-10-08 14:06:19 +01:00
495975e231
Optimisation for filter_events_for_server
...
We're better off hashing just the event_id than the whole ((type, state_key),
event_id) tuple - so use a dict instead of a set.
Also, iteritems > items.
2018-10-08 13:46:52 +01:00
8a1817f0d2
Use errback pattern and catch async failures
2018-10-08 13:29:47 +01:00
497444f1fd
Don't reuse backup versions
...
Since we don't actually delete the keys, just mark the versions
as deleted in the db rather than actually deleting them, then we
won't reuse versions.
Fixes https://github.com/vector-im/riot-web/issues/7448
2018-10-05 15:08:36 +01:00
f7199e8734
Log looping call exceptions
...
If a looping call function errors, then it kills the loop entirely.
Currently it throws away the exception logs, so we should make it
actually log them.
Fixes #3929
2018-10-05 11:24:12 +01:00
ed82043efb
Merge branch 'develop' into matthew/autocreate_autojoin
2018-10-04 17:26:59 +01:00
2dadc092b8
move logic into register, fix room alias localpart bug, tests
2018-10-04 17:00:27 +01:00
c6dbd216e6
Merge pull request #3995 from matrix-org/rav/no_deextrem_outliers
...
Fix bug in forward_extremity update logic
2018-10-04 16:27:05 +01:00
4917ff5523
Add state_group index to event_to_state_groups
...
This is needed to efficiently check for unreferenced state groups during
purge.
2018-10-04 16:03:08 +01:00
17d585753f
Delete unreferened state groups during purge
2018-10-04 16:03:06 +01:00
d86794325f
Merge branch 'master' into develop
2018-10-04 22:41:52 +10:00
dd59dfc51f
full version
2018-10-04 22:37:55 +10:00
a59d899668
Pin to prometheus_client<0.4 to avoid renaming all of our metrics
2018-10-03 17:20:15 +01:00
8935ec5a93
Merge pull request #3999 from matrix-org/erikj/fix_3pid_invite_rejetion
...
Fix handling of rejected threepid invites
2018-10-03 14:47:41 +01:00
81e2813948
Merge pull request #3996 from matrix-org/erikj/fix_bg_iteration
...
Fix exception in background metrics collection
2018-10-03 14:14:38 +01:00
52e6e815be
Sanitise error messages when user doesn't have permission to invite
2018-10-03 14:13:07 +01:00
69e857853f
Fix handling of rejected threepid invites
2018-10-03 11:57:30 +01:00
495a9d06bb
Fix exception handling in fetching remote profiles
2018-10-03 11:34:30 +01:00
7c570bff74
Fix exception in background metrics collection
...
We attempted to iterate through a list on a separate thread without
doing the necessary copying.
2018-10-03 11:28:01 +01:00
9693625e55
actually exclude outliers
2018-10-03 10:19:41 +01:00
Erik Johnston