Commit Graph

17571 Commits (bbde4038dff379fdf48b914782a73a6889135a56)

Author SHA1 Message Date
Erik Johnston db131b6b22
Change the default log config to reduce disk I/O and storage (#8040)
* Change default log config to buffer by default.

This batches up writes to the filesystem, which is more efficient for
disk I/O. This means that it can take some time for logs to get written
to disk. Note that ERROR logs (and above) immediately flush the buffer.

This only effects new installs, as we only write the log config if
started with `--generate-config` (in the same way we do for generating
signing keys).

* Default to keeping last 4 days of logs.

This hopefully reduces the amount of logs kept for new servers. Keeping
the last 1GB of logs is likely overkill for new servers, but equally may
not be enough for busy ones.

Instead, we keep the last four days worth of logs, enough so that admins
can investigate any problems that happened over e.g. a long weekend.
2020-08-11 18:09:46 +01:00
Erik Johnston 64e5bb0dc8 Newsfile 2020-08-11 18:03:26 +01:00
Erik Johnston 0f1afbe8dc Change HomeServer definition to work with typing.
Duplicating function signatures between server.py and server.pyi is
silly. This commit changes that by changing all `build_*` methods to
`get_*` methods and changing the `_make_dependency_method` to work work
as a descriptor that caches the produced value.

There are some changes in other files that were made to fix the typing
in server.py.
2020-08-11 18:00:17 +01:00
Richard van der Hoff 0cb169900e
Implement login blocking based on SAML attributes (#8052)
Hopefully this mostly speaks for itself. I also did a bit of cleaning up of the
error handling.

Fixes #8047
2020-08-11 16:08:10 +01:00
Richard van der Hoff aa827b6ad7 Merge remote-tracking branch 'origin/master' into develop 2020-08-10 23:42:12 +01:00
Richard van der Hoff 39c3f68758
Stop uploading -py3 docker images (#8056) 2020-08-10 23:41:50 +01:00
Richard van der Hoff fcbab08cbd
Add an assertion on prev_events in create_new_client_event (#8041)
I think this would have caught all the cases in
https://github.com/matrix-org/synapse/issues/7642 - and I think a 500 makes
more sense here than a 403
2020-08-10 12:29:47 +01:00
Brendan Abolivier cdbb8e6d6e
Implement new experimental push rules (#7997)
With an undocumented configuration setting to enable them for specific users.
2020-08-10 11:48:01 +01:00
Brendan Abolivier 5c43c43240
Typo 2020-08-10 11:23:24 +01:00
Brendan Abolivier 1a3aabcf3f
Lint 2020-08-10 11:13:21 +01:00
Brendan Abolivier cee6c6012e
why mypy why 2020-08-10 11:10:34 +01:00
Patrick Cloke 7f837959ea
Convert directory, e2e_room_keys, end_to_end_keys, monthly_active_users database to async (#8042) 2020-08-07 13:36:29 -04:00
Patrick Cloke f3fe6961b2
Convert additional database stores to async/await (#8045) 2020-08-07 12:17:17 -04:00
Travis Ralston 1048ed2afa
Clarify that undoing a shutdown might not be possible (#8010) 2020-08-07 17:16:24 +01:00
Richard van der Hoff de6f892065
Add a comment about SSLv23_METHOD (#8043) 2020-08-07 15:14:29 +01:00
Erik Johnston 2f9fd5ab00
Don't log OPTIONS request at INFO (#8049) 2020-08-07 14:53:05 +01:00
Patrick Cloke 4e874ed593
Remove unnecessary maybeDeferred calls (#8044) 2020-08-07 09:44:48 -04:00
Erik Johnston 7620912d84
Add health check endpoint (#8048) 2020-08-07 14:21:24 +01:00
David Vo 4dd27e6d11
Reduce unnecessary whitespace in JSON. (#7372) 2020-08-07 08:02:55 -04:00
Brendan Abolivier 367e9e6e9e
Lint 2020-08-06 17:57:58 +01:00
Brendan Abolivier bf33d5c457
Incorporate review 2020-08-06 17:52:34 +01:00
Brendan Abolivier 2ffd6783c7
Revert #7736 (#8039) 2020-08-06 17:15:35 +01:00
Patrick Cloke fe6cfc80ec
Convert some util functions to async (#8035) 2020-08-06 08:39:35 -04:00
Patrick Cloke d4a7829b12
Convert synapse.api to async/await (#8031) 2020-08-06 08:30:06 -04:00
Patrick Cloke c36228c403
Convert run_as_background_process inner function to async. (#8032) 2020-08-06 08:20:42 -04:00
Patrick Cloke 66f24449dd
Improve performance of the register endpoint (#8009) 2020-08-06 08:09:55 -04:00
Brendan Abolivier 118a9eafb3 Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/new_push_rules 2020-08-06 10:52:50 +01:00
Brendan Abolivier dd11f575a2
Incorporate review 2020-08-06 10:52:26 +01:00
Erik Johnston 079bc3c8e3
Fixup worker doc (again) (#8000) 2020-08-06 10:35:59 +01:00
Erik Johnston a7bdf98d01
Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00
Richard van der Hoff 0a86850ba3
Stop the parent process flushing the logs on exit (#8012)
This solves the problem that the first few lines are logged twice on matrix.org. Hopefully the comments explain it.
2020-08-05 09:35:17 +01:00
Richard van der Hoff 8b786db323
bug report template: move comments into comment (#8030) 2020-08-05 09:34:42 +01:00
Andrew Morgan 7cac9006d6
Spruce up the check-newsfragment CI output (#8024)
This PR:

* Reduces the amount of noise in the `check-newsfragment` CI output by hiding the dependency installation output by default.
* Prints a link to the changelog/debian changelog section of the contributing guide if an error is found.
2020-08-04 22:10:23 +01:00
Patrick Cloke 8ff2deda72
Fix async/await calls for broken media providers. (#8027) 2020-08-04 09:44:25 -04:00
Patrick Cloke 88a3ff12f0
Convert the SimpleHttpClient to async. (#8016) 2020-08-04 07:22:04 -04:00
Patrick Cloke e19de43eb5
Convert streams to async. (#8014) 2020-08-04 07:21:47 -04:00
Richard van der Hoff 916cf2d439
re-implement daemonize (#8011)
This has long been something I've wanted to do. Basically the `Daemonize` code
is both too flexible and not flexible enough, in that it offers a bunch of
features that we don't use (changing UID, closing FDs in the child, logging to
syslog) and doesn't offer a bunch that we could do with (redirecting stdout/err
to a file instead of /dev/null; having the parent not exit until the child is
running).

As a first step, I've lifted the Daemonize code and removed the bits we don't
use. This should be a non-functional change. Fixing everything else will come
later.
2020-08-04 10:03:41 +01:00
Andrew Morgan 481f76c7aa
Remove signature check on v1 identity server lookups (#8001)
We've [decided](https://github.com/matrix-org/synapse/issues/5253#issuecomment-665976308) to remove the signature check for v1 lookups.

The signature check has been removed in v2 lookups. v1 lookups are currently deprecated. As mentioned in the above linked issue, this verification was causing deployments for the vector.im and matrix.org IS deployments, and this change is the simplest solution, without being unjustified.

Implementations are encouraged to use the v2 lookup API as it has [increased privacy benefits](https://github.com/matrix-org/matrix-doc/pull/2134).
2020-08-03 21:56:43 +01:00
Andrew Morgan 5d92a1428c
Prevent join->join membership transitions changing member count (#7977)
`StatsHandler` handles updates to the `current_state_delta_stream`, and updates room stats such as the amount of state events, joined users, etc.

However, it counts every new join membership as a new user entering a room (and that user being in another room), whereas it's possible for a user's membership status to go from join -> join, for instance when they change their per-room profile information.

This PR adds a check for join->join membership transitions, and bails out early, as none of the further checks are necessary at that point.

Due to this bug, membership stats in many rooms have ended up being wildly larger than their true values. I am not sure if we also want to include a migration step which recalculates these statistics (possibly using the `_populate_stats_process_rooms` bg update).

Bug introduced in the initial implementation https://github.com/matrix-org/synapse/pull/4338.
2020-08-03 21:54:24 +01:00
Patrick Cloke 6812509807
Implement handling of HTTP HEAD requests. (#7999) 2020-08-03 08:45:42 -04:00
Patrick Cloke 2a89ce8cd4
Convert the crypto module to async/await. (#8003) 2020-08-03 08:29:01 -04:00
Michael Albert b6c6fb7950
Allow guests to operate in encrypted rooms (#7314)
Signed-off-by: Michael Albert <michael.albert@awesome-technologies.de>
2020-08-03 12:13:49 +01:00
Patrick Cloke 3b415e23a5
Convert replication code to async/await. (#7987) 2020-08-03 07:12:55 -04:00
Patrick Cloke db5970ac6d
Convert ACME code to async/await. (#7989) 2020-08-03 07:09:33 -04:00
Brendan Abolivier e2f1cccc8a
Fix PUT /pushrules to use the right rule IDs 2020-08-03 11:52:52 +01:00
Brendan Abolivier 1678057b56
Back out the database hack and replace it with a temporary config setting 2020-08-03 11:22:22 +01:00
Patrick Cloke d1008fe949
Fix some comments and types in service notices (#7996) 2020-07-31 16:22:06 -04:00
Erik Johnston 394be6a0e6
Merge pull request #8008 from matrix-org/erikj/add_rate_limiting_to_joins
Add ratelimiting on joins
2020-07-31 18:21:48 +01:00
Erik Johnston faba873d4b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/add_rate_limiting_to_joins 2020-07-31 15:07:01 +01:00
Erik Johnston 9b3ab57acd Newsfile 2020-07-31 15:06:56 +01:00