Commit Graph

126 Commits (0cc6509cfe88beec4236365e07cac5e89d70ad48)

Author SHA1 Message Date
Patrick Cloke d38d0dffc9
Use StrCollection in additional places. (#16301) 2023-09-13 07:57:19 -04:00
reivilibre e937e2111a
Add the ability to use `G` (GiB) and `T` (TiB) suffixes in configuration options that refer to numbers of bytes. (#16219)
* Add more suffixes to `parse_size`

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

---------

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2023-09-06 14:01:10 +01:00
dependabot[bot] 001fc7bd19
Bump ruff from 0.0.277 to 0.0.286 (#16198)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-29 09:41:43 -04:00
Sean Quah 68dcd2cbcb
Re-type config paths in `ConfigError`s to be `StrSequence`s (#15615)
Part of #14809.

Signed-off-by: Sean Quah <seanq@matrix.org>
2023-05-18 11:11:30 +01:00
David Robertson 796a4b7482
Prefer `type(x) is int` to `isinstance(x, int)` (#14945)
* Perfer `type(x) is int` to `isinstance(x, int)`

This covered all additional instances I could see where `x` was
user-controlled.
The remaining cases are

```
$ rg -s 'isinstance.*[^_]int'
tests/replication/_base.py
576:        if isinstance(obj, int):

synapse/util/caches/stream_change_cache.py
136:        assert isinstance(stream_pos, int)
214:        assert isinstance(stream_pos, int)
246:        assert isinstance(stream_pos, int)
267:        assert isinstance(stream_pos, int)

synapse/replication/tcp/external_cache.py
133:        if isinstance(result, int):

synapse/metrics/__init__.py
100:        if isinstance(calls, (int, float)):

synapse/handlers/appservice.py
262:        assert isinstance(new_token, int)

synapse/config/_util.py
62:        if isinstance(p, int):
```

which cover metrics, logic related to `jsonschema`, and replication and
data streams. AFAICS these are all internal to Synapse

* Changelog
2023-01-31 10:33:07 +00:00
Richard van der Hoff 5e5c8150d7
Generate missing configuration files at startup (#13615)
If things like the signing key file are missing, let's just try to generate
them on startup.

Again, this is useful for k8s-like deployments where we just want to generate
keys on the first run.
2022-08-26 11:26:06 +00:00
Andrew Morgan 6faaf76a32
Remove 'anonymised' from the phone home stats documentation (#13321) 2022-07-19 12:38:29 +00:00
Shay 493c2fc44a
Remove code generating comments in configuration file (#12941) 2022-06-14 07:53:42 -07:00
David Robertson d38d242411
Reload cache factors from disk on SIGHUP (#12673) 2022-05-11 13:43:22 +00:00
Patrick Cloke 4586119f0b
Add missing type hints to config classes. (#12402) 2022-04-11 12:07:23 -04:00
David Robertson f3f0ab10fe
Move scripts directory inside synapse, exposing as setuptools entry_points (#12118)
* Two scripts are basically entry_points already
* Move and rename scripts/* to synapse/_scripts/*.py
* Delete sync_room_to_group.pl
* Expose entry points in setup.py
* Update linter script and config
* Fixup scripts & docs mentioning scripts that moved

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2022-03-02 13:00:16 +00:00
Patrick Cloke 55669bd3de
Add missing type hints to config base classes (#11377) 2021-11-23 15:21:19 +00:00
Patrick Cloke f4b1a9a527
Require direct references to configuration variables. (#10985)
This removes the magic allowing accessing configurable
variables directly from the config object. It is now required
that a specific configuration class is used (e.g. `config.foo`
must be replaced with `config.server.foo`).
2021-10-06 10:47:41 -04:00
Patrick Cloke 94b620a5ed
Use direct references for configuration variables (part 6). (#10916) 2021-09-29 06:44:15 -04:00
Sean Quah 6c83c27107
Fix race conditions when creating media store and config directories (#10913) 2021-09-27 11:29:23 +01:00
Brendan Abolivier ae2714c1f3
Allow using several custom template directories (#10587)
Allow using several directories in read_templates.
2021-08-17 10:23:14 +00:00
Brendan Abolivier 08c8469322
Remove support for ACME v1 (#10194)
Fixes #9778

ACME v1 has been fully decommissioned for existing installs on June 1st 2021(see https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430/27), so we can now safely remove it from Synapse.
2021-06-17 18:56:48 +01:00
Jonathan de Jong 4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Richard van der Hoff eaada74075
JWT OIDC secrets for Sign in with Apple (#9549)
Apple had to be special. They want a client secret which is generated from an EC key.

Fixes #9220. Also fixes #9212 while I'm here.
2021-03-09 15:03:37 +00:00
Erik Johnston 2927921942
Clean up `ShardedWorkerHandlingConfig` (#9466)
* Split ShardedWorkerHandlingConfig

This is so that we have a type level understanding of when it is safe to
call `get_instance(..)` (as opposed to `should_handle(..)`).

* Remove special cases in ShardedWorkerHandlingConfig.

`ShardedWorkerHandlingConfig` tried to handle the various different ways
it was possible to configure federation senders and pushers. This led to
special cases that weren't hit during testing.

To fix this the handling of the different cases is moved from there and
`generic_worker` into the worker config class. This allows us to have
the logic in one place and allows the rest of the code to ignore the
different cases.
2021-02-24 13:23:18 +00:00
Patrick Cloke a1901abd6b
Add documentation and type hints to parse_duration. (#9432) 2021-02-19 08:32:21 -05:00
Eric Eastwood 0a00b7ff14
Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Richard van der Hoff 18ab35284a Merge branch 'social_login' into develop 2021-02-01 17:28:37 +00:00
Richard van der Hoff 4167494c90
Replace username picker with a template (#9275)
There's some prelimiary work here to pull out the construction of a jinja environment to a separate function.

I wanted to load the template at display time rather than load time, so that it's easy to update on the fly. Honestly, I think we should do this with all our templates: the risk of ending up with malformed templates is far outweighed by the improved turnaround time for an admin trying to update them.
2021-02-01 15:52:50 +00:00
Patrick Cloke e54746bdf7
Clean-up the template loading code. (#9200)
* Enables autoescape by default for HTML files.
* Adds a new read_template method for reading a single template.
* Some logic clean-up.
2021-01-27 10:59:50 -05:00
Richard van der Hoff fa50e4bf4d
Give `public_baseurl` a default value (#9159) 2021-01-20 12:30:41 +00:00
Richard van der Hoff ab7a24cc6b
Better formatting for config errors from modules (#8874)
The idea is that the parse_config method of extension modules can raise either a ConfigError or a JsonValidationError,
and it will be magically turned into a legible error message. There's a few components to it:

* Separating the "path" and the "message" parts of a ConfigError, so that we can fiddle with the path bit to turn it
   into an absolute path.
* Generally improving the way ConfigErrors get printed.
* Passing in the config path to load_module so that it can wrap any exceptions that get caught appropriately.
2020-12-08 14:04:35 +00:00
Patrick Cloke 34ff8da83b
Convert additional templates to Jinja (#8444)
This converts a few more of our inline HTML templates to Jinja. This is somewhat part of #7280 and should make it a bit easier to customize these in the future.
2020-10-02 11:15:53 +01:00
Andrew Morgan c77c4a2fcd Merge branch 'master' into develop 2020-09-24 17:00:33 +01:00
Andrew Morgan 3f4a2a7064
Hotfix: disable autoescape by default when rendering Jinja2 templates (#8394)
#8037 changed the default `autoescape` option when rendering Jinja2 templates from `False` to `True`. This caused some bugs, noticeably around redirect URLs being escaped in SAML2 auth confirmation templates, causing those URLs to break for users.

This change returns the previous behaviour as it stood. We may want to look at each template individually and see whether autoescaping is a good idea at some point, but for now lets just fix the breakage.
2020-09-24 16:24:08 +01:00
Erik Johnston 04cc249b43
Add experimental support for sharding event persister. Again. (#8294)
This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
2020-09-14 10:16:41 +01:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Brendan Abolivier 9f8abdcc38
Revert "Add experimental support for sharding event persister. (#8170)" (#8242)
* Revert "Add experimental support for sharding event persister. (#8170)"

This reverts commit 82c1ee1c22.

* Changelog
2020-09-04 10:19:42 +01:00
Erik Johnston 82c1ee1c22
Add experimental support for sharding event persister. (#8170)
This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
2020-09-02 15:48:37 +01:00
Andrew Morgan e04e465b4d
Use the default templates when a custom template file cannot be found (#8037)
Fixes https://github.com/matrix-org/synapse/issues/6583
2020-08-17 17:05:00 +01:00
Erik Johnston 649a7ead5c
Add ability to run multiple pusher instances (#7855)
This reuses the same scheme as federation sender sharding
2020-07-16 14:06:28 +01:00
Dagfinn Ilmari Mannsåker a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Brendan Abolivier cc9eceb00d
Don't crash when one of the configuration files is empty (#7341)
If the admin adds a `.yaml` file that's either empty or doesn't parse into a dict to a config directory (e.g. `conf.d` for debs installs), stuff like https://github.com/matrix-org/synapse/issues/7322 would happen. This PR checks that the file is correctly parsed into a dict, or ignores it with a warning if it parses into any other type (including `None` for empty files).

Fixes https://github.com/matrix-org/synapse/issues/7322
2020-04-27 15:01:03 +02:00
Richard van der Hoff 5a709630bf
Fix --help commandline argument (#7249)
I don't really remember why this was so complicated; I think it dates
back to the time when we had to instantiate the Config classes before
we could call `add_arguments` - ie before #5597. In any case, I don't
think there's a good reason for it any more, and the impact of it
being complicated is that `--help` doesn't work correctly.
2020-04-09 12:44:37 +01:00
Richard van der Hoff c165c1233b
Improve database configuration docs (#6988)
Attempts to clarify the sample config for databases, and add some stuff about
tcp keepalives to `postgres.md`.
2020-03-20 15:24:22 +00:00
Richard van der Hoff 97a42bbc3a
Add a warning about indentation to generated config (#6920)
Fixes #6916.
2020-02-14 16:22:30 +00:00
Amber Brown f743108a94
Refactor HomeserverConfig so it can be typechecked (#6137) 2019-10-10 09:39:35 +01:00
Jorik Schellekens 6d97843793
Config templating (#5900)
Template config files

* Imagine a system composed entirely of x, y, z etc and the basic operations..

Wait George, why XOR? Why not just neq?

George: Eh, I didn't think of that..

Co-Authored-By: Erik Johnston <erik@matrix.org>
2019-08-28 13:12:22 +01:00
Erik Johnston f44354e17f Clean up arg name and remove lying comment 2019-07-16 11:39:40 +01:00
Erik Johnston 03cc8c4b5d Fix invoking add_argument from homeserver.py 2019-07-15 14:25:05 +01:00
Erik Johnston fdefb9e29a Move creation of ArgumentParser to caller 2019-07-15 14:09:35 +01:00
Erik Johnston 37b524f971 Fix up comments 2019-07-15 14:09:35 +01:00
Erik Johnston 823e13ddf4 Change add_arguments to be a static method 2019-07-15 14:09:33 +01:00
Erik Johnston 9f3c0a8556 Add basic admin cmd app 2019-07-02 17:12:48 +01:00
Richard van der Hoff 3f8a252dd8 Add "--open-private-ports" cmdline option
This is helpful when generating a config file for running synapse under docker.
2019-06-24 14:15:34 +01:00