Commit Graph

216 Commits (release-v1.76)

Author SHA1 Message Date
reivilibre 22cc93afe3
Enable Faster Remote Room Joins against worker-mode Synapse. (#14752)
* Enable Complement tests for Faster Remote Room Joins on worker-mode

* (dangerous) Add an override to allow Complement to use FRRJ under workers

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

* Fix race where we didn't send out replication notification

* MORE HACKS

* Fix get_un_partial_stated_rooms_token to take instance_name

* Fix bad merge

* Remove warning

* Correctly advance un_partial_stated_room_stream

* Fix merge

* Add another notify_replication

* Fixups

* Create a separate ReplicationNotifier

* Fix test

* Fix portdb

* Create a separate ReplicationNotifier

* Fix test

* Fix portdb

* Fix presence test

* Newsfile

* Apply suggestions from code review

* Update changelog.d/14752.misc

Co-authored-by: Erik Johnston <erik@matrix.org>

* lint

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Co-authored-by: Erik Johnston <erik@matrix.org>
2023-01-22 21:10:11 +00:00
katlol cf18fea9e1
Dockerfile: Bump Python version from 3.9 to 3.11 (#14875)
Closes https://github.com/matrix-org/synapse/issues/13234

Signed-off-by: Katia Esposito <1695469+katlol@users.noreply.github.com>

Signed-off-by: Katia Esposito <1695469+katlol@users.noreply.github.com>
2023-01-20 12:07:13 +00:00
Andrew Morgan a7b54ca8d8
Implement MSC3930: polls push rules (#14787) 2023-01-19 12:47:10 +00:00
David Robertson b88cfe6d41
Require poetry>=1.3.2 (#14860)
* Upgrade to new lockfile format

Now requires poetry >= 1.2.2 to read and poetry >= 1.3.0 to write.

Cheat sheet:

```
poetry --version
poetry show > scratch/before
pipx upgrade poetry
poetry --version
poetry show > scratch/after
diff scratch{before,after} && echo "no change!"
```

* Use Poetry 1.3.2 when reading or writing lockfile

* Remove unneeded(?) poetry dep for cibuildwheel

* Update docs

* Remove redundant call to setup-python

* Remove outdated comments related to Poetry 1.x

* Remove outdated docs line

was fixed in #13082

* Minor improvements to poetry cheat sheet

* Invoke setup-python-poetry with explicit version

Not sure about this. It's hardcoding versions everywhere.

* Changelog

* Check the lockfile is version 2.0

Might one day incorporate other checks like #14742

* Typo fixes, thanks Sean

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2023-01-17 18:04:44 +00:00
David Robertson 5b3af1c7d0
Stabilise serving partial join responses (#14839)
Serving partial join responses is no longer experimental. They will only be served under the stable identifier if the the undocumented config flag experimental.msc3706_enabled is set to true.

Synapse continues to request a partial join only if the undocumented config flag experimental.faster_joins is set to true; this setting remains present and unaffected.
2023-01-17 12:44:15 +00:00
Andrew Morgan 54cd90ea60
Implement MSC3890: Remotely silence local notifications (#14775) 2023-01-13 19:32:10 +00:00
Andrew Morgan e787fb776c
Switch to our fork of dh-virtualenv for compatibility with Python 3.11 (#14774) 2023-01-04 16:26:29 +00:00
Andrew Morgan c4456114e1
Add experimental support for MSC3391: deleting account data (#14714) 2023-01-01 03:40:46 +00:00
Mathieu Velten eb2defc2f7
Add release note and update doc regarding ICU (#14712)
Fixes #14704.

Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-12-20 16:06:26 +00:00
Brendan Abolivier 2a3cd59dd0
Add optional ICU support for user search (#14464)
Fixes #13655

This change uses ICU (International Components for Unicode) to improve boundary detection in user search.

This change also adds a new dependency on libicu-dev and pkg-config for the Debian packages, which are available in all supported distros.
2022-12-12 13:21:17 +01:00
reivilibre be3a8a85e3
Add `--editable` flag to `complement.sh` which uses an editable install of Synapse for faster turn-around times whilst developing iteratively. (#14548)
Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
2022-12-07 15:45:31 +00:00
Mathieu Velten 22e91b8019
docker: remove useless cargo install with apt (#14636)
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-12-07 15:29:32 +01:00
Eric Eastwood 8f10c8b054
Move MSC3030 `/timestamp_to_event` endpoint to stable v1 location (#14471)
Fix https://github.com/matrix-org/synapse/issues/14390

 - Client API: `/_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` -> `/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>`
 - Federation API: `/_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` -> `/_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>`

Complement test changes: https://github.com/matrix-org/complement/pull/559
2022-11-28 15:54:18 -06:00
realtyem c15e9a0edb
Remove need for `worker_main_http_uri` setting to use /keys/upload. (#14400) 2022-11-16 22:16:25 +00:00
realtyem e9cbddc8e7
Modernize configure_workers_and_start.py bootstrapping script for Dockerfile-workers. (#14294) 2022-11-09 12:02:15 +00:00
realtyem d85cba1aa0
Add all Stream Writer worker types to configure_workers_and_start.py (#14197)
Co-authored-by: reivilibre <oliverw@matrix.org>
2022-11-08 13:14:00 +00:00
realtyem 69814eb282
Allow override for requesting specific worker types for Complement on command line. (#14324)
* Expose getting SYNAPSE_WORKER_TYPES from external, allowing override of workers requested.

* Add WORKER_TYPES variable option to complement.sh script that passes requested workers into start_for_complement.sh entrypoint.

* Update docs to reflect this new ability.

* Changelog

* Don't rely on soft wrapping to format long strings

Good idea dklimpel. Thanks for catching that.

Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>

* Small nits just noticed in docs.

* Fixup new line in docs.

Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
2022-11-08 12:34:09 +00:00
Patrick Cloke 59ca73006c
Enable testing MSC3874 in complement. (#14339) 2022-11-01 13:26:28 -04:00
Jonathan de Jong e440f9674a
Enable URL previews in complement homeserver config. (#14198) 2022-10-18 09:52:23 -04:00
realtyem 6c5082f3e0
Flush stdout/err in Dockerfile-workers before replacing the current process (#14195)
Also update `subprocess.check_output` to the slightly newer `subprocess.run`.

Signed-off-by: Jason Little <realtyem@gmail.com>
2022-10-18 11:56:20 +00:00
Erik Johnston 6fee2f49f3
Cache Rust build cache when building docker images (#14130) 2022-10-17 17:21:14 +00:00
realtyem c75836fe77
Strip whitespace from worker types in Dockerfile-workers (#14165) 2022-10-14 19:38:04 +00:00
Patrick Cloke bc2bd92b93 Merge remote-tracking branch 'origin/release-v1.69' into develop 2022-10-14 14:11:27 -04:00
realtyem c7446906bd
Set LD_PRELOAD to load jemalloc in Dockerfile-workers. (#14182) 2022-10-14 13:29:49 +01:00
Patrick Cloke c3e4edb4d6
Stabilize the threads API. (#14175)
Stabilize the threads API (MSC3856) by supporting (only) the v1
path for the endpoint.

This also marks the API as safe for workers since it is a read-only
API.
2022-10-14 07:16:50 -04:00
Erik Johnston 29ee4b6698
Fix docker build OOMing in CI for arm64 builds (#14173)
Co-authored-by: David Robertson <davidr@element.io>
2022-10-13 18:16:21 +00:00
Patrick Cloke c604d2c218
Mark /relations endpoint as usable on workers. (#14028)
Co-authored-by: Eric Eastwood <erice@element.io>
2022-10-12 06:46:13 -04:00
David Robertson 3f057e4c54
Use minimal Rust installation in docker images and CI (#14141) 2022-10-12 10:47:02 +01:00
Erik Johnston d94bcbced3
Fix pinning Rust deps in docker images (#14129) 2022-10-11 10:53:34 +00:00
David Robertson 0c853e0970
Fix broken links to README (#14093) 2022-10-06 22:49:51 +00:00
Richard van der Hoff 51436c8dd5
Complement test image: capture logs from nginx (#14063)
Have nginx send its logs to stderr/out, so that we can debug
https://github.com/matrix-org/synapse/issues/13334.
2022-10-05 17:37:34 +01:00
Mathieu Velten 42dd992bb7
Dockerfile for tests: align comments with current behavior (#13867)
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-09-27 15:55:43 +02:00
Mathieu Velten d5292b8017
Fix Docker build when Rust .so has been build locally first (#13811)
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-09-16 15:38:54 +00:00
Mathieu Velten 384dca53d6
complement: init postgres DB directly inside the target image (#13819)
Doing so in the base postgres image doesn't work with buildah because
changes in a declared VOLUME in the Dockerfile is supposed to be
discarded, cf https://docs.docker.com/engine/reference/builder/#volume

Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-09-16 17:12:45 +02:00
Erik Johnston c9b7e97355
Add a stub Rust crate (#12595) 2022-09-06 19:01:37 +01:00
Erik Johnston 18e4092801
Bump docker image to use stable poetry version (#13688) 2022-09-01 13:46:47 +01:00
reivilibre 2c5e2ae898
Document that the `DOCKER_BUILDKIT=1` flag is needed to build the docker image. (#13515) 2022-08-12 14:25:47 +01:00
Richard van der Hoff 166fafdf8d
synapse-workers docker: copy nginx and redis in from base images (#13447)
Part of my continuing quest to make the docker images build quicker: copy nginx and redis in from base docker images, rather than apt installing each time.
2022-08-04 12:59:27 +01:00
Jasper Spaans 503a95804e
Install cryptography build dependencies in requirements image. (#13372) 2022-08-03 11:16:32 +01:00
reivilibre 05aeeb3a80
Enable Complement CI tests in the 'latest deps' test run. (#13213)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-08-01 10:55:31 +00:00
Brendan Abolivier 10e4093839
Call out buildkit is required when building test docker images (#13338)
Co-authored-by: David Robertson <davidr@element.io>
2022-07-21 14:29:58 +02:00
David Robertson b977867358
Rate limit joins per-room (#13276) 2022-07-19 11:45:17 +00:00
David Robertson e9ce4d089b
Use and recommend poetry 1.1.14, up from 1.1.12 (#13285) 2022-07-15 16:18:47 +01:00
Richard van der Hoff 512486bbeb
Docker: copy postgres from base image (#13279)
When building the docker images for complement testing, copy a preinstalled
complement over from a base image, rather than apt installing it. This avoids
network traffic and is much faster.
2022-07-15 11:13:40 +01:00
reivilibre 65e675504f
Add the ability to set the log level using the `SYNAPSE_TEST_LOG_LEVEL` environment when using `complement.sh`. (#13152) 2022-07-05 09:46:20 +00:00
Till fa10468eb4
[Complement] Allow device_name lookup over federation (#13167) 2022-07-04 12:34:50 +00:00
reivilibre c04e25789e
Enable Complement testing in the 'Twisted Trunk' CI runs. (#13079) 2022-07-01 15:42:49 +00:00
reivilibre 9667bad55d
Improve startup times in Complement test runs against workers, particularly in CPU-constrained environments. (#13127)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-30 11:58:12 +00:00
reivilibre 3c5549e74a
Refactor the Dockerfile-workers configuration script to use Jinja2 templates in Synapse workers' Supervisord blocks. (#13054)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-27 11:43:20 +01:00
reivilibre 90cadcd403
Add a Subject Alternative Name to the certificate generated for Complement tests. (#13071) 2022-06-16 12:43:21 +01:00