1af2fcd492
Move get_or_create_user to test code ( #5628 )
...
This is only used in tests, so...
2019-07-08 23:52:26 +10:00
f05c7d62bc
Lint
2019-07-08 14:29:27 +01:00
1a807dfe68
Use application/json when querying the IS's /store-invite endpoint
2019-07-08 14:19:39 +01:00
9b1b79f3f5
Add default push rule to ignore reactions ( #5623 )
...
This adds a default push rule following the proposal in
[MSC2153](https://github.com/matrix-org/matrix-doc/pull/2153 ).
See also https://github.com/vector-im/riot-web/issues/10208
See also https://github.com/matrix-org/matrix-js-sdk/pull/976
2019-07-05 17:37:52 +01:00
ad8b909ce9
Add origin_server_ts and sender fields to m.replace ( #5613 )
...
Riot team would like some extra fields as part of m.replace, so here you go.
Fixes : #5598
2019-07-05 17:20:02 +01:00
80cc82a445
Remove support for invite_3pid_guest. ( #5625 )
...
This has never been documented, and I'm not sure it's ever been used outside
sytest.
It's quite a lot of poorly-maintained code, so I'd like to get rid of it.
For now I haven't removed the database table; I suggest we leave that for a
future clearout.
2019-07-05 16:47:58 +01:00
a175e608e9
Merge remote-tracking branch 'origin/develop' into shhs
2019-07-05 23:49:13 +10:00
b4f5416dd9
pep8
2019-07-05 14:41:29 +01:00
3d89feb438
linting
2019-07-05 23:34:24 +10:00
eadb13d2e9
Remove FileExfiltrationWriter
2019-07-05 14:15:00 +01:00
7f0d8e4288
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_exfiltrate_data
2019-07-05 14:08:21 +01:00
9ccea16d45
Assume key existence. Update docstrings
2019-07-05 14:07:56 +01:00
400bc061ca
linting
2019-07-05 22:44:22 +10:00
a6a776f3d8
remove dead transaction persist code ( #5622 )
...
this hasn't done anything for years
2019-07-05 12:59:42 +01:00
9481707a52
Fixes to the federation rate limiter ( #5621 )
...
- Put the default window_size back to 1000ms (broken by #5181 )
- Make the `rc_federation` config actually do something
- fix an off-by-one error in the 'concurrent' limit
- Avoid creating an unused `_PerHostRatelimiter` object for every single
incoming request
2019-07-05 11:10:19 +01:00
a1de642fe7
Synapse 1.1.0 (2019-07-04)
...
==========================
As of v1.1.0, Synapse no longer supports Python 2, nor Postgres version 9.4.
See the [upgrade notes](UPGRADE.rst#upgrading-to-v110) for more details.
This release also deprecates the use of environment variables to configure the
docker image. See the [docker README](https://github.com/matrix-org/synapse/blob/release-v1.1.0/docker/README.md#legacy-dynamic-configuration-file-support )
for more details.
No changes since 1.1.0rc2.
Synapse 1.1.0rc2 (2019-07-03)
=============================
Bugfixes
--------
- Fix regression in 1.1rc1 where OPTIONS requests to the media repo would fail. ([\#5593](https://github.com/matrix-org/synapse/issues/5593 ))
- Removed the `SYNAPSE_SMTP_*` docker container environment variables. Using these environment variables prevented the docker container from starting in Synapse v1.0, even though they didn't actually allow any functionality anyway. ([\#5596](https://github.com/matrix-org/synapse/issues/5596 ))
- Fix a number of "Starting txn from sentinel context" warnings. ([\#5605](https://github.com/matrix-org/synapse/issues/5605 ))
Internal Changes
----------------
- Update github templates. ([\#5552](https://github.com/matrix-org/synapse/issues/5552 ))
Synapse 1.1.0rc1 (2019-07-02)
=============================
As of v1.1.0, Synapse no longer supports Python 2, nor Postgres version 9.4.
See the [upgrade notes](UPGRADE.rst#upgrading-to-v110) for more details.
Features
--------
- Added possibilty to disable local password authentication. Contributed by Daniel Hoffend. ([\#5092](https://github.com/matrix-org/synapse/issues/5092 ))
- Add monthly active users to phonehome stats. ([\#5252](https://github.com/matrix-org/synapse/issues/5252 ))
- Allow expired user to trigger renewal email sending manually. ([\#5363](https://github.com/matrix-org/synapse/issues/5363 ))
- Statistics on forward extremities per room are now exposed via Prometheus. ([\#5384](https://github.com/matrix-org/synapse/issues/5384 ), [\#5458](https://github.com/matrix-org/synapse/issues/5458 ), [\#5461](https://github.com/matrix-org/synapse/issues/5461 ))
- Add --no-daemonize option to run synapse in the foreground, per issue #4130 . Contributed by Soham Gumaste. ([\#5412](https://github.com/matrix-org/synapse/issues/5412 ), [\#5587](https://github.com/matrix-org/synapse/issues/5587 ))
- Fully support SAML2 authentication. Contributed by [Alexander Trost](https://github.com/galexrt ) - thank you! ([\#5422](https://github.com/matrix-org/synapse/issues/5422 ))
- Allow server admins to define implementations of extra rules for allowing or denying incoming events. ([\#5440](https://github.com/matrix-org/synapse/issues/5440 ), [\#5474](https://github.com/matrix-org/synapse/issues/5474 ), [\#5477](https://github.com/matrix-org/synapse/issues/5477 ))
- Add support for handling pagination APIs on client reader worker. ([\#5505](https://github.com/matrix-org/synapse/issues/5505 ), [\#5513](https://github.com/matrix-org/synapse/issues/5513 ), [\#5531](https://github.com/matrix-org/synapse/issues/5531 ))
- Improve help and cmdline option names for --generate-config options. ([\#5512](https://github.com/matrix-org/synapse/issues/5512 ))
- Allow configuration of the path used for ACME account keys. ([\#5516](https://github.com/matrix-org/synapse/issues/5516 ), [\#5521](https://github.com/matrix-org/synapse/issues/5521 ), [\#5522](https://github.com/matrix-org/synapse/issues/5522 ))
- Add --data-dir and --open-private-ports options. ([\#5524](https://github.com/matrix-org/synapse/issues/5524 ))
- Split public rooms directory auth config in two settings, in order to manage client auth independently from the federation part of it. Obsoletes the "restrict_public_rooms_to_local_users" configuration setting. If "restrict_public_rooms_to_local_users" is set in the config, Synapse will act as if both new options are enabled, i.e. require authentication through the client API and deny federation requests. ([\#5534](https://github.com/matrix-org/synapse/issues/5534 ))
- The minimum TLS version used for outgoing federation requests can now be set with `federation_client_minimum_tls_version`. ([\#5550](https://github.com/matrix-org/synapse/issues/5550 ))
- Optimise devices changed query to not pull unnecessary rows from the database, reducing database load. ([\#5559](https://github.com/matrix-org/synapse/issues/5559 ))
- Add new metrics for number of forward extremities being persisted and number of state groups involved in resolution. ([\#5476](https://github.com/matrix-org/synapse/issues/5476 ))
Bugfixes
--------
- Fix bug processing incoming events over federation if call to `/get_missing_events` fails. ([\#5042](https://github.com/matrix-org/synapse/issues/5042 ))
- Prevent more than one room upgrade happening simultaneously on the same room. ([\#5051](https://github.com/matrix-org/synapse/issues/5051 ))
- Fix a bug where running synapse_port_db would cause the account validity feature to fail because it didn't set the type of the email_sent column to boolean. ([\#5325](https://github.com/matrix-org/synapse/issues/5325 ))
- Warn about disabling email-based password resets when a reset occurs, and remove warning when someone attempts a phone-based reset. ([\#5387](https://github.com/matrix-org/synapse/issues/5387 ))
- Fix email notifications for unnamed rooms with multiple people. ([\#5388](https://github.com/matrix-org/synapse/issues/5388 ))
- Fix exceptions in federation reader worker caused by attempting to renew attestations, which should only happen on master worker. ([\#5389](https://github.com/matrix-org/synapse/issues/5389 ))
- Fix handling of failures fetching remote content to not log failures as exceptions. ([\#5390](https://github.com/matrix-org/synapse/issues/5390 ))
- Fix a bug where deactivated users could receive renewal emails if the account validity feature is on. ([\#5394](https://github.com/matrix-org/synapse/issues/5394 ))
- Fix missing invite state after exchanging 3PID invites over federaton. ([\#5464](https://github.com/matrix-org/synapse/issues/5464 ))
- Fix intermittent exceptions on Apple hardware. Also fix bug that caused database activity times to be under-reported in log lines. ([\#5498](https://github.com/matrix-org/synapse/issues/5498 ))
- Fix logging error when a tampered event is detected. ([\#5500](https://github.com/matrix-org/synapse/issues/5500 ))
- Fix bug where clients could tight loop calling `/sync` for a period. ([\#5507](https://github.com/matrix-org/synapse/issues/5507 ))
- Fix bug with `jinja2` preventing Synapse from starting. Users who had this problem should now simply need to run `pip install matrix-synapse`. ([\#5514](https://github.com/matrix-org/synapse/issues/5514 ))
- Fix a regression where homeservers on private IP addresses were incorrectly blacklisted. ([\#5523](https://github.com/matrix-org/synapse/issues/5523 ))
- Fixed m.login.jwt using unregistred user_id and added pyjwt>=1.6.4 as jwt conditional dependencies. Contributed by Pau Rodriguez-Estivill. ([\#5555](https://github.com/matrix-org/synapse/issues/5555 ), [\#5586](https://github.com/matrix-org/synapse/issues/5586 ))
- Fix a bug that would cause invited users to receive several emails for a single 3PID invite in case the inviter is rate limited. ([\#5576](https://github.com/matrix-org/synapse/issues/5576 ))
Updates to the Docker image
---------------------------
- Add ability to change Docker containers [timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones ) with the `TZ` variable. ([\#5383](https://github.com/matrix-org/synapse/issues/5383 ))
- Update docker image to use Python 3.7. ([\#5546](https://github.com/matrix-org/synapse/issues/5546 ))
- Deprecate the use of environment variables for configuration, and make the use of a static configuration the default. ([\#5561](https://github.com/matrix-org/synapse/issues/5561 ), [\#5562](https://github.com/matrix-org/synapse/issues/5562 ), [\#5566](https://github.com/matrix-org/synapse/issues/5566 ), [\#5567](https://github.com/matrix-org/synapse/issues/5567 ))
- Increase default log level for docker image to INFO. It can still be changed by editing the generated log.config file. ([\#5547](https://github.com/matrix-org/synapse/issues/5547 ))
- Send synapse logs to the docker logging system, by default. ([\#5565](https://github.com/matrix-org/synapse/issues/5565 ))
- Open the non-TLS port by default. ([\#5568](https://github.com/matrix-org/synapse/issues/5568 ))
- Fix failure to start under docker with SAML support enabled. ([\#5490](https://github.com/matrix-org/synapse/issues/5490 ))
- Use a sensible location for data files when generating a config file. ([\#5563](https://github.com/matrix-org/synapse/issues/5563 ))
Deprecations and Removals
-------------------------
- Python 2.7 is no longer a supported platform. Synapse now requires Python 3.5+ to run. ([\#5425](https://github.com/matrix-org/synapse/issues/5425 ))
- PostgreSQL 9.4 is no longer supported. Synapse requires Postgres 9.5+ or above for Postgres support. ([\#5448](https://github.com/matrix-org/synapse/issues/5448 ))
- Remove support for cpu_affinity setting. ([\#5525](https://github.com/matrix-org/synapse/issues/5525 ))
Improved Documentation
----------------------
- Improve README section on performance troubleshooting. ([\#4276](https://github.com/matrix-org/synapse/issues/4276 ))
- Add information about how to install and run `black` on the codebase to code_style.rst. ([\#5537](https://github.com/matrix-org/synapse/issues/5537 ))
- Improve install docs on choosing server_name. ([\#5558](https://github.com/matrix-org/synapse/issues/5558 ))
Internal Changes
----------------
- Add logging to 3pid invite signature verification. ([\#5015](https://github.com/matrix-org/synapse/issues/5015 ))
- Update example haproxy config to a more compatible setup. ([\#5313](https://github.com/matrix-org/synapse/issues/5313 ))
- Track deactivated accounts in the database. ([\#5378](https://github.com/matrix-org/synapse/issues/5378 ), [\#5465](https://github.com/matrix-org/synapse/issues/5465 ), [\#5493](https://github.com/matrix-org/synapse/issues/5493 ))
- Clean up code for sending federation EDUs. ([\#5381](https://github.com/matrix-org/synapse/issues/5381 ))
- Add a sponsor button to the repo. ([\#5382](https://github.com/matrix-org/synapse/issues/5382 ), [\#5386](https://github.com/matrix-org/synapse/issues/5386 ))
- Don't log non-200 responses from federation queries as exceptions. ([\#5383](https://github.com/matrix-org/synapse/issues/5383 ))
- Update Python syntax in contrib/ to Python 3. ([\#5446](https://github.com/matrix-org/synapse/issues/5446 ))
- Update federation_client dev script to support `.well-known` and work with python3. ([\#5447](https://github.com/matrix-org/synapse/issues/5447 ))
- SyTest has been moved to Buildkite. ([\#5459](https://github.com/matrix-org/synapse/issues/5459 ))
- Demo script now uses python3. ([\#5460](https://github.com/matrix-org/synapse/issues/5460 ))
- Synapse can now handle RestServlets that return coroutines. ([\#5475](https://github.com/matrix-org/synapse/issues/5475 ), [\#5585](https://github.com/matrix-org/synapse/issues/5585 ))
- The demo servers talk to each other again. ([\#5478](https://github.com/matrix-org/synapse/issues/5478 ))
- Add an EXPERIMENTAL config option to try and periodically clean up extremities by sending dummy events. ([\#5480](https://github.com/matrix-org/synapse/issues/5480 ))
- Synapse's codebase is now formatted by `black`. ([\#5482](https://github.com/matrix-org/synapse/issues/5482 ))
- Some cleanups and sanity-checking in the CPU and database metrics. ([\#5499](https://github.com/matrix-org/synapse/issues/5499 ))
- Improve email notification logging. ([\#5502](https://github.com/matrix-org/synapse/issues/5502 ))
- Fix "Unexpected entry in 'full_schemas'" log warning. ([\#5509](https://github.com/matrix-org/synapse/issues/5509 ))
- Improve logging when generating config files. ([\#5510](https://github.com/matrix-org/synapse/issues/5510 ))
- Refactor and clean up Config parser for maintainability. ([\#5511](https://github.com/matrix-org/synapse/issues/5511 ))
- Make the config clearer in that email.template_dir is relative to the Synapse's root directory, not the `synapse/` folder within it. ([\#5543](https://github.com/matrix-org/synapse/issues/5543 ))
- Update v1.0.0 release changelog to include more information about changes to password resets. ([\#5545](https://github.com/matrix-org/synapse/issues/5545 ))
- Remove non-functioning check_event_hash.py dev script. ([\#5548](https://github.com/matrix-org/synapse/issues/5548 ))
- Synapse will now only allow TLS v1.2 connections when serving federation, if it terminates TLS. As Synapse's allowed ciphers were only able to be used in TLSv1.2 before, this does not change behaviour. ([\#5550](https://github.com/matrix-org/synapse/issues/5550 ))
- Logging when running GC collection on generation 0 is now at the DEBUG level, not INFO. ([\#5557](https://github.com/matrix-org/synapse/issues/5557 ))
- Reduce the amount of stuff we send in the docker context. ([\#5564](https://github.com/matrix-org/synapse/issues/5564 ))
- Point the reverse links in the Purge History contrib scripts at the intended location. ([\#5570](https://github.com/matrix-org/synapse/issues/5570 ))
-----BEGIN PGP SIGNATURE-----
iQJEBAABCgAuFiEEumuwyPtYLL2OMhYdOtoG7cdT0R4FAl0d9zUQHGVyaWtAbWF0
cml4Lm9yZwAKCRA62gbtx1PRHt5iD/0bBlARLQWmTTR04u3JbfPaX0DQoWPj943m
cC6WFm7WebAC/pspuadYtgQWxY+k2KoZ9WmdmZsg1Af9LbJ4DqVeCaIMzznQixDK
z7BdsZdQT0IXQdqSoPjdGnNa3NYYxD3Sc7LbhtcEihA3jHGVZd4F0aZ3c2Rrfnu9
nFuaftYmerhv09+m53614sDxn88tQUMSPz4BlPTxf9u6ifKbwIVt+UbmZ2kvEiUk
uwN6qKxzKJuT1uWHLN8l7E3bLeC6oQm7sfkWWrjb758ub9Ilh7Fb1gB5nLbL4SrI
tjZc4bDC2ChgAHjYuC/huG8iGPHojoqEkrJm9obNSAWwu87ofk2+wHWslFMtUH18
PPwMsFhNHLrUcmjlZHIxpfSkEEBuRpVlcwNXDfFp9xzLo9MudOZ2cmVIum2zQ1wb
+dBldWI5DXA3BjUVZR9Gwr57WpllYSVycWrErl69LhYE9Sf/sw8gBnyQR97Z/Nwl
xjx4CJFyMbyXp/LjrHOCRDFXB/L3HQTrHHUfQISs1R7tD644KHfEZMwPjMDM7zoh
3Vi9ULDwBpREVWKq527d2IwMNV1wdsxmbp9QcyeL7JjM5ykYB22/Dt9ZojNwo4Nr
VCzr2b55bdoKx6+x0golHUTXxrLlfewuO4+VMPk4b5AL958ogo8p45koJNA8IPk1
KKnqbU9vjg==
=veSY
-----END PGP SIGNATURE-----
Merge tag 'v1.1.0' into shhs
Synapse 1.1.0 (2019-07-04)
==========================
As of v1.1.0, Synapse no longer supports Python 2, nor Postgres version 9.4.
See the [upgrade notes](UPGRADE.rst#upgrading-to-v110) for more details.
This release also deprecates the use of environment variables to configure the
docker image. See the [docker README](https://github.com/matrix-org/synapse/blob/release-v1.1.0/docker/README.md#legacy-dynamic-configuration-file-support )
for more details.
No changes since 1.1.0rc2.
Synapse 1.1.0rc2 (2019-07-03)
=============================
Bugfixes
--------
- Fix regression in 1.1rc1 where OPTIONS requests to the media repo would fail. ([\#5593](https://github.com/matrix-org/synapse/issues/5593 ))
- Removed the `SYNAPSE_SMTP_*` docker container environment variables. Using these environment variables prevented the docker container from starting in Synapse v1.0, even though they didn't actually allow any functionality anyway. ([\#5596](https://github.com/matrix-org/synapse/issues/5596 ))
- Fix a number of "Starting txn from sentinel context" warnings. ([\#5605](https://github.com/matrix-org/synapse/issues/5605 ))
Internal Changes
----------------
- Update github templates. ([\#5552](https://github.com/matrix-org/synapse/issues/5552 ))
Synapse 1.1.0rc1 (2019-07-02)
=============================
As of v1.1.0, Synapse no longer supports Python 2, nor Postgres version 9.4.
See the [upgrade notes](UPGRADE.rst#upgrading-to-v110) for more details.
Features
--------
- Added possibilty to disable local password authentication. Contributed by Daniel Hoffend. ([\#5092](https://github.com/matrix-org/synapse/issues/5092 ))
- Add monthly active users to phonehome stats. ([\#5252](https://github.com/matrix-org/synapse/issues/5252 ))
- Allow expired user to trigger renewal email sending manually. ([\#5363](https://github.com/matrix-org/synapse/issues/5363 ))
- Statistics on forward extremities per room are now exposed via Prometheus. ([\#5384](https://github.com/matrix-org/synapse/issues/5384 ), [\#5458](https://github.com/matrix-org/synapse/issues/5458 ), [\#5461](https://github.com/matrix-org/synapse/issues/5461 ))
- Add --no-daemonize option to run synapse in the foreground, per issue #4130 . Contributed by Soham Gumaste. ([\#5412](https://github.com/matrix-org/synapse/issues/5412 ), [\#5587](https://github.com/matrix-org/synapse/issues/5587 ))
- Fully support SAML2 authentication. Contributed by [Alexander Trost](https://github.com/galexrt ) - thank you! ([\#5422](https://github.com/matrix-org/synapse/issues/5422 ))
- Allow server admins to define implementations of extra rules for allowing or denying incoming events. ([\#5440](https://github.com/matrix-org/synapse/issues/5440 ), [\#5474](https://github.com/matrix-org/synapse/issues/5474 ), [\#5477](https://github.com/matrix-org/synapse/issues/5477 ))
- Add support for handling pagination APIs on client reader worker. ([\#5505](https://github.com/matrix-org/synapse/issues/5505 ), [\#5513](https://github.com/matrix-org/synapse/issues/5513 ), [\#5531](https://github.com/matrix-org/synapse/issues/5531 ))
- Improve help and cmdline option names for --generate-config options. ([\#5512](https://github.com/matrix-org/synapse/issues/5512 ))
- Allow configuration of the path used for ACME account keys. ([\#5516](https://github.com/matrix-org/synapse/issues/5516 ), [\#5521](https://github.com/matrix-org/synapse/issues/5521 ), [\#5522](https://github.com/matrix-org/synapse/issues/5522 ))
- Add --data-dir and --open-private-ports options. ([\#5524](https://github.com/matrix-org/synapse/issues/5524 ))
- Split public rooms directory auth config in two settings, in order to manage client auth independently from the federation part of it. Obsoletes the "restrict_public_rooms_to_local_users" configuration setting. If "restrict_public_rooms_to_local_users" is set in the config, Synapse will act as if both new options are enabled, i.e. require authentication through the client API and deny federation requests. ([\#5534](https://github.com/matrix-org/synapse/issues/5534 ))
- The minimum TLS version used for outgoing federation requests can now be set with `federation_client_minimum_tls_version`. ([\#5550](https://github.com/matrix-org/synapse/issues/5550 ))
- Optimise devices changed query to not pull unnecessary rows from the database, reducing database load. ([\#5559](https://github.com/matrix-org/synapse/issues/5559 ))
- Add new metrics for number of forward extremities being persisted and number of state groups involved in resolution. ([\#5476](https://github.com/matrix-org/synapse/issues/5476 ))
Bugfixes
--------
- Fix bug processing incoming events over federation if call to `/get_missing_events` fails. ([\#5042](https://github.com/matrix-org/synapse/issues/5042 ))
- Prevent more than one room upgrade happening simultaneously on the same room. ([\#5051](https://github.com/matrix-org/synapse/issues/5051 ))
- Fix a bug where running synapse_port_db would cause the account validity feature to fail because it didn't set the type of the email_sent column to boolean. ([\#5325](https://github.com/matrix-org/synapse/issues/5325 ))
- Warn about disabling email-based password resets when a reset occurs, and remove warning when someone attempts a phone-based reset. ([\#5387](https://github.com/matrix-org/synapse/issues/5387 ))
- Fix email notifications for unnamed rooms with multiple people. ([\#5388](https://github.com/matrix-org/synapse/issues/5388 ))
- Fix exceptions in federation reader worker caused by attempting to renew attestations, which should only happen on master worker. ([\#5389](https://github.com/matrix-org/synapse/issues/5389 ))
- Fix handling of failures fetching remote content to not log failures as exceptions. ([\#5390](https://github.com/matrix-org/synapse/issues/5390 ))
- Fix a bug where deactivated users could receive renewal emails if the account validity feature is on. ([\#5394](https://github.com/matrix-org/synapse/issues/5394 ))
- Fix missing invite state after exchanging 3PID invites over federaton. ([\#5464](https://github.com/matrix-org/synapse/issues/5464 ))
- Fix intermittent exceptions on Apple hardware. Also fix bug that caused database activity times to be under-reported in log lines. ([\#5498](https://github.com/matrix-org/synapse/issues/5498 ))
- Fix logging error when a tampered event is detected. ([\#5500](https://github.com/matrix-org/synapse/issues/5500 ))
- Fix bug where clients could tight loop calling `/sync` for a period. ([\#5507](https://github.com/matrix-org/synapse/issues/5507 ))
- Fix bug with `jinja2` preventing Synapse from starting. Users who had this problem should now simply need to run `pip install matrix-synapse`. ([\#5514](https://github.com/matrix-org/synapse/issues/5514 ))
- Fix a regression where homeservers on private IP addresses were incorrectly blacklisted. ([\#5523](https://github.com/matrix-org/synapse/issues/5523 ))
- Fixed m.login.jwt using unregistred user_id and added pyjwt>=1.6.4 as jwt conditional dependencies. Contributed by Pau Rodriguez-Estivill. ([\#5555](https://github.com/matrix-org/synapse/issues/5555 ), [\#5586](https://github.com/matrix-org/synapse/issues/5586 ))
- Fix a bug that would cause invited users to receive several emails for a single 3PID invite in case the inviter is rate limited. ([\#5576](https://github.com/matrix-org/synapse/issues/5576 ))
Updates to the Docker image
---------------------------
- Add ability to change Docker containers [timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones ) with the `TZ` variable. ([\#5383](https://github.com/matrix-org/synapse/issues/5383 ))
- Update docker image to use Python 3.7. ([\#5546](https://github.com/matrix-org/synapse/issues/5546 ))
- Deprecate the use of environment variables for configuration, and make the use of a static configuration the default. ([\#5561](https://github.com/matrix-org/synapse/issues/5561 ), [\#5562](https://github.com/matrix-org/synapse/issues/5562 ), [\#5566](https://github.com/matrix-org/synapse/issues/5566 ), [\#5567](https://github.com/matrix-org/synapse/issues/5567 ))
- Increase default log level for docker image to INFO. It can still be changed by editing the generated log.config file. ([\#5547](https://github.com/matrix-org/synapse/issues/5547 ))
- Send synapse logs to the docker logging system, by default. ([\#5565](https://github.com/matrix-org/synapse/issues/5565 ))
- Open the non-TLS port by default. ([\#5568](https://github.com/matrix-org/synapse/issues/5568 ))
- Fix failure to start under docker with SAML support enabled. ([\#5490](https://github.com/matrix-org/synapse/issues/5490 ))
- Use a sensible location for data files when generating a config file. ([\#5563](https://github.com/matrix-org/synapse/issues/5563 ))
Deprecations and Removals
-------------------------
- Python 2.7 is no longer a supported platform. Synapse now requires Python 3.5+ to run. ([\#5425](https://github.com/matrix-org/synapse/issues/5425 ))
- PostgreSQL 9.4 is no longer supported. Synapse requires Postgres 9.5+ or above for Postgres support. ([\#5448](https://github.com/matrix-org/synapse/issues/5448 ))
- Remove support for cpu_affinity setting. ([\#5525](https://github.com/matrix-org/synapse/issues/5525 ))
Improved Documentation
----------------------
- Improve README section on performance troubleshooting. ([\#4276](https://github.com/matrix-org/synapse/issues/4276 ))
- Add information about how to install and run `black` on the codebase to code_style.rst. ([\#5537](https://github.com/matrix-org/synapse/issues/5537 ))
- Improve install docs on choosing server_name. ([\#5558](https://github.com/matrix-org/synapse/issues/5558 ))
Internal Changes
----------------
- Add logging to 3pid invite signature verification. ([\#5015](https://github.com/matrix-org/synapse/issues/5015 ))
- Update example haproxy config to a more compatible setup. ([\#5313](https://github.com/matrix-org/synapse/issues/5313 ))
- Track deactivated accounts in the database. ([\#5378](https://github.com/matrix-org/synapse/issues/5378 ), [\#5465](https://github.com/matrix-org/synapse/issues/5465 ), [\#5493](https://github.com/matrix-org/synapse/issues/5493 ))
- Clean up code for sending federation EDUs. ([\#5381](https://github.com/matrix-org/synapse/issues/5381 ))
- Add a sponsor button to the repo. ([\#5382](https://github.com/matrix-org/synapse/issues/5382 ), [\#5386](https://github.com/matrix-org/synapse/issues/5386 ))
- Don't log non-200 responses from federation queries as exceptions. ([\#5383](https://github.com/matrix-org/synapse/issues/5383 ))
- Update Python syntax in contrib/ to Python 3. ([\#5446](https://github.com/matrix-org/synapse/issues/5446 ))
- Update federation_client dev script to support `.well-known` and work with python3. ([\#5447](https://github.com/matrix-org/synapse/issues/5447 ))
- SyTest has been moved to Buildkite. ([\#5459](https://github.com/matrix-org/synapse/issues/5459 ))
- Demo script now uses python3. ([\#5460](https://github.com/matrix-org/synapse/issues/5460 ))
- Synapse can now handle RestServlets that return coroutines. ([\#5475](https://github.com/matrix-org/synapse/issues/5475 ), [\#5585](https://github.com/matrix-org/synapse/issues/5585 ))
- The demo servers talk to each other again. ([\#5478](https://github.com/matrix-org/synapse/issues/5478 ))
- Add an EXPERIMENTAL config option to try and periodically clean up extremities by sending dummy events. ([\#5480](https://github.com/matrix-org/synapse/issues/5480 ))
- Synapse's codebase is now formatted by `black`. ([\#5482](https://github.com/matrix-org/synapse/issues/5482 ))
- Some cleanups and sanity-checking in the CPU and database metrics. ([\#5499](https://github.com/matrix-org/synapse/issues/5499 ))
- Improve email notification logging. ([\#5502](https://github.com/matrix-org/synapse/issues/5502 ))
- Fix "Unexpected entry in 'full_schemas'" log warning. ([\#5509](https://github.com/matrix-org/synapse/issues/5509 ))
- Improve logging when generating config files. ([\#5510](https://github.com/matrix-org/synapse/issues/5510 ))
- Refactor and clean up Config parser for maintainability. ([\#5511](https://github.com/matrix-org/synapse/issues/5511 ))
- Make the config clearer in that email.template_dir is relative to the Synapse's root directory, not the `synapse/` folder within it. ([\#5543](https://github.com/matrix-org/synapse/issues/5543 ))
- Update v1.0.0 release changelog to include more information about changes to password resets. ([\#5545](https://github.com/matrix-org/synapse/issues/5545 ))
- Remove non-functioning check_event_hash.py dev script. ([\#5548](https://github.com/matrix-org/synapse/issues/5548 ))
- Synapse will now only allow TLS v1.2 connections when serving federation, if it terminates TLS. As Synapse's allowed ciphers were only able to be used in TLSv1.2 before, this does not change behaviour. ([\#5550](https://github.com/matrix-org/synapse/issues/5550 ))
- Logging when running GC collection on generation 0 is now at the DEBUG level, not INFO. ([\#5557](https://github.com/matrix-org/synapse/issues/5557 ))
- Reduce the amount of stuff we send in the docker context. ([\#5564](https://github.com/matrix-org/synapse/issues/5564 ))
- Point the reverse links in the Purge History contrib scripts at the intended location. ([\#5570](https://github.com/matrix-org/synapse/issues/5570 ))
2019-07-05 19:46:11 +10:00
0e5434264f
Make errors about email password resets much clearer ( #5616 )
...
The runtime errors that dealt with local email password resets talked about config options that users may not even have in their config file yet (if upgrading). Instead, the cryptic errors are now replaced with hopefully much more helpful ones.
2019-07-05 10:44:12 +01:00
1ee268d33d
Improve the backwards compatibility re-exports of synapse.logging.context ( #5617 )
...
* Improve the backwards compatibility re-exports of synapse.logging.context.
* reexport logformatter too
2019-07-05 02:32:02 +10:00
822a0f0435
Merge branch 'master' of github.com:matrix-org/synapse into develop
2019-07-04 14:00:27 +01:00
20332b278d
1.1.0
2019-07-04 11:44:09 +01:00
c061d4f237
Fixup from review comments.
2019-07-04 11:41:06 +01:00
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ ( #5606 )
2019-07-04 00:07:04 +10:00
d0b849c86d
Apply comment fixups from code review
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-07-03 15:03:38 +01:00
cb8d568cf9
Fix 'utime went backwards' errors on daemonization. ( #5609 )
...
* Fix 'utime went backwards' errors on daemonization.
Fixes #5608
* remove spurious debug
2019-07-03 22:40:45 +10:00
f4343c7d2b
Merge remote-tracking branch 'origin/develop' into shhs
2019-07-03 22:39:30 +10:00
463d5a8fde
1.1.0rc2
2019-07-03 10:51:47 +01:00
91753cae59
Fix a number of "Starting txn from sentinel context" warnings ( #5605 )
...
Fixes #5602 , #5603
2019-07-03 09:31:27 +01:00
0ee9076ffe
Fix media repo breaking ( #5593 )
2019-07-02 19:01:28 +01:00
9f3c0a8556
Add basic admin cmd app
2019-07-02 17:12:48 +01:00
8ee69f299c
Add basic function to get all data for a user out of synapse
2019-07-02 12:09:04 +01:00
a0fa4641c4
prepare v1.1.0rc1
2019-07-02 11:51:55 +01:00
6eecb6e500
Complete the SAML2 implementation ( #5422 )
...
* SAML2 Improvements and redirect stuff
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
* Code cleanups and simplifications.
Also: share the saml client between redirect and response handlers.
* changelog
* Revert redundant changes to static js
* Move all the saml stuff out to a centralised handler
* Add support for tracking SAML2 sessions.
This allows us to correctly handle `allow_unsolicited: False`.
* update sample config
* cleanups
* update sample config
* rename BaseSSORedirectServlet for consistency
* Address review comments
2019-07-02 11:18:11 +01:00
948488e115
Fix JWT login with new users ( #5586 )
...
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
2019-07-02 10:25:37 +01:00
4689408a35
Merge remote-tracking branch 'origin/develop' into shhs
2019-07-02 18:31:29 +10:00
b4914681a5
fix async/await consentresource ( #5585 )
...
Fixes #5582
2019-07-01 23:33:52 +10:00
b4fd86a9b4
Merge branch 'develop' into rav/saml2_client
2019-07-01 14:21:03 +01:00
0993b05ca5
improve error text when room is too large
2019-07-01 23:13:21 +10:00
3bcb13edd0
Address review comments
2019-07-01 12:13:22 +01:00
04196a4dae
Merge pull request #5507 from matrix-org/erikj/presence_sync_tighloop
...
Fix sync tightloop bug.
2019-07-01 11:43:10 +01:00
915280f1ed
Fixup comment
2019-07-01 10:22:42 +01:00
f40a7dc41f
Make the http server handle coroutine-making REST servlets ( #5475 )
2019-06-29 17:06:55 +10:00
c7ff297dde
Merge pull request #5576 from matrix-org/babolivier/3pid-invite-ratelimit
...
Don't update the ratelimiter before sending a 3PID invite
2019-06-28 17:43:48 +01:00
15d9fc31bd
Only ratelimit when sending the email
...
If we do the opposite, an event can arrive after or while sending the email and the 3PID invite event will get ratelimited.
2019-06-28 16:04:05 +01:00
01d0f8e701
Don't update the ratelimiter before sending a 3PID invite
...
This would cause emails being sent, but Synapse responding with a 429 when creating the event. The client would then retry, and with bad timing the same scenario would happen again. Some testing I did ended up sending me 10 emails for one single invite because of this.
2019-06-28 15:22:16 +01:00
071150ce19
Don't log GC 0s at INFO ( #5557 )
2019-06-28 21:45:33 +10:00
be3b901ccd
Update the TLS cipher string and provide configurability for TLS on outgoing federation ( #5550 )
2019-06-28 18:19:09 +10:00
9646a593ac
Added possibilty to disable local password authentication ( #5092 )
...
Signed-off-by: Daniel Hoffend <dh@dotlan.net>
2019-06-27 18:37:29 +01:00
c548dbc4b1
Make it clearer that the template dir is relative to synapse's root dir ( #5543 )
...
Helps address #5444
2019-06-27 18:20:17 +01:00
e79ec03165
Merge pull request #5559 from matrix-org/erikj/refactor_changed_devices
...
Refactor devices changed query to pull less from DB
2019-06-27 16:53:15 +01:00
729f5a4fb6
Review comments
2019-06-27 16:06:23 +01:00
856ea04eb3
Fix JWT login ( #5555 )
...
* Fix JWT login with register
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
* Add pyjwt conditional dependency
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
* Added changelog file
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
* Improved changelog description
Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>
2019-06-27 12:02:41 +01:00
79b9d9076d
rename BaseSSORedirectServlet for consistency
2019-06-27 00:46:57 +01:00
28db0ae537
cleanups
2019-06-27 00:37:41 +01:00
36f4953dec
Add support for tracking SAML2 sessions.
...
This allows us to correctly handle `allow_unsolicited: False`.
2019-06-26 23:50:55 +01:00
3705322103
Move all the saml stuff out to a centralised handler
2019-06-26 22:52:02 +01:00
0ade403f55
Revert redundant changes to static js
2019-06-26 22:46:23 +01:00
a4daa899ec
Merge branch 'develop' into rav/saml2_client
2019-06-26 22:34:41 +01:00
8624db3194
Refactor and comment sync device list code
2019-06-26 19:39:49 +01:00
f335e77d53
Use batch_iter and correct docstring
2019-06-26 19:39:46 +01:00
806a06daf2
Rename get_users_whose_devices_changed
2019-06-26 19:39:19 +01:00
a2f6d31a63
Refactor get_user_ids_changed to pull less from DB
...
When a client asks for users whose devices have changed since a token we
used to pull *all* users from the database since the token, which could
easily be thousands of rows for old tokens.
This PR changes this to only check for changes for users the client is
actually interested in.
Fixes #5553
2019-06-26 12:03:44 +01:00
ef8c62758c
Prevent multiple upgrades on the same room at once ( #5051 )
...
Closes #4583
Does slightly less than #5045 , which prevented a room from being upgraded multiple times, one after another. This PR still allows that, but just prevents two from happening at the same time.
Mostly just to mitigate the fact that servers are slow and it can take a moment for the room upgrade to actually complete. We don't want people sending another request to upgrade the room when really they just thought the first didn't go through.
2019-06-25 14:19:21 +01:00
4ac7ef4b67
Merge pull request #5524 from matrix-org/rav/new_cmdline_options
...
Add --data-dir and --open-private-ports options.
2019-06-24 17:25:57 +01:00
af8a962905
Merge pull request #5523 from matrix-org/rav/arg_defaults
...
Stop conflating generated config and default config
2019-06-24 17:24:35 +01:00
e59a8cd2e5
Merge pull request #5499 from matrix-org/rav/cleanup_metrics
...
Cleanups and sanity-checking in cpu and db metrics
2019-06-24 17:12:54 +01:00
deb4fe6ef3
Merge pull request #5534 from matrix-org/babolivier/federation-publicrooms
...
Split public rooms directory auth config in two
2019-06-24 16:08:02 +01:00
bfe84e051e
Split public rooms directory auth config in two
2019-06-24 15:42:31 +01:00
25433f212d
Merge pull request #5531 from matrix-org/erikj/workers_pagination_token
...
Fix /messages on workers when no from param specified.
2019-06-24 15:30:10 +01:00
3f8a252dd8
Add "--open-private-ports" cmdline option
...
This is helpful when generating a config file for running synapse under docker.
2019-06-24 14:15:34 +01:00
6a92b06cbb
Add --data-directory commandline argument
...
We don't necessarily want to put the data in the cwd.
2019-06-24 14:15:34 +01:00
16b52642e2
Don't load the generated config as the default.
...
It's too confusing.
2019-06-24 14:14:52 +01:00
7c2f8881a9
Ensure that all config options have sensible defaults
...
This will enable us to skip the unintuitive behaviour where the generated
config and default config are the same thing.
2019-06-24 14:14:52 +01:00
367a8263b3
Remove unused Config.config_dir_path attribute
...
This is no longer used and only serves to confuse.
2019-06-24 13:51:22 +01:00
edea4bb5be
Allow configuration of the path used for ACME account keys.
...
Because sticking it in the same place as the config isn't necessarily the right
thing to do.
2019-06-24 13:51:22 +01:00
c3c6b00d95
Pass config_dir_path and data_dir_path into Config.read_config. ( #5522 )
...
* Pull config_dir_path and data_dir_path calculation out of read_config_files
* Pass config_dir_path and data_dir_path into read_config
2019-06-24 11:34:45 +01:00
21bf4318b5
Factor acme bits out to a separate file ( #5521 )
...
This makes some of the conditional-import hoop-jumping easier.
2019-06-24 11:33:56 +01:00
1793de6c6d
black
2019-06-24 11:16:13 +01:00
dddf20e8e1
Fix /messages on workers when no from param specified.
...
If no `from` param is specified we calculate and use the "current
token" that inlcuded typing, presence, etc. These are unused during
pagination and are not available on workers, so we simply don't
calculate them.
2019-06-24 10:06:51 +01:00
dc94773e60
Avoid raising exceptions in metrics
...
Sentry will catch the errors if they happen, so that should be good enough, and
woun't make things explode if we hit the error condition.
2019-06-24 10:01:16 +01:00
5097aee740
Merge branch 'develop' into rav/cleanup_metrics
2019-06-24 10:00:13 +01:00
c753c098dd
Merge pull request #5498 from matrix-org/rav/fix_clock_reversal
...
Use monotonic clock where possible for metrics
2019-06-24 09:55:12 +01:00
6cda36777b
Drop support for cpu_affinity ( #5525 )
...
This has no useful purpose on python3, and is generally a source of confusion.
2019-06-22 11:01:55 +10:00
e1a795758c
Improve help and cmdline option names for --generate-config options ( #5512 )
...
* group the arguments together into a group
* add new names "--generate-missing-config" and "--config-directory" for
existing cmdline options "--generate-keys" and "--keys-dir", which better
reflect their purposes.
2019-06-21 18:50:43 +01:00
03cea2b0fe
Refactor Config parser and add some comments. ( #5511 )
...
Add some comments, and simplify `read_config_files`.
2019-06-21 17:43:38 +01:00
37933a3bf8
Improve logging when generating config files ( #5510 )
...
Make it a bit clearer what's going on.
2019-06-21 17:14:56 +01:00
5d6644efe0
Only import jinja2 when needed ( #5514 )
...
Fixes https://github.com/matrix-org/synapse/issues/5431
`jinja2` was being imported even when it wasn't strictly necessary. This made it required to run Synapse, even if the functionality that required it wasn't enabled. This was causing new Synapse installations to crash on startup.
Email modules are now required.
2019-06-21 16:52:09 +01:00
0e8b35f7b0
Fix "Unexpected entry in 'full_schemas'" log warning ( #5509 )
...
There is a README.txt which always sets off this warning, which is a bit
alarming when you first start synapse. I don't think we need to warn about
this.
2019-06-21 15:11:57 +01:00
2f8491daef
Fix logging error when a tampered event is detected. ( #5500 )
2019-06-21 15:11:42 +01:00
7eadb74056
Fix /messages on worker when no token supplied
2019-06-21 14:15:59 +01:00
8181e290a9
Fix sync tightloop bug.
...
If, for some reason, presence updates take a while to persist then it
can trigger clients to tightloop calling `/sync` due to the presence
handler returning updates but not advancing the stream token.
Fixes #5503 .
2019-06-21 11:10:27 +01:00
f3ab533374
Support pagination API in client_reader worker
2019-06-21 10:43:12 +01:00
7456698241
Merge pull request #5476 from matrix-org/erikj/histogram_extremities
...
Add metrics for length of new extremities persisted.
2019-06-21 09:58:54 +01:00
f47969f42a
Improve email notification logging ( #5502 )
2019-06-20 15:14:26 +01:00
f8bd30af2a
Black
2019-06-20 13:10:06 +01:00
45f28a9d2f
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/histogram_extremities
2019-06-20 11:59:14 +01:00
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
fe641df770
Sanity-checking for metrics updates
...
Check that our clocks go forward.
2019-06-19 21:18:38 +01:00
f682af052d
Simplify PerformanceCounters.update interface
...
we already have the duration for the update, so may as well use it rather than
passing extra params around and recalculating it.
2019-06-19 21:18:38 +01:00
68128d5626
Remove unused _get_event_counters
...
This has been redundant since cdb3757942
2019-06-19 21:14:25 +01:00
15bf32e062
Use monotonic clock where possible for metrics
...
Fixes intermittent errors observed on Apple hardware which were caused by
time.clock() appearing to go backwards when called from different threads.
Also fixes a bug where database activity times were logged as 1/1000 of their
correct ratio due to confusion between milliseconds and seconds.
2019-06-19 21:09:43 +01:00
Richard van der Hoff