Merge branch 'rav/fix_retry_reset' into matrix-org-hotfixes
commit
8569f3cdef
105
UPGRADE.rst
105
UPGRADE.rst
|
@ -2,52 +2,78 @@ Upgrading Synapse
|
|||
=================
|
||||
|
||||
Before upgrading check if any special steps are required to upgrade from the
|
||||
what you currently have installed to current version of synapse. The extra
|
||||
what you currently have installed to current version of Synapse. The extra
|
||||
instructions that may be required are listed later in this document.
|
||||
|
||||
1. If synapse was installed in a virtualenv then activate that virtualenv before
|
||||
upgrading. If synapse is installed in a virtualenv in ``~/synapse/env`` then
|
||||
run:
|
||||
* If Synapse was installed using `prebuilt packages
|
||||
<INSTALL.md#prebuilt-packages>`_, you will need to follow the normal process
|
||||
for upgrading those packages.
|
||||
|
||||
.. code:: bash
|
||||
* If Synapse was installed from source, then:
|
||||
|
||||
1. Activate the virtualenv before upgrading. For example, if Synapse is
|
||||
installed in a virtualenv in ``~/synapse/env`` then run:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
source ~/synapse/env/bin/activate
|
||||
|
||||
2. If synapse was installed using pip then upgrade to the latest version by
|
||||
running:
|
||||
2. If Synapse was installed using pip then upgrade to the latest version by
|
||||
running:
|
||||
|
||||
.. code:: bash
|
||||
.. code:: bash
|
||||
|
||||
pip install --upgrade matrix-synapse[all]
|
||||
pip install --upgrade matrix-synapse
|
||||
|
||||
# restart synapse
|
||||
synctl restart
|
||||
If Synapse was installed using git then upgrade to the latest version by
|
||||
running:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
If synapse was installed using git then upgrade to the latest version by
|
||||
running:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
# Pull the latest version of the master branch.
|
||||
git pull
|
||||
pip install --upgrade .
|
||||
|
||||
# Update synapse and its python dependencies.
|
||||
pip install --upgrade .[all]
|
||||
3. Restart Synapse:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
# restart synapse
|
||||
./synctl restart
|
||||
|
||||
|
||||
To check whether your update was successful, you can check the Server header
|
||||
returned by the Client-Server API:
|
||||
To check whether your update was successful, you can check the running server
|
||||
version with:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
# replace <host.name> with the hostname of your synapse homeserver.
|
||||
# You may need to specify a port (eg, :8448) if your server is not
|
||||
# configured on port 443.
|
||||
curl -kv https://<host.name>/_matrix/client/versions 2>&1 | grep "Server:"
|
||||
# you may need to replace 'localhost:8008' if synapse is not configured
|
||||
# to listen on port 8008.
|
||||
|
||||
curl http://localhost:8008/_synapse/admin/v1/server_version
|
||||
|
||||
Rolling back to older versions
|
||||
------------------------------
|
||||
|
||||
Rolling back to previous releases can be difficult, due to database schema
|
||||
changes between releases. Where we have been able to test the rollback process,
|
||||
this will be noted below.
|
||||
|
||||
In general, you will need to undo any changes made during the upgrade process,
|
||||
for example:
|
||||
|
||||
* pip:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
source env/bin/activate
|
||||
# replace `1.3.0` accordingly:
|
||||
pip install matrix-synapse==1.3.0
|
||||
|
||||
* Debian:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
# replace `1.3.0` and `stretch` accordingly:
|
||||
wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
|
||||
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
|
||||
|
||||
Upgrading to v1.4.0
|
||||
===================
|
||||
|
@ -99,6 +125,31 @@ Synapse will expect these files to exist inside the configured template director
|
|||
default templates, see `synapse/res/templates
|
||||
<https://github.com/matrix-org/synapse/tree/master/synapse/res/templates>`_.
|
||||
|
||||
Rolling back to v1.3.1
|
||||
----------------------
|
||||
|
||||
If you encounter problems with v1.4.0, it should be possible to roll back to
|
||||
v1.3.1, subject to the following:
|
||||
|
||||
* The 'room statistics' engine was heavily reworked in this release (see
|
||||
`#5971 <https://github.com/matrix-org/synapse/pull/5971>`_), including
|
||||
significant changes to the database schema, which are not easily
|
||||
reverted. This will cause the room statistics engine to stop updating when
|
||||
you downgrade.
|
||||
|
||||
The room statistics are essentially unused in v1.3.1 (in future versions of
|
||||
Synapse, they will be used to populate the room directory), so there should
|
||||
be no loss of functionality. However, the statistics engine will write errors
|
||||
to the logs, which can be avoided by setting the following in `homeserver.yaml`:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
stats:
|
||||
enabled: false
|
||||
|
||||
Don't forget to re-enable it when you upgrade again, in preparation for its
|
||||
use in the room directory!
|
||||
|
||||
Upgrading to v1.2.0
|
||||
===================
|
||||
|
||||
|
|
|
@ -1 +1 @@
|
|||
Drop some unused tables.
|
||||
Stop populating some unused tables.
|
||||
|
|
|
@ -1 +1 @@
|
|||
Add POST /_matrix/client/r0/account/3pid/unbind endpoint from MSC2140 for unbinding a 3PID from an identity server without removing it from the homeserver user account.
|
||||
Add POST /_matrix/client/unstable/account/3pid/unbind endpoint from MSC2140 for unbinding a 3PID from an identity server without removing it from the homeserver user account.
|
||||
|
|
|
@ -0,0 +1 @@
|
|||
Give appropriate exit codes when synctl fails.
|
|
@ -0,0 +1 @@
|
|||
Replace `trust_identity_server_for_password_resets` config option with `account_threepid_delegates`.
|
|
@ -0,0 +1,2 @@
|
|||
Stop populating some unused tables.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Add some notes on rolling back to v1.3.1.
|
|
@ -0,0 +1 @@
|
|||
Update the upgrade notes.
|
|
@ -0,0 +1 @@
|
|||
Remove POST method from password reset submit_token endpoint until we implement submit_url functionality.
|
|
@ -0,0 +1 @@
|
|||
Provide SYNAPSE_WORKER envvar to specify python module.
|
|
@ -0,0 +1 @@
|
|||
Fix logcontext spam on non-Linux platforms.
|
|
@ -0,0 +1 @@
|
|||
Add POST /_matrix/client/unstable/account/3pid/unbind endpoint from MSC2140 for unbinding a 3PID from an identity server without removing it from the homeserver user account.
|
|
@ -0,0 +1 @@
|
|||
Ensure query parameters in email validation links are URL-encoded.
|
|
@ -0,0 +1 @@
|
|||
Remove `bind` parameter from Client Server POST `/account` endpoint as per [MSC2290](https://github.com/matrix-org/matrix-doc/pull/2290/).
|
|
@ -0,0 +1 @@
|
|||
Add a 'failure_ts' column to the 'destinations' database table.
|
|
@ -89,6 +89,8 @@ The following environment variables are supported in run mode:
|
|||
`/data`.
|
||||
* `SYNAPSE_CONFIG_PATH`: path to the config file. Defaults to
|
||||
`<SYNAPSE_CONFIG_DIR>/homeserver.yaml`.
|
||||
* `SYNAPSE_WORKER`: module to execute, used when running synapse with workers.
|
||||
Defaults to `synapse.app.homeserver`, which is suitable for non-worker mode.
|
||||
* `UID`, `GID`: the user and group id to run Synapse as. Defaults to `991`, `991`.
|
||||
* `TZ`: the [timezone](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) the container will run with. Defaults to `UTC`.
|
||||
|
||||
|
|
|
@ -182,6 +182,7 @@ def main(args, environ):
|
|||
mode = args[1] if len(args) > 1 else None
|
||||
desired_uid = int(environ.get("UID", "991"))
|
||||
desired_gid = int(environ.get("GID", "991"))
|
||||
synapse_worker = environ.get("SYNAPSE_WORKER", "synapse.app.homeserver")
|
||||
if (desired_uid == os.getuid()) and (desired_gid == os.getgid()):
|
||||
ownership = None
|
||||
else:
|
||||
|
@ -245,7 +246,7 @@ def main(args, environ):
|
|||
|
||||
log("Starting synapse with config file " + config_path)
|
||||
|
||||
args = ["python", "-m", "synapse.app.homeserver", "--config-path", config_path]
|
||||
args = ["python", "-m", synapse_worker, "--config-path", config_path]
|
||||
if ownership is not None:
|
||||
args = ["su-exec", ownership] + args
|
||||
os.execv("/sbin/su-exec", args)
|
||||
|
|
|
@ -938,7 +938,7 @@ uploads_path: "DATADIR/uploads"
|
|||
# https://matrix.org/docs/spec/identity_service/latest
|
||||
#
|
||||
account_threepid_delegates:
|
||||
#email: https://example.com # Delegate email sending to matrix.org
|
||||
#email: https://example.com # Delegate email sending to example.org
|
||||
#msisdn: http://localhost:8090 # Delegate SMS sending to this local process
|
||||
|
||||
# Users who register on this homeserver will automatically be joined
|
||||
|
|
|
@ -294,7 +294,7 @@ class RegistrationConfig(Config):
|
|||
# https://matrix.org/docs/spec/identity_service/latest
|
||||
#
|
||||
account_threepid_delegates:
|
||||
#email: https://example.com # Delegate email sending to matrix.org
|
||||
#email: https://example.com # Delegate email sending to example.org
|
||||
#msisdn: http://localhost:8090 # Delegate SMS sending to this local process
|
||||
|
||||
# Users who register on this homeserver will automatically be joined
|
||||
|
|
|
@ -165,7 +165,7 @@ class Authenticator(object):
|
|||
async def _reset_retry_timings(self, origin):
|
||||
try:
|
||||
logger.info("Marking origin %r as up", origin)
|
||||
await self.store.set_destination_retry_timings(origin, 0, 0)
|
||||
await self.store.set_destination_retry_timings(origin, None, 0, 0)
|
||||
except Exception:
|
||||
logger.exception("Error resetting retry timings on %s", origin)
|
||||
|
||||
|
|
|
@ -1,4 +1,5 @@
|
|||
# Copyright 2014-2016 OpenMarket Ltd
|
||||
# Copyright 2019 The Matrix.org Foundation C.I.C.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
|
@ -42,13 +43,17 @@ try:
|
|||
# exception.
|
||||
resource.getrusage(RUSAGE_THREAD)
|
||||
|
||||
is_thread_resource_usage_supported = True
|
||||
|
||||
def get_thread_resource_usage():
|
||||
return resource.getrusage(RUSAGE_THREAD)
|
||||
|
||||
|
||||
except Exception:
|
||||
# If the system doesn't support resource.getrusage(RUSAGE_THREAD) then we
|
||||
# won't track resource usage by returning None.
|
||||
# won't track resource usage.
|
||||
is_thread_resource_usage_supported = False
|
||||
|
||||
def get_thread_resource_usage():
|
||||
return None
|
||||
|
||||
|
@ -359,7 +364,11 @@ class LoggingContext(object):
|
|||
|
||||
# When we stop, let's record the cpu used since we started
|
||||
if not self.usage_start:
|
||||
logger.warning("Called stop on logcontext %s without calling start", self)
|
||||
# Log a warning on platforms that support thread usage tracking
|
||||
if is_thread_resource_usage_supported:
|
||||
logger.warning(
|
||||
"Called stop on logcontext %s without calling start", self
|
||||
)
|
||||
return
|
||||
|
||||
utime_delta, stime_delta = self._get_cputime()
|
||||
|
|
|
@ -136,10 +136,11 @@ class Mailer(object):
|
|||
group together multiple email sending attempts
|
||||
sid (str): The generated session ID
|
||||
"""
|
||||
params = {"token": token, "client_secret": client_secret, "sid": sid}
|
||||
link = (
|
||||
self.hs.config.public_baseurl
|
||||
+ "_matrix/client/unstable/password_reset/email/submit_token"
|
||||
"?token=%s&client_secret=%s&sid=%s" % (token, client_secret, sid)
|
||||
+ "_matrix/client/unstable/password_reset/email/submit_token?%s"
|
||||
% urllib.parse.urlencode(params)
|
||||
)
|
||||
|
||||
template_vars = {"link": link}
|
||||
|
@ -163,10 +164,11 @@ class Mailer(object):
|
|||
group together multiple email sending attempts
|
||||
sid (str): The generated session ID
|
||||
"""
|
||||
params = {"token": token, "client_secret": client_secret, "sid": sid}
|
||||
link = (
|
||||
self.hs.config.public_baseurl
|
||||
+ "_matrix/client/unstable/registration/email/submit_token"
|
||||
"?token=%s&client_secret=%s&sid=%s" % (token, client_secret, sid)
|
||||
+ "_matrix/client/unstable/registration/email/submit_token?%s"
|
||||
% urllib.parse.urlencode(params)
|
||||
)
|
||||
|
||||
template_vars = {"link": link}
|
||||
|
|
|
@ -272,23 +272,6 @@ class PasswordResetSubmitTokenServlet(RestServlet):
|
|||
request.write(html.encode("utf-8"))
|
||||
finish_request(request)
|
||||
|
||||
@defer.inlineCallbacks
|
||||
def on_POST(self, request, medium):
|
||||
if medium != "email":
|
||||
raise SynapseError(
|
||||
400, "This medium is currently not supported for password resets"
|
||||
)
|
||||
|
||||
body = parse_json_object_from_request(request)
|
||||
assert_params_in_dict(body, ["sid", "client_secret", "token"])
|
||||
|
||||
valid, _ = yield self.store.validate_threepid_session(
|
||||
body["sid"], body["client_secret"], body["token"], self.clock.time_msec()
|
||||
)
|
||||
response_code = 200 if valid else 400
|
||||
|
||||
return response_code, {"success": valid}
|
||||
|
||||
|
||||
class PasswordRestServlet(RestServlet):
|
||||
PATTERNS = client_patterns("/account/password$")
|
||||
|
@ -538,15 +521,11 @@ class ThreepidRestServlet(RestServlet):
|
|||
user_id, threepid["medium"], threepid["address"], threepid["validated_at"]
|
||||
)
|
||||
|
||||
if "bind" in body and body["bind"]:
|
||||
logger.debug("Binding threepid %s to %s", threepid, user_id)
|
||||
yield self.identity_handler.bind_threepid(threepid_creds, user_id)
|
||||
|
||||
return 200, {}
|
||||
|
||||
|
||||
class ThreepidUnbindRestServlet(RestServlet):
|
||||
PATTERNS = client_patterns("/account/3pid/unbind$")
|
||||
PATTERNS = client_patterns("/account/3pid/unbind$", releases=(), unstable=True)
|
||||
|
||||
def __init__(self, hs):
|
||||
super(ThreepidUnbindRestServlet, self).__init__()
|
||||
|
|
|
@ -1,20 +0,0 @@
|
|||
/* Copyright 2019 The Matrix.org Foundation C.I.C.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
-- these tables are never used.
|
||||
DROP TABLE IF EXISTS room_names;
|
||||
DROP TABLE IF EXISTS topics;
|
||||
DROP TABLE IF EXISTS history_visibility;
|
||||
DROP TABLE IF EXISTS guest_access;
|
43
synctl
43
synctl
|
@ -71,7 +71,20 @@ def abort(message, colour=RED, stream=sys.stderr):
|
|||
sys.exit(1)
|
||||
|
||||
|
||||
def start(configfile, daemonize=True):
|
||||
def start(configfile: str, daemonize: bool = True) -> bool:
|
||||
"""Attempts to start synapse.
|
||||
Args:
|
||||
configfile: path to a yaml synapse config file
|
||||
daemonize: whether to daemonize synapse or keep it attached to the current
|
||||
session
|
||||
|
||||
Returns:
|
||||
True if the process started successfully
|
||||
False if there was an error starting the process
|
||||
|
||||
If deamonize is False it will only return once synapse exits.
|
||||
"""
|
||||
|
||||
write("Starting ...")
|
||||
args = SYNAPSE
|
||||
|
||||
|
@ -83,25 +96,40 @@ def start(configfile, daemonize=True):
|
|||
try:
|
||||
subprocess.check_call(args)
|
||||
write("started synapse.app.homeserver(%r)" % (configfile,), colour=GREEN)
|
||||
return True
|
||||
except subprocess.CalledProcessError as e:
|
||||
write(
|
||||
"error starting (exit code: %d); see above for logs" % e.returncode,
|
||||
colour=RED,
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
def start_worker(app, configfile, worker_configfile):
|
||||
def start_worker(app: str, configfile: str, worker_configfile: str) -> bool:
|
||||
"""Attempts to start a synapse worker.
|
||||
Args:
|
||||
app: name of the worker's appservice
|
||||
configfile: path to a yaml synapse config file
|
||||
worker_configfile: path to worker specific yaml synapse file
|
||||
|
||||
Returns:
|
||||
True if the process started successfully
|
||||
False if there was an error starting the process
|
||||
"""
|
||||
|
||||
args = [sys.executable, "-B", "-m", app, "-c", configfile, "-c", worker_configfile]
|
||||
|
||||
try:
|
||||
subprocess.check_call(args)
|
||||
write("started %s(%r)" % (app, worker_configfile), colour=GREEN)
|
||||
return True
|
||||
except subprocess.CalledProcessError as e:
|
||||
write(
|
||||
"error starting %s(%r) (exit code: %d); see above for logs"
|
||||
% (app, worker_configfile, e.returncode),
|
||||
colour=RED,
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
def stop(pidfile, app):
|
||||
|
@ -292,11 +320,14 @@ def main():
|
|||
write("All processes exited; now restarting...")
|
||||
|
||||
if action == "start" or action == "restart":
|
||||
error = False
|
||||
if start_stop_synapse:
|
||||
# Check if synapse is already running
|
||||
if os.path.exists(pidfile) and pid_running(int(open(pidfile).read())):
|
||||
abort("synapse.app.homeserver already running")
|
||||
start(configfile, bool(options.daemonize))
|
||||
|
||||
if not start(configfile, bool(options.daemonize)):
|
||||
error = True
|
||||
|
||||
for worker in workers:
|
||||
env = os.environ.copy()
|
||||
|
@ -307,12 +338,16 @@ def main():
|
|||
for cache_name, factor in iteritems(worker.cache_factors):
|
||||
os.environ["SYNAPSE_CACHE_FACTOR_" + cache_name.upper()] = str(factor)
|
||||
|
||||
start_worker(worker.app, configfile, worker.configfile)
|
||||
if not start_worker(worker.app, configfile, worker.configfile):
|
||||
error = True
|
||||
|
||||
# Reset env back to the original
|
||||
os.environ.clear()
|
||||
os.environ.update(env)
|
||||
|
||||
if error:
|
||||
exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
|
@ -29,3 +29,12 @@ Enabling an unknown default rule fails with 404
|
|||
|
||||
# Blacklisted due to https://github.com/matrix-org/synapse/issues/1663
|
||||
New federated private chats get full presence information (SYN-115)
|
||||
|
||||
# Blacklisted temporarily due to https://github.com/matrix-org/matrix-doc/pull/2290
|
||||
# These sytests need to be updated with new endpoints, which will come in a later PR
|
||||
# That PR will also remove this blacklist
|
||||
Can bind 3PID via home server
|
||||
Can bind and unbind 3PID via homeserver
|
||||
3PIDs are unbound after account deactivation
|
||||
Can bind and unbind 3PID via /unbind by specifying the identity server
|
||||
Can bind and unbind 3PID via /unbind without specifying the identity server
|
||||
|
|
Loading…
Reference in New Issue