Merge remote-tracking branch 'origin/develop' into erikj/record_txn_id

pull/8476/head
Erik Johnston 2020-10-07 14:58:42 +01:00
commit 801dda26c7
192 changed files with 2530 additions and 1895 deletions

View File

@ -1,41 +1,10 @@
# This file serves as a blacklist for SyTest tests that we expect will fail in
# Synapse when run under worker mode. For more details, see sytest-blacklist.
Message history can be paginated
Can re-join room if re-invited
The only membership state included in an initial sync is for all the senders in the timeline
Local device key changes get to remote servers
If remote user leaves room we no longer receive device updates
Forgotten room messages cannot be paginated
Inbound federation can get public room list
Members from the gap are included in gappy incr LL sync
Leaves are present in non-gapped incremental syncs
Old leaves are present in gapped incremental syncs
User sees updates to presence from other users in the incremental sync.
Gapped incremental syncs include all state changes
Old members are included in gappy incr LL sync if they start speaking
# new failures as of https://github.com/matrix-org/sytest/pull/732
Device list doesn't change if remote server is down
Remote servers cannot set power levels in rooms without existing powerlevels
Remote servers should reject attempts by non-creators to set the power levels
# https://buildkite.com/matrix-dot-org/synapse/builds/6134#6f67bf47-e234-474d-80e8-c6e1868b15c5
Server correctly handles incoming m.device_list_update
# this fails reliably with a torture level of 100 due to https://github.com/matrix-org/synapse/issues/6536
Outbound federation requests missing prev_events and then asks for /state_ids and resolves the state
Can get rooms/{roomId}/members at a given point

View File

@ -290,19 +290,6 @@ Testing with SyTest is recommended for verifying that changes related to the
Client-Server API are functioning correctly. See the `installation instructions
<https://github.com/matrix-org/sytest#installing>`_ for details.
Building Internal API Documentation
===================================
Before building internal API documentation install sphinx and
sphinxcontrib-napoleon::
pip install sphinx
pip install sphinxcontrib-napoleon
Building internal API documentation::
python setup.py build_sphinx
Troubleshooting
===============

View File

@ -75,6 +75,23 @@ for example:
wget https://packages.matrix.org/debian/pool/main/m/matrix-synapse-py3/matrix-synapse-py3_1.3.0+stretch1_amd64.deb
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
Upgrading to v1.22.0
====================
ThirdPartyEventRules breaking changes
-------------------------------------
This release introduces a backwards-incompatible change to modules making use of
``ThirdPartyEventRules`` in Synapse. If you make use of a module defined under the
``third_party_event_rules`` config option, please make sure it is updated to handle
the below change:
The ``http_client`` argument is no longer passed to modules as they are initialised. Instead,
modules are expected to make use of the ``http_client`` property on the ``ModuleApi`` class.
Modules are now passed a ``module_api`` argument during initialisation, which is an instance of
``ModuleApi``. ``ModuleApi`` instances have a ``http_client`` property which acts the same as
the ``http_client`` argument previously passed to ``ThirdPartyEventRules`` modules.
Upgrading to v1.21.0
====================

1
changelog.d/7658.feature Normal file
View File

@ -0,0 +1 @@
Add a configuration option for always using the "userinfo endpoint" for OpenID Connect. This fixes support for some identity providers, e.g. GitLab. Contributed by Benjamin Koch.

1
changelog.d/8292.feature Normal file
View File

@ -0,0 +1 @@
Allow `ThirdPartyEventRules` modules to query and manipulate whether a room is in the public rooms directory.

1
changelog.d/8312.feature Normal file
View File

@ -0,0 +1 @@
Add support for olm fallback keys ([MSC2732](https://github.com/matrix-org/matrix-doc/pull/2732)).

1
changelog.d/8369.feature Normal file
View File

@ -0,0 +1 @@
Allow running background tasks in a separate worker process.

1
changelog.d/8380.feature Normal file
View File

@ -0,0 +1 @@
Add support for device dehydration ([MSC2697](https://github.com/matrix-org/matrix-doc/pull/2697)).

1
changelog.d/8407.misc Normal file
View File

@ -0,0 +1 @@
Add typing information to the device handler.

1
changelog.d/8432.misc Normal file
View File

@ -0,0 +1 @@
Check for unreachable code with mypy.

1
changelog.d/8433.misc Normal file
View File

@ -0,0 +1 @@
Add unit test for event persister sharding.

1
changelog.d/8443.misc Normal file
View File

@ -0,0 +1 @@
Configure `public_baseurl` when using demo scripts.

1
changelog.d/8448.misc Normal file
View File

@ -0,0 +1 @@
Add SQL logging on queries that happen during startup.

1
changelog.d/8450.misc Normal file
View File

@ -0,0 +1 @@
Speed up unit tests when using PostgreSQL.

1
changelog.d/8452.misc Normal file
View File

@ -0,0 +1 @@
Remove redundant databae loads of stream_ordering for events we already have.

1
changelog.d/8454.bugfix Normal file
View File

@ -0,0 +1 @@
Fix a longstanding bug where invalid ignored users in account data could break clients.

1
changelog.d/8457.bugfix Normal file
View File

@ -0,0 +1 @@
Fix a bug where backfilling a room with an event that was missing the `redacts` field would break.

1
changelog.d/8461.feature Normal file
View File

@ -0,0 +1 @@
Change default room version to "6", per [MSC2788](https://github.com/matrix-org/matrix-doc/pull/2788).

1
changelog.d/8462.doc Normal file
View File

@ -0,0 +1 @@
Update the directions for using the manhole with coroutines.

1
changelog.d/8463.misc Normal file
View File

@ -0,0 +1 @@
Reduce inconsistencies between codepaths for membership and non-membership events.

1
changelog.d/8464.misc Normal file
View File

@ -0,0 +1 @@
Combine `SpamCheckerApi` with the more generic `ModuleApi`.

1
changelog.d/8465.bugfix Normal file
View File

@ -0,0 +1 @@
Don't attempt to respond to some requests if the client has already disconnected.

1
changelog.d/8467.feature Normal file
View File

@ -0,0 +1 @@
Allow `ThirdPartyEventRules` modules to query and manipulate whether a room is in the public rooms directory.

1
changelog.d/8468.misc Normal file
View File

@ -0,0 +1 @@
Additional testing for `ThirdPartyEventRules`.

1
changelog.d/8474.misc Normal file
View File

@ -0,0 +1 @@
Unblacklist some sytests.

1
changelog.d/8480.misc Normal file
View File

@ -0,0 +1 @@
Remove outdated sphinx documentation, scripts and configuration.

1
changelog.d/8486.bugfix Normal file
View File

@ -0,0 +1 @@
Fix incremental sync returning an incorrect `prev_batch` token in timeline section, which when used to paginate returned events that were included in the incremental sync. Broken since v0.16.0.

View File

@ -30,6 +30,8 @@ for port in 8080 8081 8082; do
if ! grep -F "Customisation made by demo/start.sh" -q $DIR/etc/$port.config; then
printf '\n\n# Customisation made by demo/start.sh\n' >> $DIR/etc/$port.config
echo "public_baseurl: http://localhost:$port/" >> $DIR/etc/$port.config
echo 'enable_registration: true' >> $DIR/etc/$port.config
# Warning, this heredoc depends on the interaction of tabs and spaces. Please don't

View File

@ -64,8 +64,6 @@ save as it takes a while and is very resource intensive.
- Use underscores for functions and variables.
- **Docstrings**: should follow the [google code
style](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings).
This is so that we can generate documentation with
[sphinx](http://sphinxcontrib-napoleon.readthedocs.org/en/latest/).
See the
[examples](http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html)
in the sphinx documentation.

View File

@ -35,9 +35,12 @@ This gives a Python REPL in which `hs` gives access to the
`synapse.server.HomeServer` object - which in turn gives access to many other
parts of the process.
Note that any call which returns a coroutine will need to be wrapped in `ensureDeferred`.
As a simple example, retrieving an event from the database:
```
>>> hs.get_datastore().get_event('$1416420717069yeQaw:matrix.org')
```pycon
>>> from twisted.internet import defer
>>> defer.ensureDeferred(hs.get_datastore().get_event('$1416420717069yeQaw:matrix.org'))
<Deferred at 0x7ff253fc6998 current result: <FrozenEvent event_id='$1416420717069yeQaw:matrix.org', type='m.room.create', state_key=''>>
```

View File

@ -238,13 +238,36 @@ Synapse config:
```yaml
oidc_config:
enabled: true
issuer: "https://id.twitch.tv/oauth2/"
client_id: "your-client-id" # TO BE FILLED
client_secret: "your-client-secret" # TO BE FILLED
client_auth_method: "client_secret_post"
user_mapping_provider:
config:
localpart_template: '{{ user.preferred_username }}'
display_name_template: '{{ user.name }}'
enabled: true
issuer: "https://id.twitch.tv/oauth2/"
client_id: "your-client-id" # TO BE FILLED
client_secret: "your-client-secret" # TO BE FILLED
client_auth_method: "client_secret_post"
user_mapping_provider:
config:
localpart_template: "{{ user.preferred_username }}"
display_name_template: "{{ user.name }}"
```
### GitLab
1. Create a [new application](https://gitlab.com/profile/applications).
2. Add the `read_user` and `openid` scopes.
3. Add this Callback URL: `[synapse public baseurl]/_synapse/oidc/callback`
Synapse config:
```yaml
oidc_config:
enabled: true
issuer: "https://gitlab.com/"
client_id: "your-client-id" # TO BE FILLED
client_secret: "your-client-secret" # TO BE FILLED
client_auth_method: "client_secret_post"
scopes: ["openid", "read_user"]
user_profile_method: "userinfo_endpoint"
user_mapping_provider:
config:
localpart_template: '{{ user.nickname }}'
display_name_template: '{{ user.name }}'
```

View File

@ -119,7 +119,7 @@ pid_file: DATADIR/homeserver.pid
# For example, for room version 1, default_room_version should be set
# to "1".
#
#default_room_version: "5"
#default_room_version: "6"
# The GC threshold parameters to pass to `gc.set_threshold`, if defined
#
@ -1714,6 +1714,14 @@ oidc_config:
#
#skip_verification: true
# Whether to fetch the user profile from the userinfo endpoint. Valid
# values are: "auto" or "userinfo_endpoint".
#
# Defaults to "auto", which fetches the userinfo endpoint if "openid" is included
# in `scopes`. Uncomment the following to always fetch the userinfo endpoint.
#
#user_profile_method: "userinfo_endpoint"
# Uncomment to allow a user logging in via OIDC to match a pre-existing account instead
# of failing. This could be used if switching from password logins to OIDC. Defaults to false.
#
@ -2496,6 +2504,11 @@ opentracing:
# events: worker1
# typing: worker1
# The worker that is used to run background tasks (e.g. cleaning up expired
# data). If not provided this defaults to the main process.
#
#run_background_tasks_on: worker1
# Configuration for Redis when using workers. This *must* be enabled when
# using workers (unless using old style direct TCP configuration).

View File

@ -11,7 +11,7 @@ able to be imported by the running Synapse.
The Python class is instantiated with two objects:
* Any configuration (see below).
* An instance of `synapse.spam_checker_api.SpamCheckerApi`.
* An instance of `synapse.module_api.ModuleApi`.
It then implements methods which return a boolean to alter behavior in Synapse.
@ -26,11 +26,8 @@ well as some specific methods:
The details of the each of these methods (as well as their inputs and outputs)
are documented in the `synapse.events.spamcheck.SpamChecker` class.
The `SpamCheckerApi` class provides a way for the custom spam checker class to
call back into the homeserver internals. It currently implements the following
methods:
* `get_state_events_in_room`
The `ModuleApi` class provides a way for the custom spam checker class to
call back into the homeserver internals.
### Example

View File

@ -1 +0,0 @@
TODO: how (if at all) is this actually maintained?

View File

@ -1,271 +0,0 @@
# -*- coding: utf-8 -*-
#
# Synapse documentation build configuration file, created by
# sphinx-quickstart on Tue Jun 10 17:31:02 2014.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
import sys
import os
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
sys.path.insert(0, os.path.abspath(".."))
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
# needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
"sphinx.ext.autodoc",
"sphinx.ext.intersphinx",
"sphinx.ext.coverage",
"sphinx.ext.ifconfig",
"sphinxcontrib.napoleon",
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]
# The suffix of source filenames.
source_suffix = ".rst"
# The encoding of source files.
# source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = "index"
# General information about the project.
project = "Synapse"
copyright = (
"Copyright 2014-2017 OpenMarket Ltd, 2017 Vector Creations Ltd, 2017 New Vector Ltd"
)
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = "1.0"
# The full version, including alpha/beta/rc tags.
release = "1.0"
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
# language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
# today = ''
# Else, today_fmt is used as the format for a strftime call.
# today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ["_build"]
# The reST default role (used for this markup: `text`) to use for all
# documents.
# default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
# add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
# add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
# show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = "sphinx"
# A list of ignored prefixes for module index sorting.
# modindex_common_prefix = []
# If true, keep warnings as "system message" paragraphs in the built documents.
# keep_warnings = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = "default"
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
# html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
# html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
# html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
# html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
# html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
# html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ["_static"]
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
# html_extra_path = []
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
# html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
# html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
# html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
# html_additional_pages = {}
# If false, no module index is generated.
# html_domain_indices = True
# If false, no index is generated.
# html_use_index = True
# If true, the index is split into individual pages for each letter.
# html_split_index = False
# If true, links to the reST sources are added to the pages.
# html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
# html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
# html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
# html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
# html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = "Synapsedoc"
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [("index", "Synapse.tex", "Synapse Documentation", "TNG", "manual")]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
# latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
# latex_use_parts = False
# If true, show page references after internal links.
# latex_show_pagerefs = False
# If true, show URL addresses after external links.
# latex_show_urls = False
# Documents to append as an appendix to all manuals.
# latex_appendices = []
# If false, no module index is generated.
# latex_domain_indices = True
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [("index", "synapse", "Synapse Documentation", ["TNG"], 1)]
# If true, show URL addresses after external links.
# man_show_urls = False
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(
"index",
"Synapse",
"Synapse Documentation",
"TNG",
"Synapse",
"One line description of project.",
"Miscellaneous",
)
]
# Documents to append as an appendix to all manuals.
# texinfo_appendices = []
# If false, no module index is generated.
# texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
# texinfo_show_urls = 'footnote'
# If true, do not generate a @detailmenu in the "Top" node's menu.
# texinfo_no_detailmenu = False
# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {"http://docs.python.org/": None}
napoleon_include_special_with_doc = True
napoleon_use_ivar = True

View File

@ -1,20 +0,0 @@
.. Synapse documentation master file, created by
sphinx-quickstart on Tue Jun 10 17:31:02 2014.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Synapse's documentation!
===================================
Contents:
.. toctree::
synapse
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

View File

@ -1,7 +0,0 @@
synapse
=======
.. toctree::
:maxdepth: 4
synapse

View File

@ -1,7 +0,0 @@
synapse.api.auth module
=======================
.. automodule:: synapse.api.auth
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.constants module
============================
.. automodule:: synapse.api.constants
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.dbobjects module
============================
.. automodule:: synapse.api.dbobjects
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.errors module
=========================
.. automodule:: synapse.api.errors
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.event_stream module
===============================
.. automodule:: synapse.api.event_stream
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.events.factory module
=================================
.. automodule:: synapse.api.events.factory
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.events.room module
==============================
.. automodule:: synapse.api.events.room
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,18 +0,0 @@
synapse.api.events package
==========================
Submodules
----------
.. toctree::
synapse.api.events.factory
synapse.api.events.room
Module contents
---------------
.. automodule:: synapse.api.events
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.handlers.events module
==================================
.. automodule:: synapse.api.handlers.events
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.handlers.factory module
===================================
.. automodule:: synapse.api.handlers.factory
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.handlers.federation module
======================================
.. automodule:: synapse.api.handlers.federation
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.handlers.register module
====================================
.. automodule:: synapse.api.handlers.register
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.handlers.room module
================================
.. automodule:: synapse.api.handlers.room
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,21 +0,0 @@
synapse.api.handlers package
============================
Submodules
----------
.. toctree::
synapse.api.handlers.events
synapse.api.handlers.factory
synapse.api.handlers.federation
synapse.api.handlers.register
synapse.api.handlers.room
Module contents
---------------
.. automodule:: synapse.api.handlers
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.notifier module
===========================
.. automodule:: synapse.api.notifier
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.register_events module
==================================
.. automodule:: synapse.api.register_events
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.room_events module
==============================
.. automodule:: synapse.api.room_events
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,30 +0,0 @@
synapse.api package
===================
Subpackages
-----------
.. toctree::
synapse.api.events
synapse.api.handlers
synapse.api.streams
Submodules
----------
.. toctree::
synapse.api.auth
synapse.api.constants
synapse.api.errors
synapse.api.notifier
synapse.api.storage
Module contents
---------------
.. automodule:: synapse.api
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.server module
=========================
.. automodule:: synapse.api.server
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.storage module
==========================
.. automodule:: synapse.api.storage
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.stream module
=========================
.. automodule:: synapse.api.stream
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.api.streams.event module
================================
.. automodule:: synapse.api.streams.event
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,17 +0,0 @@
synapse.api.streams package
===========================
Submodules
----------
.. toctree::
synapse.api.streams.event
Module contents
---------------
.. automodule:: synapse.api.streams
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.app.homeserver module
=============================
.. automodule:: synapse.app.homeserver
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,17 +0,0 @@
synapse.app package
===================
Submodules
----------
.. toctree::
synapse.app.homeserver
Module contents
---------------
.. automodule:: synapse.app
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,10 +0,0 @@
synapse.db package
==================
Module contents
---------------
.. automodule:: synapse.db
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.federation.handler module
=================================
.. automodule:: synapse.federation.handler
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.federation.messaging module
===================================
.. automodule:: synapse.federation.messaging
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.federation.pdu_codec module
===================================
.. automodule:: synapse.federation.pdu_codec
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.federation.persistence module
=====================================
.. automodule:: synapse.federation.persistence
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.federation.replication module
=====================================
.. automodule:: synapse.federation.replication
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,22 +0,0 @@
synapse.federation package
==========================
Submodules
----------
.. toctree::
synapse.federation.handler
synapse.federation.pdu_codec
synapse.federation.persistence
synapse.federation.replication
synapse.federation.transport
synapse.federation.units
Module contents
---------------
.. automodule:: synapse.federation
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.federation.transport module
===================================
.. automodule:: synapse.federation.transport
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.federation.units module
===============================
.. automodule:: synapse.federation.units
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,19 +0,0 @@
synapse.persistence package
===========================
Submodules
----------
.. toctree::
synapse.persistence.service
synapse.persistence.tables
synapse.persistence.transactions
Module contents
---------------
.. automodule:: synapse.persistence
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.persistence.service module
==================================
.. automodule:: synapse.persistence.service
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.persistence.tables module
=================================
.. automodule:: synapse.persistence.tables
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.persistence.transactions module
=======================================
.. automodule:: synapse.persistence.transactions
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.rest.base module
========================
.. automodule:: synapse.rest.base
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.rest.events module
==========================
.. automodule:: synapse.rest.events
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.rest.register module
============================
.. automodule:: synapse.rest.register
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.rest.room module
========================
.. automodule:: synapse.rest.room
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,20 +0,0 @@
synapse.rest package
====================
Submodules
----------
.. toctree::
synapse.rest.base
synapse.rest.events
synapse.rest.register
synapse.rest.room
Module contents
---------------
.. automodule:: synapse.rest
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,30 +0,0 @@
synapse package
===============
Subpackages
-----------
.. toctree::
synapse.api
synapse.app
synapse.federation
synapse.persistence
synapse.rest
synapse.util
Submodules
----------
.. toctree::
synapse.server
synapse.state
Module contents
---------------
.. automodule:: synapse
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.server module
=====================
.. automodule:: synapse.server
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.state module
====================
.. automodule:: synapse.state
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.util.async module
=========================
.. automodule:: synapse.util.async
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.util.dbutils module
===========================
.. automodule:: synapse.util.dbutils
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.util.http module
========================
.. automodule:: synapse.util.http
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.util.lockutils module
=============================
.. automodule:: synapse.util.lockutils
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.util.logutils module
============================
.. automodule:: synapse.util.logutils
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,21 +0,0 @@
synapse.util package
====================
Submodules
----------
.. toctree::
synapse.util.async
synapse.util.http
synapse.util.lockutils
synapse.util.logutils
synapse.util.stringutils
Module contents
---------------
.. automodule:: synapse.util
:members:
:undoc-members:
:show-inheritance:

View File

@ -1,7 +0,0 @@
synapse.util.stringutils module
===============================
.. automodule:: synapse.util.stringutils
:members:
:undoc-members:
:show-inheritance:

View File

@ -319,6 +319,23 @@ stream_writers:
events: event_persister1
```
#### Background tasks
There is also *experimental* support for moving background tasks to a separate
worker. Background tasks are run periodically or started via replication. Exactly
which tasks are configured to run depends on your Synapse configuration (e.g. if
stats is enabled).
To enable this, the worker must have a `worker_name` and can be configured to run
background tasks. For example, to move background tasks to a dedicated worker,
the shared configuration would include:
```yaml
run_background_tasks_on: background_worker
```
You might also wish to investigate the `update_user_directory` and
`media_instance_running_background_jobs` settings.
### `synapse.app.pusher`

View File

@ -6,6 +6,7 @@ check_untyped_defs = True
show_error_codes = True
show_traceback = True
mypy_path = stubs
warn_unreachable = True
files =
synapse/api,
synapse/appservice,
@ -16,6 +17,7 @@ files =
synapse/federation,
synapse/handlers/auth.py,
synapse/handlers/cas_handler.py,
synapse/handlers/device.py,
synapse/handlers/directory.py,
synapse/handlers/events.py,
synapse/handlers/federation.py,
@ -142,3 +144,6 @@ ignore_missing_imports = True
[mypy-nacl.*]
ignore_missing_imports = True
[mypy-hiredis]
ignore_missing_imports = True

View File

@ -1 +0,0 @@
sphinx-apidoc -o docs/sphinx/ synapse/ -ef

View File

@ -90,6 +90,7 @@ BOOLEAN_COLUMNS = {
"room_stats_state": ["is_federatable"],
"local_media_repository": ["safe_from_quarantine"],
"users": ["shadow_banned"],
"e2e_fallback_keys_json": ["used"],
}
@ -489,7 +490,7 @@ class Porter(object):
hs = MockHomeserver(self.hs_config)
with make_conn(db_config, engine) as db_conn:
with make_conn(db_config, engine, "portdb") as db_conn:
engine.check_database(
db_conn, allow_outdated_version=allow_outdated_version
)

View File

@ -1,8 +1,3 @@
[build_sphinx]
source-dir = docs/sphinx
build-dir = docs/build
all_files = 1
[trial]
test_suite = tests

View File

@ -16,7 +16,7 @@
"""Contains *incomplete* type hints for txredisapi.
"""
from typing import List, Optional, Union
from typing import List, Optional, Union, Type
class RedisProtocol:
def publish(self, channel: str, message: bytes): ...
@ -42,3 +42,21 @@ def lazyConnection(
class SubscriberFactory:
def buildProtocol(self, addr): ...
class ConnectionHandler: ...
class RedisFactory:
continueTrying: bool
handler: RedisProtocol
def __init__(
self,
uuid: str,
dbid: Optional[int],
poolsize: int,
isLazy: bool = False,
handler: Type = ConnectionHandler,
charset: str = "utf-8",
password: Optional[str] = None,
replyTimeout: Optional[int] = None,
convertNumbers: Optional[int] = True,
): ...

View File

@ -155,3 +155,8 @@ class EventContentFields:
class RoomEncryptionAlgorithms:
MEGOLM_V1_AES_SHA2 = "m.megolm.v1.aes-sha2"
DEFAULT = MEGOLM_V1_AES_SHA2
class AccountDataTypes:
DIRECT = "m.direct"
IGNORED_USER_LIST = "m.ignored_user_list"

View File

@ -28,6 +28,7 @@ from twisted.protocols.tls import TLSMemoryBIOFactory
import synapse
from synapse.app import check_bind_error
from synapse.app.phone_stats_home import start_phone_stats_home
from synapse.config.server import ListenerConfig
from synapse.crypto import context_factory
from synapse.logging.context import PreserveLoggingContext
@ -271,9 +272,19 @@ def start(hs: "synapse.server.HomeServer", listeners: Iterable[ListenerConfig]):
hs.get_datastore().db_pool.start_profiling()
hs.get_pusherpool().start()
# Log when we start the shut down process.
hs.get_reactor().addSystemEventTrigger(
"before", "shutdown", logger.info, "Shutting down..."
)
setup_sentry(hs)
setup_sdnotify(hs)
# If background tasks are running on the main process, start collecting the
# phone home stats.
if hs.config.run_background_tasks:
start_phone_stats_home(hs)
# We now freeze all allocated objects in the hopes that (almost)
# everything currently allocated are things that will be used for the
# rest of time. Doing so means less work each GC (hopefully).

View File

@ -208,6 +208,7 @@ def start(config_options):
# Explicitly disable background processes
config.update_user_directory = False
config.run_background_tasks = False
config.start_pushers = False
config.send_federation = False

View File

@ -128,11 +128,13 @@ from synapse.rest.key.v2 import KeyApiV2Resource
from synapse.server import HomeServer, cache_in_self
from synapse.storage.databases.main.censor_events import CensorEventsStore
from synapse.storage.databases.main.media_repository import MediaRepositoryStore
from synapse.storage.databases.main.metrics import ServerMetricsStore
from synapse.storage.databases.main.monthly_active_users import (
MonthlyActiveUsersWorkerStore,
)
from synapse.storage.databases.main.presence import UserPresenceState
from synapse.storage.databases.main.search import SearchWorkerStore
from synapse.storage.databases.main.stats import StatsStore
from synapse.storage.databases.main.ui_auth import UIAuthWorkerStore
from synapse.storage.databases.main.user_directory import UserDirectoryStore
from synapse.types import ReadReceipt
@ -454,6 +456,7 @@ class GenericWorkerSlavedStore(
# FIXME(#3714): We need to add UserDirectoryStore as we write directly
# rather than going via the correct worker.
UserDirectoryStore,
StatsStore,
UIAuthWorkerStore,
SlavedDeviceInboxStore,
SlavedDeviceStore,
@ -476,6 +479,7 @@ class GenericWorkerSlavedStore(
SlavedFilteringStore,
MonthlyActiveUsersWorkerStore,
MediaRepositoryStore,
ServerMetricsStore,
SearchWorkerStore,
BaseSlavedStore,
):

Some files were not shown because too many files have changed in this diff Show More