Merge branch 'release-v1.36' into matrix-org-hotfixes
Erik Johnston
commit
a4b573ee48
79
CHANGES.md
79
CHANGES.md
|
|
@ -1,3 +1,82 @@
|
|||
Synapse 1.36.0rc2 (2021-06-11)
|
||||
==============================
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fix a bug which caused presence updates to stop working some time after a restart, when using a presence writer worker. Broke in v1.33.0. ([\#10149](https://github.com/matrix-org/synapse/issues/10149))
|
||||
- Fix a bug when using federation sender worker where it would send out more presence updates than necessary, leading to high resource usage. Broke in v1.33.0. ([\#10163](https://github.com/matrix-org/synapse/issues/10163))
|
||||
- Fix a bug where Synapse could send the same presence update to a remote twice. ([\#10165](https://github.com/matrix-org/synapse/issues/10165))
|
||||
|
||||
|
||||
Synapse 1.36.0rc1 (2021-06-08)
|
||||
==============================
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
- Add new endpoint `/_matrix/client/r0/rooms/{roomId}/aliases` from Client-Server API r0.6.1 (previously [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432)). ([\#9224](https://github.com/matrix-org/synapse/issues/9224))
|
||||
- Improve performance of incoming federation transactions in large rooms. ([\#9953](https://github.com/matrix-org/synapse/issues/9953), [\#9973](https://github.com/matrix-org/synapse/issues/9973))
|
||||
- Rewrite logic around verifying JSON object and fetching server keys to be more performant and use less memory. ([\#10035](https://github.com/matrix-org/synapse/issues/10035))
|
||||
- Add new admin APIs for unprotecting local media from quarantine. Contributed by @dklimpel. ([\#10040](https://github.com/matrix-org/synapse/issues/10040))
|
||||
- Add new admin APIs to remove media by media ID from quarantine. Contributed by @dklimpel. ([\#10044](https://github.com/matrix-org/synapse/issues/10044))
|
||||
- Make reason and score parameters optional for reporting content. Implements [MSC2414](https://github.com/matrix-org/matrix-doc/pull/2414). Contributed by Callum Brown. ([\#10077](https://github.com/matrix-org/synapse/issues/10077))
|
||||
- Add support for routing more requests to workers. ([\#10084](https://github.com/matrix-org/synapse/issues/10084))
|
||||
- Report OpenTracing spans for database activity. ([\#10113](https://github.com/matrix-org/synapse/issues/10113), [\#10136](https://github.com/matrix-org/synapse/issues/10136), [\#10141](https://github.com/matrix-org/synapse/issues/10141))
|
||||
- Significantly reduce memory usage of joining large remote rooms. ([\#10117](https://github.com/matrix-org/synapse/issues/10117))
|
||||
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fixed a bug causing replication requests to fail when receiving a lot of events via federation. ([\#10082](https://github.com/matrix-org/synapse/issues/10082))
|
||||
- Fix a bug in the `force_tracing_for_users` option introduced in Synapse v1.35 which meant that the OpenTracing spans produced were missing most tags. ([\#10092](https://github.com/matrix-org/synapse/issues/10092))
|
||||
- Fixed a bug that could cause Synapse to stop notifying application services. Contributed by Willem Mulder. ([\#10107](https://github.com/matrix-org/synapse/issues/10107))
|
||||
- Fix bug where the server would attempt to fetch the same history in the room from a remote server multiple times in parallel. ([\#10116](https://github.com/matrix-org/synapse/issues/10116))
|
||||
- Fix a bug introduced in Synapse 1.33.0 which caused replication requests to fail when receiving a lot of very large events via federation. ([\#10118](https://github.com/matrix-org/synapse/issues/10118))
|
||||
- Fix bug when using workers where pagination requests failed if a remote server returned zero events from `/backfill`. Introduced in 1.35.0. ([\#10133](https://github.com/matrix-org/synapse/issues/10133))
|
||||
|
||||
|
||||
Improved Documentation
|
||||
----------------------
|
||||
|
||||
- Clarify security note regarding hosting Synapse on the same domain as other web applications. ([\#9221](https://github.com/matrix-org/synapse/issues/9221))
|
||||
- Update CAPTCHA documentation to mention turning off the verify origin feature. Contributed by @aaronraimist. ([\#10046](https://github.com/matrix-org/synapse/issues/10046))
|
||||
- Tweak wording of database recommendation in `INSTALL.md`. Contributed by @aaronraimist. ([\#10057](https://github.com/matrix-org/synapse/issues/10057))
|
||||
- Add initial infrastructure for rendering Synapse documentation with mdbook. ([\#10086](https://github.com/matrix-org/synapse/issues/10086))
|
||||
- Convert the remaining Admin API documentation files to markdown. ([\#10089](https://github.com/matrix-org/synapse/issues/10089))
|
||||
- Make a link in docs use HTTPS. Contributed by @RhnSharma. ([\#10130](https://github.com/matrix-org/synapse/issues/10130))
|
||||
- Fix broken link in Docker docs. ([\#10132](https://github.com/matrix-org/synapse/issues/10132))
|
||||
|
||||
|
||||
Deprecations and Removals
|
||||
-------------------------
|
||||
|
||||
- Remove the experimental `spaces_enabled` flag. The spaces features are always available now. ([\#10063](https://github.com/matrix-org/synapse/issues/10063))
|
||||
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Tell CircleCI to build Docker images from `main` branch. ([\#9906](https://github.com/matrix-org/synapse/issues/9906))
|
||||
- Simplify naming convention for release branches to only include the major and minor version numbers. ([\#10013](https://github.com/matrix-org/synapse/issues/10013))
|
||||
- Add `parse_strings_from_args` for parsing an array from query parameters. ([\#10048](https://github.com/matrix-org/synapse/issues/10048), [\#10137](https://github.com/matrix-org/synapse/issues/10137))
|
||||
- Remove some dead code regarding TLS certificate handling. ([\#10054](https://github.com/matrix-org/synapse/issues/10054))
|
||||
- Remove redundant, unmaintained `convert_server_keys` script. ([\#10055](https://github.com/matrix-org/synapse/issues/10055))
|
||||
- Improve the error message printed by synctl when synapse fails to start. ([\#10059](https://github.com/matrix-org/synapse/issues/10059))
|
||||
- Fix GitHub Actions lint for newsfragments. ([\#10069](https://github.com/matrix-org/synapse/issues/10069))
|
||||
- Update opentracing to inject the right context into the carrier. ([\#10074](https://github.com/matrix-org/synapse/issues/10074))
|
||||
- Fix up `BatchingQueue` implementation. ([\#10078](https://github.com/matrix-org/synapse/issues/10078))
|
||||
- Log method and path when dropping request due to size limit. ([\#10091](https://github.com/matrix-org/synapse/issues/10091))
|
||||
- In Github Actions workflows, summarize the Sytest results in an easy-to-read format. ([\#10094](https://github.com/matrix-org/synapse/issues/10094))
|
||||
- Make `/sync` do fewer state resolutions. ([\#10102](https://github.com/matrix-org/synapse/issues/10102))
|
||||
- Add missing type hints to the admin API servlets. ([\#10105](https://github.com/matrix-org/synapse/issues/10105))
|
||||
- Improve opentracing annotations for `Notifier`. ([\#10111](https://github.com/matrix-org/synapse/issues/10111))
|
||||
- Enable Prometheus metrics for the jaeger client library. ([\#10112](https://github.com/matrix-org/synapse/issues/10112))
|
||||
- Work to improve the responsiveness of `/sync` requests. ([\#10124](https://github.com/matrix-org/synapse/issues/10124))
|
||||
- OpenTracing: use a consistent name for background processes. ([\#10135](https://github.com/matrix-org/synapse/issues/10135))
|
||||
|
||||
|
||||
Synapse 1.35.1 (2021-06-03)
|
||||
===========================
|
||||
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
Simplify naming convention for release branches to only include the major and minor version numbers.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Rewrite logic around verifying JSON object and fetching server keys to be more performant and use less memory.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add an admin API for unprotecting local media from quarantine. Contributed by @dklimpel.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add new admin APIs to remove media by media ID from quarantine. Contributed by @dkimpel.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update CAPTCHA documentation to mention turning off the verify origin feature. Contributed by @aaronraimist.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add `parse_strings_from_args` for parsing an array from query parameters.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove some dead code regarding TLS certificate handling.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove redundant, unmaintained `convert_server_keys` script.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Tweak wording of database recommendation in `INSTALL.md`. Contributed by @aaronraimist.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Improve the error message printed by synctl when synapse fails to start.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove the experimental `spaces_enabled` flag. The spaces features are always available now.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix GitHub Actions lint for newsfragments.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update opentracing to inject the right context into the carrier.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Make reason and score parameters optional for reporting content. Implements [MSC2414](https://github.com/matrix-org/matrix-doc/pull/2414). Contributed by Callum Brown.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix up `BatchingQueue` implementation.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fixed a bug causing replication requests to fail when receiving a lot of events via federation.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add support for routing more requests to workers.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add initial infrastructure for rendering Synapse documentation with mdbook.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Convert the remaining Admin API documentation files to markdown.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Log method and path when dropping request due to size limit.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a bug in the `force_tracing_for_users` option introduced in Synapse v1.35 which meant that the OpenTracing spans produced were missing most tags.
|
||||
|
|
@ -1 +0,0 @@
|
|||
In Github Actions workflows, summarize the Sytest results in an easy-to-read format.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Make `/sync` do fewer state resolutions.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add missing type hints to the admin API servlets.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fixed a bug that could cause Synapse to stop notifying application services. Contributed by Willem Mulder.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Improve opentracing annotations for `Notifier`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Enable Prometheus metrics for the jaeger client library.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Report OpenTracing spans for database activity.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix bug where the server would attempt to fetch the same history in the room from a remote server multiple times in parallel.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Significantly reduce memory usage of joining large remote rooms.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix a bug introduced in Synapse 1.33.0 which caused replication requests to fail when receiving a lot of very large events via federation.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Work to improve the responsiveness of `/sync` requests.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Make a link in docs use HTTPS. Contributed by @RhnSharma.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix broken link in Docker docs.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix bug when using workers where pagination requests failed if a remote server returned zero events from `/backfill`. Introduced in 1.35.0.
|
||||
|
|
@ -1 +0,0 @@
|
|||
OpenTracing: use a consistent name for background processes.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Report OpenTracing spans for database activity.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add `parse_strings_from_args` for parsing an array from query parameters.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Report OpenTracing spans for database activity.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Clarify security note regarding hosting Synapse on the same domain as other web applications.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add new endpoint `/_matrix/client/r0/rooms/{roomId}/aliases` from Client-Server API r0.6.1 (previously [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432)).
|
||||
|
|
@ -1 +0,0 @@
|
|||
Tell CircleCI to build Docker images from `main` branch.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Improve performance of incoming federation transactions in large rooms.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Improve performance of incoming federation transactions in large rooms.
|
||||
|
|
@ -47,7 +47,7 @@ try:
|
|||
except ImportError:
|
||||
pass
|
||||
|
||||
__version__ = "1.35.1"
|
||||
__version__ = "1.36.0rc2"
|
||||
|
||||
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
|
||||
# We import here so that we don't have to install a bunch of deps when
|
||||
|
|
|
|||
|
|
@ -299,14 +299,14 @@ class BasePresenceHandler(abc.ABC):
|
|||
if not states:
|
||||
return
|
||||
|
||||
hosts_and_states = await get_interested_remotes(
|
||||
hosts_to_states = await get_interested_remotes(
|
||||
self.store,
|
||||
self.presence_router,
|
||||
states,
|
||||
)
|
||||
|
||||
for destinations, states in hosts_and_states:
|
||||
self._federation.send_presence_to_destinations(states, destinations)
|
||||
for destination, host_states in hosts_to_states.items():
|
||||
self._federation.send_presence_to_destinations(host_states, [destination])
|
||||
|
||||
async def send_full_presence_to_users(self, user_ids: Collection[str]):
|
||||
"""
|
||||
|
|
@ -495,9 +495,6 @@ class WorkerPresenceHandler(BasePresenceHandler):
|
|||
users=users_to_states.keys(),
|
||||
)
|
||||
|
||||
# If this is a federation sender, notify about presence updates.
|
||||
await self.maybe_send_presence_to_interested_destinations(states)
|
||||
|
||||
async def process_replication_rows(
|
||||
self, stream_name: str, instance_name: str, token: int, rows: list
|
||||
):
|
||||
|
|
@ -519,11 +516,27 @@ class WorkerPresenceHandler(BasePresenceHandler):
|
|||
for row in rows
|
||||
]
|
||||
|
||||
for state in states:
|
||||
self.user_to_current_state[state.user_id] = state
|
||||
# The list of states to notify sync streams and remote servers about.
|
||||
# This is calculated by comparing the old and new states for each user
|
||||
# using `should_notify(..)`.
|
||||
#
|
||||
# Note that this is necessary as the presence writer will periodically
|
||||
# flush presence state changes that should not be notified about to the
|
||||
# DB, and so will be sent over the replication stream.
|
||||
state_to_notify = []
|
||||
|
||||
for new_state in states:
|
||||
old_state = self.user_to_current_state.get(new_state.user_id)
|
||||
self.user_to_current_state[new_state.user_id] = new_state
|
||||
|
||||
if not old_state or should_notify(old_state, new_state):
|
||||
state_to_notify.append(new_state)
|
||||
|
||||
stream_id = token
|
||||
await self.notify_from_replication(states, stream_id)
|
||||
await self.notify_from_replication(state_to_notify, stream_id)
|
||||
|
||||
# If this is a federation sender, notify about presence updates.
|
||||
await self.maybe_send_presence_to_interested_destinations(state_to_notify)
|
||||
|
||||
def get_currently_syncing_users_for_replication(self) -> Iterable[str]:
|
||||
return [
|
||||
|
|
@ -829,15 +842,15 @@ class PresenceHandler(BasePresenceHandler):
|
|||
if to_federation_ping:
|
||||
federation_presence_out_counter.inc(len(to_federation_ping))
|
||||
|
||||
hosts_and_states = await get_interested_remotes(
|
||||
hosts_to_states = await get_interested_remotes(
|
||||
self.store,
|
||||
self.presence_router,
|
||||
list(to_federation_ping.values()),
|
||||
)
|
||||
|
||||
for destinations, states in hosts_and_states:
|
||||
for destination, states in hosts_to_states.items():
|
||||
self._federation_queue.send_presence_to_destinations(
|
||||
states, destinations
|
||||
states, [destination]
|
||||
)
|
||||
|
||||
async def _handle_timeouts(self) -> None:
|
||||
|
|
@ -1962,7 +1975,7 @@ async def get_interested_remotes(
|
|||
store: DataStore,
|
||||
presence_router: PresenceRouter,
|
||||
states: List[UserPresenceState],
|
||||
) -> List[Tuple[Collection[str], List[UserPresenceState]]]:
|
||||
) -> Dict[str, Set[UserPresenceState]]:
|
||||
"""Given a list of presence states figure out which remote servers
|
||||
should be sent which.
|
||||
|
||||
|
|
@ -1974,11 +1987,9 @@ async def get_interested_remotes(
|
|||
states: A list of incoming user presence updates.
|
||||
|
||||
Returns:
|
||||
A list of 2-tuples of destinations and states, where for
|
||||
each tuple the list of UserPresenceState should be sent to each
|
||||
destination
|
||||
A map from destinations to presence states to send to that destination.
|
||||
"""
|
||||
hosts_and_states = [] # type: List[Tuple[Collection[str], List[UserPresenceState]]]
|
||||
hosts_and_states: Dict[str, Set[UserPresenceState]] = {}
|
||||
|
||||
# First we look up the rooms each user is in (as well as any explicit
|
||||
# subscriptions), then for each distinct room we look up the remote
|
||||
|
|
@ -1990,11 +2001,12 @@ async def get_interested_remotes(
|
|||
for room_id, states in room_ids_to_states.items():
|
||||
user_ids = await store.get_users_in_room(room_id)
|
||||
hosts = {get_domain_from_id(user_id) for user_id in user_ids}
|
||||
hosts_and_states.append((hosts, states))
|
||||
for host in hosts:
|
||||
hosts_and_states.setdefault(host, set()).update(states)
|
||||
|
||||
for user_id, states in users_to_states.items():
|
||||
host = get_domain_from_id(user_id)
|
||||
hosts_and_states.append(([host], states))
|
||||
hosts_and_states.setdefault(host, set()).update(states)
|
||||
|
||||
return hosts_and_states
|
||||
|
||||
|
|
|
|||
|
|
@ -50,7 +50,7 @@ class PresenceStore(SQLBaseStore):
|
|||
instance_name=self._instance_name,
|
||||
tables=[("presence_stream", "instance_name", "stream_id")],
|
||||
sequence_name="presence_stream_sequence",
|
||||
writers=hs.config.worker.writers.to_device,
|
||||
writers=hs.config.worker.writers.presence,
|
||||
)
|
||||
else:
|
||||
self._presence_id_gen = StreamIdGenerator(
|
||||
|
|
|
|||
|
|
@ -397,6 +397,11 @@ class MultiWriterIdGenerator:
|
|||
# ... persist event ...
|
||||
"""
|
||||
|
||||
# If we have a list of instances that are allowed to write to this
|
||||
# stream, make sure we're in it.
|
||||
if self._writers and self._instance_name not in self._writers:
|
||||
raise Exception("Tried to allocate stream ID on non-writer")
|
||||
|
||||
return _MultiWriterCtxManager(self)
|
||||
|
||||
def get_next_mult(self, n: int):
|
||||
|
|
@ -406,6 +411,11 @@ class MultiWriterIdGenerator:
|
|||
# ... persist events ...
|
||||
"""
|
||||
|
||||
# If we have a list of instances that are allowed to write to this
|
||||
# stream, make sure we're in it.
|
||||
if self._writers and self._instance_name not in self._writers:
|
||||
raise Exception("Tried to allocate stream ID on non-writer")
|
||||
|
||||
return _MultiWriterCtxManager(self, n)
|
||||
|
||||
def get_next_txn(self, txn: LoggingTransaction):
|
||||
|
|
@ -416,6 +426,11 @@ class MultiWriterIdGenerator:
|
|||
# ... persist event ...
|
||||
"""
|
||||
|
||||
# If we have a list of instances that are allowed to write to this
|
||||
# stream, make sure we're in it.
|
||||
if self._writers and self._instance_name not in self._writers:
|
||||
raise Exception("Tried to allocate stream ID on non-writer")
|
||||
|
||||
next_id = self._load_next_id_txn(txn)
|
||||
|
||||
with self._lock:
|
||||
|
|
|
|||
Loading…
Reference in New Issue