Merge branch 'develop' of github.com:matrix-org/synapse into matrix-org-hotfixes
Erik Johnston
commit
578ad9fc48
72
CHANGES.md
72
CHANGES.md
|
|
@ -1,3 +1,75 @@
|
|||
Synapse 1.11.0rc1 (2020-02-19)
|
||||
==============================
|
||||
|
||||
Features
|
||||
--------
|
||||
|
||||
- Admin API to add or modify threepids of user accounts. ([\#6769](https://github.com/matrix-org/synapse/issues/6769))
|
||||
- Limit the number of events that can be requested by the backfill federation API to 100. ([\#6864](https://github.com/matrix-org/synapse/issues/6864))
|
||||
- Add ability to run some group APIs on workers. ([\#6866](https://github.com/matrix-org/synapse/issues/6866))
|
||||
- Reject device display names over 100 characters in length to prevent abuse. ([\#6882](https://github.com/matrix-org/synapse/issues/6882))
|
||||
- Add ability to route federation user device queries to workers. ([\#6873](https://github.com/matrix-org/synapse/issues/6873))
|
||||
- The result of a user directory search can now be filtered via the spam checker. ([\#6888](https://github.com/matrix-org/synapse/issues/6888))
|
||||
- Implement new `GET /_matrix/client/unstable/org.matrix.msc2432/rooms/{roomId}/aliases` endpoint as per [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432). ([\#6939](https://github.com/matrix-org/synapse/issues/6939), [\#6948](https://github.com/matrix-org/synapse/issues/6948), [\#6949](https://github.com/matrix-org/synapse/issues/6949))
|
||||
- Stop sending `m.room.alias` events wheng adding / removing aliases. Check `alt_aliases` in the latest `m.room.canonical_alias` event when deleting an alias. ([\#6904](https://github.com/matrix-org/synapse/issues/6904))
|
||||
- Change the default power levels of invites, tombstones and server ACLs for new rooms. ([\#6834](https://github.com/matrix-org/synapse/issues/6834))
|
||||
|
||||
Bugfixes
|
||||
--------
|
||||
|
||||
- Fixed third party event rules function `on_create_room`'s return value being ignored. ([\#6781](https://github.com/matrix-org/synapse/issues/6781))
|
||||
- Allow URL-encoded User IDs on `/_synapse/admin/v2/users/<user_id>[/admin]` endpoints. Thanks to @NHAS for reporting. ([\#6825](https://github.com/matrix-org/synapse/issues/6825))
|
||||
- Fix Synapse refusing to start if `federation_certificate_verification_whitelist` option is blank. ([\#6849](https://github.com/matrix-org/synapse/issues/6849))
|
||||
- Fix errors from logging in the purge jobs related to the message retention policies support. ([\#6945](https://github.com/matrix-org/synapse/issues/6945))
|
||||
- Return a 404 instead of 200 for querying information of a non-existant user through the admin API. ([\#6901](https://github.com/matrix-org/synapse/issues/6901))
|
||||
|
||||
|
||||
Updates to the Docker image
|
||||
---------------------------
|
||||
|
||||
- The deprecated "generate-config-on-the-fly" mode is no longer supported. ([\#6918](https://github.com/matrix-org/synapse/issues/6918))
|
||||
|
||||
|
||||
Improved Documentation
|
||||
----------------------
|
||||
|
||||
- Add details of PR merge strategy to contributing docs. ([\#6846](https://github.com/matrix-org/synapse/issues/6846))
|
||||
- Spell out that the last event sent to a room won't be deleted by a purge. ([\#6891](https://github.com/matrix-org/synapse/issues/6891))
|
||||
- Update Synapse's documentation to warn about the deprecation of ACME v1. ([\#6905](https://github.com/matrix-org/synapse/issues/6905), [\#6907](https://github.com/matrix-org/synapse/issues/6907), [\#6909](https://github.com/matrix-org/synapse/issues/6909))
|
||||
- Add documentation for the spam checker. ([\#6906](https://github.com/matrix-org/synapse/issues/6906))
|
||||
- Fix worker docs to point `/publicised_groups` API correctly. ([\#6938](https://github.com/matrix-org/synapse/issues/6938))
|
||||
- Clean up and update docs on setting up federation. ([\#6940](https://github.com/matrix-org/synapse/issues/6940))
|
||||
- Add a warning about indentation to generated configuration files. ([\#6920](https://github.com/matrix-org/synapse/issues/6920))
|
||||
- Databases created using the compose file in contrib/docker will now always have correct encoding and locale settings. Contributed by Fridtjof Mund. ([\#6921](https://github.com/matrix-org/synapse/issues/6921))
|
||||
- Update pip install directions in readme to avoid error when using zsh. ([\#6855](https://github.com/matrix-org/synapse/issues/6855))
|
||||
|
||||
|
||||
Deprecations and Removals
|
||||
-------------------------
|
||||
|
||||
- Remove `m.lazy_load_members` from `unstable_features` since lazy loading is in the stable Client-Server API version r0.5.0. ([\#6877](https://github.com/matrix-org/synapse/issues/6877))
|
||||
|
||||
|
||||
Internal Changes
|
||||
----------------
|
||||
|
||||
- Add type hints to `SyncHandler`. ([\#6821](https://github.com/matrix-org/synapse/issues/6821))
|
||||
- Refactoring work in preparation for changing the event redaction algorithm. ([\#6823](https://github.com/matrix-org/synapse/issues/6823), [\#6827](https://github.com/matrix-org/synapse/issues/6827), [\#6854](https://github.com/matrix-org/synapse/issues/6854), [\#6856](https://github.com/matrix-org/synapse/issues/6856), [\#6857](https://github.com/matrix-org/synapse/issues/6857), [\#6858](https://github.com/matrix-org/synapse/issues/6858))
|
||||
- Fix stacktraces when using `ObservableDeferred` and async/await. ([\#6836](https://github.com/matrix-org/synapse/issues/6836))
|
||||
- Port much of `synapse.handlers.federation` to async/await. ([\#6837](https://github.com/matrix-org/synapse/issues/6837), [\#6840](https://github.com/matrix-org/synapse/issues/6840))
|
||||
- Populate `rooms.room_version` database column at startup, rather than in a background update. ([\#6847](https://github.com/matrix-org/synapse/issues/6847))
|
||||
- Reduce amount we log at `INFO` level. ([\#6833](https://github.com/matrix-org/synapse/issues/6833), [\#6862](https://github.com/matrix-org/synapse/issues/6862))
|
||||
- Remove unused `get_room_stats_state` method. ([\#6869](https://github.com/matrix-org/synapse/issues/6869))
|
||||
- Add typing to `synapse.federation.sender` and port to async/await. ([\#6871](https://github.com/matrix-org/synapse/issues/6871))
|
||||
- Refactor `_EventInternalMetadata` object to improve type safety. ([\#6872](https://github.com/matrix-org/synapse/issues/6872))
|
||||
- Add an additional entry to the SyTest blacklist for worker mode. ([\#6883](https://github.com/matrix-org/synapse/issues/6883))
|
||||
- Fix the use of sed in the linting scripts when using BSD sed. ([\#6887](https://github.com/matrix-org/synapse/issues/6887))
|
||||
- Add type hints to the spam checker module. ([\#6915](https://github.com/matrix-org/synapse/issues/6915))
|
||||
- Convert the directory handler tests to use HomeserverTestCase. ([\#6919](https://github.com/matrix-org/synapse/issues/6919))
|
||||
- Increase DB/CPU perf of `_is_server_still_joined` check. ([\#6936](https://github.com/matrix-org/synapse/issues/6936))
|
||||
- Tiny optimisation for incoming HTTP request dispatch. ([\#6950](https://github.com/matrix-org/synapse/issues/6950))
|
||||
|
||||
|
||||
Synapse 1.10.1 (2020-02-17)
|
||||
===========================
|
||||
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
Admin API to add or modify threepids of user accounts.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fixed third party event rules function `on_create_room`'s return value being ignored.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add type hints to `SyncHandler`.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Refactoring work in preparation for changing the event redaction algorithm.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Allow URL-encoded User IDs on `/_synapse/admin/v2/users/<user_id>[/admin]` endpoints. Thanks to @NHAS for reporting.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Refactoring work in preparation for changing the event redaction algorithm.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Reducing log level to DEBUG for synapse.storage.TIME.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Change the default power levels of invites, tombstones and server ACLs for new rooms.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix stacktraces when using `ObservableDeferred` and async/await.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Port much of `synapse.handlers.federation` to async/await.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Port much of `synapse.handlers.federation` to async/await.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix an issue with cross-signing where device signatures were not sent to remote servers.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add details of PR merge strategy to contributing docs.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Populate `rooms.room_version` database column at startup, rather than in a background update.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix Synapse refusing to start if `federation_certificate_verification_whitelist` option is blank.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Refactoring work in preparation for changing the event redaction algorithm.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update pip install directiosn in readme to avoid error when using zsh.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Refactoring work in preparation for changing the event redaction algorithm.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Refactoring work in preparation for changing the event redaction algorithm.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Refactoring work in preparation for changing the event redaction algorithm.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Reduce amount we log at `INFO` level.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Limit the number of events that can be requested by the backfill federation API to 100.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add ability to run some group APIs on workers.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove unused `get_room_stats_state` method.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add typing to `synapse.federation.sender` and port to async/await.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Refactor _EventInternalMetadata object to improve type safety.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add ability to route federation user device queries to workers.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Remove `m.lazy_load_members` from `unstable_features` since lazy loading is in the stable Client-Server API version r0.5.0.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Reject device display names over 100 characters in length.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add an additional entry to the SyTest blacklist for worker mode.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix the use of sed in the linting scripts when using BSD sed.
|
||||
|
|
@ -1 +0,0 @@
|
|||
The result of a user directory search can now be filtered via the spam checker.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Spell out that the last event sent to a room won't be deleted by a purge.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Return a 404 instead of 200 for querying information of a non-existant user through the admin API.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Stop sending alias events during adding / removing aliases. Check alt_aliases in the latest canonical aliases event when deleting an alias.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update Synapse's documentation to warn about the deprecation of ACME v1.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add documentation for the spam checker.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update Synapse's documentation to warn about the deprecation of ACME v1.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Update Synapse's documentation to warn about the deprecation of ACME v1.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add type hints to the spam checker module.
|
||||
|
|
@ -1 +0,0 @@
|
|||
The deprecated "generate-config-on-the-fly" mode is no longer supported.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Convert the directory handler tests to use HomeserverTestCase.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Add a warning about indentation to generated configuration files.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Databases created using the compose file in contrib/docker will now always have correct encoding and locale settings. Contributed by Fridtjof Mund.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Increase DB/CPU perf of `_is_server_still_joined` check.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Increase perf of `get_auth_chain_ids` used in state res v2.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix worker docs to point `/publicised_groups` API correctly.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Implement `GET /_matrix/client/r0/rooms/{roomId}/aliases` endpoint as per [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432).
|
||||
|
|
@ -1 +0,0 @@
|
|||
Clean up and update docs on setting up federation.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Fix errors from logging in the purge jobs related to the message retention policies support.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Increase perf of `get_auth_chain_ids` used in state res v2.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Implement `GET /_matrix/client/r0/rooms/{roomId}/aliases` endpoint as per [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432).
|
||||
|
|
@ -1 +0,0 @@
|
|||
Implement `GET /_matrix/client/r0/rooms/{roomId}/aliases` endpoint as per [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432).
|
||||
|
|
@ -1 +0,0 @@
|
|||
Tiny optimisation for incoming HTTP request dispatch.
|
||||
|
|
@ -1 +0,0 @@
|
|||
Revert #6937.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Improve perf of v2 state res for large rooms.
|
||||
|
|
@ -0,0 +1 @@
|
|||
Reduce time spent doing GC by freezing objects on startup.
|
||||
|
|
@ -36,7 +36,7 @@ try:
|
|||
except ImportError:
|
||||
pass
|
||||
|
||||
__version__ = "1.10.1"
|
||||
__version__ = "1.11.0rc1"
|
||||
|
||||
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
|
||||
# We import here so that we don't have to install a bunch of deps when
|
||||
|
|
|
|||
|
|
@ -279,6 +279,15 @@ def start(hs, listeners=None):
|
|||
|
||||
setup_sentry(hs)
|
||||
setup_sdnotify(hs)
|
||||
|
||||
# We now freeze all allocated objects in the hopes that (almost)
|
||||
# everything currently allocated are things that will be used for the
|
||||
# rest of time. Doing so means less work each GC (hopefully).
|
||||
#
|
||||
# This only works on Python 3.7
|
||||
if sys.version_info >= (3, 7):
|
||||
gc.collect()
|
||||
gc.freeze()
|
||||
except Exception:
|
||||
traceback.print_exc(file=sys.stderr)
|
||||
reactor = hs.get_reactor()
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@
|
|||
|
||||
import logging
|
||||
from collections import namedtuple
|
||||
from typing import Dict, Iterable, List, Optional
|
||||
from typing import Dict, Iterable, List, Optional, Set
|
||||
|
||||
from six import iteritems, itervalues
|
||||
|
||||
|
|
@ -662,7 +662,7 @@ class StateResolutionStore(object):
|
|||
allow_rejected=allow_rejected,
|
||||
)
|
||||
|
||||
def get_auth_chain(self, event_ids):
|
||||
def get_auth_chain(self, event_ids: List[str], ignore_events: Set[str]):
|
||||
"""Gets the full auth chain for a set of events (including rejected
|
||||
events).
|
||||
|
||||
|
|
@ -674,11 +674,16 @@ class StateResolutionStore(object):
|
|||
presence of rejected events
|
||||
|
||||
Args:
|
||||
event_ids (list): The event IDs of the events to fetch the auth
|
||||
chain for. Must be state events.
|
||||
event_ids: The event IDs of the events to fetch the auth chain for.
|
||||
Must be state events.
|
||||
ignore_events: Set of events to exclude from the returned auth
|
||||
chain.
|
||||
|
||||
|
||||
Returns:
|
||||
Deferred[list[str]]: List of event IDs of the auth chain.
|
||||
"""
|
||||
|
||||
return self.store.get_auth_chain_ids(event_ids, include_given=True)
|
||||
return self.store.get_auth_chain_ids(
|
||||
event_ids, include_given=True, ignore_events=ignore_events,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -248,7 +248,7 @@ def _get_auth_chain_difference(state_sets, event_map, state_res_store):
|
|||
and eid not in common
|
||||
)
|
||||
|
||||
auth_chain = yield state_res_store.get_auth_chain(auth_ids)
|
||||
auth_chain = yield state_res_store.get_auth_chain(auth_ids, common)
|
||||
auth_ids.update(auth_chain)
|
||||
|
||||
auth_sets.append(auth_ids)
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@
|
|||
# limitations under the License.
|
||||
import itertools
|
||||
import logging
|
||||
from typing import List, Optional, Set
|
||||
|
||||
from six.moves import range
|
||||
from six.moves.queue import Empty, PriorityQueue
|
||||
|
|
@ -46,21 +47,37 @@ class EventFederationWorkerStore(EventsWorkerStore, SignatureWorkerStore, SQLBas
|
|||
event_ids, include_given=include_given
|
||||
).addCallback(self.get_events_as_list)
|
||||
|
||||
def get_auth_chain_ids(self, event_ids, include_given=False):
|
||||
def get_auth_chain_ids(
|
||||
self,
|
||||
event_ids: List[str],
|
||||
include_given: bool = False,
|
||||
ignore_events: Optional[Set[str]] = None,
|
||||
):
|
||||
"""Get auth events for given event_ids. The events *must* be state events.
|
||||
|
||||
Args:
|
||||
event_ids (list): state events
|
||||
include_given (bool): include the given events in result
|
||||
event_ids: state events
|
||||
include_given: include the given events in result
|
||||
ignore_events: Set of events to exclude from the returned auth
|
||||
chain. This is useful if the caller will just discard the
|
||||
given events anyway, and saves us from figuring out their auth
|
||||
chains if not required.
|
||||
|
||||
Returns:
|
||||
list of event_ids
|
||||
"""
|
||||
return self.db.runInteraction(
|
||||
"get_auth_chain_ids", self._get_auth_chain_ids_txn, event_ids, include_given
|
||||
"get_auth_chain_ids",
|
||||
self._get_auth_chain_ids_txn,
|
||||
event_ids,
|
||||
include_given,
|
||||
ignore_events,
|
||||
)
|
||||
|
||||
def _get_auth_chain_ids_txn(self, txn, event_ids, include_given):
|
||||
def _get_auth_chain_ids_txn(self, txn, event_ids, include_given, ignore_events):
|
||||
if ignore_events is None:
|
||||
ignore_events = set()
|
||||
|
||||
if include_given:
|
||||
results = set(event_ids)
|
||||
else:
|
||||
|
|
@ -80,6 +97,7 @@ class EventFederationWorkerStore(EventsWorkerStore, SignatureWorkerStore, SQLBas
|
|||
txn.execute(base_sql + clause, list(args))
|
||||
new_front.update([r[0] for r in txn])
|
||||
|
||||
new_front -= ignore_events
|
||||
new_front -= results
|
||||
|
||||
front = new_front
|
||||
|
|
|
|||
|
|
@ -603,7 +603,7 @@ class TestStateResolutionStore(object):
|
|||
|
||||
return {eid: self.event_map[eid] for eid in event_ids if eid in self.event_map}
|
||||
|
||||
def get_auth_chain(self, event_ids):
|
||||
def get_auth_chain(self, event_ids, ignore_events):
|
||||
"""Gets the full auth chain for a set of events (including rejected
|
||||
events).
|
||||
|
||||
|
|
@ -617,6 +617,8 @@ class TestStateResolutionStore(object):
|
|||
Args:
|
||||
event_ids (list): The event IDs of the events to fetch the auth
|
||||
chain for. Must be state events.
|
||||
ignore_events: Set of events to exclude from the returned auth
|
||||
chain.
|
||||
|
||||
Returns:
|
||||
Deferred[list[str]]: List of event IDs of the auth chain.
|
||||
|
|
@ -627,7 +629,7 @@ class TestStateResolutionStore(object):
|
|||
stack = list(event_ids)
|
||||
while stack:
|
||||
event_id = stack.pop()
|
||||
if event_id in result:
|
||||
if event_id in result or event_id in ignore_events:
|
||||
continue
|
||||
|
||||
result.add(event_id)
|
||||
|
|
|
|||
Loading…
Reference in New Issue