OpenCloud
/
MatrixSynapse
mirror of https://github.com/matrix-org/synapse
1
0
Fork 0
Code Issues Releases Wiki Activity

Merge remote-tracking branch 'origin/develop' into erikj/destination_retry_cache

pull/3933/head
Richard van der Hoff 2018-09-25 12:03:54 +01:00
parent 19dc676d1a 4c3e7eeec5
commit a1cd37390f
51 changed files with 128 additions and 62 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

74
CHANGES.md
View File

@ -1,3 +1,77 @@
Synapse 0.33.5.1 (2018-09-25)
=============================
Internal Changes
----------------
- Fix incompatibility with older Twisted version in tests. Thanks
@OlegGirko! ([\#3940](https://github.com/matrix-org/synapse/issues/3940))
Synapse 0.33.5 (2018-09-24)
===========================
No significant changes.
Synapse 0.33.5rc1 (2018-09-17)
==============================
Features
--------
- Python 3.5 and 3.6 support is now in beta. ([\#3576](https://github.com/matrix-org/synapse/issues/3576))
- Implement `event_format` filter param in `/sync` ([\#3790](https://github.com/matrix-org/synapse/issues/3790))
- Add synapse_admin_mau:registered_reserved_users metric to expose number of real reaserved users ([\#3846](https://github.com/matrix-org/synapse/issues/3846))
Bugfixes
--------
- Remove connection ID for replication prometheus metrics, as it creates a large number of new series. ([\#3788](https://github.com/matrix-org/synapse/issues/3788))
- guest users should not be part of mau total ([\#3800](https://github.com/matrix-org/synapse/issues/3800))
- Bump dependency on pyopenssl 16.x, to avoid incompatibility with recent Twisted. ([\#3804](https://github.com/matrix-org/synapse/issues/3804))
- Fix existing room tags not coming down sync when joining a room ([\#3810](https://github.com/matrix-org/synapse/issues/3810))
- Fix jwt import check ([\#3824](https://github.com/matrix-org/synapse/issues/3824))
- fix VOIP crashes under Python 3 (#3821) ([\#3835](https://github.com/matrix-org/synapse/issues/3835))
- Fix manhole so that it works with latest openssh clients ([\#3841](https://github.com/matrix-org/synapse/issues/3841))
- Fix outbound requests occasionally wedging, which can result in federation breaking between servers. ([\#3845](https://github.com/matrix-org/synapse/issues/3845))
- Show heroes if room name/canonical alias has been deleted ([\#3851](https://github.com/matrix-org/synapse/issues/3851))
- Fix handling of redacted events from federation ([\#3859](https://github.com/matrix-org/synapse/issues/3859))
- ([\#3874](https://github.com/matrix-org/synapse/issues/3874))
- Mitigate outbound federation randomly becoming wedged ([\#3875](https://github.com/matrix-org/synapse/issues/3875))
Internal Changes
----------------
- CircleCI tests now run on the potential merge of a PR. ([\#3704](https://github.com/matrix-org/synapse/issues/3704))
- http/ is now ported to Python 3. ([\#3771](https://github.com/matrix-org/synapse/issues/3771))
- Improve human readable error messages for threepid registration/account update ([\#3789](https://github.com/matrix-org/synapse/issues/3789))
- Make /sync slightly faster by avoiding needless copies ([\#3795](https://github.com/matrix-org/synapse/issues/3795))
- handlers/ is now ported to Python 3. ([\#3803](https://github.com/matrix-org/synapse/issues/3803))
- Limit the number of PDUs/EDUs per federation transaction ([\#3805](https://github.com/matrix-org/synapse/issues/3805))
- Only start postgres instance for postgres tests on Travis CI ([\#3806](https://github.com/matrix-org/synapse/issues/3806))
- tests/ is now ported to Python 3. ([\#3808](https://github.com/matrix-org/synapse/issues/3808))
- crypto/ is now ported to Python 3. ([\#3822](https://github.com/matrix-org/synapse/issues/3822))
- rest/ is now ported to Python 3. ([\#3823](https://github.com/matrix-org/synapse/issues/3823))
- add some logging for the keyring queue ([\#3826](https://github.com/matrix-org/synapse/issues/3826))
- speed up lazy loading by 2-3x ([\#3827](https://github.com/matrix-org/synapse/issues/3827))
- Improved Dockerfile to remove build requirements after building reducing the image size. ([\#3834](https://github.com/matrix-org/synapse/issues/3834))
- Disable lazy loading for incremental syncs for now ([\#3840](https://github.com/matrix-org/synapse/issues/3840))
- federation/ is now ported to Python 3. ([\#3847](https://github.com/matrix-org/synapse/issues/3847))
- Log when we retry outbound requests ([\#3853](https://github.com/matrix-org/synapse/issues/3853))
- Removed some excess logging messages. ([\#3855](https://github.com/matrix-org/synapse/issues/3855))
- Speed up purge history for rooms that have been previously purged ([\#3856](https://github.com/matrix-org/synapse/issues/3856))
- Refactor some HTTP timeout code. ([\#3857](https://github.com/matrix-org/synapse/issues/3857))
- Fix running merged builds on CircleCI ([\#3858](https://github.com/matrix-org/synapse/issues/3858))
- Fix typo in replication stream exception. ([\#3860](https://github.com/matrix-org/synapse/issues/3860))
- Add in flight real time metrics for Measure blocks ([\#3871](https://github.com/matrix-org/synapse/issues/3871))
- Disable buffering and automatic retrying in treq requests to prevent timeouts. ([\#3872](https://github.com/matrix-org/synapse/issues/3872))
- mention jemalloc in the README ([\#3877](https://github.com/matrix-org/synapse/issues/3877))
- Remove unmaintained "nuke-room-from-db.sh" script ([\#3888](https://github.com/matrix-org/synapse/issues/3888))
Synapse 0.33.4 (2018-09-07)
===========================

1
changelog.d/3576.feature
View File

@ -1 +0,0 @@
Python 3.5+ is now supported.

1
changelog.d/3578.bugfix Normal file
View File

@ -0,0 +1 @@
Fix problem when playing media from Chrome using direct URL (thanks @remjey!)

1
changelog.d/3704.misc
View File

@ -1 +0,0 @@
CircleCI tests now run on the potential merge of a PR.

1
changelog.d/3771.misc
View File

@ -1 +0,0 @@
http/ is now ported to Python 3.

1
changelog.d/3788.bugfix
View File

@ -1 +0,0 @@
Remove connection ID for replication prometheus metrics, as it creates a large number of new series.

1
changelog.d/3789.misc
View File

@ -1 +0,0 @@
Improve human readable error messages for threepid registration/account update

1
changelog.d/3790.feature
View File

@ -1 +0,0 @@
Implement `event_format` filter param in `/sync`

1
changelog.d/3795.misc
View File

@ -1 +0,0 @@
Make /sync slightly faster by avoiding needless copies

1
changelog.d/3800.bugfix
View File

@ -1 +0,0 @@
guest users should not be part of mau total

1
changelog.d/3803.misc
View File

@ -1 +0,0 @@
handlers/ is now ported to Python 3.

1
changelog.d/3804.bugfix
View File

@ -1 +0,0 @@
Bump dependency on pyopenssl 16.x, to avoid incompatibility with recent Twisted.

1
changelog.d/3805.misc
View File

@ -1 +0,0 @@
Limit the number of PDUs/EDUs per federation transaction

1
changelog.d/3806.misc
View File

@ -1 +0,0 @@
Only start postgres instance for postgres tests on Travis CI

1
changelog.d/3808.misc
View File

@ -1 +0,0 @@
tests/ is now ported to Python 3.

1
changelog.d/3810.bugfix
View File

@ -1 +0,0 @@
Fix existing room tags not coming down sync when joining a room

1
changelog.d/3822.misc
View File

@ -1 +0,0 @@
crypto/ is now ported to Python 3.

1
changelog.d/3823.misc
View File

@ -1 +0,0 @@
rest/ is now ported to Python 3.

1
changelog.d/3824.bugfix
View File

@ -1 +0,0 @@
Fix jwt import check

1
changelog.d/3826.misc
View File

@ -1 +0,0 @@
add some logging for the keyring queue

1
changelog.d/3827.misc
View File

@ -1 +0,0 @@
speed up lazy loading by 2-3x

1
changelog.d/3834.misc
View File

@ -1 +0,0 @@
Improved Dockerfile to remove build requirements after building reducing the image size.

1
changelog.d/3835.bugfix
View File

@ -1 +0,0 @@
fix VOIP crashes under Python 3 (#3821)

1
changelog.d/3840.misc
View File

@ -1 +0,0 @@
Disable lazy loading for incremental syncs for now

1
changelog.d/3841.bugfix
View File

@ -1 +0,0 @@
Fix manhole so that it works with latest openssh clients

1
changelog.d/3845.bugfix
View File

@ -1 +0,0 @@
Fix outbound requests occasionally wedging, which can result in federation breaking between servers.

1
changelog.d/3846.feature
View File

@ -1 +0,0 @@
Add synapse_admin_mau:registered_reserved_users metric to expose number of real reaserved users

1
changelog.d/3847.misc
View File

@ -1 +0,0 @@
federation/ is now ported to Python 3.

1
changelog.d/3851.bugfix
View File

@ -1 +0,0 @@
Show heroes if room name/canonical alias has been deleted

1
changelog.d/3853.misc
View File

@ -1 +0,0 @@
Log when we retry outbound requests

1
changelog.d/3855.misc
View File

@ -1 +0,0 @@
Removed some excess logging messages.

1
changelog.d/3856.misc
View File

@ -1 +0,0 @@
Speed up purge history for rooms that have been previously purged

1
changelog.d/3857.misc
View File

@ -1 +0,0 @@
Refactor some HTTP timeout code.

1
changelog.d/3858.misc
View File

@ -1 +0,0 @@
Fix running merged builds on CircleCI

1
changelog.d/3859.bugfix
View File

@ -1 +0,0 @@
Fix handling of redacted events from federation

1
changelog.d/3860.misc
View File

@ -1 +0,0 @@
Fix typo in replication stream exception.

1
changelog.d/3871.misc
View File

@ -1 +0,0 @@
Add in flight real time metrics for Measure blocks

1
changelog.d/3872.misc
View File

@ -1 +0,0 @@
Disable buffering and automatic retrying in treq requests to prevent timeouts.

0
changelog.d/3874.bugfix
View File

1
changelog.d/3875.bugfix
View File

@ -1 +0,0 @@
Mitigate outbound federation randomly becoming wedged

1
changelog.d/3877.misc
View File

@ -1 +0,0 @@
mention jemalloc in the README

1
changelog.d/3888.misc
View File

@ -1 +0,0 @@
Remove unmaintained "nuke-room-from-db.sh" script

1
changelog.d/3925.misc Normal file
View File

@ -0,0 +1 @@
Fix spurious exceptions when remote http client closes conncetion

1
changelog.d/3936.bugfix Normal file
View File

@ -0,0 +1 @@
Fix out-of-bounds error when LLing yourself

1
changelog.d/3947.misc Normal file
View File

@ -0,0 +1 @@
Require attrs 16.0.0 or later

2
synapse/__init__.py
View File

@ -27,4 +27,4 @@ try:
except ImportError:
pass
__version__ = "0.33.4"
__version__ = "0.33.5.1"

13
synapse/handlers/sync.py
View File

@ -713,10 +713,6 @@ class SyncHandler(object):
)
]
# always make sure we LL ourselves so we know we're in the room
# (if we are), to fix https://github.com/vector-im/riot-web/issues/7209
types.append((EventTypes.Member, sync_config.user.to_string()))
# only apply the filtering to room members
filtered_types = [EventTypes.Member]
@ -726,6 +722,13 @@ class SyncHandler(object):
}
if full_state:
if lazy_load_members:
# always make sure we LL ourselves so we know we're in the room
# (if we are) to fix https://github.com/vector-im/riot-web/issues/7209
# We only need apply this on full state syncs given we disabled
# LL for incr syncs in #3840.
types.append((EventTypes.Member, sync_config.user.to_string()))
if batch:
current_state_ids = yield self.store.get_state_ids_for_event(
batch.events[-1].event_id, types=types,
@ -794,7 +797,7 @@ class SyncHandler(object):
else:
state_ids = {}
if lazy_load_members:
if types:
if types and batch.events:
# We're returning an incremental sync, with no
# "gap" since the previous sync, so normally there would be
# no state to return.

49
synapse/http/server.py
View File

@ -84,10 +84,21 @@ def wrap_json_request_handler(h):
logger.info(
"%s SynapseError: %s - %s", request, code, e.msg
)
respond_with_json(
request, code, e.error_dict(), send_cors=True,
pretty_print=_request_user_agent_is_curl(request),
)
# Only respond with an error response if we haven't already started
# writing, otherwise lets just kill the connection
if request.startedWriting:
if request.transport:
try:
request.transport.abortConnection()
except Exception:
# abortConnection throws if the connection is already closed
pass
else:
respond_with_json(
request, code, e.error_dict(), send_cors=True,
pretty_print=_request_user_agent_is_curl(request),
)
except Exception:
# failure.Failure() fishes the original Failure out
@ -100,16 +111,26 @@ def wrap_json_request_handler(h):
request,
f.getTraceback().rstrip(),
)
respond_with_json(
request,
500,
{
"error": "Internal server error",
"errcode": Codes.UNKNOWN,
},
send_cors=True,
pretty_print=_request_user_agent_is_curl(request),
)
# Only respond with an error response if we haven't already started
# writing, otherwise lets just kill the connection
if request.startedWriting:
if request.transport:
try:
request.transport.abortConnection()
except Exception:
# abortConnection throws if the connection is already closed
pass
else:
respond_with_json(
request,
500,
{
"error": "Internal server error",
"errcode": Codes.UNKNOWN,
},
send_cors=True,
pretty_print=_request_user_agent_is_curl(request),
)
return wrap_async_request_handler(wrapped_request_handler)

4
synapse/python_dependencies.py
View File

@ -58,7 +58,9 @@ REQUIREMENTS = {
"phonenumbers>=8.2.0": ["phonenumbers"],
"six": ["six"],
"prometheus_client": ["prometheus_client"],
"attrs": ["attr"],
# we use attr.s(slots), which arrived in 16.0.0
"attrs>=16.0.0": ["attr>=16.0.0"],
"netaddr>=0.7.18": ["netaddr"],
}

1
synapse/rest/media/v1/download_resource.py
View File

@ -52,6 +52,7 @@ class DownloadResource(Resource):
b" script-src 'none';"
b" plugin-types application/pdf;"
b" style-src 'unsafe-inline';"
b" media-src 'self';"
b" object-src 'self';"
)
server_name, media_id, name = parse_media_id(request)

4
tests/http/test_fedclient.py
View File

@ -54,7 +54,7 @@ class FederationClientTests(HomeserverTestCase):
def test_client_never_connect(self):
"""
If the HTTP request is not connected and is timed out, it'll give a
ConnectingCancelledError.
ConnectingCancelledError or TimeoutError.
"""
d = self.cl.get_json("testserv:8008", "foo/bar", timeout=10000)
@ -76,7 +76,7 @@ class FederationClientTests(HomeserverTestCase):
self.reactor.advance(10.5)
f = self.failureResultOf(d)
self.assertIsInstance(f.value, ConnectingCancelledError)
self.assertIsInstance(f.value, (ConnectingCancelledError, TimeoutError))
def test_client_connect_no_response(self):
"""