OpenCloud
/
MatrixSynapse
mirror of https://github.com/matrix-org/synapse
1
0
Fork 0
Code Issues Releases Wiki Activity
10,575 Commits
842 Branches
671 Tags
284 MiB
Commit Graph

7718 Commits (d1d54d60889f94228adff55f64116aa66d07ac3a)

Author SHA1 Message Date
Neil Johnson d4b2e05852 Merge branch 'master' of https://github.com/matrix-org/synapse into develop 2018-04-13 12:16:27 +01:00
Neil Johnson 51d628d28d Bump version and Change log 2018-04-13 12:08:19 +01:00
Richard van der Hoff d3347ad485 Revert "Use sortedcontainers instead of blist" 
This reverts commit 9fbe70a7dc.

It turns out that sortedcontainers.SortedDict is not an exact match for
blist.sorteddict; in particular, `popitem()` removes things from the opposite
end of the dict.

This is trivial to fix, but I want to add some unit tests, and potentially some
more thought about it, before we do so.
 2018-04-13 11:16:43 +01:00
Richard van der Hoff fac3f9e678 Bump canonicaljson to 1.1.3 
1.1.2 was a bit broken too :/
 2018-04-13 10:21:38 +01:00
Richard van der Hoff 60f6014bb7 ResponseCache: fix handling of completed results 
Turns out that ObservableDeferred.observe doesn't return a deferred if the
result is already completed. Fix handling and improve documentation.
 2018-04-13 07:32:29 +01:00
Richard van der Hoff 119596ab8f Update canonicaljson dependency 
1.1.0 and 1.1.1 were broken, so we're updating this to help people make sure
they don't end up on a broken version.

Also, 1.1.0 is speedier...
 2018-04-12 17:31:44 +01:00
Richard van der Hoff b78395b7fe Refactor ResponseCache usage 
Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a
(get, set) pair, and then use it throughout the codebase.

This will be largely non-functional, but does include the following functional
changes:

* federation_server.on_context_state_request: drops use of _server_linearizer
  which looked redundant and could cause incorrect cache misses by yielding
  between the get and the set.
* RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks
* the wrap function includes some logging. I'm hoping this won't be too noisy
  on production.
 2018-04-12 13:02:15 +01:00
Richard van der Hoff d5c74b9f6c
Merge pull request #3092 from matrix-org/rav/response_cache_metrics 
Add metrics for ResponseCache
 2018-04-12 12:59:36 +01:00
Erik Johnston 415aeefd89 Format docstring 2018-04-12 12:07:09 +01:00
Erik Johnston 19ceb4851f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/processed_event_lag 2018-04-12 11:36:07 +01:00
Richard van der Hoff 261124396e
Merge pull request #3059 from matrix-org/rav/doc_response_cache 
Document the behaviour of ResponseCache
 2018-04-12 11:22:30 +01:00
Erik Johnston 23a7f9d7f4 Doc we raise on unknown event 2018-04-12 11:20:51 +01:00
Erik Johnston d7bf3a68f0 s/list/tuple 2018-04-12 11:19:04 +01:00
Erik Johnston f67e906e18 Set all metrics at the same time 2018-04-12 11:18:19 +01:00
Erik Johnston 971059a733
Merge pull request #3088 from matrix-org/erikj/as_parallel 
Send events to ASes concurrently
 2018-04-12 10:42:36 +01:00
Erik Johnston 4dae4a97ed Track last processed event received_ts 2018-04-11 14:27:09 +01:00
Erik Johnston 92e34615c5 Track where event stream processing have gotten up to 2018-04-11 12:13:40 +01:00
Erik Johnston ab825aa328 Add GaugeMetric 2018-04-11 12:13:40 +01:00
Richard van der Hoff 233699c42e
Merge pull request #2760 from Valodim/pypy 
Synapse on PyPy
 2018-04-11 11:20:01 +01:00
Neil Johnson 781cd8c54f bump version/changelog 2018-04-11 10:54:43 +01:00
Neil Johnson 9ef0b179e0 Merge commit '11d2609da70af797405241cdf7d9df19db5628f2' of https://github.com/matrix-org/synapse into release-v0.27.0 2018-04-11 10:51:59 +01:00
Erik Johnston 121591568b Send events to ASes concurrently 2018-04-11 09:56:00 +01:00
Richard van der Hoff b3384232a0 Add metrics for ResponseCache 2018-04-10 23:14:47 +01:00
Neil Johnson d54cfbb7a8 fix typo 2018-04-10 17:38:16 +01:00
Erik Johnston eaa2ebf20b
Merge pull request #3079 from matrix-org/erikj/limit_concurrent_sends 
Limit concurrent event sends for a room
 2018-04-10 16:43:58 +01:00
Erik Johnston 9daf82278f
Merge pull request #3078 from matrix-org/erikj/federation_sender 
Send federation events concurrently
 2018-04-10 16:43:48 +01:00
Neil Johnson dd723267b2 Merge branch 'release-v0.27.0' of https://github.com/matrix-org/synapse into develop 2018-04-10 15:03:29 +01:00
Erik Johnston a060dfa132 Use run_in_background instead 2018-04-10 14:25:11 +01:00
Erik Johnston f8e8ec013b Note why we're limiting concurrent event sends 2018-04-10 14:00:46 +01:00
Erik Johnston 1246d23710 Preserve log contexts correctly 2018-04-10 12:04:32 +01:00
Erik Johnston d49cbf712f Log event ID on exception 2018-04-10 12:03:41 +01:00
Erik Johnston ce72d590ed
Merge pull request #3082 from matrix-org/erikj/urlencode_paths 
URL quote path segments over federation
 2018-04-10 11:31:16 +01:00
Erik Johnston 11d2609da7 Ensure slashes are escaped 2018-04-10 11:24:40 +01:00
Erik Johnston dab87b84a3 URL quote path segments over federation 2018-04-10 11:16:08 +01:00
Vincent Breitmoser 6d7f0f8dd3 Don't disable GC when running on PyPy 
PyPy's incminimark GC can't be triggered manually. From what I observed
there are no obvious issues with just letting it run normally. And
unlike CPython, it actually returns unused RAM to the system.

Signed-off-by: Vincent Breitmoser <look@my.amazin.horse>
 2018-04-10 11:35:34 +02:00
Vincent Breitmoser f4284d943a In DomainSpecificString, override __repr__ in addition to __str__ 
For some reason, string interpolation on a DomainSpecificString object
like "%r" % (domainSpecificStringObj) fails under PyPy, because the
default __repr__ implementation wants to iterate over the object. I'm
not sure why that happens, but overriding __repr__ instead of __str__
fixes this problem, and is arguably the more appropriate thing to do
anyways.
 2018-04-10 11:35:29 +02:00
Richard van der Hoff d1e56cfcd1 Fix pep8 error on psycopg2cffi hack 2018-04-10 11:35:29 +02:00
Vincent Breitmoser 89de934981 Use psycopg2cffi module instead of psycopg2 if running on pypy 
The psycopg2 package isn't available for PyPy.  This commit adds a check
if the runtime is PyPy, and if it is uses psycopg2cffi module in favor
of psycopg2. This is almost a drop-in replacement, except for one place
where an additional cast to string is required.
 2018-04-10 11:29:52 +02:00
Vincent Breitmoser 9fbe70a7dc Use sortedcontainers instead of blist 
This commit drop-in replaces blist with SortedContainers. They are
written in pure python so work with pypy, but perform as good as
native implementations, at least in a couple benchmarks:

http://www.grantjenks.com/docs/sortedcontainers/performance.html
 2018-04-10 11:29:51 +02:00
Richard van der Hoff a3599dda97
Merge pull request #2996 from krombel/allow_auto_join_rooms 
move handling of auto_join_rooms to RegisterHandler
 2018-04-10 01:11:00 +01:00
Richard van der Hoff 87478c5a60
Merge pull request #3061 from NotAFile/add-some-byte-strings 
Add b prefixes to some strings that are bytes in py3
 2018-04-09 23:54:05 +01:00
Richard van der Hoff c508b2f2f0
Merge pull request #3073 from NotAFile/use-six-reraise 
Replace old-style raise with six.reraise
 2018-04-09 23:53:40 +01:00
Richard van der Hoff 37354b55c9
Merge pull request #2938 from dklug/develop 
Return 401 for invalid access_token on logout
 2018-04-09 23:52:56 +01:00
Richard van der Hoff 0e9aa1d091
Merge pull request #3074 from NotAFile/fix-py3-prints 
use python3-compatible prints
 2018-04-09 23:44:41 +01:00
Richard van der Hoff 8eaa141d8f
Merge pull request #3075 from NotAFile/six-type-checks 
Replace some type checks with six type checks
 2018-04-09 23:40:44 +01:00
Richard van der Hoff 664adb4236
Merge pull request #3016 from silkeh/improve-service-lookups 
Improve handling of SRV records for federation connections
 2018-04-09 23:40:06 +01:00
Neil Johnson 41e0611895 remove errant print 2018-04-09 18:44:20 +01:00
Neil Johnson 61b439c904 Fix msec to sec, again 2018-04-09 18:43:48 +01:00
Neil Johnson 87770300d5 Fix msec to sec 2018-04-09 18:38:59 +01:00
Neil Johnson 64bc2162ef Fix psycopg2 interpolation 2018-04-09 17:50:36 +01:00
Richard van der Hoff d2c6f4d626
Merge pull request #3080 from matrix-org/rav/fix_500_on_rejoin 
Return a 404 rather than a 500 on rejoining empty rooms
 2018-04-09 17:32:36 +01:00
Neil Johnson 5232d3bfb1 version bump v0.27.3-rc2 2018-04-09 17:25:57 +01:00
Neil Johnson 5e785d4d5b Merge branch 'develop' of https://github.com/matrix-org/synapse into release-v0.27.0 2018-04-09 17:21:34 +01:00
Erik Johnston 6e025a97b4 Handle all events in a room correctly 2018-04-09 16:02:48 +01:00
Neil Johnson d9ae2bc826 bump version to release candidate 2018-04-09 16:00:31 +01:00
Neil Johnson 072fb59446 bump version 2018-04-09 13:49:25 +01:00
Richard van der Hoff 13decdbf96 Revert "Merge pull request #3066 from matrix-org/rav/remove_redundant_metrics" 
We aren't ready to release this yet, so I'm reverting it for now.

This reverts commit d1679a4ed7, reversing
changes made to e089100c62.
 2018-04-09 12:59:12 +01:00
Richard van der Hoff f3ef60662f Return a 404 rather than a 500 on rejoining empty rooms 
Filter ourselves out of the server list before checking for an empty remote
host list, to fix 500 error

Fixes #2141
 2018-04-09 12:56:22 +01:00
Erik Johnston e5082494eb Limit concurrent event sends for a room 2018-04-09 12:07:39 +01:00
Erik Johnston 56b0589865 Use create_and_send_nonmember_event everywhere 2018-04-09 12:04:18 +01:00
Erik Johnston 11974f3787 Send federation events concurrently 2018-04-09 11:47:10 +01:00
Erik Johnston 145d14656b Handle exceptions in get_hosts_for_room when sending events over federation 2018-04-09 11:47:01 +01:00
Adrian Tschira e54c202b81 Replace some type checks with six type checks 
Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-04-07 01:02:32 +02:00
Adrian Tschira b0500d3774 use python3-compatible prints 2018-04-06 23:35:27 +02:00
Adrian Tschira 4f40d058cc Replace old-style raise with six.reraise 
The old style raise is invalid syntax in python3. As noted in the docs,
this adds one more frame in the traceback, but I think this is
acceptable:

    <ipython-input-7-bcc5cba3de3f> in <module>()
         16     except:
         17         pass
    ---> 18     six.reraise(*x)

    /usr/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
        691             if value.__traceback__ is not tb:
        692                 raise value.with_traceback(tb)
    --> 693             raise value
        694         finally:
        695             value = None

    <ipython-input-7-bcc5cba3de3f> in <module>()
          9
         10 try:
    ---> 11     x()
         12 except:
         13     x = sys.exc_info()

Also note that this uses six, which is not formally a dependency yet,
but is included indirectly since most packages depend on it.

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-04-06 23:06:24 +02:00
Luke Barnard 135fc5b9cd
Merge pull request #3046 from matrix-org/dbkr/join_group 
Implement group join API
 2018-04-06 16:24:32 +01:00
Luke Barnard 020a501354 de-lint, quote consistency 2018-04-06 16:02:06 +01:00
Luke Barnard db2fd801f7 Explicitly grab individual columns from group object 2018-04-06 15:57:25 +01:00
Luke Barnard 7945435587 When exposing group state, return is_openly_joinable 
as opposed to join_policy, which is really only pertinent to the
synapse implementation of the group server.

By doing this we keep the group server concept extensible by
allowing arbitrarily complex rules for deciding whether a group
is openly joinable.
 2018-04-06 15:43:27 +01:00
Luke Barnard 6bd1b7053e By default, join policy is "invite" 2018-04-06 15:43:27 +01:00
Luke Barnard b4478e586f add_user -> _add_user 2018-04-06 15:43:27 +01:00
Luke Barnard 112c2253e2 pep8 2018-04-06 15:43:27 +01:00
Luke Barnard 6850f8aea3 Get group_info from existing call to check_group_is_ours 2018-04-06 15:43:27 +01:00
Luke Barnard cd087a265d Don't use redundant inlineCallbacks 2018-04-06 15:43:27 +01:00
Luke Barnard 87c864b698 join_rule -> join_policy 2018-04-06 15:43:27 +01:00
Luke Barnard ae85c7804e is_joinable -> join_rule 2018-04-06 15:43:27 +01:00
Luke Barnard f8d1917fce Fix federation client `set_group_joinable` typo 2018-04-06 15:43:27 +01:00
Luke Barnard 6eb3aa94b6 Factor out add_user from accept_invite and join_group 2018-04-06 15:43:27 +01:00
David Baker edb45aae38 pep8 2018-04-06 15:43:27 +01:00
David Baker b370fe61c0 Implement group join API 2018-04-06 15:43:27 +01:00
Erik Johnston 7b824f1475 Add response size metrics 2018-04-06 13:20:11 +01:00
Krombel 1d71f484d4 use PUT instead of POST for federating groups/m.join_policy 2018-04-06 12:54:09 +02:00
Richard van der Hoff 15e8ed874f more verbosity in synctl 2018-04-06 09:28:36 +01:00
Richard van der Hoff c7f0969731
Merge pull request #2986 from jplatte/join_reponse_room_id 
Add room_id to the response of `rooms/{roomId}/join`
 2018-04-05 17:29:06 +01:00
Richard van der Hoff 3449da3bc7
Merge pull request #3068 from matrix-org/rav/fix_cache_invalidation 
Improve database cache performance
 2018-04-05 17:21:44 +01:00
Richard van der Hoff d1679a4ed7
Merge pull request #3066 from matrix-org/rav/remove_redundant_metrics 
Remove redundant metrics which were deprecated in 0.27.0.
 2018-04-05 17:21:18 +01:00
Richard van der Hoff 01afc563c3 Fix overzealous cache invalidation 
Fixes an issue where a cache invalidation would invalidate *all* pending
entries, rather than just the entry that we intended to invalidate.
 2018-04-05 16:24:04 +01:00
Luke Barnard e089100c62
Merge pull request #3045 from matrix-org/dbkr/group_joinable 
Add joinability for groups
 2018-04-05 15:57:49 +01:00
Neil Johnson 68b0ee4e8d
Merge pull request #3041 from matrix-org/r30_stats 
R30 stats
 2018-04-05 15:37:37 +01:00
Richard van der Hoff 22284a6f65
Merge pull request #3060 from matrix-org/rav/kill_event_content 
Remove uses of events.content
 2018-04-05 15:02:17 +01:00
Luke Barnard 917380e89d NON NULL -> NOT NULL 2018-04-05 14:32:12 +01:00
Luke Barnard 104c0bc1d5 Use "/settings/" (plural) 2018-04-05 14:07:16 +01:00
Luke Barnard 700e5e7198 Use DEFAULT join_policy of "invite" in db 2018-04-05 14:01:17 +01:00
Luke Barnard b214a04ffc Document set_group_join_policy 2018-04-05 13:29:16 +01:00
Neil Johnson 0e5f479fc0 Review comments 
Use iteritems over item to loop over dict
formatting
 2018-04-05 12:16:46 +01:00
Richard van der Hoff 518f6de088 Remove redundant metrics which were deprecated in 0.27.0. 2018-04-04 19:46:28 +01:00
Jan Christian Grünhage e4570c53dd phone home cache size configurations 2018-04-04 16:46:58 +01:00
Adrian Tschira 6168351877 Add b prefixes to some strings that are bytes in py3 
This has no effect on python2

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-04-04 13:48:51 +02:00
Richard van der Hoff f92963f5db
Revert "improve mxid check performance" 2018-04-04 12:08:29 +01:00
Silke 72251d1b97 Remove address resolution of hosts in SRV records 
Signed-off-by: Silke Hofstra <silke@slxh.eu>
 2018-04-04 12:26:50 +02:00