ef1c524bb3
Improve error msg when key-fetch fails ( #5896 )
...
There's no point doing a raise_from here, because the exception is always
logged at warn with no stacktrace in the caller. Instead, let's try to give
better messages to reduce confusion.
In particular, this means that we won't log 'Failed to connect to remote
server' when we don't even attempt to connect to the remote server due to
blacklisting.
2019-08-22 10:42:06 +01:00
814f253f1b
make isort happy
2019-08-21 13:22:15 -07:00
7c3abc6572
apply PR review suggestions
2019-08-21 13:19:35 -07:00
4dab867288
Drop some unused tables. ( #5893 )
...
These tables are never used, so we may as well drop them.
2019-08-21 13:16:28 +01:00
97cbc96093
Only sign when we respond to remote key requests
2019-08-21 11:21:58 +01:00
5906be8589
Add config option for keys to use to sign keys
...
This allows servers to separate keys that are used to sign remote keys
when acting as a notary server.
2019-08-21 10:44:58 +01:00
72bc285669
Refactor the Appservice scheduler code ( #5886 )
...
Get rid of the labyrinthine `recoverer_fn` code, and clean up the startup code
(it seemed to be previously inexplicably split between
`ApplicationServiceScheduler.start` and `_Recoverer.start`).
Add some docstrings too.
2019-08-20 17:42:45 +01:00
baa3f4a80d
Avoid deep recursion in appservice recovery ( #5885 )
...
Hopefully, this will fix a stack overflow when recovering an appservice.
The recursion here leads to a huge chain of deferred callbacks, which then
overflows the stack when the chain completes. `inlineCallbacks` makes a better
job of this if we use iteration instead.
Clean up the code a bit too, while we're there.
2019-08-20 17:39:38 +01:00
c886f976e0
Opentracing doc update ( #5776 )
...
Update opentracing docs to use the unified 'trace' method
2019-08-20 13:56:03 +01:00
74f016d343
Remove now unused pick_server_from_list
2019-08-20 12:37:08 +01:00
1f9df1cc7b
Fixup _sort_server_list to be slightly more efficient
...
Also document that we are using the algorithm described in RFC2782 and
ensure we handle zero weight correctly.
2019-08-20 12:36:11 +01:00
5019945828
Refactor the Appservice scheduler code
...
Get rid of the labyrinthine `recoverer_fn` code, and clean up the startup code
(it seemed to be previously inexplicably split between
`ApplicationServiceScheduler.start` and `_Recoverer.start`).
Add some docstrings too.
2019-08-20 11:50:23 +01:00
1dec31560e
Change jitter to be a factor rather than absolute value
2019-08-20 11:46:00 +01:00
bb29bc2937
Use MSC2197 on stable prefix as it has almost finished FCP
...
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2019-08-20 08:49:31 +01:00
d514dac0b2
Merge pull request #5860 from matrix-org/erikj/update_5704_comments
...
Remove logging for #5407 and update comments
2019-08-19 10:20:59 +01:00
bdd201ea7f
Merge branch 'master' into develop
2019-08-17 10:50:42 +01:00
74fb729213
1.3.1
2019-08-17 09:16:17 +01:00
412c6e21a8
Drop dependency on sdnotify ( #5871 )
...
... to save OSes which don't use it from having to maintain a port.
Fixes #5865 .
2019-08-17 09:09:52 +01:00
8a5f6ed130
Merge pull request #5857 from matrix-org/uhoreg/fix_e2e_room_keys_index
...
add the version field to the index for e2e_room_keys
2019-08-16 17:45:50 -07:00
6d86df73f1
Fix issue with Synapse not starting up. Fixes #5866 .
...
Signed-off-by: Chris Moos <chris@chrismoos.com>
2019-08-16 22:16:13 +01:00
87fa26006b
Opentracing misc ( #5856 )
...
Add authenticated_entity and servlet_names tags.
Functionally:
- Add a tag for authenticated_entity
- Add a tag for servlet_names
Stylistically:
Moved to importing methods directly from opentracing.
2019-08-16 16:13:25 +01:00
861d663c15
Fixup changelog and remove debug logging
2019-08-16 13:15:26 +01:00
b13cac896d
Fix up password reset template config names ( #5863 )
...
Fixes #5833
The emailconfig code was attempting to pull incorrect config file names. This corrects that, while also marking a difference between a config file variable that's a filepath versus a str containing HTML.
2019-08-15 16:27:11 +01:00
f299c5414c
Refactor MatrixFederationAgent to retry SRV.
...
This refactors MatrixFederationAgent to move the SRV lookup into the
endpoint code, this has two benefits:
1. Its easier to retry different host/ports in the same way as
HostnameEndpoint.
2. We avoid SRV lookups if we have a free connection in the pool
2019-08-15 15:43:22 +01:00
ce5f1cb98c
Merge branch 'master' into develop
2019-08-15 12:38:21 +01:00
fb5acd7039
1.3.0
2019-08-15 12:05:24 +01:00
748aa38378
Remove logging for #5407 and update comments
2019-08-15 12:02:18 +01:00
2253b083d9
Add support for inbound MSC2197 requests on unstable Federation API
...
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2019-08-15 11:06:21 +01:00
6fadb560fc
Support MSC2197 outbound with unstable prefix
...
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2019-08-15 10:59:37 +01:00
baee288fb4
Don't create broken room when power_level_content_override.users does not contain creator_id. ( #5633 )
2019-08-15 09:45:57 +01:00
e6e136decc
Retry well known on fail.
...
If we have recently seen a valid well-known for a domain we want to
retry on (non-final) errors a few times, to handle temporary blips in
networking/etc.
2019-08-15 09:28:58 +01:00
c058aeb88d
update set_e2e_room_key to agree with fixed index
2019-08-14 18:02:58 -07:00
b7f7cc7ace
add the version field to the index for e2e_room_keys
2019-08-14 17:14:40 -07:00
1b63ccd848
Wrap `get_local_public_room_list` call in `maybeDeferred` because it
...
is cached and so does not always return a `Deferred`.
`await` does not silently pass-through non-Deferreds like `yield` used to.
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2019-08-14 14:53:49 +01:00
09f6152a11
Merge pull request #5844 from matrix-org/erikj/retry_well_known_lookup
...
Retry well-known lookup before expiry.
2019-08-14 09:53:33 +01:00
f70d0a1dd9
1.3.0rc1
2019-08-13 18:20:09 +01:00
18bdac8ee4
fix config being a dict, actually
2019-08-14 02:06:42 +10:00
17e1e80726
Retry well-known lookup before expiry.
...
This gives a bit of a grace period where we can attempt to refetch a
remote `well-known`, while still using the cached result if that fails.
Hopefully this will make the well-known resolution a bit more torelant
of failures, rather than it immediately treating failures as "no result"
and caching that for an hour.
2019-08-13 16:20:38 +01:00
af187805b3
Merge pull request #5809 from matrix-org/erikj/handle_pusher_stop
...
Handle pusher being deleted during processing.
2019-08-13 14:08:29 +01:00
96bdd661b8
Remove redundant return
2019-08-13 12:50:36 +01:00
0b6fbb28a8
Don't load the media repo when configured to use an external media repo ( #5754 )
2019-08-13 21:49:28 +10:00
e9906b0772
Merge pull request #5836 from matrix-org/erikj/lower_bound_ttl_well_known
...
Add a lower bound to well-known TTL.
2019-08-13 12:41:16 +01:00
fb3469f53a
Clarify docstring
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-08-13 10:17:23 +01:00
f218705d2a
Make default well known cache global again.
2019-08-13 10:06:51 +01:00
2546f32b90
Merge pull request #5826 from matrix-org/erikj/reduce_event_pauses
...
Don't unnecessarily block notifying of new events.
2019-08-13 09:36:25 +01:00
f63ba7a795
Cross-signing [1/4] -- hidden devices ( #5759 )
...
* allow devices to be marked as "hidden"
This is a prerequisite for cross-signing, as it allows us to create other things
that live within the device namespace, so they can be used for signatures.
2019-08-12 15:14:37 -07:00
fb86217553
Merge pull request #5788 from matrix-org/rav/metaredactions
...
Fix handling of redactions of redactions
2019-08-12 12:25:19 +01:00
107ad133fc
Move well known lookup into a separate clas
2019-08-07 15:36:38 +01:00
af9f1c0764
Add a lower bound for TTL on well known results.
...
It costs both us and the remote server for us to fetch the well known
for every single request we send, so we add a minimum cache period. This
is set to 5m so that we still honour the basic premise of "refetch
frequently".
2019-08-06 17:01:23 +01:00
d1b5b055be
Merge pull request #5825 from matrix-org/erikj/fix_empty_limited_sync
...
Handle TimelineBatch being limited and empty.
2019-08-06 15:39:44 +01:00
edeae53221
Return 404 instead of 403 when retrieving an event without perms ( #5798 )
...
Part of fixing matrix-org/sytest#652
Sytest PR: matrix-org/sytest#667
2019-08-06 13:33:55 +01:00
bf4db42920
Don't unnecessarily block notifying of new events.
...
When persisting events we calculate new stream orderings up front.
Before we notify about an event all events with lower stream orderings
must have finished being persisted.
This PR moves the assignment of stream ordering till *after* calculated
the new current state and split the batch of events into separate chunks
for persistence. This means that if it takes a long time to calculate
new current state then it will not block events in other rooms being
notified about.
This should help reduce some global pauses in the events stream which
can last for tens of seconds (if not longer), caused by some
particularly expensive state resolutions.
2019-08-06 13:32:02 +01:00
6881f21f3e
Handle TimelineBatch being limited and empty.
...
This hopefully addresses #5407 by gracefully handling an empty but
limited TimelineBatch. We also add some logging to figure out how this
is happening.
2019-08-06 12:59:00 +01:00
fac1cdc562
make changes from PR review
2019-08-01 21:51:19 -04:00
336c546d6a
Merge branch 'cross-signing_hidden' into cross-signing_keys
2019-08-01 16:31:40 -04:00
8ed9e63432
Account validity: allow defining HTML templates to serve the us… ( #5807 )
...
Account validity: allow defining HTML templates to serve the user on account renewal attempt
2019-08-01 16:09:25 +02:00
d55bc4a8bf
Merge pull request #5810 from matrix-org/erikj/no_server_reachable
...
Return 502 not 500 when failing to reach any remote server.
2019-08-01 14:19:39 +01:00
5d018d23f0
Have ClientReaderSlavedStore inherit RegistrationStore ( #5806 )
...
Fixes #5803
2019-08-01 13:54:56 +01:00
3c076c79c5
Merge pull request #5808 from matrix-org/erikj/parse_decode_error
...
Handle incorrectly encoded query params correctly
2019-08-01 13:48:10 +01:00
a8f40a8302
Return 502 not 500 when failing to reach any remote server.
2019-08-01 13:47:31 +01:00
55a0c98d16
Merge pull request #5805 from matrix-org/erikj/validate_state
...
Validate well known state events are state events.
2019-08-01 13:45:48 +01:00
0b36decfb6
Merge pull request #5801 from matrix-org/erikj/recursive_tombstone
...
Don't allow clients to send tombstones that reference the same room
2019-08-01 13:45:35 +01:00
d02e41dcb2
Handle pusher being deleted during processing.
...
Instead of throwing a StoreError lets break out of processing loop and
mark the pusher as stopped.
2019-08-01 13:44:12 +01:00
d2e3d5b9db
Handle incorrectly encoded query params correctly
2019-08-01 13:23:00 +01:00
76a58fdcce
Fix spelling.
...
Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2019-08-01 13:17:55 +01:00
58af30a6c7
Merge pull request #5802 from matrix-org/erikj/deny_redacting_different_room
...
Deny redaction of events in a different room.
2019-08-01 13:14:46 +01:00
0f632f3a57
Merge pull request #5790 from matrix-org/erikj/groups_request_errors
...
Handle RequestSendFailed exception correctly in more places.
2019-08-01 13:14:08 +01:00
f25f638c35
Lint
2019-08-01 12:19:08 +02:00
a4a9ded4d0
Allow defining HTML templates to serve the user on account renewal
2019-08-01 11:59:27 +02:00
d78a4fe156
don't need to return the hidden column any more
2019-08-01 02:16:09 -04:00
73b26f827c
really fix queries to work with Postgres
...
(by going back to not using SQL directly)
2019-07-31 18:37:05 -04:00
430ea08186
PostgreSQL, Y U no like?
2019-07-31 15:38:11 -04:00
185188be03
remove extra SQL query param
2019-07-31 15:18:15 -04:00
dc4d74e44a
Validate well-known state events are state events.
...
Lets disallow sending things like memberships, topics etc as non-state
events.
2019-07-31 16:36:20 +01:00
2e697d3013
Explicitly check that tombstone is a state event before notifying.
2019-07-31 16:32:03 +01:00
cf89266b98
Deny redaction of events in a different room.
...
We already correctly filter out such redactions, but we should also deny
them over the CS API.
2019-07-31 16:12:27 +01:00
f31d4cb7a2
Don't allow clients to send tombstones that reference the same room
2019-07-31 15:52:27 +01:00
72167fb394
Change user deactivated errcode to USER_DEACTIVATED and use it ( #5686 )
...
This is intended as an amendment to #5674 as using M_UNKNOWN as the errcode makes it hard for clients to differentiate between an invalid password and a deactivated user (the problem we were trying to solve in the first place).
M_UNKNOWN was originally chosen as it was presumed than an MSC would have to be carried out to add a new code, but as Synapse often is the testing bed for new MSC implementations, it makes sense to try it out first in the wild and then add it into the spec if it is successful. Thus this PR return a new M_USER_DEACTIVATED code when a deactivated user attempts to login.
2019-07-31 15:19:06 +01:00
58a755cdc3
Remove duplicate return statement
2019-07-31 13:24:51 +01:00
a9bcae9f50
Share SSL options for well-known requests
2019-07-31 10:39:24 +01:00
c1f0a5636c
Merge branch 'develop' into cross-signing_hidden
2019-07-30 23:33:06 -04:00
bc95890822
Merge branch 'cross-signing_hidden' into cross-signing_keys
2019-07-30 23:21:26 -04:00
781ade836b
apply changes from PR review
2019-07-30 23:09:50 -04:00
123c04daa7
Don't recreate column
2019-07-30 16:42:48 +01:00
15056ca208
Fix current_state_events membership background update.
...
Turns out not all rooms are in `rooms`, so lets fetch the room list from
`current_state_events`. We move the delta file to force it to be run
again.
2019-07-30 14:51:41 +01:00
b4d5ff0af7
Don't log as exception when failing durig backfill
2019-07-30 13:19:22 +01:00
1ec7d656dd
Unwrap error
2019-07-30 13:09:02 +01:00
458e51df7a
Fix error handling when fetching remote device keys
2019-07-30 13:07:02 +01:00
8c97f6414c
Remove non-functional 'expire_access_token' setting ( #5782 )
...
The `expire_access_token` didn't do what it sounded like it should do. What it
actually did was make Synapse enforce the 'time' caveat on macaroons used as
access tokens, but since our access token macaroons never contained such a
caveat, it was always a no-op.
(The code to add 'time' caveats was removed back in v0.18.5, in #1656 )
2019-07-30 08:25:02 +01:00
4e97eb89e5
Handle loops in redaction events
2019-07-30 00:00:34 +01:00
448bcfd0f9
recursively fetch redactions
2019-07-30 00:00:34 +01:00
e6a6c4fbab
split _get_events_from_db out of _enqueue_events
2019-07-29 23:15:15 +01:00
c9964ba600
Return dicts from _fetch_event_list
2019-07-29 23:15:15 +01:00
865077f1d1
Room Complexity Client Implementation ( #5783 )
2019-07-30 02:47:27 +10:00
aecae8f397
Correctly handle errors doing requests to group servers
2019-07-29 17:21:57 +01:00
7c8c3b8437
Merge pull request #5774 from matrix-org/erikj/fix_rejected_membership
...
Fix room summary when rejected events are in state
2019-07-29 17:15:15 +01:00
3e013b7c8e
Merge pull request #5752 from matrix-org/erikj/forgotten_user
...
Remove some more joins on room_memberships
2019-07-29 17:15:01 +01:00
2a12d76646
Merge pull request #5770 from matrix-org/erikj/fix_current_state_event_sqlite
...
Fix current_state bg update to work on old SQLite
2019-07-29 17:09:01 +01:00
97a8b4caf7
Move some timeout checking logs to DEBUG #5785
2019-07-30 02:02:18 +10:00
df3a5db629
Expand comment
2019-07-29 16:40:25 +01:00
85b0bd8fe0
Update the device list cache when keys/query is called ( #5693 )
2019-07-29 16:34:44 +01:00
105e7f6ed3
Remove lost comment
2019-07-29 16:09:48 +01:00
45df38e61b
Fix current_state bg update to work on old SQLite
2019-07-29 13:04:10 +01:00
244953be3f
Add kwargs and doc
2019-07-29 10:03:14 +02:00
08352d44f8
Add ability to pass arguments to looping calls
2019-07-29 09:54:37 +02:00
d74595e2ca
Merge branch 'master' into develop
2019-07-26 12:39:33 +01:00
1a93daf353
Merge pull request #5744 from matrix-org/erikj/log_leave_origin_mismatch
...
Log when we receive a /make_* request from a different origin
2019-07-26 12:38:37 +01:00
dde6ea7ff6
1.2.1
2019-07-26 11:33:16 +01:00
14c24c9037
Fix room summary when rejected events are in state
...
Annoyingly, `current_state_events` table can include rejected events,
in which case the membership column will be null. To work around this
lets just always filter out null membership for now.
2019-07-26 10:11:36 +01:00
a0ee2ec458
Merge branch 'erikj/log_leave_origin_mismatch' into release-v1.2.1
2019-07-26 10:09:36 +01:00
d1020653fc
Log when we receive a /make_* request from a different origin
2019-07-26 10:08:22 +01:00
1f8bae7724
Log when we receive receipt from a different origin
2019-07-26 07:55:25 +01:00
0f2ecb961e
Fix DoS when there is a cycle in redaction events
...
Make sure that synapse doesn't explode when a redaction redacts itself, or
there is a larger cycle.
2019-07-26 06:36:48 +00:00
26d742fed6
Merge pull request #5767 from matrix-org/rav/redactions/cross_room_id
...
log when a redaction attempts to redact an event in a different room
2019-07-25 18:49:56 +01:00
b1605cdd23
log when a redaction attempts to redact an event in a different room
2019-07-25 18:26:20 +01:00
c659b9f94f
allow uploading keys for cross-signing
2019-07-25 11:08:24 -04:00
618bd1ee76
Fix some error cases in the caching layer. ( #5749 )
...
There was some inconsistent behaviour in the caching layer around how
exceptions were handled - particularly synchronously-thrown ones.
This seems to be most easily handled by pushing the creation of
ObservableDeferreds down from CacheDescriptor to the Cache.
2019-07-25 15:59:45 +01:00
f16aa3a44b
Merge branch 'master' into develop
2019-07-25 15:19:22 +01:00
c0a1301ccd
1.2.0
2019-07-25 14:10:32 +01:00
d1c7c2a98a
allow devices to be marked as "hidden"
...
This is a prerequisite for cross-signing, as it allows us to create other things
that live within the device namespace, so they can be used for signatures.
2019-07-24 23:21:52 -04:00
baf081cd3b
Bugfixes
...
--------
- Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. ([\#5734](https://github.com/matrix-org/synapse/issues/5734 ))
-----BEGIN PGP SIGNATURE-----
iQJHBAABCgAxFiEEgQG31Z317NrSMt0QiISIDS7+X/QFAl04Ur0THGFuZHJld0Bh
bW9yZ2FuLnh5egAKCRCIhIgNLv5f9F4oD/0TY6S/SEd2uAmzor64ojmbX5BOwPzf
j/wzUTrfvuf40EvkNPDpnejNZSvy/ysbaGQaQusv0SQKlV3xrvdn4RuMvnOWVWck
kBsO+lvzOaUTR0KHDxN4y9F5eI2NdPbub4847PPVzyqSIHAd+kolxXS8kSBBhwpL
yfaICWV/AOy5L7xN+JZ9IQpnegVAvUj5DmgXzDHd6VdeiHDVJuARaBgrR5uCkwVS
ZoLRqZ95XV/qiguMAUvPOwyEqht2mwO64989MswP16YYm8oMkB5QA6I5nYnACsTP
qk9YcN/oNvEfQXUhttku6MxK1/4yUMPUhEoDBDH7ebc0440QDtWN+IHTdA6oPVZB
IuStL9YGY16m7Ltx37ZUA4URfNMiSeLHo3zKc/mCAcwxN4HyOjJewtxbG5zKQAOZ
SMs8UcDwGR4zL1hnt8ZDNYtWwfzJBQIdGjoHvjXJEY7/1csTv2lmAwewFTXiqSAr
30GW5ews94kotqBK53zZT6V0F5gHNqgGHniOz1ZpqLLxYLqO3LSAGe97CrqlWUdX
GkhA9tZyweknociD9fyyBmKdcFJ4mL4a+oGI5CMnSMph8UvCY8Y5XMb1T+iYEABI
tA9G3mBvgkLPj+5V+8QggNkBafSigW2Q4FX7enGsDmiiskZOtfeKrAcVkapD4ooi
3I7IW5aetZr2IQ==
=+JBn
-----END PGP SIGNATURE-----
Merge tag 'v1.2.0rc2' into develop
Bugfixes
--------
- Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. ([\#5734](https://github.com/matrix-org/synapse/issues/5734 ))
2019-07-24 13:47:51 +01:00
2d573e2e2b
1.2.0rc2
2019-07-24 13:43:40 +01:00
2276936bac
Merge pull request #5743 from matrix-org/erikj/log_origin_receipts_mismatch
...
Log when we receive receipt from a different origin
2019-07-24 13:27:57 +01:00
f30a71a67b
Stop trying to fetch events with event_id=None. ( #5753 )
...
`None` is not a valid event id, so queuing up a database fetch for it seems
like a silly thing to do.
I considered making `get_event` return `None` if `event_id is None`, but then
its interaction with `allow_none` seemed uninituitive, and strong typing ftw.
2019-07-24 13:16:18 +01:00
cf2972c818
Fix servlet metric names ( #5734 )
...
* Fix servlet metric names
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
* Remove redundant check
* Cover all return paths
2019-07-24 13:07:35 +01:00
0c4a99607e
Remove join when calculating room summaries.
2019-07-24 11:49:15 +01:00
62921fb53e
Remove join on room_memberships when fetching rooms for user.
2019-07-24 11:45:58 +01:00
32768e96d4
Add function to get all forgotten rooms for user
...
This will allow us to efficiently filter out rooms that have been
forgotten in other queries without having to join against the
`room_memberships` table.
2019-07-24 11:44:23 +01:00
418635e68a
Add a prometheus metric for active cache lookups. ( #5750 )
...
* Add a prometheus metric for active cache lookups.
* changelog
2019-07-24 11:33:13 +01:00
3641784e8c
Make Jaeger fully configurable ( #5694 )
...
* Allow Jaeger to be configured
* Update sample config
2019-07-23 15:46:04 +01:00
4806651744
Replace returnValue with return ( #5736 )
2019-07-23 23:00:55 +10:00
18a466b84e
Opentracing Utils ( #5722 )
...
* Add decerators for tracing functions
* Use the new clean contexts
* Context and edu utils
* Move opentracing setters
* Move whitelisting
* Sectioning comments
* Better args wrapper
* Docstrings
Co-Authored-By: Erik Johnston <erik@matrix.org>
* Remove unused methods.
* Don't use global
* One tracing decorator to rule them all.
2019-07-23 13:31:16 +01:00
3db1377b26
Log when we receive receipt from a different origin
2019-07-23 13:31:03 +01:00
841b12867e
Merge pull request #5732 from matrix-org/erikj/sdnotify
...
Add process hooks to tell systemd our state.
2019-07-23 13:06:53 +01:00
b2a629ef49
Speed up current state background update.
...
Turns out that storing huge JSON arrays in the progress JSON isn't
something that postgres particularly likes.
2019-07-23 10:01:30 +01:00
0d0f6d12bc
Fix logging in workers ( #5729 )
...
This also adds a worker blacklist.
2019-07-22 16:05:00 +01:00
80cfad233e
Call startup commands as system triggers.
...
This helps ensures that we only consider ourselves "up" once all the
startup functions have completed.
2019-07-22 15:22:14 +01:00
720d30469f
Merge pull request #5730 from matrix-org/erikj/cache_versions
...
Cache get_version_string.
2019-07-22 14:52:52 +01:00
c560b791e1
Add process hooks to tell systemd our state.
...
Fixes #5676 .
2019-07-22 14:52:18 +01:00
8e513e7afc
Merge pull request #5731 from matrix-org/jaywink/admin-user-list-user-type
...
Add `user_type` to returned fields in admin API user list endpoints
2019-07-22 16:28:51 +03:00
0cb72812f9
Fix stack overflow in Keyring ( #5724 )
...
* Refactor Keyring._start_key_lookups
There's an awful lot of deferreds and dictionaries flying around here. The
whole thing can be made much simpler and achieve the same effect.
* Add a delay to key lookup lock release to fix stack overflow
A tactical call_later here should fix #5723
* changelog
2019-07-22 13:51:22 +01:00
f477ce4b1a
v1.2.0rc1
...
Features
--------
- Add support for opentracing. ([\#5544](https://github.com/matrix-org/synapse/issues/5544 ), [\#5712](https://github.com/matrix-org/synapse/issues/5712 ))
- Add ability to pull all locally stored events out of synapse that a particular user can see. ([\#5589](https://github.com/matrix-org/synapse/issues/5589 ))
- Add a basic admin command app to allow server operators to run Synapse admin commands separately from the main production instance. ([\#5597](https://github.com/matrix-org/synapse/issues/5597 ))
- Add `sender` and `origin_server_ts` fields to `m.replace`. ([\#5613](https://github.com/matrix-org/synapse/issues/5613 ))
- Add default push rule to ignore reactions. ([\#5623](https://github.com/matrix-org/synapse/issues/5623 ))
- Include the original event when asking for its relations. ([\#5626](https://github.com/matrix-org/synapse/issues/5626 ))
- Implement `session_lifetime` configuration option, after which access tokens will expire. ([\#5660](https://github.com/matrix-org/synapse/issues/5660 ))
- Return "This account has been deactivated" when a deactivated user tries to login. ([\#5674](https://github.com/matrix-org/synapse/issues/5674 ))
- Enable aggregations support by default ([\#5714](https://github.com/matrix-org/synapse/issues/5714 ))
Bugfixes
--------
- Fix 'utime went backwards' errors on daemonization. ([\#5609](https://github.com/matrix-org/synapse/issues/5609 ))
- Various minor fixes to the federation request rate limiter. ([\#5621](https://github.com/matrix-org/synapse/issues/5621 ))
- Forbid viewing relations on an event once it has been redacted. ([\#5629](https://github.com/matrix-org/synapse/issues/5629 ))
- Fix requests to the `/store_invite` endpoint of identity servers being sent in the wrong format. ([\#5638](https://github.com/matrix-org/synapse/issues/5638 ))
- Fix newly-registered users not being able to lookup their own profile without joining a room. ([\#5644](https://github.com/matrix-org/synapse/issues/5644 ))
- Fix bug in #5626 that prevented the original_event field from actually having the contents of the original event in a call to `/relations`. ([\#5654](https://github.com/matrix-org/synapse/issues/5654 ))
- Fix 3PID bind requests being sent to identity servers as `application/x-form-www-urlencoded` data, which is deprecated. ([\#5658](https://github.com/matrix-org/synapse/issues/5658 ))
- Fix some problems with authenticating redactions in recent room versions. ([\#5699](https://github.com/matrix-org/synapse/issues/5699 ), [\#5700](https://github.com/matrix-org/synapse/issues/5700 ), [\#5707](https://github.com/matrix-org/synapse/issues/5707 ))
- Ignore redactions of m.room.create events. ([\#5701](https://github.com/matrix-org/synapse/issues/5701 ))
Updates to the Docker image
---------------------------
- Base Docker image on a newer Alpine Linux version (3.8 -> 3.10). ([\#5619](https://github.com/matrix-org/synapse/issues/5619 ))
- Add missing space in default logging file format generated by the Docker image. ([\#5620](https://github.com/matrix-org/synapse/issues/5620 ))
Improved Documentation
----------------------
- Add information about nginx normalisation to reverse_proxy.rst. Contributed by @skalarproduktraum - thanks! ([\#5397](https://github.com/matrix-org/synapse/issues/5397 ))
- --no-pep517 should be --no-use-pep517 in the documentation to setup the development environment. ([\#5651](https://github.com/matrix-org/synapse/issues/5651 ))
- Improvements to Postgres setup instructions. Contributed by @Lrizika - thanks! ([\#5661](https://github.com/matrix-org/synapse/issues/5661 ))
- Minor tweaks to postgres documentation. ([\#5675](https://github.com/matrix-org/synapse/issues/5675 ))
Deprecations and Removals
-------------------------
- Remove support for the `invite_3pid_guest` configuration setting. ([\#5625](https://github.com/matrix-org/synapse/issues/5625 ))
Internal Changes
----------------
- Move logging code out of `synapse.util` and into `synapse.logging`. ([\#5606](https://github.com/matrix-org/synapse/issues/5606 ), [\#5617](https://github.com/matrix-org/synapse/issues/5617 ))
- Add a blacklist file to the repo to blacklist certain sytests from failing CI. ([\#5611](https://github.com/matrix-org/synapse/issues/5611 ))
- Make runtime errors surrounding password reset emails much clearer. ([\#5616](https://github.com/matrix-org/synapse/issues/5616 ))
- Remove dead code for persiting outgoing federation transactions. ([\#5622](https://github.com/matrix-org/synapse/issues/5622 ))
- Add `lint.sh` to the scripts-dev folder which will run all linting steps required by CI. ([\#5627](https://github.com/matrix-org/synapse/issues/5627 ))
- Move RegistrationHandler.get_or_create_user to test code. ([\#5628](https://github.com/matrix-org/synapse/issues/5628 ))
- Add some more common python virtual-environment paths to the black exclusion list. ([\#5630](https://github.com/matrix-org/synapse/issues/5630 ))
- Some counter metrics exposed over Prometheus have been renamed, with the old names preserved for backwards compatibility and deprecated. See `docs/metrics-howto.rst` for details. ([\#5636](https://github.com/matrix-org/synapse/issues/5636 ))
- Unblacklist some user_directory sytests. ([\#5637](https://github.com/matrix-org/synapse/issues/5637 ))
- Factor out some redundant code in the login implementation. ([\#5639](https://github.com/matrix-org/synapse/issues/5639 ))
- Update ModuleApi to avoid register(generate_token=True). ([\#5640](https://github.com/matrix-org/synapse/issues/5640 ))
- Remove access-token support from `RegistrationHandler.register`, and rename it. ([\#5641](https://github.com/matrix-org/synapse/issues/5641 ))
- Remove access-token support from `RegistrationStore.register`, and rename it. ([\#5642](https://github.com/matrix-org/synapse/issues/5642 ))
- Improve logging for auto-join when a new user is created. ([\#5643](https://github.com/matrix-org/synapse/issues/5643 ))
- Remove unused and unnecessary check for FederationDeniedError in _exception_to_failure. ([\#5645](https://github.com/matrix-org/synapse/issues/5645 ))
- Fix a small typo in a code comment. ([\#5655](https://github.com/matrix-org/synapse/issues/5655 ))
- Clean up exception handling around client access tokens. ([\#5656](https://github.com/matrix-org/synapse/issues/5656 ))
- Add a mechanism for per-test homeserver configuration in the unit tests. ([\#5657](https://github.com/matrix-org/synapse/issues/5657 ))
- Inline issue_access_token. ([\#5659](https://github.com/matrix-org/synapse/issues/5659 ))
- Update the sytest BuildKite configuration to checkout Synapse in `/src`. ([\#5664](https://github.com/matrix-org/synapse/issues/5664 ))
- Add a `docker` type to the towncrier configuration. ([\#5673](https://github.com/matrix-org/synapse/issues/5673 ))
- Convert `synapse.federation.transport.server` to `async`. Might improve some stack traces. ([\#5689](https://github.com/matrix-org/synapse/issues/5689 ))
- Documentation for opentracing. ([\#5703](https://github.com/matrix-org/synapse/issues/5703 ))
-----BEGIN PGP SIGNATURE-----
iQJHBAABCAAxFiEEgQG31Z317NrSMt0QiISIDS7+X/QFAl01rA4THGFuZHJld0Bh
bW9yZ2FuLnh5egAKCRCIhIgNLv5f9GYqD/sHgmTXL33/TL6ytdWfcrngo3+zyqz3
eMTvLggwFojTEzE2pO9R3UsaEZI+gMZEKfm+EEU4dXEnXxRxppMniHX3mMball7Q
Xz7TgmediE6nW+m8FDbAuHsMYa7GY3p+u5ZyleO78zIjIu8YahcpKwX40JCJNCE1
Vzz1T24i+lIsGB+3ahzabxkG1QHMppfClEFEpC8jYb+DL8BQzdMl0HAvdw4tAbDW
ONPAdGkjtPUv6TcfBDXoMXVdoAA4CFyDuCWR0O57I3Re5b7ARy6Dx0vcQbMCTEgQ
GnT1/QWtrpkX04OezMre2FAI+VQpQu/Hlhrl3+h8uQT1jBqYpWfR9jy/Y/iJWH0a
Dam0XzvZrMM1wrzd6exLMWS76W8EWKT9IV7xR9kf4HwnOxGrmusshzkCEUAxwxJX
jmFfTUw2f0yKt6lfNWaa67wP3suOlb++g33becFMKEM2DIb3iFxdk8U9p50406fL
asCfMsNWNKrcIkAnsgvPXfZMhUX0bgHBtkiZxTZ8/cJK01N+9pVvlb6ZZ7j30aIH
r4I6MAfaA57QOIla9Ml3LrT49dRskJpbReIqPiPxfH47W7hZhaCt0mF95SXzogF0
cP6t36ths/ywJ85EQUYY+9VgYK26jObm2xFTN2rUnZ1hnCEhKY4/Lwz0jPRQ8au3
2XHFZfqeAVpvvw==
=ZrnB
-----END PGP SIGNATURE-----
Merge tag 'v1.2.0rc1' into develop
v1.2.0rc1
Features
--------
- Add support for opentracing. ([\#5544](https://github.com/matrix-org/synapse/issues/5544 ), [\#5712](https://github.com/matrix-org/synapse/issues/5712 ))
- Add ability to pull all locally stored events out of synapse that a particular user can see. ([\#5589](https://github.com/matrix-org/synapse/issues/5589 ))
- Add a basic admin command app to allow server operators to run Synapse admin commands separately from the main production instance. ([\#5597](https://github.com/matrix-org/synapse/issues/5597 ))
- Add `sender` and `origin_server_ts` fields to `m.replace`. ([\#5613](https://github.com/matrix-org/synapse/issues/5613 ))
- Add default push rule to ignore reactions. ([\#5623](https://github.com/matrix-org/synapse/issues/5623 ))
- Include the original event when asking for its relations. ([\#5626](https://github.com/matrix-org/synapse/issues/5626 ))
- Implement `session_lifetime` configuration option, after which access tokens will expire. ([\#5660](https://github.com/matrix-org/synapse/issues/5660 ))
- Return "This account has been deactivated" when a deactivated user tries to login. ([\#5674](https://github.com/matrix-org/synapse/issues/5674 ))
- Enable aggregations support by default ([\#5714](https://github.com/matrix-org/synapse/issues/5714 ))
Bugfixes
--------
- Fix 'utime went backwards' errors on daemonization. ([\#5609](https://github.com/matrix-org/synapse/issues/5609 ))
- Various minor fixes to the federation request rate limiter. ([\#5621](https://github.com/matrix-org/synapse/issues/5621 ))
- Forbid viewing relations on an event once it has been redacted. ([\#5629](https://github.com/matrix-org/synapse/issues/5629 ))
- Fix requests to the `/store_invite` endpoint of identity servers being sent in the wrong format. ([\#5638](https://github.com/matrix-org/synapse/issues/5638 ))
- Fix newly-registered users not being able to lookup their own profile without joining a room. ([\#5644](https://github.com/matrix-org/synapse/issues/5644 ))
- Fix bug in #5626 that prevented the original_event field from actually having the contents of the original event in a call to `/relations`. ([\#5654](https://github.com/matrix-org/synapse/issues/5654 ))
- Fix 3PID bind requests being sent to identity servers as `application/x-form-www-urlencoded` data, which is deprecated. ([\#5658](https://github.com/matrix-org/synapse/issues/5658 ))
- Fix some problems with authenticating redactions in recent room versions. ([\#5699](https://github.com/matrix-org/synapse/issues/5699 ), [\#5700](https://github.com/matrix-org/synapse/issues/5700 ), [\#5707](https://github.com/matrix-org/synapse/issues/5707 ))
- Ignore redactions of m.room.create events. ([\#5701](https://github.com/matrix-org/synapse/issues/5701 ))
Updates to the Docker image
---------------------------
- Base Docker image on a newer Alpine Linux version (3.8 -> 3.10). ([\#5619](https://github.com/matrix-org/synapse/issues/5619 ))
- Add missing space in default logging file format generated by the Docker image. ([\#5620](https://github.com/matrix-org/synapse/issues/5620 ))
Improved Documentation
----------------------
- Add information about nginx normalisation to reverse_proxy.rst. Contributed by @skalarproduktraum - thanks! ([\#5397](https://github.com/matrix-org/synapse/issues/5397 ))
- --no-pep517 should be --no-use-pep517 in the documentation to setup the development environment. ([\#5651](https://github.com/matrix-org/synapse/issues/5651 ))
- Improvements to Postgres setup instructions. Contributed by @Lrizika - thanks! ([\#5661](https://github.com/matrix-org/synapse/issues/5661 ))
- Minor tweaks to postgres documentation. ([\#5675](https://github.com/matrix-org/synapse/issues/5675 ))
Deprecations and Removals
-------------------------
- Remove support for the `invite_3pid_guest` configuration setting. ([\#5625](https://github.com/matrix-org/synapse/issues/5625 ))
Internal Changes
----------------
- Move logging code out of `synapse.util` and into `synapse.logging`. ([\#5606](https://github.com/matrix-org/synapse/issues/5606 ), [\#5617](https://github.com/matrix-org/synapse/issues/5617 ))
- Add a blacklist file to the repo to blacklist certain sytests from failing CI. ([\#5611](https://github.com/matrix-org/synapse/issues/5611 ))
- Make runtime errors surrounding password reset emails much clearer. ([\#5616](https://github.com/matrix-org/synapse/issues/5616 ))
- Remove dead code for persiting outgoing federation transactions. ([\#5622](https://github.com/matrix-org/synapse/issues/5622 ))
- Add `lint.sh` to the scripts-dev folder which will run all linting steps required by CI. ([\#5627](https://github.com/matrix-org/synapse/issues/5627 ))
- Move RegistrationHandler.get_or_create_user to test code. ([\#5628](https://github.com/matrix-org/synapse/issues/5628 ))
- Add some more common python virtual-environment paths to the black exclusion list. ([\#5630](https://github.com/matrix-org/synapse/issues/5630 ))
- Some counter metrics exposed over Prometheus have been renamed, with the old names preserved for backwards compatibility and deprecated. See `docs/metrics-howto.rst` for details. ([\#5636](https://github.com/matrix-org/synapse/issues/5636 ))
- Unblacklist some user_directory sytests. ([\#5637](https://github.com/matrix-org/synapse/issues/5637 ))
- Factor out some redundant code in the login implementation. ([\#5639](https://github.com/matrix-org/synapse/issues/5639 ))
- Update ModuleApi to avoid register(generate_token=True). ([\#5640](https://github.com/matrix-org/synapse/issues/5640 ))
- Remove access-token support from `RegistrationHandler.register`, and rename it. ([\#5641](https://github.com/matrix-org/synapse/issues/5641 ))
- Remove access-token support from `RegistrationStore.register`, and rename it. ([\#5642](https://github.com/matrix-org/synapse/issues/5642 ))
- Improve logging for auto-join when a new user is created. ([\#5643](https://github.com/matrix-org/synapse/issues/5643 ))
- Remove unused and unnecessary check for FederationDeniedError in _exception_to_failure. ([\#5645](https://github.com/matrix-org/synapse/issues/5645 ))
- Fix a small typo in a code comment. ([\#5655](https://github.com/matrix-org/synapse/issues/5655 ))
- Clean up exception handling around client access tokens. ([\#5656](https://github.com/matrix-org/synapse/issues/5656 ))
- Add a mechanism for per-test homeserver configuration in the unit tests. ([\#5657](https://github.com/matrix-org/synapse/issues/5657 ))
- Inline issue_access_token. ([\#5659](https://github.com/matrix-org/synapse/issues/5659 ))
- Update the sytest BuildKite configuration to checkout Synapse in `/src`. ([\#5664](https://github.com/matrix-org/synapse/issues/5664 ))
- Add a `docker` type to the towncrier configuration. ([\#5673](https://github.com/matrix-org/synapse/issues/5673 ))
- Convert `synapse.federation.transport.server` to `async`. Might improve some stack traces. ([\#5689](https://github.com/matrix-org/synapse/issues/5689 ))
- Documentation for opentracing. ([\#5703](https://github.com/matrix-org/synapse/issues/5703 ))
2019-07-22 13:49:16 +01:00
66f5ff72fd
Add `user_type` to returned fields in admin API user list endpoints
...
Mostly user type will be empty (normal user) but there is also the
"support" user type.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-07-22 15:29:18 +03:00
5ea773c505
Cache get_version_string.
...
The version of a module isn't going to change over the lifetime of the
process (assuming no funky hot reloading is going on, which it isn't),
so let's just cache the result to avoid spawning lots of git
subprocesses.
Fixes #5672 .
2019-07-22 13:15:08 +01:00
54437c48ca
1.2.0rc1
2019-07-22 12:59:04 +01:00
0fd171770a
Merge branch 'release-v1.2.0' into develop
2019-07-22 11:18:50 +01:00
826e6ec3bd
Opentracing Documentation ( #5703 )
...
* Opentracing survival guide
* Update decorator names in doc
* Doc cleanup
These are all alterations as a result of comments in #5703 , it
includes mostly typos and clarifications. The most interesting
changes are:
- Split developer and user docs into two sections
- Add a high level description of OpenTracing
* newsfile
* Move contributer specific info to docstring.
* Sample config.
* Trailing whitespace.
* Update 5703.misc
* Apply suggestions from code review
Mostly just rewording parts of the docs for clarity.
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-07-22 11:15:21 +01:00
dcca56baba
Add a delay to key lookup lock release to fix stack overflow
...
A tactical call_later here should fix #5723
2019-07-19 17:57:00 +01:00
c7095be913
Refactor Keyring._start_key_lookups
...
There's an awful lot of deferreds and dictionaries flying around here. The
whole thing can be made much simpler and achieve the same effect.
2019-07-19 17:49:19 +01:00
7704873cb8
Merge pull request #5720 from matrix-org/erikj/transactions_upsert
...
Use upsert when updating destination retry interval
2019-07-19 16:51:16 +01:00
Richard van der Hoff