0836cbb9f5
Factor out an "assert_requester_is_admin" function ( #5120 )
...
Rather than copying-and-pasting the same four lines hundreds of times
2019-05-02 10:45:52 +01:00
f203c98794
fix examples
2019-05-01 17:49:56 +01:00
40e576e29c
Move admin api impl to its own package
...
It doesn't really belong under rest/client/v1 any more.
2019-05-01 15:44:30 +01:00
8e9ca83537
Move admin API to a new prefix
2019-05-01 15:44:30 +01:00
579b637b6c
Move admin API away from ClientV1RestServlet
2019-05-01 15:16:04 +01:00
8c5b1e30d4
Add a default .m.rule.tombstone push rule ( #4867 )
...
* Add a default .m.rule.tombstone push rule
In support of MSC1930: https://github.com/matrix-org/matrix-doc/pull/1930
* changelog
* Appease the changelog linter
2019-04-29 15:40:31 -06:00
b31cc1c613
Merge pull request #5100 from matrix-org/rav/verification_hackery
...
Improve logging when event-signature checking fails
2019-04-29 13:19:32 +01:00
d6118c5be6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ratelimit_3pid_invite
2019-04-26 18:14:23 +01:00
28a81ed62f
Ratelimit 3pid invites
...
We do ratelimit sending the 3PID invite events, but that happens after
spamming the identity server.
2019-04-26 18:06:25 +01:00
bd0d45ca69
Fix infinite loop in presence handler
...
Fixes #5102
2019-04-26 11:14:49 +01:00
837d7f85a9
more logging improvements
2019-04-25 22:17:59 +01:00
fd8fb32bdd
remove extraneous exception logging
2019-04-25 22:02:03 +01:00
7ca638c761
Clarify logging when PDU signature checking fails
2019-04-25 20:55:12 +01:00
6824ddd93d
Config option for verifying federation certificates (MSC 1711) ( #4967 )
2019-04-25 14:22:49 +01:00
788163e204
Remove log error for .well-known/matrix/client ( #4972 )
2019-04-24 17:44:06 +01:00
6b2b9a58c4
Prevent "producer not unregistered" message ( #5009 )
2019-04-24 17:37:32 +01:00
f8826d31cd
Don't crash on lack of expiry templates
2019-04-18 14:50:05 +01:00
ca90336a69
Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration
2019-04-17 19:44:40 +01:00
eaf41a943b
Add management endpoints for account validity
2019-04-17 19:34:45 +01:00
91934025b9
Merge pull request #5047 from matrix-org/babolivier/account_expiration
...
Send out emails with links to extend an account's validity period
2019-04-17 14:57:39 +01:00
20f0617e87
Send out emails with links to extend an account's validity period
2019-04-17 14:42:20 +01:00
49ff74da9b
Merge pull request #5071 from matrix-org/babolivier/3pid-check
...
Make sure we're not registering the same 3pid twice
2019-04-17 14:37:42 +01:00
600ec04739
Make sure we're not registering the same 3pid twice
2019-04-17 14:23:01 +01:00
fd2fcb817c
Merge pull request #5070 from matrix-org/erikj/postpath
...
Remove usage of request.postpath
2019-04-17 09:03:26 +01:00
a1eb4c6d2f
Merge pull request #5065 from matrix-org/erikj/fix_versions
...
VersionRestServlet doesn't take a param
2019-04-16 17:52:36 +01:00
ad010f6306
Remove usage of request.postpath
...
This is an undocumented variable in twisted, and relies on the servlet
being mounted in the right way.
This also breaks getting push rules on workers.
2019-04-16 17:42:50 +01:00
a137f4eac0
Add systemd-python to optional dependencies ( #4339 )
...
Using systemd-python allows for logging to the systemd journal,
as is documented in: `synapse/contrib/systemd/log_config.yaml`.
Signed-off-by: Silke Hofstra <silke@slxh.eu>
2019-04-16 20:41:17 +10:00
38642614cf
VersionRestServlet doesn't take a param
2019-04-15 19:39:47 +01:00
6e27a8620f
Merge pull request #5063 from matrix-org/erikj/move_endpoints
...
Move some rest endpoints to client reader
2019-04-15 18:55:01 +01:00
ec638a1602
Only handle GET requests for /push_rules
2019-04-15 18:51:48 +01:00
d5adf297e6
Move some rest endpoints to client reader
2019-04-15 17:21:03 +01:00
caa76e6021
Remove periods from copyright headers ( #5046 )
2019-04-11 17:08:13 +01:00
e446921def
Merge pull request #5033 from matrix-org/erikj/fix_schema_delta
...
Fix schema upgrade when dropping tables
2019-04-10 10:22:35 +01:00
329688c161
Fix disappearing exceptions in manhole. ( #5035 )
...
Avoid sending syntax errors from the manhole to sentry.
2019-04-10 07:23:48 +01:00
a0fc256d65
Limit in flight DNS requests
...
This is to work around a bug in twisted where a large number of
concurrent DNS requests cause it to tight loop forever.
c.f. https://twistedmatrix.com/trac/ticket/9620#ticket
2019-04-09 17:23:42 +01:00
bfc8fdf1fc
Merge pull request #5027 from matrix-org/babolivier/account_expiration
...
Add time-based account expiration
2019-04-09 17:02:41 +01:00
747aa9f8ca
Add account expiration feature
2019-04-09 16:46:04 +01:00
50d2a3059d
Fix schema upgrade when dropping tables
...
We need to drop tables in the correct order due to foreign table
constraints (on `application_services`), otherwise the DROP TABLE
command will fail.
Introduced in #4992 .
2019-04-09 14:39:18 +01:00
644b86677f
Merge pull request #5030 from matrix-org/rav/rewrite_g_s_v_k
...
Rewrite Datastore.get_server_verify_keys
2019-04-09 11:30:23 +01:00
4abf5aa81a
Bump psycopg requirement ( #5032 )
2019-04-09 11:29:50 +01:00
18b69be00f
Rewrite Datastore.get_server_verify_keys
...
Rewrite this so that it doesn't hammer the database.
2019-04-09 00:00:10 +01:00
f50efcb65d
Replace SlavedKeyStore with a shim
...
since we're pulling everything out of KeyStore anyway, we may as well simplify
it.
2019-04-08 23:59:07 +01:00
f88a9e6323
Remove redundant merged_keys dict
...
There's no point in collecting a merged dict of keys: it is sufficient to
consider just the new keys which have been fetched by the most recent
key_fetch_fns.
2019-04-08 22:36:18 +01:00
3352baac4b
Remove unused server_tls_certificates functions ( #5028 )
...
These have been unused since #4120 , and with the demise of perspectives, it is
unlikely that they will ever be used again.
2019-04-08 21:50:18 +01:00
b25e387c0d
add context to phonehome stats ( #5020 )
...
add context to phonehome stats
2019-04-08 15:47:39 +01:00
67d7b44784
Merge pull request #5024 from matrix-org/rav/record_correct_server_in_serverkeys
...
Fix from_server buglet in get_keys_from_perspectives
2019-04-08 15:40:37 +01:00
2d951686a7
drop tables listed in #1830 ( #4992 )
...
Tables dropped:
* application_services,
* application_services_regex,
* transaction_id_to_pdu,
* stats_reporting
* current_state_resets
* event_content_hashes
* event_destinations
* event_edge_hashes
* event_signatures
* feedback
* room_hosts
* state_forward_extremities
2019-04-08 15:37:26 +01:00
7d2a0c848e
Fix from_server buglet in get_keys_from_perspectives
...
make sure we store the name of the server the keys came from, rather than the
origin server, after doing a fetch-from-perspectives.
2019-04-08 12:51:16 +01:00
7fc1e17f4c
Merge pull request #5001 from matrix-org/rav/keyring_cleanups
...
Cleanups in the Keyring
2019-04-08 12:47:09 +01:00
6ae9361510
Hoist server_name check out of process_v2_response
...
It's easier to check it in the caller than to complicate the interface with an
extra param.
2019-04-04 19:12:54 +01:00
ef27d434d1
Clean up Keyring.process_v2_response
...
Make this just return the key dict, rather than a single-entry dict mapping the
server name to the key dict. It's easy for the caller to get the server name
from from the response object anyway.
2019-04-04 19:12:54 +01:00
b43d9a920b
Fix docstring on get_server_keys_json
2019-04-04 18:54:03 +01:00
8e85493b0c
Add config option to block users from looking up 3PIDs ( #5010 )
2019-04-04 17:25:47 +01:00
a33a5abc4c
Clean up the database pagination code ( #5007 )
...
* rewrite & simplify
* changelog
* cleanup potential sql injection
2019-04-05 00:21:16 +11:00
616e6a10bd
Merge pull request #5002 from matrix-org/erikj/delete_group
...
Add delete group admin API
2019-04-04 14:15:41 +01:00
db265f0642
Prevent kicking users who aren't in the room ( #4999 )
...
Prevent kick events from succeeding if the user is not currently in the room.
2019-04-04 13:05:51 +01:00
9f5d206c4a
Avoid redundant URL encoding ( #4555 )
...
* Do not double encode fallback redirect URL
Signed-off-by: Marcel Fabian Krüger <zauguin@gmail.com>
2019-04-04 12:05:56 +01:00
c192bf8970
Add admin API for group deletion
2019-04-03 16:29:52 +01:00
4a2e13631d
Add functions to delete a group
2019-04-03 16:29:52 +01:00
4a4d5c4fd6
Fix grammar and document get_current_users_in_room ( #4998 )
2019-04-03 14:32:20 +01:00
e8419554ff
Remove presence lists ( #4989 )
...
Remove presence list support as per MSC 1819
2019-04-03 11:11:15 +01:00
8f549c1177
Merge pull request #4982 from matrix-org/erikj/msc1915
...
Implement MSC1915 - 3PID unbind APIs
2019-04-03 11:07:09 +01:00
7efd1d87c2
Run black on the rest of the storage module ( #4996 )
2019-04-03 10:07:29 +01:00
3039d61baf
Merge pull request #4991 from matrix-org/erikj/stagger_push_startup
...
Make starting pushers faster during start up
2019-04-02 18:23:32 +01:00
66e78700a2
Transfer related groups on room upgrade ( #4990 )
...
Transfers the m.room.related_groups state event on room upgrade.
2019-04-02 17:15:24 +01:00
8530090b16
Add config.signing_key_path. ( #4974 )
...
As requested by @andrewshadura
2019-04-02 16:59:27 +01:00
5bec8d660d
Make starting pushers faster during start up
...
We start all pushers on start up and immediately start a background
process to fetch push to send. This makes start up incredibly painful
when dealing with many pushers.
Instead, let's do a quick fast DB check to see if there *may* be push to
send and only start the background processes for those pushers. We also
stagger starting up and doing those checks so that we don't try and
handle all pushers at once.
2019-04-02 16:59:13 +01:00
297bf2547e
Fix sync bug when accepting invites ( #4956 )
...
Hopefully this time we really will fix #4422 .
We need to make sure that the cache on
`get_rooms_for_user_with_stream_ordering` is invalidated *before* the
SyncHandler is notified for the new events, and we can now do so reliably via
the `events` stream.
2019-04-02 12:42:39 +01:00
4ef5d17b96
Correctly handle id_server param
2019-04-02 11:20:09 +01:00
24232514bf
Remove threepid binding if id server returns 400/404/501
2019-04-02 11:20:09 +01:00
c75e2017f1
Fixup docstrings
2019-04-02 11:20:06 +01:00
862d6e5ba5
Add unbind API to /r0 as it is now stabalised
2019-04-01 15:25:19 +01:00
3715c124b3
Grandfather in existing user threepids
...
We assume, as we did before, that users bound their threepid to one of
the trusted identity servers. So we simply fill the new table with all
threepids in `user_threepids` joined with the trusted identity servers.
2019-04-01 15:25:19 +01:00
057715aaa2
Allowing specifying IS to use in unbind API.
...
By default the homeserver will use the identity server used during the
binding of the 3PID to unbind the 3PID. However, we need to allow
clients to explicitly ask the homeserver to unbind via a particular
identity server, for the case where the 3PID was bound out of band from
the homeserver.
Implements MSC915.
2019-04-01 15:25:18 +01:00
9fbbc3d9e5
For unbind poke IS used during binding of 3PID
...
This changes the behaviour from using the server specified trusted
identity server to using the IS that used during the binding of the
3PID, if known.
This is the behaviour specified by MSC1915.
2019-04-01 15:23:30 +01:00
1666c0696a
Track IS used to bind 3PIDs
...
This will then be used to know which IS to default to when unbinding the
threepid.
2019-04-01 15:23:01 +01:00
62988f73fd
Merge branch 'master' into develop
2019-04-01 14:08:53 +01:00
35442efb75
0.99.3
2019-04-01 12:49:03 +00:00
54a87a7b08
Collect room-version variations into one place ( #4969 )
...
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
902cdc63b6
Merge pull request #4955 from matrix-org/rav/merge_state_into_events
...
Combine the CurrentStateDeltaStream into the EventStream
2019-03-28 18:32:13 +00:00
d688a51736
Merge pull request #4954 from matrix-org/rav/refactor_parse_row
...
Refactors to replication stream row update/parsing
2019-03-28 18:31:17 +00:00
c7296bcb98
remove log line for password ( #4965 )
...
Remove log line for password.
2019-03-28 17:38:01 +00:00
7a91b9d81c
Allow password providers to bind emails ( #4947 )
...
This PR allows password provider modules to bind email addresses when a user is registering and is motivated by matrix-org/matrix-synapse-ldap3#58
2019-03-28 15:48:07 +00:00
248014379e
Merge pull request #4942 from matrix-org/erikj/fix_presence
...
Use event streams to calculate presence
2019-03-28 14:38:31 +00:00
4e5f0f7ca0
Use an assert
2019-03-28 14:05:05 +00:00
40e56997bc
Review comments
2019-03-28 13:48:41 +00:00
d035d62f6b
Merge remote-tracking branch 'origin/develop' into rav/refactor_parse_row
2019-03-28 13:45:14 +00:00
4eeb2c2f07
Merge pull request #4953 from matrix-org/rav/refactor_replication_streams
...
Split up replication.tcp.streams into smaller files
2019-03-28 13:43:25 +00:00
2e060774ad
Run `black` on some storage modules that the stats branch touches ( #4959 )
2019-03-29 00:37:16 +11:00
4b91c313a9
Combine the CurrentStateDeltaStream into the EventStream
2019-03-27 22:07:05 +00:00
1f6d6f918a
Make EventStream rows have a type
...
... as a precursor to combining it with the CurrentStateDelta stream.
2019-03-27 22:07:05 +00:00
015b3622eb
Skip building a ROW_TYPE when building updates
...
We're about to turn it straight into a JSON object anyway so building a
ROW_TYPE is a bit pointless, and reduces flexibility in the update_function.
2019-03-27 21:58:03 +00:00
f570916a3e
Add parse_row method to replication stream class
...
This will allow individual stream classes to override how a row is parsed.
2019-03-27 21:32:33 +00:00
71dcb275f1
move FederationStream out to its own file
2019-03-27 21:13:14 +00:00
aa1e017864
move EventsStream out to its own file
2019-03-27 21:13:14 +00:00
a5798de067
Move replication.tcp.streams into a package
2019-03-27 21:13:14 +00:00
acaa18f7dd
Fix/improve some docstrings in the replication code. ( #4949 )
2019-03-27 21:12:36 +00:00
197fae1639
Use event streams to calculate presence
...
Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.
We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).
Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.
Fixes #3962
2019-03-27 13:41:36 +00:00
4aa914369b
bump version
2019-03-27 10:23:03 +00:00
bbd244c7b2
Support 3PID login in password providers ( #4931 )
...
Adds a new method, check_3pid_auth, which gives password providers
the chance to allow authentication with third-party identifiers such
as email or msisdn.
2019-03-26 17:48:30 +00:00
903f04c21f
Use the state event amount for userdir import batching, not room count ( #4944 )
2019-03-27 02:49:28 +11:00
4a125be138
Make federation endpoints more tolerant of trailing slashes v2 ( #4935 )
...
Redo of https://github.com/matrix-org/synapse/pull/4840
2019-03-26 11:35:29 +00:00
8cbbedaa2b
Fix ClientReplicationStreamProtocol.__str__ ( #4929 )
...
`__str__` depended on `self.addr`, which was absent from
ClientReplicationStreamProtocol, so attempting to call str on such an object
would raise an exception.
We can calculate the peer addr from the transport, so there is no need for addr
anyway.
2019-03-25 16:41:51 +00:00
9bde730ef8
Fix bug where read-receipts lost their timestamps ( #4927 )
...
Make sure that they are sent correctly over the replication stream.
Fixes : #4898
2019-03-25 16:38:05 +00:00
ac396a0d32
Refactor out state delta handling into its own class ( #4917 )
2019-03-25 20:37:08 +11:00
5fee9d8067
Merge pull request #4869 from matrix-org/erikj/yaml_load
...
Fix yaml warnings by using safe_load
2019-03-22 11:58:13 +00:00
3677548a82
Use yaml safe_load
2019-03-22 10:20:17 +00:00
7bef97dfb7
Remove trailing slashes from outbound federation requests and retry on 400 ( #4840 )
...
As per #3622 , we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
2019-03-21 15:07:28 +00:00
b41c2eaadc
Clean up backoff_on_404 and metehod calls
2019-03-21 14:32:47 +00:00
01e6b405be
Merge pull request #4908 from matrix-org/erikj/block_peek_on_blocked_rooms
...
Deny peeking into rooms that have been blocked
2019-03-21 14:07:17 +00:00
3959858eaa
Merge pull request #4904 from matrix-org/erikj/fix_shutdown
...
Fixup shutdown room API
2019-03-21 11:24:42 +00:00
536a266520
Deny peeking into rooms that have been blocked
2019-03-21 11:20:13 +00:00
3ecec5ede2
Fix upsert
2019-03-21 10:21:15 +00:00
09f991a63d
Merge pull request #4896 from matrix-org/erikj/disable_room_directory
...
Add option to disable search room lists
2019-03-21 10:16:54 +00:00
cd62981a6a
Revert spurious delete
2019-03-20 17:51:27 +00:00
8d8834d3e7
comment block_room
2019-03-20 17:49:56 +00:00
aa959a6c07
Use flags
2019-03-20 17:40:29 +00:00
7d47cc1305
Move requester check into assert_accepted_privacy_policy
2019-03-20 17:08:36 +00:00
72a14860ab
Gracefully handle failing to kick user
2019-03-20 16:54:21 +00:00
6b28890543
Log new room ID
2019-03-20 16:52:28 +00:00
74c46d81fa
Only require consent for events with an associated request
...
There are a number of instances where a server or admin may puppet a
user to join/leave rooms, which we don't want to fail if the user has
not consented to the privacy policy. We fix this by adding a check to
test if the requester has an associated access_token, which is used as a
proxy to answer the question of whether the action is being done on
behalf of a real request from the user.
2019-03-20 16:50:23 +00:00
67d618e111
Allow blocking a room multiple times
2019-03-20 16:50:05 +00:00
263f2c9ce1
Merge pull request #4895 from matrix-org/erikj/disable_user_search
...
Add option to disable searching in the user dir
2019-03-20 16:47:15 +00:00
4d53017432
Batching in the user directory import ( #4900 )
2019-03-21 03:06:36 +11:00
cdb8036161
Add a config option for torture-testing worker replication. ( #4902 )
...
Setting this to 50 or so makes a bunch of sytests fail in worker mode.
2019-03-20 16:04:35 +00:00
a902d13180
Batch up outgoing read-receipts to reduce federation traffic. ( #4890 )
...
Rate-limit outgoing read-receipts as per #4730 .
2019-03-20 16:02:25 +00:00
cd8c5b91ad
Fix up sample config
2019-03-20 14:35:41 +00:00
ab20f85c59
Update synapse/config/user_directory.py
...
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-03-20 14:33:11 +00:00
cc197a61a1
Disable publishing to room list when its disabled
2019-03-20 14:30:36 +00:00
2c90422146
Pull out config option
2019-03-20 14:25:58 +00:00
7529038e66
Return before we log
2019-03-20 14:25:28 +00:00
926f29ea6d
Fix up config comments
2019-03-20 14:24:53 +00:00
2150151abe
kwargs doesn't like commas on calling funcs either. TIL
2019-03-20 14:13:32 +00:00
bb52a2e653
lint
2019-03-20 14:08:57 +00:00
cd36a1283b
New test, fix issues
2019-03-20 14:00:39 +00:00
c69df5d5d3
Fix comments. v0.99.2 -> v0.99.3
2019-03-20 11:27:18 +00:00
551ea11559
Just return if not doing any trailing slash shennanigans
2019-03-20 11:07:36 +00:00
213c98c00a
Add option to disable search room lists
...
This disables both local and remote room list searching.
2019-03-19 17:10:52 +00:00
320667a479
Add option to disable searching in the user dir
...
We still populate it, as it can still be accessed via the admin API.
2019-03-19 16:40:19 +00:00
13bc1e0746
Use a regular HomeServerConfig object for unit tests
...
Rather than using a Mock for the homeserver config, use a genuine
HomeServerConfig object. This makes for a more realistic test, and means that
we don't have to keep remembering to add things to the mock config every time
we add a new config setting.
2019-03-19 11:44:43 +00:00
45c4e19c74
Merge remote-tracking branch 'origin/develop' into HEAD
2019-03-19 11:43:46 +00:00
88f0675967
fix test_auto_create_auto_join_where_no_consent ( #4886 )
2019-03-19 11:38:59 +00:00
0dbfae03f9
Enforce hs_disabled_message correctly
...
Fixes a bug where hs_disabled_message was not enforced for 3pid-based requests
if there was no server_notices_mxid configured.
2019-03-19 11:30:54 +00:00
e9eeca1314
Fix user directory background update ( #4887 )
2019-03-19 22:13:53 +11:00
fd463b4f5d
Comment out most options in the generated config. ( #4863 )
...
Make it so that most options in the config are optional, and commented out in
the generated config.
The reasons this is a good thing are as follows:
* If we decide that we should change the default for an option, we can do so,
and only those admins that have deliberately chosen to override that option
will be stuck on the old setting.
* It moves us towards a point where we can get rid of the super-surprising
feature of synapse where the default settings for the config come from the
generated yaml.
* It makes setting up a test config for unit testing an order of magnitude
easier (see forthcoming PR).
* It makes the generated config more consistent, and hopefully easier for users
to understand.
2019-03-19 10:06:40 +00:00
282c97327f
Migrate the user directory initial population to a background task ( #4864 )
2019-03-19 04:50:24 +11:00
a8ad39eec7
lint
2019-03-18 17:47:39 +00:00
621e7f37f1
Better exception handling
2019-03-18 17:45:54 +00:00
651ad8bc96
Add ratelimiting on failed login attempts ( #4865 )
2019-03-18 12:57:20 +00:00
899e523d6d
Add ratelimiting on login ( #4821 )
...
Add two ratelimiters on login (per-IP address and per-userID).
2019-03-15 17:46:16 +00:00
Richard van der Hoff