Commit Graph

212 Commits (release-v1.76)

Author SHA1 Message Date
Sean Quah 2b82ec425f
Add type hints for most `HomeServer` parameters (#11095) 2021-10-22 18:15:41 +01:00
Dirk Klimpel ef7fe09778
Fix setting a user's external_id via the admin API returns 500 and deletes users existing external mappings if that external ID is already mapped (#11051)
Fixes #10846
2021-10-21 09:52:32 +01:00
Aaron R 2c61a318cc
Show error when timestamp in seconds is provided to the /purge_media_cache API (#11101) 2021-10-20 14:41:48 +00:00
Patrick Cloke a0f48ee89d
Use direct references for configuration variables (part 7). (#10959) 2021-10-04 07:18:54 -04:00
Patrick Cloke bb7fdd821b
Use direct references for configuration variables (part 5). (#10897) 2021-09-24 07:25:21 -04:00
Callum Brown 90d9fc7505
Allow `.` and `~` chars in registration tokens (#10887)
Per updates to MSC3231 in order to use the same grammar
as other identifiers.
2021-09-23 17:58:12 +00:00
Patrick Cloke e584534403
Use direct references for some configuration variables (part 3) (#10885)
This avoids the overhead of searching through the various
configuration classes by directly referencing the class that
the attributes are in.

It also improves type hints since mypy can now resolve the
types of the configuration variables.
2021-09-23 07:13:34 -04:00
Andrew Morgan aa2c027792
Remove unnecessary parentheses around tuples returned from methods (#10889) 2021-09-23 11:59:07 +01:00
Hillery Shay 5fca3c8ae6
Allow Synapse Admin API's Room Search to accept non-ASCII characters (#10859)
* add tests for checking if room search works with non-ascii char

* change encoding on parse_string to UTF-8

* lints

* properly encode search term

* lints

* add changelog file

* update changelog number

* set changelog entry filetype to .bugfix

* Revert "set changelog entry filetype to .bugfix"

This reverts commit be8e5a3142.

* update changelog message and file type

* change parse_string default encoding back to ascii and update room search admin api calll to parse string

* refactor tests

* Update tests/rest/admin/test_room.py

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-09-21 08:04:35 -07:00
Patrick Cloke b93259082c
Add missing type hints to non-client REST servlets. (#10817)
Including admin, consent, key, synapse, and media. All REST servlets
(the synapse.rest module) now require typed method definitions.
2021-09-15 08:45:32 -04:00
Patrick Cloke 01c88a09cd
Use direct references for some configuration variables (#10798)
Instead of proxying through the magic getter of the RootConfig
object. This should be more performant (and is more explicit).
2021-09-13 13:07:12 -04:00
Patrick Cloke ecbfa4fe4f
Additional type hints for client REST servlets (part 5) (#10736)
Additionally this enforce type hints on all function signatures inside
of the synapse.rest.client package.
2021-09-03 09:22:22 -04:00
Dirk Klimpel e62cdbef1a
Improve ServerNoticeServlet to avoid duplicate requests (#10679)
Fixes: #9544
2021-08-27 09:16:40 +00:00
Callum Brown 947dbbdfd1
Implement MSC3231: Token authenticated registration (#10142)
Signed-off-by: Callum Brown <callum@calcuode.com>

This is part of my GSoC project implementing [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).
2021-08-21 22:14:43 +01:00
Dirk Klimpel 220f901229
Remove not needed database updates in modify user admin API (#10627) 2021-08-19 10:25:05 +01:00
Erik Johnston 78a70a2e0b Merge branch 'release-v1.41' into develop 2021-08-18 17:02:47 +01:00
Andrew Morgan 3692f7fd33
Mount /_synapse/admin/v1/users/{userId}/media admin API on media workers only (#10628)
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
2021-08-18 13:25:12 +01:00
Dirk Klimpel 430241a1e9
Remove deprecated Shutdown Room and Purge Room Admin API (#8830) 2021-08-17 21:19:13 +01:00
reivilibre 642a42edde
Flatten the synapse.rest.client package (#10600) 2021-08-17 11:57:58 +00:00
Dirk Klimpel 3bcd525b46
Allow to edit `external_ids` by Edit User admin API (#10598)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-08-17 11:56:11 +01:00
Will Hunt a933c2c7d8
Add an admin API to check if a username is available (#10578)
This adds a new API GET /_synapse/admin/v1/username_available?username=foo to check if a username is available. It is the counterpart to https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available, except that it works even if registration is disabled.
2021-08-17 10:52:38 +01:00
Michael Telatynski 0ace38b7b3
Experimental support for MSC3266 Room Summary API. (#10394) 2021-08-16 14:49:12 +00:00
Dirk Klimpel 915b37e5ef
Admin API to delete media for a specific user (#10558) 2021-08-11 19:29:59 +00:00
Dirk Klimpel 89c4ca81bb
Add `creation_ts` to list users admin API (#10448)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-07-22 16:05:16 +02:00
Patrick Cloke 5db118626b
Add a return type to parse_string. (#10438)
And set the required attribute in a few places which will error if
a parameter is not provided.
2021-07-21 09:47:56 -04:00
Jonathan de Jong 98aec1cc9d
Use inline type hints in `handlers/` and `rest/`. (#10382) 2021-07-16 18:22:36 +01:00
Brendan Abolivier 36dc15412d
Add a module type for account validity (#9884)
This adds an API for third-party plugin modules to implement account validity, so they can provide this feature instead of Synapse. The module implementing the current behaviour for this feature can be found at https://github.com/matrix-org/synapse-email-account-validity.

To allow for a smooth transition between the current feature and the new module, hooks have been added to the existing account validity endpoints to allow their behaviours to be overridden by a module.
2021-07-16 18:11:53 +02:00
Dirk Klimpel d9b3637e44
Bugfix `make_room_admin` fails for users that have left a private room (#10367)
Fixes: #10338
2021-07-13 11:53:45 +01:00
Patrick Cloke 9e4610cc27
Correct type hints for parse_string(s)_from_args. (#10137) 2021-06-08 08:30:48 -04:00
Dirk Klimpel d558292548
Add missing type hints to the admin API servlets (#10105) 2021-06-07 15:12:34 +01:00
Dirk Klimpel 0284d2a297
Add new admin APIs to remove media by media ID from quarantine. (#10044)
Related to: #6681, #5956, #10040

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-06-02 18:50:35 +01:00
Dirk Klimpel 65e6c64d83
Add an admin API for unprotecting local media from quarantine (#10040)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-05-26 11:19:47 +01:00
Andrew Morgan 4d6e5a5e99
Use a database table to hold the users that should have full presence sent to them, instead of something in-memory (#9823) 2021-05-18 14:13:45 +01:00
ThibF 0085dc5abc
Delete room endpoint (#9889)
Support the delete of a room through DELETE request and mark
previous request as deprecated through documentation.

Signed-off-by: Thibault Ferrante <thibault.ferrante@pm.me>
2021-04-29 10:31:45 +01:00
Andrew Morgan fe604a022a
Remove various bits of compatibility code for Python <3.6 (#9879)
I went through and removed a bunch of cruft that was lying around for compatibility with old Python versions. This PR also will now prevent Synapse from starting unless you're running Python 3.6+.
2021-04-27 13:13:07 +01:00
Jonathan de Jong 4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Dirk Klimpel 1fc97ee876
Add an admin API to manage ratelimit for a specific user (#9648) 2021-04-13 10:26:37 +01:00
Dirk Klimpel 48a1f4db31
Remove old admin API `GET /_synapse/admin/v1/users/<user_id>` (#9401)
Related: #8334
Deprecated in: #9429 - Synapse 1.28.0 (2021-02-25)

`GET /_synapse/admin/v1/users/<user_id>` has no
- unit tests
- documentation

API in v2 is available (#5925 - 12/2019, v1.7.0).
API is misleading. It expects `user_id` and returns a list of all users.

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-04-09 09:44:40 +01:00
Dirk Klimpel bb0fe02a52
Add `order_by` to list user admin API (#9691) 2021-04-01 11:28:53 +01:00
Erik Johnston b5efcb577e
Make it possible to use dmypy (#9692)
Running `dmypy run` will do a `mypy` check while spinning up a daemon
that makes rerunning `dmypy run` a lot faster.

`dmypy` doesn't support `follow_imports = silent` and has
`local_partial_types` enabled, so this PR enables those options and
fixes the issues that were newly raised. Note that `local_partial_types`
will be enabled by default in upcoming mypy releases.
2021-03-26 16:49:46 +00:00
Patrick Cloke b7748d3c00
Import HomeServer from the proper module. (#9665) 2021-03-23 07:12:48 -04:00
Dirk Klimpel 8dd2ea65a9
Consistently check whether a password may be set for a user. (#9636) 2021-03-18 12:54:08 -04:00
Andrew Morgan 1b0eaed21f
Prevent bundling aggregations for state events (#9619)
There's no need to do aggregation bundling for state events. Doing so can cause performance issues.
2021-03-16 10:27:51 +00:00
Patrick Cloke 55da8df078
Fix additional type hints from Twisted 21.2.0. (#9591) 2021-03-12 11:37:57 -05:00
Dirk Klimpel 70d1b6abff
Re-Activating account when local passwords are disabled (#9587)
Fixes: #8393
2021-03-11 13:52:32 +00:00
Dirk Klimpel 7076eee4b9
Add type hints to purge room and server notice admin API. (#9520) 2021-03-08 10:34:38 -05:00
Dirk Klimpel d790d0d314
Add type hints to user admin API. (#9521) 2021-03-03 08:09:39 -05:00
Dirk Klimpel f00c4e7af0
Add type hints to device and event report admin API (#9519) 2021-03-02 09:31:12 +00:00
Patrick Cloke ad8589d392
Fix a bug when a room alias is given to the admin join endpoint (#9506) 2021-03-01 13:59:01 -05:00
Patrick Cloke a0bc9d387e
Use the proper Request in type hints. (#9515)
This also pins the Twisted version in the mypy job for CI until
proper type hints are fixed throughout Synapse.
2021-03-01 12:23:46 -05:00
Dirk Klimpel 71c9f8de6d
Add an `order_by` field to list users' media admin API. (#8978) 2021-02-22 14:38:51 -05:00
Eric Eastwood 0a00b7ff14
Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Patrick Cloke 7950aa8a27 Fix some typos. 2021-02-12 11:14:12 -05:00
David Teller b0b2cac057
Merge pull request #9150 from Yoric/develop-context
New API /_synapse/admin/rooms/{roomId}/context/{eventId}
2021-02-08 15:53:44 +01:00
Travis Ralston b60bb28bbc
Add an admin API to get the current room state (#9168)
This could arguably replace the existing admin API for `/members`, however that is out of scope of this change.

This sort of endpoint is ideal for moderation use cases as well as other applications, such as needing to retrieve various bits of information about a room to perform a task (like syncing power levels between two places). This endpoint exposes nothing more than an admin would be able to access with a `select *` query on their database.
2021-02-02 11:16:29 +00:00
David Teller 31d072aea0 FIXUP: linter 2021-01-28 16:53:40 +01:00
David Teller 93f84e0373 FIXUP: Making get_event_context a bit more paranoid 2021-01-28 12:31:07 +01:00
David Teller a764869623 FIXUP: Doc 2021-01-28 12:31:07 +01:00
David Teller b859919acc FIXUP: Now testing that the user is admin! 2021-01-28 12:31:07 +01:00
David Teller 10332c175c New API /_synapse/admin/rooms/{roomId}/context/{eventId}
Signed-off-by: David Teller <davidt@element.io>
2021-01-28 12:29:49 +01:00
Pankaj Yadav 2e537a0280
Check if a user is in the room before sending a PowerLevel event on their behalf (#9235) 2021-01-27 17:38:08 +00:00
Jason Robinson cee4010f94 Merge branch 'develop' into jaywink/admin-forward-extremities
# Conflicts:
#	synapse/rest/admin/__init__.py
2021-01-26 10:15:32 +02:00
Patrick Cloke 4a55d267ee
Add an admin API for shadow-banning users. (#9209)
This expands the current shadow-banning feature to be usable via
the admin API and adds documentation for it.

A shadow-banned users receives successful responses to their
client-server API requests, but the events are not propagated into rooms.

Shadow-banning a user should be used as a tool of last resort and may lead
to confusing or broken behaviour for the client.
2021-01-25 14:49:39 -05:00
Jason Robinson 8965b6cfec Merge branch 'develop' into jaywink/admin-forward-extremities 2021-01-23 21:41:35 +02:00
Dirk Klimpel c55e62548c
Add tests for List Users Admin API (#9045) 2021-01-21 09:18:46 -05:00
Patrick Cloke 3e4cdfe5d9
Add an admin API endpoint to protect media. (#9086)
Protecting media stops it from being quarantined when
e.g. all media in a room is quarantined. This is useful
for sticker packs and other media that is uploaded by
server administrators, but used by many people.
2021-01-15 11:18:09 -05:00
Dirk Klimpel 7a2e9b549d
Remove user's avatar URL and displayname when deactivated. (#8932)
This only applies if the user's data is to be erased.
2021-01-12 16:30:15 -05:00
Jason Robinson da16d06301 Address pr feedback
* docs updates
* prettify SQL
* add missing copyright
* cursor_to_dict
* update touched files copyright years

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-11 23:43:58 +02:00
David Teller b161528fcc
Also support remote users on the joined_rooms admin API. (#8948)
For remote users, only the rooms which the server knows about are returned.
Local users have all of their joined rooms returned.
2021-01-11 14:32:17 -05:00
Jason Robinson 90ad4d443a Implement clearing cache after deleting forward extremities
Also run linter.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-09 21:57:41 +02:00
Jason Robinson 85c0999bfb Add Rooms admin forward extremities DELETE endpoint
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-08 00:12:23 +02:00
Jason Robinson c91045f56c Move unknown room ID error into resolve_room_id
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-07 23:03:54 +02:00
Jason Robinson b849e46139 Add forward extremities endpoint to rooms admin API
GET /_synapse/admin/v1/rooms/<identifier>/forward_extremities now gets forward extremities for a room, returning count and the list of extremities.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-07 23:01:59 +02:00
Erik Johnston d781a81e69
Allow server admin to get admin bit in rooms where local user is an admin (#8756)
This adds an admin API that allows a server admin to get power in a room if a local user has power in a room. Will also invite the user if they're not in the room and its a private room. Can specify another user (rather than the admin user) to be granted power.

Co-authored-by: Matthew Hodgson <matthew@matrix.org>
2020-12-18 15:37:19 +00:00
Patrick Cloke bd30cfe86a
Convert internal pusher dicts to attrs classes. (#8940)
This improves type hinting and should use less memory.
2020-12-16 11:25:30 -05:00
Dirk Klimpel 0a34cdfc66
Add number of local devices to Room Details Admin API (#8886) 2020-12-11 10:42:47 +00:00
Will Hunt ff1f0ee094
Call set_avatar_url with target_user, not user_id (#8872)
* Call set_avatar_url with target_user, not user_id

Fixes https://github.com/matrix-org/synapse/issues/8871

* Create 8872.bugfix

* Update synapse/rest/admin/users.py

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Testing

* Update changelog.d/8872.bugfix

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2020-12-07 19:13:07 +00:00
David Florness c4675e1b24
Add additional validation for the admin register endpoint. (#8837)
Raise a proper 400 error if the `mac` field is missing.
2020-12-02 10:01:15 -05:00
Richard van der Hoff a090b86209
Add `force_purge` option to delete-room admin api. (#8843) 2020-11-30 16:48:12 +00:00
Dirk Klimpel 3f0ff53158
Remove deprecated `/_matrix/client/*/admin` endpoints (#8785)
These are now only available via `/_synapse/admin/v1`.
2020-11-25 16:26:11 -05:00
Erik Johnston f737368a26
Add admin API for logging in as a user (#8617) 2020-11-17 10:51:25 +00:00
Dirk Klimpel c3119d1536
Add an admin API for users' media statistics (#8700)
Add `GET /_synapse/admin/v1/statistics/users/media` to get statisics about local media usage by users.
Related to #6094
It is the first API for statistics.
Goal is to avoid/reduce usage of sql queries like [Wiki analyzing Synapse](https://github.com/matrix-org/synapse/wiki/SQL-for-analyzing-Synapse-PostgreSQL-database-stats)

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-11-05 18:59:12 +00:00
Dirk Klimpel e4676bd877
Add `displayname` to Shared-Secret Registration for admins (#8722)
Add `displayname` to Shared-Secret Registration for admins to `POST /_synapse/admin/v1/register`
2020-11-05 13:55:45 +00:00
Dirk Klimpel 2239813278
Add an admin APIs to allow server admins to list users' pushers (#8610)
Add an admin API `GET /_synapse/admin/v1/users/<user_id>/pushers` like https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-pushers
2020-10-28 15:02:42 +00:00
Dirk Klimpel 9b7c28283a
Add admin API to list users' local media (#8647)
Add admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information of users' uploaded files.
2020-10-27 14:12:31 +00:00
Dirk Klimpel 66e6801c3e
Split admin API for reported events into a detail and a list view (#8539)
Split admin API for reported events in detail und list view.
API was introduced with #8217 in synapse v.1.21.0.

It makes the list (`GET /_synapse/admin/v1/event_reports`) less complex and provides a better overview.
The details can be queried with: `GET /_synapse/admin/v1/event_reports/<report_id>`.
It is similar to room and users API.

It is a kind of regression in `GET /_synapse/admin/v1/event_reports`.  `event_json` was removed. But the api was introduced one version before and it is an admin API (not under spec).

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-10-26 18:16:37 +00:00
Dirk Klimpel 49d72dea2a
Add an admin api to delete local media. (#8519)
Related to: #6459, #3479

Add `DELETE /_synapse/admin/v1/media/<server_name>/<media_id>` to delete
a single file from server.
2020-10-26 17:02:28 +00:00
Dirk Klimpel 4ac3a8c5dc
Fix a bug in the joined_rooms admin API (#8643)
If the user was not in any rooms then the API returned the same error
as if the user did not exist.
2020-10-26 12:25:48 -04:00
Dirk Klimpel 913f8a06e4
Add field `total` to device list in admin API (#8644) 2020-10-26 14:07:51 +00:00
Patrick Cloke c9c0ad5e20
Remove the deprecated Handlers object (#8494)
All handlers now available via get_*_handler() methods on the HomeServer.
2020-10-09 07:24:34 -04:00
Richard van der Hoff f31f8e6319
Remove stream ordering from Metadata dict (#8452)
There's no need for it to be in the dict as well as the events table. Instead,
we store it in a separate attribute in the EventInternalMetadata object, and
populate that on load.

This means that we can rely on it being correctly populated for any event which
has been persited to the database.
2020-10-05 14:43:14 +01:00
Erik Johnston 7941372ec8
Make token serializing/deserializing async (#8427)
The idea is that in future tokens will encode a mapping of instance to position. However, we don't want to include the full instance name in the string representation, so instead we'll have a mapping between instance name and an immutable integer ID in the DB that we can use instead. We'll then do the lookup when we serialize/deserialize the token (we could alternatively pass around an `Instance` type that includes both the name and ID, but that turns out to be a lot more invasive).
2020-09-30 20:29:19 +01:00
Erik Johnston ea70f1c362
Various clean ups to room stream tokens. (#8423) 2020-09-29 21:48:33 +01:00
Dirk Klimpel 4da01f9c61
Admin API for reported events (#8217)
Add an admin API to read entries of table `event_reports`. API: `GET /_synapse/admin/v1/event_reports`
2020-09-22 18:15:04 +01:00
Dirk Klimpel d688b4bafc
Admin API for querying rooms where a user is a member (#8306)
Add a new admin API `GET /_synapse/admin/v1/users/<user_id>/joined_rooms` to
list all rooms where a user is a member.
2020-09-18 15:26:36 +01:00
Patrick Cloke 8a4a4186de
Simplify super() calls to Python 3 syntax. (#8344)
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Patrick Cloke c3c9732c53
Use admin_patterns for all admin APIs. (#8331)
This reduces duplication of the admin prefix in regular expressions.
2020-09-17 07:04:15 -04:00
Manuel Stahl 97962ad17b
Search in columns 'name' and 'displayname' in the admin users endpoint (#7377)
* Search in columns 'name' and 'displayname' in the admin users endpoint

Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
2020-08-25 14:18:14 +01:00
Patrick Cloke e259d63f73
Stop shadow-banned users from sending invites. (#8095) 2020-08-20 15:07:42 -04:00
Erik Johnston a7bdf98d01
Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00