Commit Graph

2068 Commits (8ca120df7c2964c73390dbfab9ceba03f76eadfe)

Author SHA1 Message Date
Patrick Cloke 79bfe966e0
Improve error checking for OIDC/SAML mapping providers (#8774)
Checks that the localpart returned by mapping providers for SAML and
OIDC are valid before registering new users.

Extends the OIDC tests for existing users and invalid data.
2020-11-19 14:25:17 -05:00
Richard van der Hoff 950bb0305f
Consistently use room_id from federation request body (#8776)
* Consistently use room_id from federation request body

Some federation APIs have a redundant `room_id` path param (see
https://github.com/matrix-org/matrix-doc/issues/2330). We should make sure we
consistently use either the path param or the body param, and the body param is
easier.

* Kill off some references to "context"

Once upon a time, "rooms" were known as "contexts". I think this kills of the
last references to "contexts".
2020-11-19 10:05:33 +00:00
Erik Johnston 03e392f787
Fix tests on develop (#8777)
This was broken due to #8617 and #8761.
2020-11-18 15:43:11 +00:00
Richard van der Hoff deff8f628d
Merge pull request #8761 from matrix-org/rav/test_request_rendering
Make `make_request` actually render the request
2020-11-17 15:17:04 +00:00
Patrick Cloke ee382025b0
Abstract shared SSO code. (#8765)
De-duplicates code between the SAML and OIDC implementations.
2020-11-17 09:46:23 -05:00
Erik Johnston f737368a26
Add admin API for logging in as a user (#8617) 2020-11-17 10:51:25 +00:00
Richard van der Hoff acfe3b3065 Remove redundant `HomeserverTestCase.render` 2020-11-16 18:24:08 +00:00
Richard van der Hoff be8fa65d0b Remove redundant calls to `render()` 2020-11-16 18:24:08 +00:00
Richard van der Hoff 129ae841e5 Make `make_request` actually render the request
remove the stubbing out of `request.process`, so that `requestReceived` also renders the request via the appropriate resource.

Replace render() with a stub for now.
2020-11-16 18:24:00 +00:00
Richard van der Hoff 1f41422c98 Fix the URL in the URL preview tests
the preview resource is mointed at preview_url, not url_preview
2020-11-16 18:24:00 +00:00
Richard van der Hoff 3dc1871219
Merge pull request #8757 from matrix-org/rav/pass_site_to_make_request
Pass a Site into `make_request`
2020-11-16 18:22:24 +00:00
Richard van der Hoff f125895475
Move `wait_until_result` into `FakeChannel` (#8758)
FakeChannel has everything we need, and this more accurately models the real
flow.
2020-11-16 18:21:47 +00:00
Richard van der Hoff c3e3552ec4 fixup test 2020-11-16 15:51:47 +00:00
Richard van der Hoff bebfb9a97b
Merge branch 'develop' into rav/pass_site_to_make_request 2020-11-16 15:22:40 +00:00
Richard van der Hoff 791d7cd6f0
Rename `create_test_json_resource` to `create_test_resource` (#8759)
The root resource isn't necessarily a JsonResource, so rename this method
accordingly, and update a couple of test classes to use the method rather than
directly manipulating self.resource.
2020-11-16 14:45:52 +00:00
Richard van der Hoff ebc405446e
Add a `custom_headers` param to `make_request` (#8760)
Some tests want to set some custom HTTP request headers, so provide a way to do
that before calling requestReceived().
2020-11-16 14:45:22 +00:00
Richard van der Hoff cfd895a22e use global make_request() directly where we have a custom Resource
Where we want to render a request against a specific Resource, call the global
make_request() function rather than the one in HomeserverTestCase, allowing us
to pass in an appropriate `Site`.
2020-11-15 23:09:03 +00:00
Richard van der Hoff 70c0d47989 fix dict handling for make_request() 2020-11-15 23:09:03 +00:00
Richard van der Hoff 9debe657a3 pass a Site into make_request 2020-11-15 23:09:03 +00:00
Richard van der Hoff d3523e3e97 pass a Site into RestHelper 2020-11-15 23:09:03 +00:00
Dirk Klimpel c3119d1536
Add an admin API for users' media statistics (#8700)
Add `GET /_synapse/admin/v1/statistics/users/media` to get statisics about local media usage by users.
Related to #6094
It is the first API for statistics.
Goal is to avoid/reduce usage of sql queries like [Wiki analyzing Synapse](https://github.com/matrix-org/synapse/wiki/SQL-for-analyzing-Synapse-PostgreSQL-database-stats)

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-11-05 18:59:12 +00:00
Dirk Klimpel e4676bd877
Add `displayname` to Shared-Secret Registration for admins (#8722)
Add `displayname` to Shared-Secret Registration for admins to `POST /_synapse/admin/v1/register`
2020-11-05 13:55:45 +00:00
Andrew Morgan 6abb1ad0be
Consolidate purge table lists to prevent desyncronisation (#8713)
I idly noticed that these lists were out of sync with each other, causing us to miss a table in a test case (`local_invites`). Let's consolidate this list instead to prevent this from happening in the future.
2020-11-04 11:26:05 +00:00
Erik Johnston 243d427fbc
Block clients from sending server ACLs that lock the local server out. (#8708)
Fixes #4042
2020-11-03 12:13:48 +00:00
Richard van der Hoff cbc82aa09f
Implement and use an @lru_cache decorator (#8595)
We don't always need the full power of a DeferredCache.
2020-10-30 11:43:17 +00:00
Patrick Cloke fd7c743445
Fail test cases if they fail to await all awaitables (#8690) 2020-10-30 07:15:07 -04:00
Erik Johnston 46f4be94b4
Fix race for concurrent downloads of remote media. (#8682)
Fixes #6755
2020-10-30 10:55:24 +00:00
Erik Johnston ef2d627015
Fix unit tests (#8689)
* Fix unit tests

* Newsfile
2020-10-29 18:21:49 +00:00
Patrick Cloke 8b42a4eefd
Gracefully handle a pending logging connection during shutdown. (#8685) 2020-10-29 12:53:57 -04:00
Erik Johnston f21e24ffc2
Add ability for access tokens to belong to one user but grant access to another user. (#8616)
We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't).

A future PR will add an API for creating such a token.

When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
2020-10-29 15:58:44 +00:00
Patrick Cloke 00b24aa545
Support generating structured logs in addition to standard logs. (#8607)
This modifies the configuration of structured logging to be usable from
the standard Python logging configuration.

This also separates the formatting of logs from the transport allowing
JSON logs to files or standard logs to sockets.
2020-10-29 07:27:37 -04:00
Erik Johnston 9a7e0d2ea6
Don't require hiredis to run unit tests (#8680) 2020-10-29 11:17:35 +00:00
Dirk Klimpel 2239813278
Add an admin APIs to allow server admins to list users' pushers (#8610)
Add an admin API `GET /_synapse/admin/v1/users/<user_id>/pushers` like https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-pushers
2020-10-28 15:02:42 +00:00
Dan Callahan aff1eb7c67
Tell Black to format code for Python 3.5 (#8664)
This allows trailing commas in multi-line arg lists.

Minor, but we might as well keep our formatting current with regard to
our minimum supported Python version.

Signed-off-by: Dan Callahan <danc@element.io>
2020-10-27 23:26:36 +00:00
Dirk Klimpel 9b7c28283a
Add admin API to list users' local media (#8647)
Add admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information of users' uploaded files.
2020-10-27 14:12:31 +00:00
Dirk Klimpel 66e6801c3e
Split admin API for reported events into a detail and a list view (#8539)
Split admin API for reported events in detail und list view.
API was introduced with #8217 in synapse v.1.21.0.

It makes the list (`GET /_synapse/admin/v1/event_reports`) less complex and provides a better overview.
The details can be queried with: `GET /_synapse/admin/v1/event_reports/<report_id>`.
It is similar to room and users API.

It is a kind of regression in `GET /_synapse/admin/v1/event_reports`.  `event_json` was removed. But the api was introduced one version before and it is an admin API (not under spec).

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-10-26 18:16:37 +00:00
Dirk Klimpel 49d72dea2a
Add an admin api to delete local media. (#8519)
Related to: #6459, #3479

Add `DELETE /_synapse/admin/v1/media/<server_name>/<media_id>` to delete
a single file from server.
2020-10-26 17:02:28 +00:00
Dirk Klimpel 4ac3a8c5dc
Fix a bug in the joined_rooms admin API (#8643)
If the user was not in any rooms then the API returned the same error
as if the user did not exist.
2020-10-26 12:25:48 -04:00
Will Hunt e8dbbcb64c
Fix get|set_type_stream_id_for_appservice store functions (#8648) 2020-10-26 10:51:33 -04:00
Dirk Klimpel 913f8a06e4
Add field `total` to device list in admin API (#8644) 2020-10-26 14:07:51 +00:00
Erik Johnston 2b7c180879
Start fewer opentracing spans (#8640)
#8567 started a span for every background process. This is good as it means all Synapse code that gets run should be in a span (unless in the sentinel logging context), but it means we generate about 15x the number of spans as we did previously.

This PR attempts to reduce that number by a) not starting one for send commands to Redis, and b) deferring starting background processes until after we're sure they're necessary.

I don't really know how much this will help.
2020-10-26 09:30:19 +00:00
Erik Johnston c850dd9a8e
Fix handling of User-Agent headers with bad utf-8. (#8632) 2020-10-23 17:12:59 +01:00
Erik Johnston db9ef792f0
Fix email notifications for invites without local state. (#8627)
This can happen if e.g. the room invited into is no longer on the
server (or if all users left the room).
2020-10-23 10:41:32 +01:00
Erik Johnston 2ac908f377
Don't instansiate Requester directly (#8614) 2020-10-22 10:11:06 +01:00
Richard van der Hoff 9146a8a691
Merge pull request #8572 from matrix-org/rav/cache_hacking/2
Push some deferred wrangling down into DeferredCache
2020-10-21 17:55:04 +01:00
Richard van der Hoff 6d3905c7c7 Add some more tests 2020-10-21 15:39:25 +01:00
Richard van der Hoff 1f4269700c Push some deferred wrangling down into DeferredCache 2020-10-21 15:39:25 +01:00
Richard van der Hoff 7b71695388 Combine the two sets of tests for CacheDescriptor 2020-10-21 15:38:29 +01:00
Will Hunt 70259d8c8c
Limit AS transactions to 100 events (#8606)
* Limit AS transactions to 100 events

* Update changelog.d/8606.feature

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Add tests

* Update synapse/appservice/scheduler.py

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2020-10-21 15:36:53 +01:00
Patrick Cloke 20a67aa70d
Separate the TCP and terse JSON formatting code. (#8587)
This should (theoretically) allow for using the TCP code with a different output type
and make it easier to use the JSON code with files / console.
2020-10-21 06:59:54 -04:00