Compare commits

...

9 Commits

Author SHA1 Message Date
Andrew Morgan e8d0853739
Generalise _maybe_store_room_on_invite (#8754)
There's a handy function called maybe_store_room_on_invite which allows us to create an entry in the rooms table for a room and its version for which we aren't joined to yet, but we can reference when ingesting events about.

This is currently used for invites where we receive some stripped state about the room and pass it down via /sync to the client, without us being in the room yet.

There is a similar requirement for knocking, where we will eventually do the same thing, and need an entry in the rooms table as well. Thus, reusing this function works, however its name needs to be generalised a bit.

Separated out from #6739.
2020-11-13 16:24:04 +00:00
Erik Johnston 34226ec761 Fix changelog 2020-11-13 14:14:09 +00:00
Erik Johnston 0a5185495b Fix changelog 2020-11-13 14:06:52 +00:00
Erik Johnston 4a54b821bb 1.23.0rc1 2020-11-13 13:59:58 +00:00
Dirk Klimpel 023f791143
Migrate documentation `docs/admin_api/event_reports` to markdown (#8742)
Related to #8714. `event_reports.rst` was introduced in Synapse 1.21.0.
2020-11-13 13:57:55 +00:00
Marcus Schopen 68fc0dcb5a
SAML: add <mdui:UIInfo> element examples (#8718)
add some mdui:UIInfo element examples for saml2_config in homeserver.yaml
2020-11-13 12:07:50 +00:00
chagai95 69147ed158
Updating README.rst (#8746)
Minor corrections and advice... Should help beginners.
2020-11-13 12:07:09 +00:00
Erik Johnston 427ede619f
Add metrics for tracking 3PID /requestToken requests. (#8712)
The main use case is to see how many requests are being made, and how
many are second/third/etc attempts. If there are large number of retries
then that likely indicates a delivery problem.
2020-11-13 12:03:51 +00:00
Erik Johnston 1b15a3d92c
Fix port script so that it can be run again after failure. (#8755)
If the script fails (or is CTRL-C'ed) between porting some of the events table and copying of the sequences then the port script will immediately die if run again due to the postgres DB having inconsistencies between sequences and tables.

The fix is to move the porting of sequences to before porting the tables, so that there is never a period where the Postgres DB is inconsistent. To do that we need to change how we port the sequences so that it calculates the values from the SQLite DB rather than the Postgres DB.

Fixes #8619
2020-11-13 11:53:51 +00:00
79 changed files with 417 additions and 279 deletions

View File

@ -1,3 +1,89 @@
Synapse 1.23.0rc1 (2020-11-13)
==============================
Features
--------
- Add a push rule that highlights when a jitsi conference is created in a room. ([\#8286](https://github.com/matrix-org/synapse/issues/8286))
- Add an admin api to delete a single file or files that were not used for a defined time from server. Contributed by @dklimpel. ([\#8519](https://github.com/matrix-org/synapse/issues/8519))
- Split admin API for reported events (`GET /_synapse/admin/v1/event_reports`) into detail and list endpoints. This is a breaking change to #8217 which was introduced in Synapse v1.21.0. Those who already use this API should check their scripts. Contributed by @dklimpel. ([\#8539](https://github.com/matrix-org/synapse/issues/8539))
- Support generating structured logs via the standard logging configuration. ([\#8607](https://github.com/matrix-org/synapse/issues/8607), [\#8685](https://github.com/matrix-org/synapse/issues/8685))
- Add an admin API to allow server admins to list users' pushers. Contributed by @dklimpel. ([\#8610](https://github.com/matrix-org/synapse/issues/8610), [\#8689](https://github.com/matrix-org/synapse/issues/8689))
- Add an admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information about uploaded media. Contributed by @dklimpel. ([\#8647](https://github.com/matrix-org/synapse/issues/8647))
- Add an admin API for local user media statistics. Contributed by @dklimpel. ([\#8700](https://github.com/matrix-org/synapse/issues/8700))
- Add `displayname` to Shared-Secret Registration for admins. ([\#8722](https://github.com/matrix-org/synapse/issues/8722))
Bugfixes
--------
- Fix fetching of E2E cross signing keys over federation when only one of the master key and device signing key is cached already. ([\#8455](https://github.com/matrix-org/synapse/issues/8455))
- Fix a bug where Synapse would blindly forward bad responses from federation to clients when retrieving profile information. ([\#8580](https://github.com/matrix-org/synapse/issues/8580))
- Fix a bug where the account validity endpoint would silently fail if the user ID did not have an expiration time. It now returns a 400 error. ([\#8620](https://github.com/matrix-org/synapse/issues/8620))
- Fix email notifications for invites without local state. ([\#8627](https://github.com/matrix-org/synapse/issues/8627))
- Fix handling of invalid group IDs to return a 400 rather than log an exception and return a 500. ([\#8628](https://github.com/matrix-org/synapse/issues/8628))
- Fix handling of User-Agent headers that are invalid UTF-8, which caused user agents of users to not get correctly recorded. ([\#8632](https://github.com/matrix-org/synapse/issues/8632))
- Fix a bug in the `joined_rooms` admin API if the user has never joined any rooms. The bug was introduced, along with the API, in v1.21.0. ([\#8643](https://github.com/matrix-org/synapse/issues/8643))
- Fix exception during handling multiple concurrent requests for remote media when using multiple media repositories. ([\#8682](https://github.com/matrix-org/synapse/issues/8682))
- Fix bug that prevented Synapse from recovering after losing connection to the database. ([\#8726](https://github.com/matrix-org/synapse/issues/8726))
- Fix bug where the `/_synapse/admin/v1/send_server_notice` API could send notices to non-notice rooms. ([\#8728](https://github.com/matrix-org/synapse/issues/8728))
- Fix PostgreSQL port script fails when DB has no backfilled events. Broke in v1.21.0. ([\#8729](https://github.com/matrix-org/synapse/issues/8729))
- Fix PostgreSQL port script to correctly handle foreign key constraints. Broke in v1.21.0. ([\#8730](https://github.com/matrix-org/synapse/issues/8730))
- Fix PostgreSQL port script so that it can be run again after a failure. Broke in v1.21.0. ([\#8755](https://github.com/matrix-org/synapse/issues/8755))
Improved Documentation
----------------------
- Instructions for Azure AD in the OpenID Connect documentation. Contributed by peterk. ([\#8582](https://github.com/matrix-org/synapse/issues/8582))
- Improve the sample configuration for single sign-on providers. ([\#8635](https://github.com/matrix-org/synapse/issues/8635))
- Fix the filepath of Dex's example config and the link to Dex's Getting Started guide in the OpenID Connect docs. ([\#8657](https://github.com/matrix-org/synapse/issues/8657))
- Note support for Python 3.9. ([\#8665](https://github.com/matrix-org/synapse/issues/8665))
- Minor updates to docs on running tests. ([\#8666](https://github.com/matrix-org/synapse/issues/8666))
- Interlink prometheus/grafana documentation. ([\#8667](https://github.com/matrix-org/synapse/issues/8667))
- Notes on SSO logins and media_repository worker. ([\#8701](https://github.com/matrix-org/synapse/issues/8701))
- Document experimental support for running multiple event persisters. ([\#8706](https://github.com/matrix-org/synapse/issues/8706))
- Add information regarding the various sources of, and expected contributions to, Synapse's documentation to `CONTRIBUTING.md`. ([\#8714](https://github.com/matrix-org/synapse/issues/8714))
- Migrate documentation `docs/admin_api/event_reports` to markdown. ([\#8742](https://github.com/matrix-org/synapse/issues/8742))
- Add some helpful hints to the README for new Synapse developers. Contributed by @chagai95. ([\#8746](https://github.com/matrix-org/synapse/issues/8746))
Internal Changes
----------------
- Optimise `/createRoom` with multiple invited users. ([\#8559](https://github.com/matrix-org/synapse/issues/8559))
- Implement and use an @lru_cache decorator. ([\#8595](https://github.com/matrix-org/synapse/issues/8595))
- Don't instansiate Requester directly. ([\#8614](https://github.com/matrix-org/synapse/issues/8614))
- Type hints for `RegistrationStore`. ([\#8615](https://github.com/matrix-org/synapse/issues/8615))
- Change schema to support access tokens belonging to one user but granting access to another. ([\#8616](https://github.com/matrix-org/synapse/issues/8616))
- Remove unused OPTIONS handlers. ([\#8621](https://github.com/matrix-org/synapse/issues/8621))
- Run `mypy` as part of the lint.sh script. ([\#8633](https://github.com/matrix-org/synapse/issues/8633))
- Correct Synapse's PyPI package name in the OpenID Connect installation instructions. ([\#8634](https://github.com/matrix-org/synapse/issues/8634))
- Catch exceptions during initialization of `password_providers`. Contributed by Nicolai Søborg. ([\#8636](https://github.com/matrix-org/synapse/issues/8636))
- Fix typos and spelling errors in the code. ([\#8639](https://github.com/matrix-org/synapse/issues/8639))
- Reduce number of OpenTracing spans started. ([\#8640](https://github.com/matrix-org/synapse/issues/8640), [\#8668](https://github.com/matrix-org/synapse/issues/8668), [\#8670](https://github.com/matrix-org/synapse/issues/8670))
- Add field `total` to device list in admin API. ([\#8644](https://github.com/matrix-org/synapse/issues/8644))
- Add more type hints to the application services code. ([\#8655](https://github.com/matrix-org/synapse/issues/8655), [\#8693](https://github.com/matrix-org/synapse/issues/8693))
- Tell Black to format code for Python 3.5. ([\#8664](https://github.com/matrix-org/synapse/issues/8664))
- Don't pull event from DB when handling replication traffic. ([\#8669](https://github.com/matrix-org/synapse/issues/8669))
- Abstract some invite-related code in preparation for landing knocking. ([\#8671](https://github.com/matrix-org/synapse/issues/8671), [\#8688](https://github.com/matrix-org/synapse/issues/8688))
- Clarify representation of events in logfiles. ([\#8679](https://github.com/matrix-org/synapse/issues/8679))
- Don't require `hiredis` package to be installed to run unit tests. ([\#8680](https://github.com/matrix-org/synapse/issues/8680))
- Fix typing info on cache call signature to accept `on_invalidate`. ([\#8684](https://github.com/matrix-org/synapse/issues/8684))
- Fail tests if they do not await coroutines. ([\#8690](https://github.com/matrix-org/synapse/issues/8690))
- Improve start time by adding an index to `e2e_cross_signing_keys.stream_id`. ([\#8694](https://github.com/matrix-org/synapse/issues/8694))
- Re-organize the structured logging code to separate the TCP transport handling from the JSON formatting. ([\#8697](https://github.com/matrix-org/synapse/issues/8697))
- Use Python 3.8 in Docker images by default. ([\#8698](https://github.com/matrix-org/synapse/issues/8698))
- Remove the "draft" status of the Room Details Admin API. ([\#8702](https://github.com/matrix-org/synapse/issues/8702))
- Improve the error returned when a non-string displayname or avatar_url is used when updating a user's profile. ([\#8705](https://github.com/matrix-org/synapse/issues/8705))
- Block attempts by clients to send server ACLs, or redactions of server ACLs, that would result in the local server being blocked from the room. ([\#8708](https://github.com/matrix-org/synapse/issues/8708))
- Add metrics the allow the local sysadmin to track 3PID `/requestToken` requests. ([\#8712](https://github.com/matrix-org/synapse/issues/8712))
- Consolidate duplicated lists of purged tables that are checked in tests. ([\#8713](https://github.com/matrix-org/synapse/issues/8713))
- Add some `mdui:UIInfo` element examples for `saml2_config` in the homeserver config. ([\#8718](https://github.com/matrix-org/synapse/issues/8718))
- Improve the error message returned when a remote server incorrectly sets the `Content-Type` header in response to a JSON request. ([\#8719](https://github.com/matrix-org/synapse/issues/8719))
- Speed up repeated state resolutions on the same room by caching event ID to auth event ID lookups. ([\#8752](https://github.com/matrix-org/synapse/issues/8752))
Synapse 1.22.1 (2020-10-30)
===========================

View File

@ -261,18 +261,22 @@ to install using pip and a virtualenv::
pip install -e ".[all,test]"
This will run a process of downloading and installing all the needed
dependencies into a virtual env.
dependencies into a virtual env. If any dependencies fail to install,
try installing the failing modules individually::
Once this is done, you may wish to run Synapse's unit tests, to
check that everything is installed as it should be::
pip install -e "module-name"
Once this is done, you may wish to run Synapse's unit tests to
check that everything is installed correctly::
python -m twisted.trial tests
This should end with a 'PASSED' result::
This should end with a 'PASSED' result (note that exact numbers will
differ)::
Ran 1266 tests in 643.930s
Ran 1337 tests in 716.064s
PASSED (skips=15, successes=1251)
PASSED (skips=15, successes=1322)
Running the Integration Tests
=============================

View File

@ -1 +0,0 @@
Add a push rule that highlights when a jitsi conference is created in a room.

View File

@ -1 +0,0 @@
Fix fetching of E2E cross signing keys over federation when only one of the master key and device signing key is cached already.

View File

@ -1 +0,0 @@
Add an admin api to delete a single file or files were not used for a defined time from server. Contributed by @dklimpel.

View File

@ -1 +0,0 @@
Split admin API for reported events (`GET /_synapse/admin/v1/event_reports`) into detail and list endpoints. This is a breaking change to #8217 which was introduced in Synapse v1.21.0. Those who already use this API should check their scripts. Contributed by @dklimpel.

View File

@ -1 +0,0 @@
Optimise `/createRoom` with multiple invited users.

View File

@ -1 +0,0 @@
Fix a bug where Synapse would blindly forward bad responses from federation to clients when retrieving profile information.

View File

@ -1 +0,0 @@
Instructions for Azure AD in the OpenID Connect documentation. Contributed by peterk.

View File

@ -1 +0,0 @@
Implement and use an @lru_cache decorator.

View File

@ -1 +0,0 @@
Support generating structured logs via the standard logging configuration.

View File

@ -1 +0,0 @@
Add an admin APIs to allow server admins to list users' pushers. Contributed by @dklimpel.

View File

@ -1 +0,0 @@
Don't instansiate Requester directly.

View File

@ -1 +0,0 @@
Type hints for `RegistrationStore`.

View File

@ -1 +0,0 @@
Change schema to support access tokens belonging to one user but granting access to another.

View File

@ -1 +0,0 @@
Fix a bug where the account validity endpoint would silently fail if the user ID did not have an expiration time. It now returns a 400 error.

View File

@ -1 +0,0 @@
Remove unused OPTIONS handlers.

View File

@ -1 +0,0 @@
Fix email notifications for invites without local state.

View File

@ -1 +0,0 @@
Fix handling of invalid group IDs to return a 400 rather than log an exception and return a 500.

View File

@ -1 +0,0 @@
Fix handling of User-Agent headers that are invalid UTF-8, which caused user agents of users to not get correctly recorded.

View File

@ -1 +0,0 @@
Run `mypy` as part of the lint.sh script.

View File

@ -1 +0,0 @@
Correct Synapse's PyPI package name in the OpenID Connect installation instructions.

View File

@ -1 +0,0 @@
Improve the sample configuration for single sign-on providers.

View File

@ -1 +0,0 @@
Catch exceptions during initialization of `password_providers`. Contributed by Nicolai Søborg.

View File

@ -1 +0,0 @@
Fix typos and spelling errors in the code.

View File

@ -1 +0,0 @@
Reduce number of OpenTracing spans started.

View File

@ -1 +0,0 @@
Fix a bug in the `joined_rooms` admin API if the user has never joined any rooms. The bug was introduced, along with the API, in v1.21.0.

View File

@ -1 +0,0 @@
Add field `total` to device list in admin API.

View File

@ -1 +0,0 @@
Add an admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information about uploaded media. Contributed by @dklimpel.

View File

@ -1 +0,0 @@
Add more type hints to the application services code.

View File

@ -1 +0,0 @@
Fix the filepath of Dex's example config and the link to Dex's Getting Started guide in the OpenID Connect docs.

View File

@ -1 +0,0 @@
Tell Black to format code for Python 3.5.

View File

@ -1 +0,0 @@
Note support for Python 3.9.

View File

@ -1 +0,0 @@
Minor updates to docs on running tests.

View File

@ -1 +0,0 @@
Interlink prometheus/grafana documentation.

View File

@ -1 +0,0 @@
Reduce number of OpenTracing spans started.

View File

@ -1 +0,0 @@
Don't pull event from DB when handling replication traffic.

View File

@ -1 +0,0 @@
Reduce number of OpenTracing spans started.

View File

@ -1 +0,0 @@
Abstract some invite-related code in preparation for landing knocking.

View File

@ -1 +0,0 @@
Clarify representation of events in logfiles.

View File

@ -1 +0,0 @@
Don't require `hiredis` package to be installed to run unit tests.

View File

@ -1 +0,0 @@
Fix exception during handling multiple concurrent requests for remote media when using multiple media repositories.

View File

@ -1 +0,0 @@
Fix typing info on cache call signature to accept `on_invalidate`.

View File

@ -1 +0,0 @@
Support generating structured logs via the standard logging configuration.

View File

@ -1 +0,0 @@
Abstract some invite-related code in preparation for landing knocking.

View File

@ -1 +0,0 @@
Add an admin APIs to allow server admins to list users' pushers. Contributed by @dklimpel.

View File

@ -1 +0,0 @@
Fail tests if they do not await coroutines.

View File

@ -1 +0,0 @@
Add more type hints to the application services code.

View File

@ -1 +0,0 @@
Improve start time by adding an index to `e2e_cross_signing_keys.stream_id`.

View File

@ -1 +0,0 @@
Re-organize the structured logging code to separate the TCP transport handling from the JSON formatting.

View File

@ -1 +0,0 @@
Use Python 3.8 in Docker images by default.

View File

@ -1 +0,0 @@
Add an admin API for local user media statistics. Contributed by @dklimpel.

View File

@ -1 +0,0 @@
Notes on SSO logins and media_repository worker.

View File

@ -1 +0,0 @@
Remove the "draft" status of the Room Details Admin API.

View File

@ -1 +0,0 @@
Improve the error returned when a non-string displayname or avatar_url is used when updating a user's profile.

View File

@ -1 +0,0 @@
Document experimental support for running multiple event persisters.

View File

@ -1 +0,0 @@
Block attempts by clients to send server ACLs, or redactions of server ACLs, that would result in the local server being blocked from the room.

View File

@ -1 +0,0 @@
Consolidate duplicated lists of purged tables that are checked in tests.

View File

@ -1 +0,0 @@
Add information regarding the various sources of, and expected contributions to, Synapse's documentation to `CONTRIBUTING.md`.

View File

@ -1 +0,0 @@
Improve the error message returned when a remote server incorrectly sets the `Content-Type` header in response to a JSON request.

View File

@ -1 +0,0 @@
Add `displayname` to Shared-Secret Registration for admins.

View File

@ -1 +0,0 @@
Fix bug where Synapse would not recover after losing connection to the database.

View File

@ -1 +0,0 @@
Fix bug where the `/_synapse/admin/v1/send_server_notice` API could send notices to non-notice rooms.

View File

@ -1 +0,0 @@
Fix port script fails when DB has no backfilled events. Broke in v1.21.0.

View File

@ -1 +0,0 @@
Fix port script to correctly handle foreign key constraints. Broke in v1.21.0.

View File

@ -1 +0,0 @@
Speed up repeated state resolutions on the same room by caching event ID to auth event ID lookups.

1
changelog.d/8754.misc Normal file
View File

@ -0,0 +1 @@
Generalise `RoomStore.maybe_store_room_on_invite` to handle other, non-invite membership events.

View File

@ -0,0 +1,172 @@
# Show reported events
This API returns information about reported events.
The api is:
```
GET /_synapse/admin/v1/event_reports?from=0&limit=10
```
To use it, you will need to authenticate by providing an `access_token` for a
server admin: see [README.rst](README.rst).
It returns a JSON body like the following:
```json
{
"event_reports": [
{
"event_id": "$bNUFCwGzWca1meCGkjp-zwslF-GfVcXukvRLI1_FaVY",
"id": 2,
"reason": "foo",
"score": -100,
"received_ts": 1570897107409,
"canonical_alias": "#alias1:matrix.org",
"room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
"name": "Matrix HQ",
"sender": "@foobar:matrix.org",
"user_id": "@foo:matrix.org"
},
{
"event_id": "$3IcdZsDaN_En-S1DF4EMCy3v4gNRKeOJs8W5qTOKj4I",
"id": 3,
"reason": "bar",
"score": -100,
"received_ts": 1598889612059,
"canonical_alias": "#alias2:matrix.org",
"room_id": "!eGvUQuTCkHGVwNMOjv:matrix.org",
"name": "Your room name here",
"sender": "@foobar:matrix.org",
"user_id": "@bar:matrix.org"
}
],
"next_token": 2,
"total": 4
}
```
To paginate, check for `next_token` and if present, call the endpoint again with `from`
set to the value of `next_token`. This will return a new page.
If the endpoint does not return a `next_token` then there are no more reports to
paginate through.
**URL parameters:**
* `limit`: integer - Is optional but is used for pagination, denoting the maximum number
of items to return in this call. Defaults to `100`.
* `from`: integer - Is optional but used for pagination, denoting the offset in the
returned results. This should be treated as an opaque value and not explicitly set to
anything other than the return value of `next_token` from a previous call. Defaults to `0`.
* `dir`: string - Direction of event report order. Whether to fetch the most recent
first (`b`) or the oldest first (`f`). Defaults to `b`.
* `user_id`: string - Is optional and filters to only return users with user IDs that
contain this value. This is the user who reported the event and wrote the reason.
* `room_id`: string - Is optional and filters to only return rooms with room IDs that
contain this value.
**Response**
The following fields are returned in the JSON response body:
* `id`: integer - ID of event report.
* `received_ts`: integer - The timestamp (in milliseconds since the unix epoch) when this
report was sent.
* `room_id`: string - The ID of the room in which the event being reported is located.
* `name`: string - The name of the room.
* `event_id`: string - The ID of the reported event.
* `user_id`: string - This is the user who reported the event and wrote the reason.
* `reason`: string - Comment made by the `user_id` in this report. May be blank.
* `score`: integer - Content is reported based upon a negative score, where -100 is
"most offensive" and 0 is "inoffensive".
* `sender`: string - This is the ID of the user who sent the original message/event that
was reported.
* `canonical_alias`: string - The canonical alias of the room. `null` if the room does not
have a canonical alias set.
* `next_token`: integer - Indication for pagination. See above.
* `total`: integer - Total number of event reports related to the query
(`user_id` and `room_id`).
# Show details of a specific event report
This API returns information about a specific event report.
The api is:
```
GET /_synapse/admin/v1/event_reports/<report_id>
```
To use it, you will need to authenticate by providing an `access_token` for a
server admin: see [README.rst](README.rst).
It returns a JSON body like the following:
```jsonc
{
"event_id": "$bNUFCwGzWca1meCGkjp-zwslF-GfVcXukvRLI1_FaVY",
"event_json": {
"auth_events": [
"$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M",
"$oggsNXxzPFRE3y53SUNd7nsj69-QzKv03a1RucHu-ws"
],
"content": {
"body": "matrix.org: This Week in Matrix",
"format": "org.matrix.custom.html",
"formatted_body": "<strong>matrix.org</strong>:<br><a href=\"https://matrix.org/blog/\"><strong>This Week in Matrix</strong></a>",
"msgtype": "m.notice"
},
"depth": 546,
"hashes": {
"sha256": "xK1//xnmvHJIOvbgXlkI8eEqdvoMmihVDJ9J4SNlsAw"
},
"origin": "matrix.org",
"origin_server_ts": 1592291711430,
"prev_events": [
"$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M"
],
"prev_state": [],
"room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
"sender": "@foobar:matrix.org",
"signatures": {
"matrix.org": {
"ed25519:a_JaEG": "cs+OUKW/iHx5pEidbWxh0UiNNHwe46Ai9LwNz+Ah16aWDNszVIe2gaAcVZfvNsBhakQTew51tlKmL2kspXk/Dg"
}
},
"type": "m.room.message",
"unsigned": {
"age_ts": 1592291711430,
}
},
"id": <report_id>,
"reason": "foo",
"score": -100,
"received_ts": 1570897107409,
"canonical_alias": "#alias1:matrix.org",
"room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
"name": "Matrix HQ",
"sender": "@foobar:matrix.org",
"user_id": "@foo:matrix.org"
}
```
**URL parameters:**
* `report_id`: string - The ID of the event report.
**Response**
The following fields are returned in the JSON response body:
* `id`: integer - ID of event report.
* `received_ts`: integer - The timestamp (in milliseconds since the unix epoch) when this
report was sent.
* `room_id`: string - The ID of the room in which the event being reported is located.
* `name`: string - The name of the room.
* `event_id`: string - The ID of the reported event.
* `user_id`: string - This is the user who reported the event and wrote the reason.
* `reason`: string - Comment made by the `user_id` in this report. May be blank.
* `score`: integer - Content is reported based upon a negative score, where -100 is
"most offensive" and 0 is "inoffensive".
* `sender`: string - This is the ID of the user who sent the original message/event that
was reported.
* `canonical_alias`: string - The canonical alias of the room. `null` if the room does not
have a canonical alias set.
* `event_json`: object - Details of the original event that was reported.

View File

@ -1,165 +0,0 @@
Show reported events
====================
This API returns information about reported events.
The api is::
GET /_synapse/admin/v1/event_reports?from=0&limit=10
To use it, you will need to authenticate by providing an ``access_token`` for a
server admin: see `README.rst <README.rst>`_.
It returns a JSON body like the following:
.. code:: jsonc
{
"event_reports": [
{
"event_id": "$bNUFCwGzWca1meCGkjp-zwslF-GfVcXukvRLI1_FaVY",
"id": 2,
"reason": "foo",
"score": -100,
"received_ts": 1570897107409,
"canonical_alias": "#alias1:matrix.org",
"room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
"name": "Matrix HQ",
"sender": "@foobar:matrix.org",
"user_id": "@foo:matrix.org"
},
{
"event_id": "$3IcdZsDaN_En-S1DF4EMCy3v4gNRKeOJs8W5qTOKj4I",
"id": 3,
"reason": "bar",
"score": -100,
"received_ts": 1598889612059,
"canonical_alias": "#alias2:matrix.org",
"room_id": "!eGvUQuTCkHGVwNMOjv:matrix.org",
"name": "Your room name here",
"sender": "@foobar:matrix.org",
"user_id": "@bar:matrix.org"
}
],
"next_token": 2,
"total": 4
}
To paginate, check for ``next_token`` and if present, call the endpoint again
with ``from`` set to the value of ``next_token``. This will return a new page.
If the endpoint does not return a ``next_token`` then there are no more
reports to paginate through.
**URL parameters:**
- ``limit``: integer - Is optional but is used for pagination,
denoting the maximum number of items to return in this call. Defaults to ``100``.
- ``from``: integer - Is optional but used for pagination,
denoting the offset in the returned results. This should be treated as an opaque value and
not explicitly set to anything other than the return value of ``next_token`` from a previous call.
Defaults to ``0``.
- ``dir``: string - Direction of event report order. Whether to fetch the most recent first (``b``) or the
oldest first (``f``). Defaults to ``b``.
- ``user_id``: string - Is optional and filters to only return users with user IDs that contain this value.
This is the user who reported the event and wrote the reason.
- ``room_id``: string - Is optional and filters to only return rooms with room IDs that contain this value.
**Response**
The following fields are returned in the JSON response body:
- ``id``: integer - ID of event report.
- ``received_ts``: integer - The timestamp (in milliseconds since the unix epoch) when this report was sent.
- ``room_id``: string - The ID of the room in which the event being reported is located.
- ``name``: string - The name of the room.
- ``event_id``: string - The ID of the reported event.
- ``user_id``: string - This is the user who reported the event and wrote the reason.
- ``reason``: string - Comment made by the ``user_id`` in this report. May be blank.
- ``score``: integer - Content is reported based upon a negative score, where -100 is "most offensive" and 0 is "inoffensive".
- ``sender``: string - This is the ID of the user who sent the original message/event that was reported.
- ``canonical_alias``: string - The canonical alias of the room. ``null`` if the room does not have a canonical alias set.
- ``next_token``: integer - Indication for pagination. See above.
- ``total``: integer - Total number of event reports related to the query (``user_id`` and ``room_id``).
Show details of a specific event report
=======================================
This API returns information about a specific event report.
The api is::
GET /_synapse/admin/v1/event_reports/<report_id>
To use it, you will need to authenticate by providing an ``access_token`` for a
server admin: see `README.rst <README.rst>`_.
It returns a JSON body like the following:
.. code:: jsonc
{
"event_id": "$bNUFCwGzWca1meCGkjp-zwslF-GfVcXukvRLI1_FaVY",
"event_json": {
"auth_events": [
"$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M",
"$oggsNXxzPFRE3y53SUNd7nsj69-QzKv03a1RucHu-ws"
],
"content": {
"body": "matrix.org: This Week in Matrix",
"format": "org.matrix.custom.html",
"formatted_body": "<strong>matrix.org</strong>:<br><a href=\"https://matrix.org/blog/\"><strong>This Week in Matrix</strong></a>",
"msgtype": "m.notice"
},
"depth": 546,
"hashes": {
"sha256": "xK1//xnmvHJIOvbgXlkI8eEqdvoMmihVDJ9J4SNlsAw"
},
"origin": "matrix.org",
"origin_server_ts": 1592291711430,
"prev_events": [
"$YK4arsKKcc0LRoe700pS8DSjOvUT4NDv0HfInlMFw2M"
],
"prev_state": [],
"room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
"sender": "@foobar:matrix.org",
"signatures": {
"matrix.org": {
"ed25519:a_JaEG": "cs+OUKW/iHx5pEidbWxh0UiNNHwe46Ai9LwNz+Ah16aWDNszVIe2gaAcVZfvNsBhakQTew51tlKmL2kspXk/Dg"
}
},
"type": "m.room.message",
"unsigned": {
"age_ts": 1592291711430,
}
},
"id": <report_id>,
"reason": "foo",
"score": -100,
"received_ts": 1570897107409,
"canonical_alias": "#alias1:matrix.org",
"room_id": "!ERAgBpSOcCCuTJqQPk:matrix.org",
"name": "Matrix HQ",
"sender": "@foobar:matrix.org",
"user_id": "@foo:matrix.org"
}
**URL parameters:**
- ``report_id``: string - The ID of the event report.
**Response**
The following fields are returned in the JSON response body:
- ``id``: integer - ID of event report.
- ``received_ts``: integer - The timestamp (in milliseconds since the unix epoch) when this report was sent.
- ``room_id``: string - The ID of the room in which the event being reported is located.
- ``name``: string - The name of the room.
- ``event_id``: string - The ID of the reported event.
- ``user_id``: string - This is the user who reported the event and wrote the reason.
- ``reason``: string - Comment made by the ``user_id`` in this report. May be blank.
- ``score``: integer - Content is reported based upon a negative score, where -100 is "most offensive" and 0 is "inoffensive".
- ``sender``: string - This is the ID of the user who sent the original message/event that was reported.
- ``canonical_alias``: string - The canonical alias of the room. ``null`` if the room does not have a canonical alias set.
- ``event_json``: object - Details of the original event that was reported.

View File

@ -1560,6 +1560,28 @@ saml2_config:
#description: ["My awesome SP", "en"]
#name: ["Test SP", "en"]
#ui_info:
# display_name:
# - lang: en
# text: "Display Name is the descriptive name of your service."
# description:
# - lang: en
# text: "Description should be a short paragraph explaining the purpose of the service."
# information_url:
# - lang: en
# text: "https://example.com/terms-of-service"
# privacy_statement_url:
# - lang: en
# text: "https://example.com/privacy-policy"
# keywords:
# - lang: en
# text: ["Matrix", "Element"]
# logo:
# - lang: en
# text: "https://example.com/logo.svg"
# width: "200"
# height: "80"
#organization:
# name: Example com
# display_name:

View File

@ -619,7 +619,18 @@ class Porter(object):
"create_port_table", create_port_table
)
# Step 2. Get tables.
# Step 2. Set up sequences
#
# We do this before porting the tables so that event if we fail half
# way through the postgres DB always have sequences that are greater
# than their respective tables. If we don't then creating the
# `DataStore` object will fail due to the inconsistency.
self.progress.set_state("Setting up sequence generators")
await self._setup_state_group_id_seq()
await self._setup_user_id_seq()
await self._setup_events_stream_seqs()
# Step 3. Get tables.
self.progress.set_state("Fetching tables")
sqlite_tables = await self.sqlite_store.db_pool.simple_select_onecol(
table="sqlite_master", keyvalues={"type": "table"}, retcol="name"
@ -634,7 +645,7 @@ class Porter(object):
tables = set(sqlite_tables) & set(postgres_tables)
logger.info("Found %d tables", len(tables))
# Step 3. Figure out what still needs copying
# Step 4. Figure out what still needs copying
self.progress.set_state("Checking on port progress")
setup_res = await make_deferred_yieldable(
defer.gatherResults(
@ -651,7 +662,7 @@ class Porter(object):
# of: `postgres_size`, `table_size`, `forward_chunk`, `backward_chunk`.
tables_to_port_info_map = {r[0]: r[1:] for r in setup_res}
# Step 4. Do the copying.
# Step 5. Do the copying.
#
# This is slightly convoluted as we need to ensure tables are ported
# in the correct order due to foreign key constraints.
@ -685,12 +696,6 @@ class Porter(object):
tables_ported.update(tables_to_port)
# Step 5. Set up sequences
self.progress.set_state("Setting up sequence generators")
await self._setup_state_group_id_seq()
await self._setup_user_id_seq()
await self._setup_events_stream_seqs()
self.progress.done()
except Exception as e:
global end_error_exec_info
@ -848,43 +853,62 @@ class Porter(object):
return done, remaining + done
def _setup_state_group_id_seq(self):
async def _setup_state_group_id_seq(self):
curr_id = await self.sqlite_store.db_pool.simple_select_one_onecol(
table="state_groups", keyvalues={}, retcol="MAX(id)", allow_none=True
)
if not curr_id:
return
def r(txn):
txn.execute("SELECT MAX(id) FROM state_groups")
curr_id = txn.fetchone()[0]
if not curr_id:
return
next_id = curr_id + 1
txn.execute("ALTER SEQUENCE state_group_id_seq RESTART WITH %s", (next_id,))
return self.postgres_store.db_pool.runInteraction("setup_state_group_id_seq", r)
await self.postgres_store.db_pool.runInteraction("setup_state_group_id_seq", r)
async def _setup_user_id_seq(self):
curr_id = await self.sqlite_store.db_pool.runInteraction(
"setup_user_id_seq", find_max_generated_user_id_localpart
)
def _setup_user_id_seq(self):
def r(txn):
next_id = find_max_generated_user_id_localpart(txn) + 1
next_id = curr_id + 1
txn.execute("ALTER SEQUENCE user_id_seq RESTART WITH %s", (next_id,))
return self.postgres_store.db_pool.runInteraction("setup_user_id_seq", r)
def _setup_events_stream_seqs(self):
def r(txn):
txn.execute("SELECT MAX(stream_ordering) FROM events")
curr_id = txn.fetchone()[0]
if curr_id:
next_id = curr_id + 1
async def _setup_events_stream_seqs(self):
"""Set the event stream sequences to the correct values.
"""
# We get called before we've ported the events table, so we need to
# fetch the current positions from the SQLite store.
curr_forward_id = await self.sqlite_store.db_pool.simple_select_one_onecol(
table="events", keyvalues={}, retcol="MAX(stream_ordering)", allow_none=True
)
curr_backward_id = await self.sqlite_store.db_pool.simple_select_one_onecol(
table="events",
keyvalues={},
retcol="MAX(-MIN(stream_ordering), 1)",
allow_none=True,
)
def _setup_events_stream_seqs_set_pos(txn):
if curr_forward_id:
txn.execute(
"ALTER SEQUENCE events_stream_seq RESTART WITH %s", (next_id,)
"ALTER SEQUENCE events_stream_seq RESTART WITH %s",
(curr_forward_id + 1,),
)
txn.execute("SELECT GREATEST(-MIN(stream_ordering), 1) FROM events")
curr_id = txn.fetchone()[0]
next_id = curr_id + 1
txn.execute(
"ALTER SEQUENCE events_backfill_stream_seq RESTART WITH %s", (next_id,),
"ALTER SEQUENCE events_backfill_stream_seq RESTART WITH %s",
(curr_backward_id + 1,),
)
return self.postgres_store.db_pool.runInteraction(
"_setup_events_stream_seqs", r
return await self.postgres_store.db_pool.runInteraction(
"_setup_events_stream_seqs", _setup_events_stream_seqs_set_pos,
)

View File

@ -48,7 +48,7 @@ try:
except ImportError:
pass
__version__ = "1.22.1"
__version__ = "1.23.0rc1"
if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
# We import here so that we don't have to install a bunch of deps when

View File

@ -271,6 +271,28 @@ class SAML2Config(Config):
#description: ["My awesome SP", "en"]
#name: ["Test SP", "en"]
#ui_info:
# display_name:
# - lang: en
# text: "Display Name is the descriptive name of your service."
# description:
# - lang: en
# text: "Description should be a short paragraph explaining the purpose of the service."
# information_url:
# - lang: en
# text: "https://example.com/terms-of-service"
# privacy_statement_url:
# - lang: en
# text: "https://example.com/privacy-policy"
# keywords:
# - lang: en
# text: ["Matrix", "Element"]
# logo:
# - lang: en
# text: "https://example.com/logo.svg"
# width: "200"
# height: "80"
#organization:
# name: Example com
# display_name:

View File

@ -67,7 +67,7 @@ from synapse.replication.http.devices import ReplicationUserDevicesResyncRestSer
from synapse.replication.http.federation import (
ReplicationCleanRoomRestServlet,
ReplicationFederationSendEventsRestServlet,
ReplicationStoreRoomOnInviteRestServlet,
ReplicationStoreRoomOnOutlierMembershipRestServlet,
)
from synapse.state import StateResolutionStore
from synapse.storage.databases.main.events_worker import EventRedactBehaviour
@ -152,12 +152,14 @@ class FederationHandler(BaseHandler):
self._user_device_resync = ReplicationUserDevicesResyncRestServlet.make_client(
hs
)
self._maybe_store_room_on_invite = ReplicationStoreRoomOnInviteRestServlet.make_client(
self._maybe_store_room_on_outlier_membership = ReplicationStoreRoomOnOutlierMembershipRestServlet.make_client(
hs
)
else:
self._device_list_updater = hs.get_device_handler().device_list_updater
self._maybe_store_room_on_invite = self.store.maybe_store_room_on_invite
self._maybe_store_room_on_outlier_membership = (
self.store.maybe_store_room_on_outlier_membership
)
# When joining a room we need to queue any events for that room up.
# For each room, a list of (pdu, origin) tuples.
@ -1617,7 +1619,7 @@ class FederationHandler(BaseHandler):
# keep a record of the room version, if we don't yet know it.
# (this may get overwritten if we later get a different room version in a
# join dance).
await self._maybe_store_room_on_invite(
await self._maybe_store_room_on_outlier_membership(
room_id=event.room_id, room_version=room_version
)

View File

@ -502,6 +502,16 @@ build_info.labels(
last_ticked = time.time()
# 3PID send info
threepid_send_requests = Histogram(
"synapse_threepid_send_requests_with_tries",
documentation="Number of requests for a 3pid token by try count. Note if"
" there is a request with try count of 4, then there would have been one"
" each for 1, 2 and 3",
buckets=(1, 2, 3, 4, 5, 10),
labelnames=("type", "reason"),
)
class ReactorLastSeenMetric:
def collect(self):

View File

@ -254,20 +254,20 @@ class ReplicationCleanRoomRestServlet(ReplicationEndpoint):
return 200, {}
class ReplicationStoreRoomOnInviteRestServlet(ReplicationEndpoint):
class ReplicationStoreRoomOnOutlierMembershipRestServlet(ReplicationEndpoint):
"""Called to clean up any data in DB for a given room, ready for the
server to join the room.
Request format:
POST /_synapse/replication/store_room_on_invite/:room_id/:txn_id
POST /_synapse/replication/store_room_on_outlier_membership/:room_id/:txn_id
{
"room_version": "1",
}
"""
NAME = "store_room_on_invite"
NAME = "store_room_on_outlier_membership"
PATH_ARGS = ("room_id",)
def __init__(self, hs):
@ -282,7 +282,7 @@ class ReplicationStoreRoomOnInviteRestServlet(ReplicationEndpoint):
async def _handle_request(self, request, room_id):
content = parse_json_object_from_request(request)
room_version = KNOWN_ROOM_VERSIONS[content["room_version"]]
await self.store.maybe_store_room_on_invite(room_id, room_version)
await self.store.maybe_store_room_on_outlier_membership(room_id, room_version)
return 200, {}
@ -291,4 +291,4 @@ def register_servlets(hs, http_server):
ReplicationFederationSendEduRestServlet(hs).register(http_server)
ReplicationGetQueryRestServlet(hs).register(http_server)
ReplicationCleanRoomRestServlet(hs).register(http_server)
ReplicationStoreRoomOnInviteRestServlet(hs).register(http_server)
ReplicationStoreRoomOnOutlierMembershipRestServlet(hs).register(http_server)

View File

@ -38,6 +38,7 @@ from synapse.http.servlet import (
parse_json_object_from_request,
parse_string,
)
from synapse.metrics import threepid_send_requests
from synapse.push.mailer import Mailer
from synapse.util.msisdn import phone_number_to_msisdn
from synapse.util.stringutils import assert_valid_client_secret, random_string
@ -143,6 +144,10 @@ class EmailPasswordRequestTokenRestServlet(RestServlet):
# Wrap the session id in a JSON object
ret = {"sid": sid}
threepid_send_requests.labels(type="email", reason="password_reset").observe(
send_attempt
)
return 200, ret
@ -411,6 +416,10 @@ class EmailThreepidRequestTokenRestServlet(RestServlet):
# Wrap the session id in a JSON object
ret = {"sid": sid}
threepid_send_requests.labels(type="email", reason="add_threepid").observe(
send_attempt
)
return 200, ret
@ -481,6 +490,10 @@ class MsisdnThreepidRequestTokenRestServlet(RestServlet):
next_link,
)
threepid_send_requests.labels(type="msisdn", reason="add_threepid").observe(
send_attempt
)
return 200, ret

View File

@ -45,6 +45,7 @@ from synapse.http.servlet import (
parse_json_object_from_request,
parse_string,
)
from synapse.metrics import threepid_send_requests
from synapse.push.mailer import Mailer
from synapse.util.msisdn import phone_number_to_msisdn
from synapse.util.ratelimitutils import FederationRateLimiter
@ -163,6 +164,10 @@ class EmailRegisterRequestTokenRestServlet(RestServlet):
# Wrap the session id in a JSON object
ret = {"sid": sid}
threepid_send_requests.labels(type="email", reason="register").observe(
send_attempt
)
return 200, ret
@ -234,6 +239,10 @@ class MsisdnRegisterRequestTokenRestServlet(RestServlet):
next_link,
)
threepid_send_requests.labels(type="msisdn", reason="register").observe(
send_attempt
)
return 200, ret

View File

@ -1240,13 +1240,15 @@ class RoomStore(RoomBackgroundUpdateStore, RoomWorkerStore, SearchStore):
logger.error("store_room with room_id=%s failed: %s", room_id, e)
raise StoreError(500, "Problem creating room.")
async def maybe_store_room_on_invite(self, room_id: str, room_version: RoomVersion):
async def maybe_store_room_on_outlier_membership(
self, room_id: str, room_version: RoomVersion
):
"""
When we receive an invite over federation, store the version of the room if we
don't already know the room version.
When we receive an invite or any other event over federation that may relate to a room
we are not in, store the version of the room if we don't already know the room version.
"""
await self.db_pool.simple_upsert(
desc="maybe_store_room_on_invite",
desc="maybe_store_room_on_outlier_membership",
table="rooms",
keyvalues={"room_id": room_id},
values={},