Commit Graph

19978 Commits (d41c4654db03dc76812e98222feb62668a1bf94c)

Author SHA1 Message Date
Richard van der Hoff d41c4654db
Use buildkit's cache feature to speed up docker builds (#11691)
Having spent much of the last week attempting to run complement tests from somewhere with damp string instead of internet... something had to be done.
2022-01-12 10:37:57 +00:00
Michael Kaye 338e70c617
Complement environment variable name change and update .gitignore. (#11718) 2022-01-10 23:18:56 +00:00
Andrew Morgan 7c3408d1a8
Document the `SYNAPSE_TEST_PERSIST_SQLITE_DB` unit test env var (#11715) 2022-01-10 17:06:42 +00:00
reivilibre ffd227c382
Fix docstring on `add_account_data_for_user`. (#11716) 2022-01-10 15:38:22 +00:00
David Robertson c43dd4d01b
Deal with mypy errors w/ type-hinted pynacl 1.5.0 (#11714)
* Deal with mypy errors w/ type-hinted pynacl 1.5.0

Fixes #11644.

I really don't like that we're monkey patching pynacl SignedKey
instances with alg and version objects. But I'm too scared to make the
changes necessary right now.

(Ideally I would replace `signedjson.types.SingingKey` with a runtime class which
wraps or inherits from `nacl.signing.SigningKey`.) C.f. https://github.com/matrix-org/python-signedjson/issues/16
2022-01-10 13:40:46 +00:00
kegsay 3be63654e4
Prettier complement logs (#11707)
* Prettier complement logs

* Changelog
2022-01-10 11:46:40 +00:00
Patrick Cloke 8e57584a58
Support spaces with > 50 rooms in the /hierarchy endpoint. (#11695)
By returning all of the m.space.child state of the space, not just
the first 50. The number of rooms returned is still capped at 50.

For the federation API this implies that the requesting server will
need to individually query for any other rooms it is not joined to.
2022-01-07 19:27:58 -05:00
David Robertson d3cf0730f8
Optionally use an on-disk sqlite db in tests (#11702)
* Optionally use an on-disk sqlite db in tests

When debugging a test it is sometimes useful to inspect the state of the
DB. This is not easy when the db is in-memory: one cannot attach the
sqlite CLI to another process's DB.

With this change, if SYNAPSE_TEST_PERSIST_SQLITE_DB is set, we use
 `_trial_temp/test.db` as our sqlite database. One can then use 
`sqlite3 _trial_temp/test.db` and query to your heart's content.

The DB is destroyed and recreated between different test cases.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-01-07 19:13:41 +00:00
David Robertson 2bb4bd1269
Test that bans win a join against a race when computing `/sync` response (#11701) 2022-01-07 16:43:21 +00:00
Olivier Wilkinson (reivilibre) 6a04767439 Merge branch 'release-v1.50' into develop 2022-01-07 14:22:13 +00:00
Patrick Cloke 6bf81a7a61
Bundle aggregations outside of the serialization method. (#11612)
This makes the serialization of events synchronous (and it no
longer access the database), but we must manually calculate and
provide the bundled aggregations.

Overall this should cause no change in behavior, but is prep work
for other improvements.
2022-01-07 09:10:46 -05:00
Olivier Wilkinson (reivilibre) 7fe7c45438 Move changelog entry for drop of support to 'Deprecations and Removals' 2022-01-07 12:51:20 +00:00
Patrick Cloke 6c68e874b1
Remove the /send_relation endpoint. (#11682)
This was removed from MSC2674 before that was approved
and is not used by any known clients.
2022-01-06 19:00:34 +00:00
Patrick Cloke 201c48c8de
Remove a Python 2-ism and improve type hints. (#11699)
On Python 2, indexing a byte-string gives back a byte-string,
while on Python 3 it gives back the ASCII equivalent as an int.
2022-01-06 18:08:48 +00:00
Olivier Wilkinson (reivilibre) e87540abb1 Re-run Towncrier to add in the changelog entry for the Mjolnir workaround 2022-01-06 17:54:21 +00:00
Shay 70ce9aea71
Strip unauthorized fields from `unsigned` object in events received over federation (#11530)
* add some tests to verify we are stripping unauthorized fields out of unsigned

* add function to strip unauthorized fields from the unsigned object of event

* newsfragment

* update newsfragment number

* add check to on_send_membership_event

* refactor tests

* fix lint error

* slightly refactor tests and add some comments

* slight refactor

* refactor tests

* fix import error

* slight refactor

* remove unsigned filtration code from synapse/handlers/federation_event.py

* lint

* move unsigned filtering code to event base

* refactor tests

* update newsfragment

* requested changes

* remove unused retun values
2022-01-06 09:09:30 -08:00
reivilibre eec34b1f2a
Work around Mjolnir compatibility issue by adding an import for `glob_to_regex` in `synapse.util`, where it moved from. (#11696) 2022-01-06 16:36:26 +00:00
lukasdenk 2ef1fea8d2
Make room creations denied by `user_may_create_room` cause an `M_FORBIDDEN` error to be returned, not `M_UNKNOWN` (#11672)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-01-06 13:16:42 +00:00
David Robertson c9eb678b73
Remove debug logging for #4422 (#11693)
as per
https://github.com/matrix-org/synapse/pull/11532#discussion_r769123269
2022-01-06 12:44:36 +00:00
Travis Ralston feb3e006d7
Fix space hierarchy endpoint to match MSC2946 (#11667)
Fixes minor discrepancies between the /hierarchy endpoint described
in MSC2946 and the implementation.

Note that the changes impact the stable and unstable /hierarchy and
unstable /spaces endpoints for both client and federation APIs.
2022-01-05 20:33:43 +00:00
Dirk Klimpel 3b51c763ba
Fix get federation status of destination if no error occured (#11593) 2022-01-05 14:46:50 -05:00
Olivier Wilkinson (reivilibre) daea7bcc34 Tweak changelog for #11677 2022-01-05 18:16:10 +00:00
Shay d8f94eeec2
Run `pyupgrade --py37-plus --keep-percent-format` on Synapse (#11685)
* newsfragment

* fix newsfragment number

* update changelog

* remove extra space
2022-01-05 09:53:05 -08:00
Patrick Cloke 83acdb23fe Re-run towncrier. 2022-01-05 12:09:15 -05:00
Patrick Cloke 877b45e812
Include `io.element.thread` capability for MSC3440. (#11690) 2022-01-05 12:08:03 -05:00
Olivier Wilkinson (reivilibre) 102f4d3598 Mention drop of support in changelog 2022-01-05 14:14:57 +00:00
Olivier Wilkinson (reivilibre) ffd71029ab Add support removal notice to the upgrade notes 2022-01-05 14:14:16 +00:00
Olivier Wilkinson (reivilibre) 5f98d8e6fd Tweak changelog 2022-01-05 14:05:15 +00:00
David Robertson 88a78c6577
Cache empty responses from `/user/devices` (#11587)
If we've never made a request to a remote homeserver, we should cache the response---even if the response is "this user has no devices".
2022-01-05 13:33:28 +00:00
Olivier Wilkinson (reivilibre) 9be5aacc2d 1.50.0rc1 2022-01-05 12:39:48 +00:00
Richard van der Hoff 0fb3dd0830
Refactor the way we set `outlier` (#11634)
* `_auth_and_persist_outliers`: mark persisted events as outliers

Mark any events that get persisted via `_auth_and_persist_outliers` as, well,
outliers.

Currently this will be a no-op as everything will already be flagged as an
outlier, but I'm going to change that.

* `process_remote_join`: stop flagging as outlier

The events are now flagged as outliers later on, by `_auth_and_persist_outliers`.

* `send_join`: remove `outlier=True`

The events created here are returned in the result of `send_join` to
`FederationHandler.do_invite_join`. From there they are passed into
`FederationEventHandler.process_remote_join`, which passes them to
`_auth_and_persist_outliers`... which sets the `outlier` flag.

* `get_event_auth`: remove `outlier=True`

stop flagging the events returned by `get_event_auth` as outliers. This method
is only called by `_get_remote_auth_chain_for_event`, which passes the results
into `_auth_and_persist_outliers`, which will flag them as outliers.

* `_get_remote_auth_chain_for_event`: remove `outlier=True`

we pass all the events into `_auth_and_persist_outliers`, which will now flag
the events as outliers.

* `_check_sigs_and_hash_and_fetch`: remove unused `outlier` parameter

This param is now never set to True, so we can remove it.

* `_check_sigs_and_hash_and_fetch_one`: remove unused `outlier` param

This is no longer set anywhere, so we can remove it.

* `get_pdu`: remove unused `outlier` parameter

... and chase it down into `get_pdu_from_destination_raw`.

* `event_from_pdu_json`: remove redundant `outlier` param

This is never set to `True`, so can be removed.

* changelog

* update docstring
2022-01-05 12:26:11 +00:00
Philipp Matthias Schäfer eedb4527f1
Fix link from generated configuration file to documentation (#11678)
Co-authored-by: reivilibre <olivier@librepush.net>
Co-authored-by: reivilibre <oliverw@matrix.org>
2022-01-05 12:16:52 +00:00
Fr3shTea 0201c6371c
Fix SimpleHttpClient not sending Accept header in `get_json` (#11677)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-01-05 11:59:29 +00:00
Dirk Klimpel 7a1cefc6e3
Add admin API to get users' account data (#11664)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-01-05 11:49:06 +00:00
reivilibre 84bfe47b01
Re-apply: Move glob_to_regex and re_word_boundary to matrix-python-common #11505 (#11687)
Co-authored-by: Sean Quah <seanq@element.io>
2022-01-05 11:41:49 +00:00
reivilibre 84d790a32e
Clarify SSO mapping provider documentation by writing `def` or `async def` before the names of methods, as appropriate. (#11681) 2022-01-05 11:25:32 +00:00
Callum Macdonald 7013e06e2f
Improve Docker docs for use with Postgres (#11640) 2022-01-05 10:50:28 +00:00
Donny Johnson 0715e77b06
Correct Synapse install command for FreeBSD. (#11267)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-01-05 10:38:51 +00:00
Richard van der Hoff 79f6d3550a
update ngnix reverse-proxy example (#11680)
this should not be a case-insensitive match.
2022-01-04 22:31:45 +00:00
Richard van der Hoff b38bdae3a2
Fix AssertionErrors after purging events (#11642)
* Fix AssertionErrors after purging events

If you purged a bunch of events from your database, and then restarted synapse
without receiving more events, then you would get a bunch of AssertionErrors on
restart.

This fixes the situation by rewinding the stream processors.

* `check-newsfragment`: ignore deleted newsfiles
2022-01-04 16:36:33 +00:00
Richard van der Hoff 878aa55293
`FederationClient.backfill`: stop flagging events as outliers (#11632)
Events returned by `backfill` should not be flagged as outliers.

Fixes:

```
AssertionError: null
  File "synapse/handlers/federation.py", line 313, in try_backfill
    dom, room_id, limit=100, extremities=extremities
  File "synapse/handlers/federation_event.py", line 517, in backfill
    await self._process_pulled_events(dest, events, backfilled=True)
  File "synapse/handlers/federation_event.py", line 642, in _process_pulled_events
    await self._process_pulled_event(origin, ev, backfilled=backfilled)
  File "synapse/handlers/federation_event.py", line 669, in _process_pulled_event
    assert not event.internal_metadata.is_outlier()
```

See https://sentry.matrix.org/sentry/synapse-matrixorg/issues/231992

Fixes #8894.
2022-01-04 16:31:32 +00:00
Richard van der Hoff 2359ee3864
Remove redundant `get_current_events_token` (#11643)
* Push `get_room_{min,max_stream_ordering}` into StreamStore

Both implementations of this are identical, so we may as well push it down and
get rid of the abstract base class nonsense.

* Remove redundant `StreamStore` class

This is empty now

* Remove redundant `get_current_events_token`

This was an exact duplicate of `get_room_max_stream_ordering`, so let's get rid
of it.

* newsfile
2022-01-04 16:10:27 +00:00
Richard van der Hoff bd9821f7f1
Better error messages from `get_create_event_for_room` (#11638)
"Unknown room" can mean a multitude of things here. To help with debugging, add
some more words to the exception text.
2022-01-04 16:10:05 +00:00
Patrick Cloke 8422a7f7f6
Include the topic event in the prejoin state, per MSC3173. (#11666)
Invites and knocks will now include the topic in the stripped state
send to clients before joining the room.
2022-01-04 11:08:08 -05:00
Shay 13c974ed35
Drop Bionic from Debian builds (#11633)
* update Trove classifiers to remove py36

* stop building bionic

* update dh-virtualenv

* newsfragment

* fix newsfragment

* update version refs

* another try at correct tag

* Update changelog
2022-01-03 11:17:16 -08:00
Patrick Cloke cbd82d0b2d
Convert all namedtuples to attrs. (#11665)
To improve type hints throughout the code.
2021-12-30 18:47:12 +00:00
Dirk Klimpel 07a3b5daba
Add type hints to `synapse/storage/databases/main/events_bg_updates.py` (#11654) 2021-12-30 07:22:31 -05:00
reivilibre 2c7f5e74e5
Fix a type annotation in `test_account_data.py` and remove it from the Mypy exclusion list. (#11657)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-12-29 15:12:30 +00:00
Dirk Klimpel f82d38ed2e
Improve type hints in storage classes. (#11652)
By using cast and making ignores more specific.
2021-12-29 13:04:28 +00:00
Patrick Cloke f58b300d27
Do not attempt to bundled aggregations for /members and /state. (#11623)
Both of those APIs return state events, which will not have bundled
aggregations added anyway.
2021-12-29 08:02:03 -05:00