Commit Graph

24223 Commits (matrix-org-hotfixes)

Author SHA1 Message Date
David Robertson 32a59a6495
Keep track of `user_ips` and `monthly_active_users` when delegating auth (#16672)
* Describe `insert_client_ip`
* Pull out client_ips and MAU tracking to BaseAuth
* Define HAS_AUTHLIB once in tests

sick of copypasting

* Track ips and token usage when delegating auth
* Test that we track MAU and user_ips
* Don't track `__oidc_admin`
2023-11-23 12:35:37 +00:00
Charles Wright 1a5f9bb651
Enable refreshable tokens on the admin registration endpoint (#16642)
Signed-off-by: Charles Wright <cvwright@futo.org>
2023-11-22 15:01:09 +00:00
V02460 f2430b16d1
Bump pyo3 (0.20), pythonize (0.20), pyo3-log (0.9) (#16673)
Signed-off-by: Kai A. Hiller <V02460@gmail.com>
2023-11-22 14:55:43 +00:00
Mathieu Velten c432d8f18f
Admin API for server notice: consistently bypass rate limits (#16670)
* Admin API for server notice: disable rate limit for all calls

* Add changelog

* Update changelog.d/16670.bugfix
2023-11-22 13:47:29 +00:00
dependabot[bot] c8118ba8c9
Bump pydantic from 2.4.2 to 2.5.1 (#16663)
Bumps [pydantic](https://github.com/pydantic/pydantic) from 2.4.2 to 2.5.1.
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/main/HISTORY.md)
- [Commits](https://github.com/pydantic/pydantic/compare/v2.4.2...v2.5.1)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-22 13:14:00 +00:00
Jason Little 460743da16
Filter out auth chain queries that don't exist (#16552) 2023-11-22 10:59:16 +00:00
David Robertson 8d5c1fe921
Merge branch 'release-v1.97' into develop 2023-11-21 14:07:08 +00:00
David Robertson 536f9c96d9
fix changelog typo 2023-11-21 13:27:32 +00:00
David Robertson bb86eb9814
1.97.0rc1 2023-11-21 12:38:46 +00:00
dependabot[bot] 7611df705e
Bump sentry-sdk from 1.32.0 to 1.35.0 (#16666)
Bumps [sentry-sdk](https://github.com/getsentry/sentry-python) from 1.32.0 to 1.35.0.
- [Release notes](https://github.com/getsentry/sentry-python/releases)
- [Changelog](https://github.com/getsentry/sentry-python/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-python/compare/1.32.0...1.35.0)

---
updated-dependencies:
- dependency-name: sentry-sdk
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-20 12:14:06 +00:00
dependabot[bot] e934e7f7e7
Bump pyopenssl from 23.2.0 to 23.3.0 (#16662)
Bumps [pyopenssl](https://github.com/pyca/pyopenssl) from 23.2.0 to 23.3.0.
- [Changelog](https://github.com/pyca/pyopenssl/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pyca/pyopenssl/compare/23.2.0...23.3.0)

---
updated-dependencies:
- dependency-name: pyopenssl
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-20 12:11:15 +00:00
dependabot[bot] d792e0f2d9
Bump types-pillow from 10.1.0.0 to 10.1.0.2 (#16664)
Bumps [types-pillow](https://github.com/python/typeshed) from 10.1.0.0 to 10.1.0.2.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-pillow
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-20 12:08:25 +00:00
dependabot[bot] 89d9ab0a0a
Bump types-psycopg2 from 2.9.21.15 to 2.9.21.16 (#16665)
Bumps [types-psycopg2](https://github.com/python/typeshed) from 2.9.21.15 to 2.9.21.16.
- [Commits](https://github.com/python/typeshed/commits)

---
updated-dependencies:
- dependency-name: types-psycopg2
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-20 12:07:34 +00:00
Erik Johnston 9c3b906b3a Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2023-11-20 10:09:33 +00:00
Erik Johnston 6088303efb
Speed up how quickly we launch new tasks (#16660)
Now that we're reducing concurrency (#16656), this is more important.
2023-11-17 16:36:02 +00:00
Patrick Cloke d9dcfe2a35
Bump requests-toolbelt from 0.10.1 to 1.0.0. (#16659) 2023-11-17 10:23:07 -05:00
Erik Johnston 9c02ef21e0
Speed up purge room by adding index (#16657)
What it says on the tin
2023-11-17 14:15:44 +00:00
Erik Johnston 6fec2d035f
Also discard 'caches' and 'backfill' stream POSITIONS (#16655)
Follow on from #16640
2023-11-17 14:14:29 +00:00
Patrick Cloke bdb0cbc5ca Merge branch 'master' into develop 2023-11-17 08:43:47 -05:00
Michael Weimann 518e4de758
Update admin user API return types in docs. (#16654) 2023-11-17 13:38:25 +00:00
Erik Johnston 700c8a0de5
Reduce task concurrency (#16656) 2023-11-17 13:14:26 +00:00
Patrick Cloke c4f5522189 Tweaks from review. 2023-11-17 08:01:13 -05:00
Patrick Cloke 6a1352e564 Move the forking note to 1.96.1. 2023-11-17 07:52:54 -05:00
Patrick Cloke 76f990c244 1.96.1 2023-11-17 07:51:59 -05:00
Patrick Cloke 47c682101f
Fix building wheels in CI. (#16653)
pip was using a vendored setuptools that was incompatible with
Python 3.12. Upgrading cibuildwheels to a version with a newer
version of pip (and thus a newer version of setuptools) fixes
the issue.
2023-11-17 07:42:49 -05:00
Patrick Cloke 2de2258bd2 Add blogpost link to changelog. 2023-11-16 13:01:32 -05:00
Patrick Cloke ff0148a165 1.96.0 2023-11-16 12:58:00 -05:00
Erik Johnston f8f7355678 Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2023-11-16 16:57:54 +00:00
Erik Johnston 4d6b800385
Revert "Fix test not detecting tables with missing primary keys and missing replica identities, then add more replica identities. (#16647)" (#16652)
This reverts commit 830988ae72.
2023-11-16 16:57:26 +00:00
Erik Johnston 01b2477b10 Revert "Fix test not detecting tables with missing primary keys and missing replica identities, then add more replica identities. (#16647)"
This reverts commit 830988ae72.
2023-11-16 16:53:04 +00:00
Erik Johnston a96086e9ed Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2023-11-16 16:50:37 +00:00
Erik Johnston ef5329a9f9
Revert "Add a Postgres `REPLICA IDENTITY` to tables that do not have an implicit one. This should allow use of Postgres logical replication. (#16456)" (#16651)
This reverts commit 69afe3f7a0.
2023-11-16 16:48:48 +00:00
Erik Johnston b20bdd3997 Merge remote-tracking branch 'origin/develop' into matrix-org-hotfixes 2023-11-16 16:27:21 +00:00
Erik Johnston 3e8531d3ba
Speed up deleting device messages (#16643)
Keeping track of a lower bound of stream ID where we've deleted everything below makes the queries much faster. Otherwise, every time we scan for rows to delete we'd re-scan across all the rows that have previously deleted (until the next table VACUUM).
2023-11-16 15:19:35 +00:00
Erik Johnston 1b238e8837
Speed up persisting large number of outliers (#16649)
Recalculating the roots tuple every iteration could be very expensive, so instead let's do a topological sort.
2023-11-16 14:25:35 +00:00
Erik Johnston fef08cbee8
Fix sending out of order `POSITION` over replication (#16639)
If a worker reconnects to Redis we send out the current positions of all our streams. However, if we're also trying to send out a backlog of RDATA at the same time then we can end up sending a `POSITION` with the current token *before* we've sent all the RDATA before the current token.

This doesn't cause actual bugs as the receiving servers see the POSITION, fetch the relevant rows from the DB, and then ignore the old RDATA as they come in. However, this is inefficient so it'd be better if we didn't  send out-of-order positions
2023-11-16 13:05:09 +00:00
Erik Johnston 898655fd12
More efficiently handle no-op POSITION (#16640)
We may receive `POSITION` commands where we already know that worker has
advanced past that position, so there is no point in handling it.
2023-11-16 12:32:17 +00:00
reivilibre 830988ae72
Fix test not detecting tables with missing primary keys and missing replica identities, then add more replica identities. (#16647)
* Fix the CI query that did not detect all cases of missing primary keys

* Add more missing REPLICA IDENTITY entries

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

---------

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2023-11-16 12:26:27 +00:00
David Robertson 43d1aa75e8
Add an Admin API to temporarily grant the ability to update an existing cross-signing key without UIA (#16634) 2023-11-15 17:28:10 +00:00
Sumner Evans 999bd77d3a
Asynchronous Uploads (#15503)
Support asynchronous uploads as defined in MSC2246.
2023-11-15 09:19:24 -05:00
Patrick Cloke 80922dc46e
Add links to pre-1.0 changelog issue/PR references. (#16638) 2023-11-15 13:31:24 +00:00
Patrick Cloke f2f2c7c1f0
Use full GitHub links instead of bare issue numbers. (#16637) 2023-11-15 08:02:11 -05:00
Will Hunt 4dd18bdc2e
Improve documentation for `/_synapse/admin/v1/rooms/<room_id>/timestamp_to_event` (#16631) 2023-11-14 11:43:44 -05:00
Nick Mills-Barrett 0e36a57b60
Remove whole table locks on push rule add/delete (#16051)
The statements are already executed within a transaction thus a table
level lock is unnecessary.
2023-11-13 16:57:44 +00:00
reivilibre 69afe3f7a0
Add a Postgres `REPLICA IDENTITY` to tables that do not have an implicit one. This should allow use of Postgres logical replication. (#16456)
* Add Postgres replica identities to tables that don't have an implicit one

Fixes #16224

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

* Move the delta to version 83 as we missed the boat for 82

* Add a test that all tables have a REPLICA IDENTITY

* Extend the test to include when indices are deleted

* isort

* black

* Fully qualify `oid` as it is a 'hidden attribute' in Postgres 11

* Update tests/storage/test_database.py

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Add missed tables

---------

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2023-11-13 16:03:22 +00:00
David Robertson fb2554b11f
Fix outbound_federation_restricted_to docs & note when added (#16628) 2023-11-13 14:26:49 +00:00
dependabot[bot] 7455b9e27d
Bump serde from 1.0.190 to 1.0.192 (#16627)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-13 11:10:28 +00:00
dependabot[bot] 35fac66d20
Bump prometheus-client from 0.17.1 to 0.18.0 (#16626)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-13 11:09:30 +00:00
dependabot[bot] 69d1ee3feb
Bump treq from 22.2.0 to 23.11.0 (#16623)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-13 11:08:31 +00:00
dependabot[bot] f92af19fa5
Bump types-pyopenssl from 23.2.0.2 to 23.3.0.0 (#16625)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-13 11:06:10 +00:00