649a7ead5c
Add ability to run multiple pusher instances ( #7855 )
...
This reuses the same scheme as federation sender sharding
2020-07-16 14:06:28 +01:00
12528dc42f
Remove obsolete comment.
...
It was correct at the time of our friend Jorik writing it (checking
git blame), but the world has moved now and it is no longer a
generator.
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2020-07-16 11:12:48 +01:00
35450519de
Ensure that calls to `json.dumps` are compatible with the standard library json. ( #7836 )
2020-07-15 13:40:54 -04:00
66a4af8d96
Do not use canonicaljson to magically handle decoding bytes from JSON. ( #7802 )
2020-07-10 14:30:08 -04:00
f299441cc6
Add ability to shard the federation sender ( #7798 )
2020-07-10 18:26:36 +01:00
38e1fac886
Fix some spelling mistakes / typos. ( #7811 )
2020-07-09 09:52:58 -04:00
67593b1728
Add `HomeServer.signing_key` property ( #7805 )
...
... instead of duplicating `config.signing_key[0]` everywhere
2020-07-08 17:51:56 +01:00
5cdca53aa0
Merge different Resource implementation classes ( #7732 )
2020-07-03 19:02:19 +01:00
1e03513f9a
Fix new metric where we used ms instead of seconds ( #7771 )
...
Introduced in #7755 , not yet released.
2020-07-01 15:23:58 +01:00
a99658074d
Add some metrics for inbound and outbound federation processing times ( #7755 )
2020-06-30 16:58:06 +01:00
a3f11567d9
Replace all remaining six usage with native Python 3 equivalents ( #7704 )
2020-06-16 08:51:47 -04:00
bd6dc17221
Replace iteritems/itervalues/iterkeys with native versions. ( #7692 )
2020-06-15 07:03:36 -04:00
075375bbc9
add a comment
2020-05-21 13:25:41 +01:00
56b66db78a
Strictly enforce canonicaljson requirements in a new room version ( #7381 )
2020-05-14 13:24:01 -04:00
16b1a34e80
Fix typing annotations in synapse/federation ( #7382 )
...
We're pretty close to having mypy working for `synapse.federation`, so let's
finish the job.
2020-05-05 14:27:13 +01:00
d5aa7d93ed
Fix catchup-on-reconnect for the Federation Stream ( #7374 )
...
looks like we managed to break this during the refactorathon.
2020-05-05 14:15:57 +01:00
6b22921b19
async/await is_server_admin ( #7363 )
2020-05-01 15:15:36 +01:00
2e8955f4a6
Further improvements to requesting the public rooms list on a homeserver which has it set to private ( #7368 )
2020-05-01 15:15:08 +01:00
2aa5bf13c8
Merge branch 'release-v1.12.4' into develop
2020-04-22 13:09:23 +01:00
f89ad3b6df
Query missing cross-signing keys on local sig upload ( #7289 )
2020-04-22 12:29:36 +01:00
461f01ad43
Merge branch 'release-v1.12.4' into develop
2020-04-20 17:55:39 +01:00
d41c8f6d4d
Revert "Query missing cross-signing keys on local sig upload"
...
This was incorrectly merged to the release branch before it was ready.
This reverts commit 72fe2affb6
2020-04-20 17:54:35 +01:00
a46ff43319
Merge branch 'release-v1.12.4' into develop
2020-04-20 17:42:27 +01:00
72fe2affb6
Query missing cross-signing keys on local sig upload
...
Add changelog
Save retrieved keys to the db
lint
Fix and de-brittle remote result dict processing
Use query_user_devices instead, assume only master, self_signing key types
Make changelog more useful
Remove very specific exception handling
Wrap get_verify_key_from_cross_signing_key in a try/except
Note that _get_e2e_cross_signing_verify_key can raise a SynapseError
lint
Add comment explaining why this is useful
Only fetch master and self_signing key types
Fix log statements, docstrings
Remove extraneous items from remote query try/except
lint
Factor key retrieval out into a separate function
Send device updates, modeled after SigningKeyEduUpdater._handle_signing_key_updates
Update method docstring
2020-04-17 15:47:49 +01:00
d78cb31588
Add typing information to federation_server. ( #7219 )
2020-04-07 15:03:23 -04:00
4cff617df1
Move catchup of replication streams to worker. ( #7024 )
...
This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.
2020-03-25 14:54:01 +00:00
39230d2171
Clean up some LoggingContext stuff ( #7120 )
...
* Pull Sentinel out of LoggingContext
... and drop a few unnecessary references to it
* Factor out LoggingContext.current_context
move `current_context` and `set_context` out to top-level functions.
Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.
* move copy-to-parent into `stop`
this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.
* Replace `LoggingContext.alive` with `finished`
Turn `alive` into `finished` and make it a bit better defined.
2020-03-24 14:45:33 +00:00
a564b92d37
Convert `*StreamRow` classes to inner classes ( #7116 )
...
This just helps keep the rows closer to their streams, so that it's easier to
see what the format of each stream is.
2020-03-23 13:59:11 +00:00
c2db6599c8
Fix a bug in the federation API which could cause occasional "Failed to get PDU" errors ( #7089 ).
2020-03-19 08:22:56 -04:00
7581d30e9f
Remove unused federation endpoint (`query_auth`) ( #7026 )
2020-03-17 08:04:49 -04:00
12d4259000
Add some type annotations to the federation base & client classes ( #6995 )
2020-02-28 07:31:07 -05:00
ab0073a6c0
Merge remote-tracking branch 'origin/release-v1.11.1' into develop
2020-02-27 13:47:44 -05:00
380122866f
Cast a coroutine into a Deferred in the federation base ( #6996 )
...
Properly convert a coroutine into a Deferred in federation_base to fix an error when joining a room.
2020-02-26 11:32:13 -05:00
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 ( #6957 )
...
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
21db35f77e
Add support for putting fed user query API on workers ( #6873 )
2020-02-07 15:45:39 +00:00
799001f2c0
Add a `make_event_from_dict` method ( #6858 )
...
... and use it in places where it's trivial to do so.
This will make it easier to pass room versions into the FrozenEvent
constructors.
2020-02-07 15:30:04 +00:00
b08b0a22d5
Add typing to synapse.federation.sender ( #6871 )
2020-02-07 13:56:38 +00:00
928edef979
Pass room_version into `event_from_pdu_json`
...
It's called from all over the shop, so this one's a bit messy.
2020-02-06 16:08:27 +00:00
b0c8bdd49d
pass room version into FederationClient.send_join ( #6854 )
...
... which allows us to sanity-check the create event.
2020-02-06 15:50:39 +00:00
ed630ea17c
Reduce amount of logging at INFO level. ( #6862 )
...
A lot of the things we log at INFO are now a bit superfluous, so lets
make them DEBUG logs to reduce the amount we log by default.
Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
2020-02-06 13:31:05 +00:00
ee0525b2b2
Simplify `room_version` handling in `FederationClient.send_invite`
2020-02-05 17:43:57 +00:00
f84700fba8
Pass room version object into `FederationClient.get_pdu`
2020-02-05 17:25:46 +00:00
6bbd890f05
make FederationClient._do_send_invite async
2020-02-05 15:50:31 +00:00
146fec0820
Apply suggestions from code review
...
Co-Authored-By: Erik Johnston <erik@matrix.org>
2020-02-05 15:47:00 +00:00
ea23210b2d
make FederationClient.send_invite async
2020-02-03 22:29:49 +00:00
6deeefb68c
make FederationClient.get_missing_events async
2020-02-03 22:28:45 +00:00
abadf44eb2
make FederationClient._do_send_leave async
2020-02-03 22:28:45 +00:00
e88b90aaeb
make FederationClient.send_leave.send_request async
2020-02-03 22:28:45 +00:00
638001116d
make FederationClient._do_send_join async
2020-02-03 22:28:45 +00:00
3960527c2e
make FederationClient.send_join.send_request async
2020-02-03 22:28:45 +00:00
ad09ee9262
make FederationClient.make_membership_event.send_request async
2020-02-03 22:28:45 +00:00
1330c311b7
make FederationClient._try_destination_list async
2020-02-03 22:28:45 +00:00
a46fabf17b
make FederationClient.send_leave async
2020-02-03 22:28:45 +00:00
8af9f11bea
make FederationClient.send_join async
2020-02-03 22:28:45 +00:00
3f11cbb404
make FederationClient.make_membership_event async
2020-02-03 22:28:45 +00:00
24d814ca23
make FederationClient.get_event_auth async
2020-02-03 22:28:45 +00:00
d73683c363
make FederationClient.get_room_state_ids async
2020-02-03 22:28:45 +00:00
0cb0c7bcd5
make FederationClient.get_pdu async
2020-02-03 22:28:45 +00:00
0536d0c9be
make FederationClient.backfill async
2020-02-03 22:28:45 +00:00
b0d112e78b
Fix `room_version` in `on_invite_request` flow ( #6827 )
...
I messed this up a bit in #6805 , but fortunately we weren't actually doing
anything with the room_version so it didn't matter that it was a str not a RoomVersion.
2020-02-03 13:15:23 +00:00
a5afdd15e5
Merge pull request #6806 from matrix-org/rav/redact_changes/3
...
Pass room_version into add_hashes_and_signatures
2020-01-31 10:57:03 +00:00
d7bf793cc1
s/get_room_version/get_room_version_id/
...
... to make way for a forthcoming get_room_version which returns a RoomVersion
object.
2020-01-31 10:06:21 +00:00
54f3f369bd
Pass room_version into create_local_event_from_event_dict
2020-01-30 22:15:50 +00:00
46a446828d
pass room version into FederationHandler.on_invite_request ( #6805 )
2020-01-30 22:13:02 +00:00
c3d4ad8afd
Fix sending server up commands from workers ( #6811 )
...
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2020-01-30 16:42:11 +00:00
a855b7c3a8
Remove unused DeviceRow class ( #6800 )
2020-01-29 12:06:31 +00:00
6b9e1014cf
Fix race in federation sender that delayed device updates. ( #6799 )
...
We were sending device updates down both the federation stream and
device streams. This mean there was a race if the federation sender
worker processed the federation stream first, as when the sender checked
if there were new device updates the slaved ID generator hadn't been
updated with the new stream IDs and so returned nothing.
This situation is correctly handled by events/receipts/etc by not
sending updates down the federation stream and instead having the
federation sender worker listen on the other streams and poke the
transaction queues as appropriate.
2020-01-29 11:23:01 +00:00
8df862e45d
Add `rooms.room_version` column ( #6729 )
...
This is so that we don't have to rely on pulling it out from `current_state_events` table.
2020-01-27 14:30:57 +00:00
a8a50f5b57
Wake up transaction queue when remote server comes back online ( #6706 )
...
This will be used to retry outbound transactions to a remote server if
we think it might have come back up.
2020-01-17 10:27:19 +00:00
d386f2f339
Add StateMap type alias ( #6715 )
2020-01-16 13:31:22 +00:00
48c3a96886
Port synapse.replication.tcp to async/await ( #6666 )
...
* Port synapse.replication.tcp to async/await
* Newsfile
* Correctly document type of on_<FOO> functions as async
* Don't be overenthusiastic with the asyncing....
2020-01-16 09:16:12 +00:00
b5ce7f5874
Process EDUs in parallel with PDUs. ( #6697 )
...
This means that things like to device messages don't get blocked behind
processing PDUs, which can potentially take *ages*.
2020-01-14 14:08:35 +00:00
54ae52ba96
Merge pull request #6349 from matrix-org/babolivier/msc1802
...
Implement v2 APIs for send_join and send_leave
2019-12-11 11:41:47 +00:00
f8bc2ae883
Move get_state methods into FederationHandler ( #6503 )
...
This is a non-functional refactor as a precursor to some other work.
2019-12-10 17:42:46 +00:00
c3dda2874d
Refactor get_events_from_store_or_dest to return a dict ( #6501 )
...
There was a bunch of unnecessary conversion back and forth between dict and
list going on here. We can simplify a bunch of the code.
2019-12-10 16:22:00 +00:00
e519489fc4
Remove fallback for missing /federation/v1/state_ids API ( #6488 )
...
This API was added way back in 0.17.0; the code here is annoying to maintain
and entirely redundant.
2019-12-09 11:37:26 +00:00
e126d83f74
Merge branch 'develop' into babolivier/msc1802
2019-12-05 21:00:43 +00:00
0f87b912ab
Implementation of MSC2314 ( #6176 )
2019-11-28 08:54:07 +11:00
3916e1b97a
Clean up newline quote marks around the codebase ( #6362 )
2019-11-21 12:00:14 +00:00
bc29a19731
Replace instance variations of homeserver with correct case/spacing
2019-11-12 13:08:12 +00:00
94cdd6fffe
Lint
2019-11-11 16:56:55 +00:00
edc4c7d4c5
Lint
2019-11-11 16:51:54 +00:00
5e18dc7955
Fix prefix for v2/send_leave
2019-11-11 16:46:09 +00:00
74897de01f
Add server-side support to the v2 API
2019-11-11 16:40:45 +00:00
1e202a90f1
Implement v2 API for send_leave
2019-11-11 16:26:53 +00:00
92527d7b21
Add missing yield
2019-11-11 16:20:53 +00:00
4c131b2c78
Implement v2 API for send_join
2019-11-11 15:47:47 +00:00
ace947e8da
Depublish a room from the public rooms list when it is upgraded ( #6232 )
2019-11-01 10:28:09 +00:00
6f4bc6d01d
Merge branch 'develop' into cross-signing_federation
2019-10-31 22:38:21 -04:00
020add5099
Update black to 19.10b0 ( #6304 )
...
* update version of black and also fix the mypy config being overridden
2019-11-01 02:43:24 +11:00
54fef094b3
Remove usage of deprecated logger.warn method from codebase ( #6271 )
...
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
bb6cec27a5
rename get_devices_by_remote to get_device_updates_by_remote
2019-10-30 14:57:34 -04:00
7d7eac61be
Merge branch 'develop' into cross-signing_federation
2019-10-30 10:17:10 -04:00
7955abeaac
Fix small typo in comment ( #6269 )
2019-10-30 11:16:19 +00:00
770d1ef673
Merge pull request #6280 from matrix-org/erikj/receipts_async_await
...
Port receipt and read markers to async/wait
2019-10-30 11:44:18 +01:00
a287f1e804
Don't return coroutines
2019-10-29 16:36:46 +00:00
2c35ffead2
Port receipt and read markers to async/wait
2019-10-29 15:08:22 +00:00
3f33879be4
Port federation_server to async/await
2019-10-29 14:13:08 +00:00
bcfc647e4d
Synapse 1.5.0rc2 (2019-10-28)
...
=============================
Bugfixes
--------
- Update list of boolean columns in `synapse_port_db`. ([\#6247](https://github.com/matrix-org/synapse/issues/6247 ))
- Fix /keys/query API on workers. ([\#6256](https://github.com/matrix-org/synapse/issues/6256 ))
- Improve signature checking on some federation APIs. ([\#6262](https://github.com/matrix-org/synapse/issues/6262 ))
Internal Changes
----------------
- Move schema delta files to the correct data store. ([\#6248](https://github.com/matrix-org/synapse/issues/6248 ))
- Small performance improvement by removing repeated config lookups in room stats calculation. ([\#6255](https://github.com/matrix-org/synapse/issues/6255 ))
-----BEGIN PGP SIGNATURE-----
iQEzBAABCAAdFiEEv27Axt/F4vrTL/8QOSor00I9eP8FAl225VkACgkQOSor00I9
eP/7wwf6AkQE1dGYDPpmS29XYocfD3EbjT8FojaHTz1gbkIt5rEF88QHzwRo4P/x
1S1kQZpEKxKrwmLMDmSbw9STK/76LKGk9k8VycVHfTdy8SICCiL50pVPpSTJ4nXV
905E7nY0zau3wunWJD4gEjqll967W0AtNKwCHO0D6vNP4pcQNhTkSjhnmypXCFGo
bvV9r0wq8mtq/FS1GX0Y2CEwbbV1MoPrvlXW37t/gBPk3P1QB/vPjakxO8LgeuZf
JzAamP5jOYZJpPCpZxwDm6ZNEQ7T4jORdA623wp1gh1nTG88cObCLXJe2jCnpjnJ
QVb1YlJiwcehXkclIoywgIzj13iIsw==
=7ctT
-----END PGP SIGNATURE-----
Merge tag 'v1.5.0rc2' into develop
Synapse 1.5.0rc2 (2019-10-28)
=============================
Bugfixes
--------
- Update list of boolean columns in `synapse_port_db`. ([\#6247](https://github.com/matrix-org/synapse/issues/6247 ))
- Fix /keys/query API on workers. ([\#6256](https://github.com/matrix-org/synapse/issues/6256 ))
- Improve signature checking on some federation APIs. ([\#6262](https://github.com/matrix-org/synapse/issues/6262 ))
Internal Changes
----------------
- Move schema delta files to the correct data store. ([\#6248](https://github.com/matrix-org/synapse/issues/6248 ))
- Small performance improvement by removing repeated config lookups in room stats calculation. ([\#6255](https://github.com/matrix-org/synapse/issues/6255 ))
2019-10-28 12:59:13 +00:00
172f264ed3
Improve signature checking on some federation APIs ( #6262 )
...
Make sure that we check that events sent over /send_join, /send_leave, and
/invite, are correctly signed and come from the expected servers.
2019-10-28 12:43:23 +00:00
Erik Johnston