6f68e32bfb
to_device updates could be dropped when consuming the replication stream ( #15349 )
...
Co-authored-by: reivilibre <oliverw@matrix.org>
2023-03-30 19:41:14 +02:00
9bb2eac719
Bump black from 22.12.0 to 23.1.0 ( #15103 )
2023-02-22 15:29:09 -05:00
65d0386693
Always notify replication when a stream advances ( #14877 )
...
This ensures that all other workers are told about stream updates in a timely manner, without having to remember to manually poke replication.
2023-01-20 18:02:18 +00:00
db1cfe9c80
Update all stream IDs after processing replication rows ( #14723 )
...
This creates a new store method, `process_replication_position` that
is called after `process_replication_rows`. By moving stream ID advances
here this guarantees any relevant cache invalidations will have been
applied before the stream is advanced.
This avoids race conditions where Python switches between threads mid
way through processing the `process_replication_rows` method where stream
IDs may be advanced before caches are invalidated due to class resolution
ordering.
See this comment/issue for further discussion:
https://github.com/matrix-org/synapse/issues/14158#issuecomment-1344048703
2023-01-04 11:49:26 +00:00
cb59e08062
Improve logging and opentracing for to-device message handling ( #14598 )
...
A batch of changes intended to make it easier to trace to-device messages through the system.
The intention here is that a client can set a property org.matrix.msgid in any to-device message it sends. That ID is then included in any tracing or logging related to the message. (Suggestions as to where this field should be documented welcome. I'm not enthusiastic about speccing it - it's very much an optional extra to help with debugging.)
I've also generally improved the data we send to opentracing for these messages.
2022-12-06 09:52:55 +00:00
50122754c8
Add missing types to opentracing. ( #13345 )
...
After this change `synapse.logging` is fully typed.
2022-07-21 12:01:52 +00:00
f721f1baba
Revert "Make all `process_replication_rows` methods async ( #13304 )" ( #13312 )
...
This reverts commit 5d4028f217
2022-07-18 14:28:14 +01:00
5d4028f217
Make all `process_replication_rows` methods async ( #13304 )
...
More prep work for asyncronous caching, also makes all process_replication_rows methods consistent (presence handler already is so).
Signed off by Nick @ Beeper (@Fizzadar)
2022-07-17 22:19:43 +01:00
53b77b203a
Replace noop background updates with DELETE. ( #12954 )
...
Removes the `register_noop_background_update` and deletes the background
updates directly in a delta file.
2022-06-13 14:06:27 -04:00
b76f1a4d5f
Add some type hints to datastore ( #12485 )
2022-04-27 13:05:00 +01:00
0752ab7a36
Reduce to-device queries for /sync. ( #12163 )
2022-03-04 17:57:27 +00:00
79fb64e417
Fix to-device being dropped in limited sync in SQLite. ( #11966 )
...
If ther are more than 100 to-device messages pending for a device
`/sync` will only return the first 100, however the next batch token was
incorrectly calculated and so all other pending messages would be
dropped.
This is due to `txn.rowcount` only returning the number of rows that
*changed*, rather than the number *selected* in SQLite.
2022-02-11 13:38:05 +00:00
64ec45fc1b
Send to-device messages to application services ( #11215 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-02-01 14:13:38 +00:00
3e0536cd2a
Replace uses of simple_insert_many with simple_insert_many_values. ( #11742 )
...
This should be (slightly) more efficient and it is simpler
to have a single method for inserting multiple values.
2022-01-13 19:44:18 -05:00
f82d38ed2e
Improve type hints in storage classes. ( #11652 )
...
By using cast and making ignores more specific.
2021-12-29 13:04:28 +00:00
5305a5e881
Type hint the constructors of the data store classes ( #11555 )
2021-12-13 17:05:00 +00:00
0d88c4f903
Improve performance of `remove_{hidden,deleted}_devices_from_device_inbox` ( #11421 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-11-25 15:14:54 +00:00
64ef25391d
Add type hints to some storage classes ( #11307 )
2021-11-11 08:47:31 -05:00
a026695083
Clarifications and small fixes to to-device related code ( #11247 )
...
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-09 14:31:15 +00:00
4535532526
Delete messages for hidden devices from `device_inbox` ( #11199 )
2021-11-02 13:18:30 +00:00
29ffd680bf
Stop synapse from saving messages in device_inbox for hidden devices. ( #10097 )
...
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-11-01 10:40:41 +00:00
bfd7a9b65c
Fix comments referencing v1.46.0 from PR #10969 . ( #11212 )
...
#10969 was merged after 1.46.0rc1 was cut and will be included
in v1.47.0rc1 instead.
2021-10-29 13:43:51 -04:00
8d46fac98e
Delete messages from `device_inbox` table when deleting device ( #10969 )
...
Fixes : #9346
2021-10-27 16:01:18 +01:00
2b82ec425f
Add type hints for most `HomeServer` parameters ( #11095 )
2021-10-22 18:15:41 +01:00
aa2c027792
Remove unnecessary parentheses around tuples returned from methods ( #10889 )
2021-09-23 11:59:07 +01:00
95e47b2e78
[pyupgrade] `synapse/` ( #10348 )
...
This PR is tantamount to running
```
pyupgrade --py36-plus --keep-percent-format `find synapse/ -type f -name "*.py"`
```
Part of #9744
2021-07-19 15:28:05 +01:00
b378d98c8f
Add debug logging for issue #9533 ( #9959 )
...
Hopefully this will help us track down where to-device messages are getting
lost/delayed.
2021-05-11 11:04:03 +01:00
4b965c862d
Remove redundant "coding: utf-8" lines ( #9786 )
...
Part of #9744
Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.
`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
7e8dc9934e
Add a type hints for service notices to the HomeServer object. ( #9675 )
2021-03-24 06:48:46 -04:00
7950aa8a27
Fix some typos.
2021-02-12 11:14:12 -05:00
6633a4015a
Allow moving account data and receipts streams off master ( #9104 )
2021-01-18 15:47:59 +00:00
b530eaa262
Allow running sendToDevice on workers ( #9044 )
2021-01-07 20:19:26 +00:00
63593134a1
Some cleanups to device inbox store. ( #9041 )
2021-01-07 17:20:44 +00:00
cbabb312e0
Use `async with` for ID gens ( #8383 )
...
This will allow us to hit the DB after we've finished using the generated stream ID.
2020-09-23 16:11:18 +01:00
8a4a4186de
Simplify super() calls to Python 3 syntax. ( #8344 )
...
This converts calls like super(Foo, self) -> super().
Generated with:
sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
5c03134d0f
Convert additional database code to async/await. ( #8195 )
2020-08-28 07:54:27 -04:00
2231dffee6
Make StreamIdGen `get_next` and `get_next_mult` async ( #8161 )
...
This is mainly so that `StreamIdGenerator` and `MultiWriterIdGenerator`
will have the same interface, allowing them to be used interchangeably.
2020-08-25 15:10:08 +01:00
d68e10f308
Convert account data, device inbox, and censor events databases to async/await ( #8063 )
2020-08-12 09:29:06 -04:00
4dd27e6d11
Reduce unnecessary whitespace in JSON. ( #7372 )
2020-08-07 08:02:55 -04:00
a7bdf98d01
Rename database classes to make some sense ( #8033 )
2020-08-05 21:38:57 +01:00
Mathieu Velten