Commit Graph

3267 Commits (9d06fb9cb105b33762a55b1d385da81c84507f71)

Author SHA1 Message Date
Erik Johnston 9d06fb9cb1 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/postgres_any 2019-10-10 14:38:21 +01:00
Erik Johnston b161786c14 Replace IN usage with helper funcs 2019-10-10 13:15:49 +01:00
Erik Johnston b4fbf71187 Add helper funcs to use postgres ANY
This means that we can write queries with `col = ANY(?)`, which helps
postgres.
2019-10-10 13:15:24 +01:00
Erik Johnston 9970f955ce Merge branch 'develop' of github.com:matrix-org/synapse into erikj/patch_inner 2019-10-10 11:51:50 +01:00
Richard van der Hoff a139420a3c
Fix races in room stats (and other) updates. (#6187)
Hopefully this will fix the occasional failures we were seeing in the room directory.

The problem was that events are not necessarily persisted (and `current_state_delta_stream` updated) in the same order as their stream_id. So for instance current_state_delta 9 might be persisted *before* current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it had done everything up to 9, and never came back to do stream_id 8.

We can solve this easily by only processing up to the stream_id where we know all events have been persisted.
2019-10-10 11:29:01 +01:00
Richard van der Hoff 562b4e51dd
Rewrite the user_filter migration again (#6184)
you can't plausibly ALTER TABLE in sqlite, so we create the new table with the
right schema to start with.
2019-10-10 11:28:23 +01:00
Erik Johnston 5c1f886c75 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/patch_inner 2019-10-09 16:52:21 +01:00
Erik Johnston def5413480
Merge pull request #6185 from matrix-org/erikj/fix_censored_evnets
Fix inserting bytes as text in `censor_redactions`
2019-10-09 15:39:13 +01:00
Erik Johnston e7631d84e6 Fix existing hex encoded json values in DB 2019-10-08 16:18:43 +01:00
Erik Johnston ced4784592 Fix inserting bytes as text 2019-10-08 16:18:43 +01:00
Brendan Abolivier c69324ffb5
Fix RegistrationStore 2019-10-08 14:48:33 +01:00
Brendan Abolivier b1c0a4ceb3
Cleanup client_ips 2019-10-08 14:38:14 +01:00
Brendan Abolivier 8f1b385acc
Don't end up with 4 classes in registration 2019-10-08 14:36:33 +01:00
Brendan Abolivier 66ebea1723 Lint 2019-10-07 17:44:41 +01:00
Brendan Abolivier 0496eafbf4 Move roommember's bg updates to a dedicated store 2019-10-07 17:35:01 +01:00
Brendan Abolivier e106a0e4db Move user_directory's bg updates to a dedicated store 2019-10-07 17:34:45 +01:00
Brendan Abolivier cfccd2d78a Move state's bg updates to a dedicated store 2019-10-07 17:34:39 +01:00
Brendan Abolivier 841054ad96 Move search's bg updates to a dedicated store 2019-10-07 17:34:35 +01:00
Brendan Abolivier 81e6ffb536 Move registration's bg updates to a dedicated store 2019-10-07 17:34:29 +01:00
Brendan Abolivier 54f87e0734 Move media_repository's bg updates to a dedicated store 2019-10-07 17:34:26 +01:00
Brendan Abolivier cef9f6753e Move devices's bg updates to a dedicated store 2019-10-07 17:34:20 +01:00
Brendan Abolivier 2d3b4f42f0 Move deviceinbox's bg updates to a dedicated store 2019-10-07 17:34:16 +01:00
Brendan Abolivier 88957199e7 Move client_ips's bg updates to a dedicated store 2019-10-07 17:34:12 +01:00
Richard van der Hoff 276ae5c63e
add some logging to the rooms stats updates, to try to track down a flaky test (#6167) 2019-10-07 14:41:39 +01:00
Brendan Abolivier 97e2722723
Merge pull request #6175 from matrix-org/babolivier/fix_unique_user_filter_index
Fix unique_user_filter_index schema update
2019-10-07 13:48:43 +01:00
Brendan Abolivier c8e6c308c6
Fix unique_user_filter_index schema update 2019-10-07 13:15:35 +01:00
Erik Johnston 86f4705866
Merge pull request #6159 from matrix-org/erikj/cache_memberships
Cache room membership lookups in _get_joined_users_from_context
2019-10-07 13:15:00 +01:00
Erik Johnston 5119a4cac7 Fix bug where we didn't pull out event ID 2019-10-07 12:21:17 +01:00
Alexander Maznev 13c4345c84 Update `user_filters` table to have a unique index, and non-null columns (#1172) 2019-10-04 10:34:16 +01:00
Erik Johnston 6511071837 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cache_memberships 2019-10-03 17:51:06 +01:00
Richard van der Hoff 66537e10ce
add some metrics on the federation sender (#6160) 2019-10-03 17:47:20 +01:00
Erik Johnston 91f61fc6d7 Use the right error.... 2019-10-03 17:28:31 +01:00
Erik Johnston 84691da6c3 pep8 2019-10-03 17:27:18 +01:00
Erik Johnston a9610cdf02 Fixup names and comments 2019-10-03 17:26:56 +01:00
Erik Johnston d89ebf7c25 cachedList descriptor doesn't like typing 2019-10-03 17:23:11 +01:00
Erik Johnston c8145af8a9 Cache room membership lookups in _get_joined_users_from_context 2019-10-03 17:11:04 +01:00
Erik Johnston dca7e32d3d
Merge pull request #6154 from matrix-org/erikj/fix_appservice_pagination
Fix appservice room list pagination
2019-10-02 16:50:32 +01:00
Erik Johnston 22a6ffdf91
Merge pull request #6153 from matrix-org/erikj/fix_room_list_non_federatable
Fix not showing non-federatable rooms to remote room list queries
2019-10-02 16:09:15 +01:00
Erik Johnston 5d8ffdfe61
Merge pull request #6148 from matrix-org/erikj/find_next_generated
Bound find_next_generated_user_id DB query.
2019-10-02 16:09:02 +01:00
Erik Johnston 7a5f080f91 Fix appservice room list pagination 2019-10-02 15:47:22 +01:00
Erik Johnston 4c4f44930d Fix not showing non-federatable rooms to remote room list queries 2019-10-02 15:20:36 +01:00
Erik Johnston 03cf4385e0 Fix public room list pagination.
We incorrectly used `room_id` as to bound the result set, even though we
order by `joined_members, room_id`, leading to incorrect results after
pagination.
2019-10-02 15:11:17 +01:00
Erik Johnston a5166e4d5f
Land improved room list based on room stats (#6019)
Use room_stats and room_state for room directory search
2019-10-02 14:08:35 +01:00
Andrew Morgan 2a1470cd05
Fix yields and copy instead of move push rules on room upgrade (#6144)
Copy push rules during a room upgrade from the old room to the new room, instead of deleting them from the old room.

For instance, we've defined upgrading of a room multiple times to be possible, and push rules won't be transferred on the second upgrade if they're deleted during the first.

Also fix some missing yields that probably broke things quite a bit.
2019-10-02 12:04:22 +01:00
Erik Johnston d69fd53f74 Bound find_next_generated_user_id DB query.
We can easily bound the set of user IDs we pull out of the DB, so lets
do that.
2019-10-02 11:45:31 +01:00
Erik Johnston ecd254bc49 Merge branch 'release-v1.4.0' of github.com:matrix-org/synapse into develop 2019-10-02 11:08:07 +01:00
Erik Johnston b4fe7e19c0
Merge pull request #6146 from matrix-org/erikj/fix_destination_retry_timings
Fix errors storing large retry intervals.
2019-10-02 11:05:22 +01:00
Erik Johnston 35f392bb29
Merge pull request #6145 from matrix-org/erikj/fix_censored_redactions
Fix fetching censored redactions from DB
2019-10-02 10:57:30 +01:00
Erik Johnston f44f1d2e83 Fix errors storing large retry intervals.
We have set the max retry interval to a value larger than a postgres or
sqlite int can hold, which caused exceptions when updating the
destinations table.

To fix postgres we need to change the column to a bigint, and for sqlite
we lower the max interval to 2**62 (which is still incredibly long).
2019-10-02 10:36:27 +01:00
Erik Johnston ce7a3e7e27 Fix fetching censored redactions from DB
Fetching a censored redactions caused an exception due to the code
expecting redactions to have a `redact` key, which redacted redactions
don't have.
2019-10-02 10:14:01 +01:00