Commit Graph

3276 Commits (fca3a541e7e5845d61c519be7223a035374ed698)

Author SHA1 Message Date
Neil Johnson a0d0ba7862
Fix MAU reaping where reserved users are specified. (#6168) 2019-10-11 09:38:26 +01:00
Erik Johnston 83d86106a8
Merge pull request #6156 from matrix-org/erikj/postgres_any
Use Postgres ANY for selecting many values.
2019-10-10 16:41:36 +01:00
Erik Johnston b54b1e759a Fix SQLite take 2 2019-10-10 16:19:40 +01:00
Erik Johnston afb6d9d53b Fix SQLite 2019-10-10 15:55:41 +01:00
Erik Johnston 3bc687508f Remove add_in_list_sql_clause 2019-10-10 15:35:46 +01:00
Erik Johnston 9d06fb9cb1 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/postgres_any 2019-10-10 14:38:21 +01:00
Erik Johnston 8bc529c04d Merge branch 'develop' of github.com:matrix-org/synapse into erikj/disable_sql_bytes 2019-10-10 14:30:51 +01:00
Erik Johnston b161786c14 Replace IN usage with helper funcs 2019-10-10 13:15:49 +01:00
Erik Johnston b4fbf71187 Add helper funcs to use postgres ANY
This means that we can write queries with `col = ANY(?)`, which helps
postgres.
2019-10-10 13:15:24 +01:00
Erik Johnston 91f43dca39 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/disable_sql_bytes 2019-10-10 13:10:57 +01:00
Erik Johnston 9970f955ce Merge branch 'develop' of github.com:matrix-org/synapse into erikj/patch_inner 2019-10-10 11:51:50 +01:00
Richard van der Hoff a139420a3c
Fix races in room stats (and other) updates. (#6187)
Hopefully this will fix the occasional failures we were seeing in the room directory.

The problem was that events are not necessarily persisted (and `current_state_delta_stream` updated) in the same order as their stream_id. So for instance current_state_delta 9 might be persisted *before* current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it had done everything up to 9, and never came back to do stream_id 8.

We can solve this easily by only processing up to the stream_id where we know all events have been persisted.
2019-10-10 11:29:01 +01:00
Richard van der Hoff 562b4e51dd
Rewrite the user_filter migration again (#6184)
you can't plausibly ALTER TABLE in sqlite, so we create the new table with the
right schema to start with.
2019-10-10 11:28:23 +01:00
Erik Johnston 5c1f886c75 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/patch_inner 2019-10-09 16:52:21 +01:00
Erik Johnston 7f18b3d526 Do the update as a background index 2019-10-09 16:03:24 +01:00
Erik Johnston def5413480
Merge pull request #6185 from matrix-org/erikj/fix_censored_evnets
Fix inserting bytes as text in `censor_redactions`
2019-10-09 15:39:13 +01:00
Erik Johnston 1d3858371e Disable bytes usage with postgres
More often than not passing bytes to `txn.execute` is a bug (where we
meant to pass a string) that just happens to work if `BYTEA_OUTPUT` is
set to `ESCAPE`. However, this is a bit of a footgun so we want to
instead error when this happens, and force using `bytearray` if we
actually want to use bytes.
2019-10-08 16:28:57 +01:00
Erik Johnston e7631d84e6 Fix existing hex encoded json values in DB 2019-10-08 16:18:43 +01:00
Erik Johnston ced4784592 Fix inserting bytes as text 2019-10-08 16:18:43 +01:00
Brendan Abolivier c69324ffb5
Fix RegistrationStore 2019-10-08 14:48:33 +01:00
Brendan Abolivier b1c0a4ceb3
Cleanup client_ips 2019-10-08 14:38:14 +01:00
Brendan Abolivier 8f1b385acc
Don't end up with 4 classes in registration 2019-10-08 14:36:33 +01:00
Brendan Abolivier 66ebea1723 Lint 2019-10-07 17:44:41 +01:00
Brendan Abolivier 0496eafbf4 Move roommember's bg updates to a dedicated store 2019-10-07 17:35:01 +01:00
Brendan Abolivier e106a0e4db Move user_directory's bg updates to a dedicated store 2019-10-07 17:34:45 +01:00
Brendan Abolivier cfccd2d78a Move state's bg updates to a dedicated store 2019-10-07 17:34:39 +01:00
Brendan Abolivier 841054ad96 Move search's bg updates to a dedicated store 2019-10-07 17:34:35 +01:00
Brendan Abolivier 81e6ffb536 Move registration's bg updates to a dedicated store 2019-10-07 17:34:29 +01:00
Brendan Abolivier 54f87e0734 Move media_repository's bg updates to a dedicated store 2019-10-07 17:34:26 +01:00
Brendan Abolivier cef9f6753e Move devices's bg updates to a dedicated store 2019-10-07 17:34:20 +01:00
Brendan Abolivier 2d3b4f42f0 Move deviceinbox's bg updates to a dedicated store 2019-10-07 17:34:16 +01:00
Brendan Abolivier 88957199e7 Move client_ips's bg updates to a dedicated store 2019-10-07 17:34:12 +01:00
Richard van der Hoff 276ae5c63e
add some logging to the rooms stats updates, to try to track down a flaky test (#6167) 2019-10-07 14:41:39 +01:00
Brendan Abolivier 97e2722723
Merge pull request #6175 from matrix-org/babolivier/fix_unique_user_filter_index
Fix unique_user_filter_index schema update
2019-10-07 13:48:43 +01:00
Brendan Abolivier c8e6c308c6
Fix unique_user_filter_index schema update 2019-10-07 13:15:35 +01:00
Erik Johnston 86f4705866
Merge pull request #6159 from matrix-org/erikj/cache_memberships
Cache room membership lookups in _get_joined_users_from_context
2019-10-07 13:15:00 +01:00
Erik Johnston 5119a4cac7 Fix bug where we didn't pull out event ID 2019-10-07 12:21:17 +01:00
Alexander Maznev 13c4345c84 Update `user_filters` table to have a unique index, and non-null columns (#1172) 2019-10-04 10:34:16 +01:00
Erik Johnston 6511071837 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cache_memberships 2019-10-03 17:51:06 +01:00
Richard van der Hoff 66537e10ce
add some metrics on the federation sender (#6160) 2019-10-03 17:47:20 +01:00
Erik Johnston 91f61fc6d7 Use the right error.... 2019-10-03 17:28:31 +01:00
Erik Johnston 84691da6c3 pep8 2019-10-03 17:27:18 +01:00
Erik Johnston a9610cdf02 Fixup names and comments 2019-10-03 17:26:56 +01:00
Erik Johnston d89ebf7c25 cachedList descriptor doesn't like typing 2019-10-03 17:23:11 +01:00
Erik Johnston c8145af8a9 Cache room membership lookups in _get_joined_users_from_context 2019-10-03 17:11:04 +01:00
Erik Johnston dca7e32d3d
Merge pull request #6154 from matrix-org/erikj/fix_appservice_pagination
Fix appservice room list pagination
2019-10-02 16:50:32 +01:00
Erik Johnston 22a6ffdf91
Merge pull request #6153 from matrix-org/erikj/fix_room_list_non_federatable
Fix not showing non-federatable rooms to remote room list queries
2019-10-02 16:09:15 +01:00
Erik Johnston 5d8ffdfe61
Merge pull request #6148 from matrix-org/erikj/find_next_generated
Bound find_next_generated_user_id DB query.
2019-10-02 16:09:02 +01:00
Erik Johnston 7a5f080f91 Fix appservice room list pagination 2019-10-02 15:47:22 +01:00
Erik Johnston 4c4f44930d Fix not showing non-federatable rooms to remote room list queries 2019-10-02 15:20:36 +01:00