Commit Graph

6118 Commits (0cd01f5c9c1a25c9356f2666dfec83c7b55bfd08)

Author SHA1 Message Date
Richard van der Hoff d84bd51e95 Refactor logger config for workers
- to make it easier to add more config options.
2017-03-10 15:34:01 +00:00
Richard van der Hoff 9072a8c627 Reread log config on SIGHUP
When we are using a log_config file, reread it on SIGHUP.
2017-03-10 15:29:55 +00:00
Erik Johnston 8f267fa8a8 Fix it for the workers 2017-03-10 11:22:25 +00:00
Erik Johnston 64d62e41b8 Noop repated delete device inbox calls from /sync 2017-03-10 10:36:43 +00:00
Erik Johnston 3545e17f43 Add setdefault key to ExpiringCache 2017-03-10 10:30:49 +00:00
Richard van der Hoff 29235901b8 Move FederationServer._handle_new_pdu to FederationHandler
Unfortunately this significantly increases the size of the already-rather-big
FederationHandler, but the code fits more naturally here, and it paves the way
for the tighter integration that I need between handling incoming PDUs and
doing the join dance.

Other than renaming the existing `FederationHandler.on_receive_pdu` to
`_process_received_pdu` to make way for it, this just consists of the move, and
replacing `self.handler` with `self` and `self` with `self.replication_layer`.
2017-03-09 16:20:13 +00:00
Richard van der Hoff e8b1721290 Move sig check out of _handle_new_pdu
When we receive PDUs via `get_missing_events`, we have already checked their
sigs, so there is no need to do it again.
2017-03-09 15:50:44 +00:00
Richard van der Hoff 3406333a58 Factor _get_missing_events_for_pdu out of _handle_new_pdu
This should be functionally identical: it just seeks to improve readability by
reducing indentation.
2017-03-09 15:50:44 +00:00
Richard van der Hoff 45d173a59a Fix docstring 2017-03-09 15:50:29 +00:00
David Baker 663396e45d Merge pull request #1971 from matrix-org/dbkr/msisdn_signin
Support registration & login with phone number
2017-03-09 10:18:51 +00:00
David Baker ece7e00048 Comment when our 3pids would be incomplete 2017-03-08 19:07:18 +00:00
David Baker 9d0d40fc15 Docs 2017-03-08 19:05:29 +00:00
David Baker 3edc57296d Incorrectly copied copyright
This file post-dates OM
2017-03-08 19:00:51 +00:00
David Baker 727124a762 Not any more, it doesn't 2017-03-08 19:00:23 +00:00
Richard van der Hoff 6ad71cc29d Remove spurious SQL logging (#1972)
looks like the upsert function was accidentally sending sql logging to the
general logger. We already log the sql in `txn.execute`.
2017-03-08 18:00:44 +00:00
David Baker d4d3629aaf Better error message 2017-03-08 17:01:26 +00:00
Erik Johnston 3170c56e07 Bump changelog and version 2017-03-08 13:00:47 +00:00
David Baker 1c99934b28 pep8 2017-03-08 11:58:20 +00:00
David Baker a9e2b9ec16 Add msisdn util file 2017-03-08 11:53:36 +00:00
David Baker 85bb322333 Pull out datastore in initialiser 2017-03-08 11:51:25 +00:00
David Baker 65d43f3ca5 Minor fixes from PR feedback 2017-03-08 11:48:43 +00:00
David Baker 0e0aee25c4 Fix log line 2017-03-08 11:46:22 +00:00
David Baker 82c5e7de25 Typos 2017-03-08 11:42:44 +00:00
David Baker 2e27339add Refector out assert_params_in_request
and replace requestEmailToken where we meant requestMsisdnToken
2017-03-08 11:37:34 +00:00
David Baker 88df6c0c9a Factor out msisdn canonicalisation
Plus a couple of other minor fixes
2017-03-08 11:03:39 +00:00
David Baker 402a7bf63d Fix pep8 2017-03-08 09:33:40 +00:00
David Baker 00466e2feb Support new login format
https://docs.google.com/document/d/1-6ZSSW5YvCGhVFDyD2QExAUAdpCWjccvJT5xiyTTG2Y/edit#
2017-03-07 16:37:23 +00:00
Erik Johnston ac5491f563 Select distinct devices from DB
Otherwise we might pull out tonnes of duplicate user_ids and this can
make synapse sad.
2017-03-06 11:10:14 +00:00
David Baker b0effa2160 Add msisdns as 3pids during registration
and support binding them with the bind_msisdn param
2017-03-03 18:34:39 +00:00
Erik Johnston 82f7f1543b Merge pull request #1964 from matrix-org/erikj/device_list_update_fix
Fix device list update to not constantly resync
2017-03-03 16:18:16 +00:00
Erik Johnston f2581ee8b8 Don't keep around old stream IDs forever 2017-03-03 16:02:53 +00:00
Erik Johnston 9834367eea Spelling 2017-03-03 15:31:57 +00:00
Erik Johnston da52d3af31 Fix up 2017-03-03 15:29:13 +00:00
David Baker ad882cd54d Just return the deferred straight off
defer.returnValue doth not maketh a generator: it would need a
yield to be a generator, and this doesn't need a yield.
2017-03-01 18:08:51 +00:00
David Baker 3557cf34dc Merge remote-tracking branch 'origin/develop' into dbkr/msisdn_signin 2017-03-01 17:20:37 +00:00
Erik Johnston d766343668 Add index to device_lists_stream 2017-03-01 15:56:30 +00:00
Erik Johnston 36be39b8b3 Fix device list update to not constantly resync 2017-03-01 14:12:11 +00:00
Erik Johnston 3365117151 Clobber old device list stream entries 2017-03-01 10:21:30 +00:00
Erik Johnston e933a2712d Don't log unknown cache warnings in workers 2017-02-28 16:22:41 +00:00
Erik Johnston d638a7484b Merge pull request #1959 from matrix-org/erikj/intern_once
Intern table column names once
2017-02-28 15:15:16 +00:00
Erik Johnston b84907bdbb Intern table column names once 2017-02-28 14:38:16 +00:00
Erik Johnston e4919b9329 Add stream_id index to device_lists_outbound_pokes
As this is used for replication streaming
2017-02-28 11:19:06 +00:00
Erik Johnston 8a12b6f1eb Fix up txn name 2017-02-28 10:15:50 +00:00
Erik Johnston 848cf95ea0 Pop with default value to stop throwing 2017-02-28 10:02:54 +00:00
Erik Johnston eda96586ca Merge pull request #1955 from matrix-org/erikj/current_state_query_bypass
Don't fetch current state in common case
2017-02-27 19:15:59 +00:00
Erik Johnston 64a2cef9bb Pop rather than del from dict 2017-02-27 19:15:36 +00:00
Erik Johnston a41dce8f8a Remove needless check 2017-02-27 18:54:43 +00:00
Erik Johnston c0d6045776 It should be all 2017-02-27 18:45:24 +00:00
Erik Johnston 49f4bc4709 Don't fetch current state in common case
Currently we fetch the list of current state events whenever we send
something in a room. This is overkill for the common case of persisting
a simple chain of non-state events, so lets handle that case specially.
2017-02-27 18:33:41 +00:00
Erik Johnston fc5d876dba Merge pull request #1954 from matrix-org/erikj/cache_device2
Cache get_user_devices_from_cache
2017-02-27 16:36:40 +00:00
Erik Johnston f58dbb02a6 Cache get_user_devices_from_cache 2017-02-27 16:22:12 +00:00
Erik Johnston acf6d4d2e3 Merge pull request #1945 from jkolo/fix_dynamic_thumbnails_aspect
Fix #1677 (dynamic thumbnails aspect)
2017-02-27 09:51:52 +00:00
Jurek aea5461488 Fix dynamic thumbnails aspect 2017-02-24 22:43:27 +01:00
Erik Johnston 1a4f8022e6 Strip newlines from SQL queries 2017-02-23 11:15:31 +00:00
Erik Johnston b2d20e94fa Remove lock from rotate notifs 2017-02-22 14:24:02 +00:00
Erik Johnston 7455ba436a Ensure we pass positive ints to delay function 2017-02-22 12:08:14 +00:00
Erik Johnston b7442c3e2b Store looping call 2017-02-21 13:59:25 +00:00
Erik Johnston a3708a1885 Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-02-21 13:46:27 +00:00
Erik Johnston 30ecfef5a3 Bump version and changelog 2017-02-21 13:43:36 +00:00
Richard van der Hoff 0c4cf9372b Fix a race in transaction queue
It was theoretically possible for a PDU to get queued and not sent for ages. On
closer inspection I think there were bigger problems elsewhere, but we might as
well fix this since it's easy.
2017-02-20 16:46:25 +00:00
Erik Johnston 6226a27bf8 Remove unused param 2017-02-20 16:01:54 +00:00
Erik Johnston efff39c030 Fix /context/ visibiltiy rules 2017-02-20 16:01:49 +00:00
Erik Johnston 17673404fb Remove unused param 2017-02-20 15:02:01 +00:00
Erik Johnston 7f026792e1 Fix /context/ visibiltiy rules 2017-02-20 14:54:50 +00:00
Erik Johnston 699be7d1be Fix up notif rotation 2017-02-18 14:42:39 +00:00
David Baker ec8d7603e6 Merge pull request #1925 from matrix-org/dbkr/pushers_lang_lengthen
Make the pushers lang field column longer
2017-02-17 11:29:06 +00:00
David Baker 8c87bb550e Merge pull request #1922 from matrix-org/dbkr/allow_forget_for_ban
Allow forgetting rooms you're banned from
2017-02-17 10:52:30 +00:00
David Baker 4aa29508af Use TEXT rather than VARCHAR
While we're changing anyway
2017-02-17 10:51:49 +00:00
David Baker b4017539d4 Make the pushers lang field column longer
To accommodate things like zh-Hans-CN

Fixes https://github.com/vector-im/riot-ios/issues/1031
2017-02-17 10:42:57 +00:00
Erik Johnston 138e030cfe Comment 2017-02-16 15:03:36 +00:00
Erik Johnston 502ae6c663 Comment 2017-02-16 14:47:11 +00:00
Erik Johnston e6acf0c399 Store the default push actions in a more efficient manner 2017-02-16 14:40:24 +00:00
Erik Johnston 04eca2589d Merge pull request #1916 from matrix-org/erikj/push_actions_delete
Aggregate event push actions
2017-02-16 15:28:58 +01:00
David Baker 474c9aadbe Allow forgetting rooms you're banned from 2017-02-15 19:32:20 +00:00
Richard van der Hoff 7dcbcca68c Merge pull request #1921 from matrix-org/rav/fix_key_changes
Fix bugs in the /keys/changes api
2017-02-15 11:25:16 +00:00
David Baker fa467e62a9 Merge pull request #1917 from matrix-org/dbkr/make_ban_reasons_work
Make kick & ban reasons work
2017-02-14 16:10:06 +00:00
David Baker 355d62c499 Make kick & ban reasons work
We somehow specced APIs with reason strings, preserve the content
in the events  and even have the clients display them, but failed
to actually pass the parameter through to the event content.
2017-02-14 15:10:55 +00:00
David Baker ce3e583d94 WIP support for msisdn 3pid proxy methods 2017-02-14 15:05:55 +00:00
Richard van der Hoff fc2f29c1d0 Fix bugs in the /keys/changes api
* `get_forward_extremeties_for_room` takes a numeric `stream_ordering`. We were
  passing a `RoomStreamToken`, which meant that it returned the *current*
  extremities, rather than those corresponding to the `from_token`. However:
* `get_state_ids_for_events` required a second ('types') parameter; this meant
  that a `TypeError` was thrown and we ended up acting as though there was *no*
  prev state.
* `get_state_ids_for_events` actually returns a map from event_id to state
  dictionary - just looking up the state keys in it again meant that we acted
  as though there was no prev state. We now check if each member's state has
  changed since *any* of the extremities.

Also add/fix some comments.
2017-02-14 13:59:50 +00:00
Erik Johnston ce3c8df6df Less aggressive timers 2017-02-14 13:41:24 +00:00
Erik Johnston 095b45c165 Aggregate event push actions 2017-02-14 13:39:41 +00:00
Erik Johnston 795f8e3fe7 Merge pull request #1873 from matrix-org/erikj/delete_push_actions
Be more agressive about purging old room event_push_actions
2017-02-14 14:29:04 +01:00
Erik Johnston d7457c7661 Merge pull request #1914 from matrix-org/erikj/cache_presence
Cache get_presence storage
2017-02-13 16:59:19 +01:00
Erik Johnston 9e617cd4c2 Cache get_presence storage 2017-02-13 13:50:03 +00:00
Kegan Dougal d0497425f8 Ordering is important on errbacks so add the cleanup func before creating an ObservableDeferred 2017-02-13 13:49:44 +00:00
Kegan Dougal 808ddf0ae7 Pop the txn from the map in case it has already been deleted somehow 2017-02-13 13:36:15 +00:00
Kegan Dougal feb15dc99f Don't cache errors at all 2017-02-13 13:33:12 +00:00
Kegan Dougal ecd7e36047 http txns: Do not cache error responses
Previously we did. This meant that, amongst other errors, rate-limiting errors
would be cached and prevent messages with that txn ID being sent.
2017-02-13 13:16:48 +00:00
Erik Johnston 6bba80241c Merge pull request #1912 from matrix-org/markjh/roominitialsync
Add db functions needed for room initial sync to slave
2017-02-13 12:20:21 +01:00
Mark Haines 3a46280ca3 Add db functions needed for room initial sync to slave 2017-02-13 11:16:53 +00:00
Tyler Smith df4407d665 Fix typo in config comments.
Signed-off-by: Tyler Smith <tylersmith.me@gmail.com>
2017-02-11 23:02:57 -08:00
Erik Johnston 1beda9c8a7 Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-02-09 10:46:58 +00:00
Erik Johnston 505bfd82bb Update version and changelog 2017-02-09 10:41:02 +00:00
Erik Johnston 52cd019a54 Make None check explicit 2017-02-08 16:04:29 +00:00
Erik Johnston f20cd34858 Merge pull request #1892 from matrix-org/erikj/rejection_fwd_extrem
Ignore new rejected events when working out forward extremities.
2017-02-08 16:59:06 +01:00
Erik Johnston 7723b4caa4 Ignore new rejected events when working out forward extremeties. 2017-02-08 14:48:06 +00:00
David Baker 063a1251a9 Remove a few aspirational but unused constants
from the Kegan era
2017-02-08 11:36:08 +00:00
Erik Johnston af6da6db2d Merge pull request #1784 from morteza-araby/user-admin
Administration functionalities
2017-02-06 16:21:10 +01:00
Erik Johnston 38434a7fbb Bump changelog and version 2017-02-04 08:27:51 +00:00
Erik Johnston 84f600b2ee Bump changelog and version 2017-02-02 18:58:33 +00:00
Erik Johnston aec1708c53 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.0 2017-02-02 18:57:05 +00:00
Erik Johnston f3c8658217 Merge pull request #1879 from matrix-org/erikj/bump_cache_factors
Bump cache sizes for common membership queries
2017-02-02 18:56:04 +00:00
Erik Johnston 38258a0976 Bump cache sizes for common membership queries 2017-02-02 18:45:55 +00:00
Erik Johnston a597994fb6 Measure new device list stuff 2017-02-02 18:36:17 +00:00
Erik Johnston 82b3e0851c Bump version and changelog 2017-02-02 17:15:17 +00:00
Erik Johnston f8c407a13b Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.0 2017-02-02 16:50:28 +00:00
Erik Johnston 1232ae41cf Use new get_users_who_share_room_with_user 2017-02-02 15:25:00 +00:00
Erik Johnston 99fa03e8b5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/device_list_fixes 2017-02-02 15:23:45 +00:00
Erik Johnston a8331897aa Merge pull request #1876 from matrix-org/erikj/shared_member_store
Make presence.get_new_events a bit faster
2017-02-02 15:20:14 +00:00
Erik Johnston 0f3e296cb7 Fix replication 2017-02-02 15:02:03 +00:00
Erik Johnston 6826593b81 sets aren't JSON serializable 2017-02-02 14:55:54 +00:00
Erik Johnston 6b61060b51 Comment 2017-02-02 14:47:15 +00:00
Erik Johnston 46ecd9fd6d Use stream_ordering_to_exterm for /keys/changes 2017-02-02 14:27:19 +00:00
Erik Johnston 9efcc3f3be Comment 2017-02-02 13:50:22 +00:00
Erik Johnston 832e9c52ca Comment 2017-02-02 13:09:56 +00:00
Erik Johnston 54a79c1d37 Make presence.get_new_events a bit faster
We do this by caching the set of users a user shares rooms with.
2017-02-02 13:07:18 +00:00
Morteza Araby 2849d3f29d admin,storage: added more administrator functionalities
administrators can now:
 - Set displayname of users
 - Update user avatars
 - Search for users by user_id
 - Browse all users in a paginated API
 - Reset user passwords
 - Deactivate users

Helpers for doing paginated queries has also been added to storage

Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>
2017-02-02 14:02:26 +01:00
Erik Johnston bfe3f5815f Update changelog 2017-02-02 11:10:41 +00:00
Erik Johnston cc01eae332 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.0 2017-02-02 11:05:57 +00:00
Erik Johnston 51adaac953 Fix email push in pusher worker
This was broken when device list updates were implemented, as Mailer
could no longer instantiate an AuthHandler due to a dependency on
federation sending.
2017-02-02 10:53:36 +00:00
Erik Johnston 10e0737569 Bump version and changelog 2017-02-02 10:02:28 +00:00
Erik Johnston fac3c03087 Be more agressive about purging old room event_push_actions 2017-02-01 18:27:24 +00:00
Erik Johnston fbfe44bb4d Doc args 2017-02-01 17:52:57 +00:00
Erik Johnston d61a04583e Comment 2017-02-01 17:35:23 +00:00
Erik Johnston 7e919bdbd0 Include newly joined users in /keys/changes API 2017-02-01 17:33:16 +00:00
Erik Johnston df4ecff5a9 Correctly raise exceptions for ratelimitng. Ratelimit on 401 2017-02-01 15:42:19 +00:00
Erik Johnston 6d6591880e Wake sync up for device changes 2017-02-01 15:15:16 +00:00
Erik Johnston bd84387ac6 Merge pull request #1869 from matrix-org/erikj/device_list_stream
Implement /keys/changes
2017-02-01 13:25:26 +00:00
Erik Johnston ebfaff84c9 Merge pull request #1870 from matrix-org/erikj/cache_get_all_new_events
Add a small cache get_all_new_events
2017-02-01 13:22:02 +00:00
Erik Johnston 73d676dc8b Comment 2017-02-01 13:17:17 +00:00
Erik Johnston 62f6b86ba7 Merge pull request #1868 from matrix-org/erikj/replication_cache
Only invalidate membership caches based on the cache stream
2017-02-01 13:12:30 +00:00
Erik Johnston f6124311fd Add m.room.member type to query 2017-02-01 11:59:17 +00:00
Erik Johnston 88a4d54883 Merge pull request #1867 from matrix-org/erikj/member_index
Add an index to make membership queries faster
2017-02-01 11:44:27 +00:00
Erik Johnston 368c88c487 Add a small cache get_all_new_events 2017-02-01 10:50:44 +00:00
Erik Johnston 5deaf9e30b Up get_latest_event_ids_in_room cache 2017-02-01 10:39:41 +00:00
Erik Johnston acb501c46d Comment 2017-02-01 10:32:49 +00:00
Erik Johnston 97479d0c54 Implement /keys/changes 2017-02-01 10:30:03 +00:00
Erik Johnston 06567ec513 Merge pull request #1866 from matrix-org/erikj/device_list_fixes
Better handle 404 response for federation /send/
2017-02-01 09:44:14 +00:00
Erik Johnston 458b6f4733 Only invalidate membership caches based on the cache stream
Before we completely invalidated get_users_in_room whenever we updated
any current_state_events table. This was way too aggressive.
2017-01-31 16:09:03 +00:00
Erik Johnston fe08db2713 Remove explicit < 400 check as apparently this is confusing 2017-01-31 15:21:32 +00:00
Erik Johnston 21b7375778 Add an index to make membership queries faster 2017-01-31 15:15:57 +00:00
Erik Johnston 4c0ec15bdc Comment 2017-01-31 13:53:46 +00:00
Erik Johnston 85c590105f Comment 2017-01-31 13:46:38 +00:00
Erik Johnston ae7a132f38 Better handle 404 response for federation /send/ 2017-01-31 13:40:09 +00:00
Erik Johnston ac001dabdc Merge pull request #1864 from matrix-org/erikj/device_list_fixes
Fix clearing out old device list outbound pokes
2017-01-31 13:35:35 +00:00
Erik Johnston bfb3d255b1 Merge pull request #1862 from matrix-org/erikj/presence_update
Use DB cache of joined users for presence
2017-01-31 13:23:24 +00:00
Erik Johnston ab55794b6f Fix deletion of old sent devices correctly 2017-01-31 13:22:41 +00:00
Erik Johnston d3169e8d28 Only fetch with row ts and count > 1 2017-01-31 11:20:03 +00:00
Erik Johnston 05b9f48ee5 Fix clearing out old device list outbound pokes 2017-01-31 10:08:55 +00:00
Erik Johnston 4b3403ca9b Stream cache invalidations for room membership storage functions 2017-01-30 17:28:22 +00:00
Erik Johnston 1c13c9f6b6 Don't have such a large cache 2017-01-30 17:12:14 +00:00
Erik Johnston c7a26b7c32 Fix unit tests 2017-01-30 17:11:24 +00:00
Erik Johnston fd1c18c088 Use DB cache of joined users for presence 2017-01-30 17:00:24 +00:00
Erik Johnston c2c9a78db9 Noop device key changes if they're the same 2017-01-30 16:55:04 +00:00
Erik Johnston e75a779d9e Fix query 2017-01-30 16:38:20 +00:00
Erik Johnston 828db669ec Use get_users_in_room and declare it iterable 2017-01-30 16:37:22 +00:00
Erik Johnston 9636b2407d Merge pull request #1857 from matrix-org/erikj/device_list_stream
Implement device lists updates over federation
2017-01-30 14:35:21 +00:00
Erik Johnston 3670025e64 Rename func 2017-01-30 14:11:31 +00:00
Erik Johnston 4ac363a168 Remove debug logging 2017-01-30 14:10:12 +00:00
Erik Johnston d360c97ae1 Clear out old destination pokes. 2017-01-30 10:14:37 +00:00
Erik Johnston 76100203ab Always use the latest stream_id, sent or unsent 2017-01-30 10:14:25 +00:00
Erik Johnston d1e1fd6210 Add ts column to device_lists_outbound_pokes 2017-01-27 15:23:48 +00:00
Erik Johnston 252b503fc8 Hook device list updates to replication 2017-01-27 14:31:35 +00:00
Erik Johnston 84a35f32c7 Comment 2017-01-27 10:35:12 +00:00
Erik Johnston c517a19c2d Comment 2017-01-27 10:33:26 +00:00
Erik Johnston 738a2867c8 SQL param ordering 2017-01-27 10:31:29 +00:00
Erik Johnston 755adff0e4 User if rather than for 2017-01-27 10:31:06 +00:00
Erik Johnston 888c59c955 Better name 2017-01-27 10:29:47 +00:00
Erik Johnston f25a4a4692 Remove unused param 2017-01-27 10:27:39 +00:00
Erik Johnston 31aca5589c Fix on sqlite: use left rather than outer join 2017-01-26 16:55:50 +00:00
Erik Johnston 76d40f4904 Handle users leaving rooms 2017-01-26 16:39:33 +00:00
Erik Johnston fbfad76c03 Add comments 2017-01-26 16:33:21 +00:00
Erik Johnston c974116f19 Implement device key caching over federation 2017-01-26 16:07:24 +00:00
Erik Johnston 51e9fe36e4 Fix up sending of m.device_list_update edus 2017-01-25 16:55:21 +00:00
Erik Johnston 2367c5568c Add basic implementation of local device list changes 2017-01-25 14:27:27 +00:00
Paul "LeoNerd" Evans 10e48d8310 Don't clobber a displayname or avatar_url if provided by an m.room.member event 2017-01-24 18:06:07 +00:00
Erik Johnston ba8e144554 Merge branch 'erikj/current_state_fix' into develop 2017-01-23 16:15:10 +00:00
Erik Johnston f5b46482f4 Merge pull request #1840 from matrix-org/erikj/current_state_fix
Insert delta of current_state_events to be more efficient
2017-01-23 16:14:34 +00:00
Erik Johnston fdf2a31a51 Typo 2017-01-23 16:14:14 +00:00
Erik Johnston 41dab8a222 Fix bug where current_state_events renamed to current_state_ids 2017-01-23 15:22:48 +00:00
Erik Johnston c77b24c092 Refactor to calculate state delta outside transaction 2017-01-23 14:51:33 +00:00
Erik Johnston 5d2134d485 Comments 2017-01-20 17:13:24 +00:00
Erik Johnston a55fa2047f Insert delta of current_state_events to be more efficient 2017-01-20 17:10:18 +00:00
Erik Johnston 3d9d48fffb Merge pull request #1836 from matrix-org/erikj/current_state_fix
Derive current_state_events from state groups
2017-01-20 15:14:05 +00:00
Erik Johnston d0897dead5 Spelling 2017-01-20 15:05:11 +00:00
Erik Johnston 567aa35b67 Update all call sites after rename 2017-01-20 14:40:31 +00:00
Erik Johnston f2f40e64a9 Comments 2017-01-20 14:38:13 +00:00
Erik Johnston 4c6a31cd6e Calculate the forward extremeties once 2017-01-20 14:28:53 +00:00
Erik Johnston 86063d4321 Merge pull request #1835 from matrix-org/erikj/fix_workers
Make worker listener config backwards compat
2017-01-20 11:55:56 +00:00
Erik Johnston 09eb08f910 Derive current_state_events from state groups 2017-01-20 11:52:51 +00:00
Erik Johnston 97efe99ae9 Make worker listener config backwards compat 2017-01-20 11:45:29 +00:00
Marvin Steadfast 86e6165687 Added default config for turn username and password 2017-01-19 14:35:55 +01:00
Marvin Steadfast 1e38be3a7a Added username and password for turn server
It makes it possible to use a turn server that needs a username and
password instead of a token.
2017-01-19 14:08:20 +01:00
Erik Johnston 841c228533 Merge pull request #1828 from matrix-org/erikj/iterable_cache_size
Update LruCache size estimate on clear
2017-01-18 14:57:54 +00:00
Erik Johnston c430111d0e Update LruCache size estimate on clear 2017-01-18 14:55:23 +00:00
David Baker 97d3918377 Merge pull request #1811 from aperezdc/unhardcode-riot-urls
Allow configuring the Riot URL used in notification emails
2017-01-18 14:38:49 +00:00
David Baker 8c5009b628 Lowercase all email addresses before querying db
Since we store all emails in the DB in lowercase
(https://github.com/matrix-org/synapse/pull/1170)
2017-01-18 13:25:56 +00:00
Erik Johnston ae7b4da4cc Merge pull request #1823 from matrix-org/erikj/load_events_logs
Remove loading events logs
2017-01-18 11:07:58 +00:00
Erik Johnston fc7cae8aa3 Merge pull request #1824 from matrix-org/erikj/retry_host_log
Lower the not retrying host log line to debug
2017-01-18 11:07:51 +00:00
Erik Johnston f9058ca785 Merge pull request #1822 from matrix-org/erikj/statE_logging
Change resolve_state_groups call site logging to DEBUG
2017-01-18 11:02:03 +00:00
Erik Johnston f648313f98 Merge pull request #1821 from matrix-org/erikj/cache_metrics_string_intern
Measure metrics of string_cache
2017-01-18 10:57:39 +00:00
Erik Johnston 15f012032c Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split
Optimise state resolution
2017-01-18 10:53:00 +00:00
Erik Johnston 4ec1cf49e2 Lower loading events log to DEBUG 2017-01-17 17:28:32 +00:00
Erik Johnston f878f64f43 Lower the not retrying host log line to debug 2017-01-17 17:20:39 +00:00
Erik Johnston 5f027d1fc5 Change resolve_state_groups call site logging to DEBUG 2017-01-17 17:07:15 +00:00
Erik Johnston 380dba1020 Measure metrics of string_cache 2017-01-17 17:04:46 +00:00
Erik Johnston ed4d176152 PEP8 2017-01-17 15:27:28 +00:00
Mark Haines c6064a7ba6 Only construct sets when necessary 2017-01-17 15:23:07 +00:00
Erik Johnston a8594fd19f Use better names 2017-01-17 14:59:03 +00:00
Erik Johnston 7fae460402 Merge pull request #1820 from matrix-org/erikj/push_tools
Get state at event rather than for room in push
2017-01-17 14:58:26 +00:00
Erik Johnston 37b4c7d8a9 Fix typo in return type 2017-01-17 14:43:32 +00:00
Erik Johnston e5d2df9c34 Use better variable name 2017-01-17 14:32:53 +00:00
Erik Johnston 04006bb7f0 Get state at event rather than for room in push 2017-01-17 14:31:21 +00:00
Erik Johnston ce59a2faad Correctly handle case of rejected events in state res 2017-01-17 14:18:53 +00:00
Erik Johnston 633f97151c Check event is in state_map 2017-01-17 13:33:54 +00:00
Erik Johnston e6153e1bd1 Fix couple of federation state bugs 2017-01-17 13:22:34 +00:00
Erik Johnston 5d6bad1b3c Optimise state resolution 2017-01-17 13:22:19 +00:00
Erik Johnston e8ecbb6f20 Merge pull request #1812 from matrix-org/erikj/state_auth_splitout_split
Split out static state methods from StateHandler
2017-01-17 11:55:18 +00:00
Erik Johnston d6c75cb7c2 Rename and comment tree_to_leaves_iterator 2017-01-17 11:47:03 +00:00
Erik Johnston 1ccd5676e3 Remove needless call to evict() 2017-01-17 11:42:26 +00:00
Erik Johnston d906206049 Increase state_group_cache_size 2017-01-17 11:31:08 +00:00
Erik Johnston f85b6ca494 Speed up cache size calculation
Instead of calculating the size of the cache repeatedly, which can take
a long time now that it can use a callback, instead cache the size and
update that on insertion and deletion.

This requires changing the cache descriptors to have two caches, one for
pending deferreds and the other for the actual values. There's no reason
to evict from the pending deferreds as they won't take up any more
memory.
2017-01-17 11:18:13 +00:00
Erik Johnston 6d00213e80 Use OrderedDict in ExpiringCache 2017-01-16 15:33:22 +00:00
Erik Johnston 897f8752da Up cache max entries for state 2017-01-16 15:08:17 +00:00
Erik Johnston beda469bc6 Put staticmethods at module level 2017-01-16 15:05:24 +00:00
Erik Johnston 46aebbbcbf Add support for 'iterable' to ExpiringCache 2017-01-16 14:57:23 +00:00
Erik Johnston 01521299c7 Increase cache size limit 2017-01-16 11:56:51 +00:00
Erik Johnston 2fae34bd2c Optionally measure size of cache by sum of length of values 2017-01-13 17:46:17 +00:00
Erik Johnston 95a22ae194 Merge pull request #1810 from matrix-org/erikj/state_auth_splitout_split
Split out static auth methods from Auth object
2017-01-13 16:32:27 +00:00
Erik Johnston ec0a523ac3 Split out static state methods from StateHandler 2017-01-13 15:25:06 +00:00
Erik Johnston e178feca3f Remove unused function 2017-01-13 15:16:45 +00:00
Erik Johnston f0325a9ccc Merge pull request #1793 from matrix-org/erikj/change_device_inbox_index
Change device_inbox stream index to include user
2017-01-13 15:14:51 +00:00
Erik Johnston c050f493dd Add comment 2017-01-13 15:14:41 +00:00
Adrian Perez de Castro a3e4a198e3
Allow configuring the Riot URL used in notification emails
The URLs used for notification emails were hardcoded to use either matrix.to
or vector.im; but for self-hosted setups where Riot is also self-hosted it
may be desirable to allow configuring an alternative Riot URL.

Fixes #1809.

Signed-off-by: Adrian Perez de Castro <aperez@igalia.com>
2017-01-13 17:12:04 +02:00
Erik Johnston 8b2fa38256 Split event auth code into seperate module 2017-01-13 15:07:32 +00:00
Erik Johnston 641ccdbb14 Merge pull request #1795 from matrix-org/erikj/port_defaults
Restore default bind address
2017-01-13 13:02:59 +00:00
Erik Johnston ebf94aff8d Fix spurious Unhandled Error log lines 2017-01-12 17:19:47 +00:00
Erik Johnston bf5c9706d9 Remove full_twisted_stacktraces option
The debug 'full_twisted_stacktraces' flag caused synapse to rewrite
twisted deferreds to always fire the callback on the next reactor tick.
This was to force the deferred to always store the stacktraces on
exceptions, and thus be more likely to have a full stacktrace when it
reaches the final error handlers and gets printed to the logs.

Dynamically rewriting things is generally bad, and in particular this
change violates assumptions of various bits of Twisted. This wouldn't
necessarily be so bad, but it turns out this option has been turned on
on some production servers.

Turning the option can cause e.g. #1778.

For now, lets just entirely nuke this option.
2017-01-12 10:32:52 +00:00
Erik Johnston 7b62d0bc70 Add missing None check 2017-01-11 10:57:03 +00:00
Erik Johnston 7e6c2937c3 Split out static auth methods from Auth object 2017-01-10 18:16:54 +00:00
Erik Johnston b1dfd20292 Pop bind_address 2017-01-10 17:23:18 +00:00
Erik Johnston edd6cdfc9a Restore default bind address 2017-01-10 17:21:41 +00:00
Erik Johnston 8a0fddfd73 Remove spurious for..else.. 2017-01-10 16:30:53 +00:00
Erik Johnston d524bc9110 Merge pull request #1792 from matrix-org/erikj/limit_cache_prefill_device
Limit number of entries to prefill from cache
2017-01-10 15:42:00 +00:00
Erik Johnston d2b00d0866 Merge pull request #1790 from matrix-org/erikj/linearizer
Add paranoia exception catch in Linearizer
2017-01-10 15:38:30 +00:00
Erik Johnston ab655dca33 Explicitly close the cursor 2017-01-10 15:15:25 +00:00
Erik Johnston 5a32e9273e Don't disable autocommit 2017-01-10 15:11:27 +00:00
Erik Johnston caddadfc5a Change device_inbox stream index to include user
This makes fetching the nost recently changed users much tricker, and
brings it in line with e.g. presence_stream indices.
2017-01-10 15:04:57 +00:00
Erik Johnston dd52d4de4c Limit number of entries to prefill from cache
Some tables, like device_inbox, take a long time to query at startup for
the stream change cache prefills. This is likely because they are slower
growing streams and so are more fragmented on disk. For now, lets pull
fewer entries out to make startup quicker.

In future, we should add a better index to make it even faster.
2017-01-10 14:34:50 +00:00
Mark Haines 32019c9897 Log which files we saved attachments to in the media_repository 2017-01-10 14:19:50 +00:00