Commit Graph

5780 Commits (acb501c46d75247329f49a1eef3baf6d8af0cba1)

Author SHA1 Message Date
Erik Johnston 7b62d0bc70 Add missing None check 2017-01-11 10:57:03 +00:00
Erik Johnston 7e6c2937c3 Split out static auth methods from Auth object 2017-01-10 18:16:54 +00:00
Erik Johnston b1dfd20292 Pop bind_address 2017-01-10 17:23:18 +00:00
Erik Johnston edd6cdfc9a Restore default bind address 2017-01-10 17:21:41 +00:00
Erik Johnston 8a0fddfd73 Remove spurious for..else.. 2017-01-10 16:30:53 +00:00
Erik Johnston d524bc9110 Merge pull request #1792 from matrix-org/erikj/limit_cache_prefill_device
Limit number of entries to prefill from cache
2017-01-10 15:42:00 +00:00
Erik Johnston d2b00d0866 Merge pull request #1790 from matrix-org/erikj/linearizer
Add paranoia exception catch in Linearizer
2017-01-10 15:38:30 +00:00
Erik Johnston ab655dca33 Explicitly close the cursor 2017-01-10 15:15:25 +00:00
Erik Johnston 5a32e9273e Don't disable autocommit 2017-01-10 15:11:27 +00:00
Erik Johnston caddadfc5a Change device_inbox stream index to include user
This makes fetching the nost recently changed users much tricker, and
brings it in line with e.g. presence_stream indices.
2017-01-10 15:04:57 +00:00
Erik Johnston dd52d4de4c Limit number of entries to prefill from cache
Some tables, like device_inbox, take a long time to query at startup for
the stream change cache prefills. This is likely because they are slower
growing streams and so are more fragmented on disk. For now, lets pull
fewer entries out to make startup quicker.

In future, we should add a better index to make it even faster.
2017-01-10 14:34:50 +00:00
Mark Haines 32019c9897 Log which files we saved attachments to in the media_repository 2017-01-10 14:19:50 +00:00
Erik Johnston 657488113e Merge pull request #1789 from matrix-org/erikj/decouple_presence
Don't block messages sending on bumping presence
2017-01-10 14:06:05 +00:00
Erik Johnston 3b4de17d2b Comment 2017-01-10 14:05:53 +00:00
Erik Johnston 7d0981b312 Merge pull request #1787 from matrix-org/erikj/linearize_member
Linearize updates to membership via PUT /state/
2017-01-10 14:04:54 +00:00
Erik Johnston f477370c0c Add paranoia exception catch in Linearizer 2017-01-10 14:04:13 +00:00
Erik Johnston 586f474a44 Don't block messages sending on bumping presence 2017-01-10 12:46:00 +00:00
Erik Johnston 6823fe5241 Linearize updates to membership via PUT /state/ 2017-01-09 18:25:13 +00:00
Erik Johnston f7085ac84f Name linearizer's for better logs 2017-01-09 17:17:10 +00:00
Erik Johnston 9898bbd9dc Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-01-09 14:51:17 +00:00
Erik Johnston 9a8ae6f1bf Bump version and changelog 2017-01-09 14:47:56 +00:00
Matthew Hodgson 2f4b2f4783 gah, fix mangled merge of 0.18.7 into develop 2017-01-07 04:00:42 +00:00
Matthew 6d363cea9d Merge branch 'release-v0.18.7' into develop 2017-01-07 03:46:16 +00:00
Matthew f0e4bac64e bump changelog & version 2017-01-07 03:45:38 +00:00
Matthew 4304e7e593 do the discard check in the right place to avoid grabbing dependent events 2017-01-07 03:44:18 +00:00
Matthew Hodgson 8c48971b51 Merge branch 'release-v0.18.7' into develop 2017-01-07 02:23:37 +00:00
Matthew e10c527930 Discard PDUs from invalid origins due to #1753 in 0.18.[56] 2017-01-07 02:13:14 +00:00
Matthew Hodgson 2f5be2d8dc oops, this should have been rc1 2017-01-07 01:11:56 +00:00
Matthew Hodgson 4086026524 move logging to right place 2017-01-07 00:41:46 +00:00
Matthew 19e2fb4386 bump version 2017-01-06 23:38:22 +00:00
Matthew b2850e62db fix typo breaking the fix to #1753 2017-01-06 23:23:37 +00:00
Mark Haines 06c00bd19b Merge branch 'release-v0.18.6' into develop 2017-01-06 14:46:27 +00:00
Mark Haines b42a972b71 Bump version and changelog 2017-01-06 14:44:28 +00:00
Patrik Oldsberg 1ef6084b75 handlers/room_member: fix guest access check when joining rooms
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2017-01-06 14:36:56 +00:00
Matthew Hodgson bd85434cb3 Merge branch 'release-v0.18.6' into develop 2017-01-05 13:58:19 +00:00
Mark Haines c18f7fc410 Fix flake8 and update changelog 2017-01-05 13:50:22 +00:00
Matthew Hodgson dafd50d178 Merge pull request #1767 from matrix-org/matthew/resolve_state_group_logging
log call paths for resolve_state_group
2017-01-05 13:47:42 +00:00
Matthew Hodgson 883ff92a7f Fix case 2017-01-05 13:45:02 +00:00
Matthew Hodgson d79d165761 add logging for all the places we call resolve_state_groups. my kingdom for a backtrace that actually works. 2017-01-05 13:40:39 +00:00
Matthew Hodgson 8cfc0165e9 fix annoying typos 2017-01-05 13:39:43 +00:00
Mark Haines 62451800e7 Bump version and changelog to v0.18.6-rc3 2017-01-05 13:36:10 +00:00
Matthew Hodgson b31ed22738 Merge branch 'release-v0.18.6' into develop 2017-01-05 13:03:02 +00:00
Mark Haines dd3df11c55 More logging for the linearizer and for get_events 2017-01-05 12:32:47 +00:00
Matthew Hodgson 468749c9fc fix comment 2017-01-05 12:00:11 +00:00
Matthew Hodgson eedf400d05 limit total timeout for get_missing_events to 10s 2017-01-05 11:58:15 +00:00
Matthew Hodgson 8e82611f37 fix comment 2017-01-05 11:44:44 +00:00
Mark Haines f784980d2b Only send events that originate on this server.
Or events that are sent via the federation "send_join" API.

This should match the behaviour from before v0.18.5 and #1635 landed.
2017-01-05 11:26:30 +00:00
Mark Haines e02bdaf08b Get the destinations from the state from before the event
Rather than the state after then event.
2017-01-04 15:17:15 +00:00
Mark Haines b6b67715ed Send ALL membership events to the server that was affected.
Send all membership changes to the server that was affected.
This ensures that if the last member of a room on a server
was kicked or banned they get told about it.
2017-01-04 13:56:20 +00:00
Matthew Hodgson 555d702e34 limit total timeout for get_missing_events to 10s 2016-12-31 15:21:37 +00:00
Matthew Hodgson 899a3a1268 Merge branch 'release-v0.18.6' into develop 2016-12-31 02:38:26 +00:00
Mark Haines f3de4f8cb7 Bump version and changelog 2016-12-30 20:21:04 +00:00
Mark Haines 62ce3034f3 s/aquire/acquire/g 2016-12-30 20:04:44 +00:00
Mark Haines 0aff09f6c9 Add more useful logging when we block fetching events 2016-12-30 20:00:44 +00:00
Mark Haines cc50b1ae53 Remove fallback from get_missing_events.
get_missing_events used to fallback to fetching the missing events
individually requesting from every server in the room, one by one.e

This could be unacceptably slow, possibly causing #1732
2016-12-30 18:13:15 +00:00
Mark Haines f576c34594 Merge remote-tracking branch 'origin/release-v0.18.6' into develop 2016-12-30 15:13:49 +00:00
Mark Haines 822cb39dfa Use the new twisted logging framework.
Hopefully adding an observer to the new framework will avoid a memory
leak https://twistedmatrix.com/trac/ticket/8164
2016-12-30 11:09:24 +00:00
Mark Haines 342fb8dae9 Merge branch 'release-v0.18.6' into develop 2016-12-29 17:33:46 +00:00
Mark Haines f023be9293 Bump changelog and version 2016-12-29 16:18:04 +00:00
Mark Haines 97ffc5690b Manually abort the underlying TLS connection.
The abort() method calls loseConnection() which tries to shutdown the
TLS connection cleanly. We now call abortConnection() directly which
should promptly close both the TLS connection and the underlying TCP
connection.

I also added some TODO markers to consider cancelling the old previous
timeout rather than checking time.time(). But given how urgently we want
to get this code released I'd rather leave the existing code with the
duplicate timeouts and the time.time() check.
2016-12-29 15:51:04 +00:00
Erik Johnston b4bc6fef5b Respect long_retries param and default to off 2016-12-29 00:58:34 +00:00
Erik Johnston 68030fd37b Spelling and comments 2016-12-29 00:10:49 +00:00
Erik Johnston b7336ff32d Clean up 2016-12-29 00:09:33 +00:00
Erik Johnston 5b6672c66d Wrap connections in an N minute timeout to ensure they get reaped correctly 2016-12-29 00:06:53 +00:00
David Baker 84cf00c645 Fix another comment typo 2016-12-21 09:51:43 +00:00
David Baker 0c88ab1844 Add /account/3pid/delete endpoint
Also fix a typo in a comment
2016-12-20 18:27:30 +00:00
Johannes Löthberg 702c020e58 Fix check for bind_address
The empty string is a valid setting for the bind_address option, so
explicitly check for None here instead.

Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-20 01:37:50 +01:00
Matthew Hodgson 09f15918be Merge pull request #1711 from matrix-org/matthew/utf8-password-change
fix ability to change password to a non-ascii one
2016-12-20 00:02:13 +00:00
Matthew Hodgson da2c8f3c94 Merge pull request #1709 from kyrias/bind_addresses
Add support for specifying multiple bind addresses
2016-12-19 23:49:34 +00:00
Matthew Hodgson a58e4e0d48 Merge pull request #1696 from kyrias/ipv6
IPv6 support
2016-12-19 23:49:07 +00:00
Matthew Hodgson f2a5aebf98 fix ability to change password to a non-ascii one
https://github.com/vector-im/riot-web/issues/2658
2016-12-18 22:25:21 +00:00
Johannes Löthberg a9c1b419a9 Bump twisted dependency
At least 16.0.0 is needed for wrapClientTLS support.

Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-18 23:16:43 +01:00
Johannes Löthberg f5cd5ebd7b Add IPv6 comment to default config
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-18 23:14:32 +01:00
Johannes Löthberg c95e9fff99 Make default homeserver config use bind_addresses
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-18 21:51:56 +01:00
Johannes Löthberg 7dfd70fc83 Add support for specifying multiple bind addresses
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-18 21:51:56 +01:00
Erik Johnston b2f8642d3d Cache network room list queries. 2016-12-16 16:11:43 +00:00
Erik Johnston b9b6d17ab1 Bump version and changelog 2016-12-16 10:18:02 +00:00
Richard van der Hoff edc6a1e4f9 Add some logging for syncing to_device events
Attempt to track down the loss of to_device events
(https://github.com/vector-im/riot-web/issues/2711 etc).
2016-12-15 18:16:10 +00:00
Erik Johnston 35129ac998 Merge pull request #1698 from matrix-org/erikj/room_list
Fix caching on public room list
2016-12-15 15:40:28 +00:00
Erik Johnston aa1336c00a Merge pull request #1700 from matrix-org/erikj/backfill_filter
Fix /backfill returning events it shouldn't
2016-12-15 14:21:30 +00:00
Marcin Bachry 24c16fc349 Fix crash in url preview when html tag has no text
Signed-off-by: Marcin Bachry <hegel666@gmail.com>
2016-12-14 22:38:18 +01:00
Erik Johnston b8255eba26 Comment 2016-12-14 13:49:54 +00:00
Erik Johnston b2999a7055 Fix /backfill returning events it shouldn't 2016-12-14 13:41:45 +00:00
Erik Johnston c3208e45c9 Fixup membership query 2016-12-14 10:46:58 +00:00
Erik Johnston 9d95351cad Merge branch 'develop' of github.com:matrix-org/synapse into erikj/room_list 2016-12-13 17:57:08 +00:00
Erik Johnston 1de53a7a1a Fix caching on public room list 2016-12-13 17:33:24 +00:00
Erik Johnston b3d398343e Bump changelog and version 2016-12-13 11:07:27 +00:00
Johannes Löthberg 0648e76979 Remove spurious newline
Apparently I just removed the spaces instead...

Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-12 18:41:30 +01:00
Erik Johnston 8588d0eb3d Merge pull request #1697 from matrix-org/erikj/fix_bg_member
Fix background update that prematurely stopped
2016-12-12 17:20:17 +00:00
Erik Johnston 1574b839e0 Merge pull request #1676 from matrix-org/erikj/room_list
Add new API appservice specific public room list
2016-12-12 17:00:10 +00:00
Erik Johnston 7ec2bf9b77 Fix background update that prematurely stopped 2016-12-12 16:54:58 +00:00
Erik Johnston 2bf5a47b3e Rename network_id to instance_id on client side 2016-12-12 16:05:45 +00:00
Johannes Löthberg d3bd94805f Fixup for #1689 and #1690
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-12 16:32:47 +01:00
Erik Johnston 09cbcb78d3 Add cache to get_public_room_ids_at_stream_id 2016-12-12 14:41:51 +00:00
Erik Johnston 631376e2ac Notify replication. Use correct network_id 2016-12-12 14:28:15 +00:00
Richard van der Hoff abed247182 Remove unspecced GET endpoints for e2e keys
GET /keys/claim is a terrible idea, since it isn't idempotent; also it throws
500 errors if you call it without all the right params.

GET /keys/query is arguable, but it's unspecced, so let's get rid of it too to
stop people relying on unspecced APIs.
2016-12-12 12:31:40 +00:00
Erik Johnston d45c984653 Docstring 2016-12-12 11:00:27 +00:00
Erik Johnston d53a80af25 Merge pull request #1620 from matrix-org/erikj/concurrent_room_access
Limit the number of events that can be created on a given room concurrently
2016-12-12 10:30:23 +00:00
Richard van der Hoff 85cd30b1fd Merge pull request #1686 from matrix-org/rav/fix_federation_key_fails
E2E key query: handle federation fails
2016-12-12 09:33:39 +00:00
Richard van der Hoff deca951241 Remove unused import 2016-12-12 09:24:35 +00:00
Glyph 9f07f4c559 IPv6 support for endpoint.py
Similar to https://github.com/matrix-org/synapse/pull/1689, but for endpoint.py
2016-12-11 11:10:32 +01:00
Glyph 6e18805ac2 IPv6 support for client.py
This is an (untested) general sketch of how to use wrapClientTLS to implement TLS over IPv6, as well as faster connections over IPv4.
2016-12-11 11:10:32 +01:00
Richard van der Hoff efa4ccfaee E2E key query: handle federation fails
Don't fail the whole request if we can't connect to a particular server.
2016-12-09 18:31:01 +00:00
Erik Johnston fbaf868f62 Correctly handle timeout errors 2016-12-09 16:30:29 +00:00
Erik Johnston be14c24cea Fix rare notifier bug where listeners dont timeout
There was a race condition that caused the notifier to 'miss' the
timeout notification, since there were no other checks for the timeout
this caused listeners to get stuck in a loop until something happened.
2016-12-09 15:43:18 +00:00
Erik Johnston 1697f6a323 Merge pull request #1680 from matrix-org/erikj/joined_rooms
Add new room membership APIs
2016-12-09 11:35:48 +00:00
Erik Johnston 52d12ca782 Add /room/<room_id>/joined_members API
This returns the currently joined members in the room with their display
names and avatar urls. This is more efficient than /members for large
rooms where you don't need the full events.
2016-12-08 13:32:07 +00:00
Erik Johnston c45d8e9ba2 Add profile data to the room_membership table for joins 2016-12-08 13:08:41 +00:00
Richard van der Hoff b08f76bd23 Fix ignored read-receipts
Don't ignore read-receipts which arrive in the same EDU as a read-receipt for
an old event.
2016-12-08 12:13:01 +00:00
Richard van der Hoff bd07a35c29 Fix result of insert_receipt
This should fix the absence of notifications when new receipts arrive.
2016-12-08 12:11:34 +00:00
Erik Johnston de796f27e6 Add joined_rooms servlet 2016-12-08 11:39:03 +00:00
Erik Johnston 2687af82d4 Comments 2016-12-07 09:58:33 +00:00
Erik Johnston 3727d66a0e Don't include appservice id 2016-12-06 17:04:26 +00:00
Erik Johnston f32fb65552 Add new API appservice specific public room list 2016-12-06 16:12:27 +00:00
Richard van der Hoff 1529c19675 Prevent user tokens being used as guest tokens (#1675)
Make sure that a user cannot pretend to be a guest by adding 'guest = True'
caveats.
2016-12-06 15:31:37 +00:00
Erik Johnston 9609c91e7d Merge pull request #653 from matrix-org/erikj/preset_guest_join
Enable guest access for private rooms by default
2016-12-05 17:47:14 +00:00
Erik Johnston 338df4f409 Merge pull request #1649 from matrix-org/dbkr/log_ui_auth_args
Log the args that we have on UI auth completion
2016-12-05 16:40:58 +00:00
pik c46e7a9c9b Bugfix: Console logging handler missing default filter 2016-12-03 20:14:58 -03:00
Johannes Löthberg 32c8b5507c preview_url_resource: Ellipsis must be in unicode string
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-01 13:12:13 +01:00
Richard van der Hoff 6841d8ff55 Fix doc-string
Remove refresh_token reference
2016-12-01 11:42:17 +00:00
Richard van der Hoff aa09d6b8f0 Rip out more refresh_token code
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.

Still TODO: removing the table from the schema.
2016-11-30 17:40:18 +00:00
Richard van der Hoff dc4b23e1a1 Merge branch 'develop' into rav/no_more_refresh_tokens 2016-11-30 17:10:04 +00:00
Richard van der Hoff 8379a741cc Merge pull request #1660 from matrix-org/rav/better_content_type_validation
More intelligent Content-Type parsing
2016-11-30 16:54:03 +00:00
Richard van der Hoff 321fe5c44c Merge pull request #1656 from matrix-org/rav/remove_time_caveat
Stop putting a time caveat on access tokens
2016-11-30 16:53:20 +00:00
Richard van der Hoff b5b3a7e867 More intelligent Content-Type parsing
Content-Type is allowed to contain options (`; charset=utf-8`, for
instance). We should allow that.
2016-11-30 15:07:32 +00:00
Richard van der Hoff 4febfe47f0 Comments
Update comments in verify_macaroon
2016-11-30 07:36:32 +00:00
Richard van der Hoff 77eca2487c Merge pull request #1653 from matrix-org/rav/guest_e2e
Implement E2E for guests
2016-11-29 17:41:35 +00:00
Richard van der Hoff 1c4f05db41 Stop putting a time caveat on access tokens
The 'time' caveat on the access tokens was something of a lie, since we weren't
enforcing it; more pertinently its presence stops us ever adding useful time
caveats.

Let's move in the right direction by not lying in our caveats.
2016-11-29 16:49:41 +00:00
Richard van der Hoff 5c4edc83b5 Stop generating refresh tokens
Since we're not doing refresh tokens any more, we should start killing off the
dead code paths. /tokenrefresh itself is a bit of a thornier subject, since
there might be apps out there using it, but we can at least not generate
refresh tokens on new logins.
2016-11-28 10:13:01 +00:00
Richard van der Hoff b6146537d2 Merge pull request #1655 from matrix-org/rav/remove_redundant_macaroon_checks
Remove redundant list of known caveat prefixes
2016-11-25 16:57:19 +00:00
Richard van der Hoff f62b69e32a Allow guest access to endpoints for E2E
Expose /devices, /keys, and /sendToDevice to guest users, so that they can use
E2E.
2016-11-25 15:26:34 +00:00
Richard van der Hoff 7f02e4d008 Give guest users a device_id
We need to create devices for guests so that they can use e2e, but we don't
have anywhere to store it, so just use a fixed one.
2016-11-25 15:25:30 +00:00
Erik Johnston 11bfe438a2 Use correct var 2016-11-24 15:26:53 +00:00
Erik Johnston aaecffba3a Correctly handle 500's and 429 on federation 2016-11-24 15:04:49 +00:00
Richard van der Hoff e1d7c96814 Remove redundant list of known caveat prefixes
Also add some comments.
2016-11-24 12:38:17 +00:00
Erik Johnston 7e03f9a484 Bump version and changelog 2016-11-24 12:29:58 +00:00
Erik Johnston 46ca345b06 Don't send old events as federation 2016-11-24 12:29:02 +00:00
Erik Johnston f36ea03741 Bump changelog and version 2016-11-24 11:08:01 +00:00
David Baker c9d4e7b716 Clarify that creds doesn not contain passwords. 2016-11-24 10:54:59 +00:00
David Baker f681aab895 Log the args that we have on UI auth completion
This will be super helpful for debugging if we have more
registration woes.
2016-11-24 10:11:45 +00:00
Erik Johnston 1985860c6e Comment 2016-11-23 15:59:59 +00:00
Erik Johnston 2ac516850b More efficient notif count queries 2016-11-23 15:57:04 +00:00
Erik Johnston 302fbd218d Merge pull request #1635 from matrix-org/erikj/split_out_fed_txn
Split out federation transaction sending to a worker
2016-11-23 15:39:12 +00:00
Erik Johnston b2d6e63b79 Merge pull request #1641 from matrix-org/erikj/as_pushers
Ignore AS users when fetching push rules
2016-11-23 15:21:52 +00:00
Erik Johnston feec718265 Shuffle receipt handler around so that worker apps don't need to load it 2016-11-23 15:14:24 +00:00
Erik Johnston 26072df6af Ensure only main or federation_sender process can send federation traffic 2016-11-23 14:09:47 +00:00
Erik Johnston b69f76c106 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_fed_txn 2016-11-23 11:31:53 +00:00
Erik Johnston 4d9b5c60f9 Comment 2016-11-23 11:11:41 +00:00
Erik Johnston 0163466d72 Ignore AS users when fetching push rules
By ignoring AS users early on when fetching push rules for a room we can
avoid needlessly hitting the DB and filling up the caches.
2016-11-23 11:01:01 +00:00
Erik Johnston 4c79a63fd7 Explicit federation ack 2016-11-23 10:40:44 +00:00