Commit Graph

1243 Commits (06c0d0ed081b56716135c4881e600861b2e8cad5)

Author SHA1 Message Date
Erik Johnston 46826bb078 Comment and remove spurious logging 2017-04-27 17:25:44 +01:00
Erik Johnston d3c150411c Merge pull request #2130 from APwhitehat/roomexists
Check that requested room_id exists
2017-04-27 09:20:26 +01:00
Erik Johnston 34e682d385 Fix invite state to always include all events 2017-04-26 16:18:08 +01:00
Erik Johnston d9aa645f86 Reduce size of joined_user cache
The _get_joined_users_from_context cache stores a mapping from user_id
to avatar_url and display_name. Instead of storing those in a dict,
store them in a namedtuple as that uses much less memory.

We also try converting the string to ascii to further reduce the size.
2017-04-25 14:38:51 +01:00
Mark Haines e6e2627636 Fix code for reporting old verify keys in synapse 2017-04-24 18:51:25 +01:00
Luke Barnard 3fb8784c92 m.read_marker -> m.fully_read (#2128)
Also:
 - change the REST endpoint to have a "S" on the end (so it's now /read_markers)
 - change the content of the m.read_up_to event to have the key "event_id" instead of "marker".
2017-04-18 17:46:15 +01:00
Anant Prakash 574d573ac2 Check that requested room_id exists 2017-04-14 23:50:59 +05:30
Luke Barnard cf6121e3da More null-guard changes 2017-04-12 14:02:03 +01:00
Luke Barnard c0aba0a23e Remove Unused ref to hs 2017-04-12 10:52:11 +01:00
Luke Barnard b9676a75f6 Move a space 2017-04-12 10:51:17 +01:00
Luke Barnard 0127423027 flake8 2017-04-11 17:07:07 +01:00
Luke Barnard d892079844 Finish implementing RM endpoint
- This change causes a 405 to be sent if "m.read_marker" is set via /account_data
 - This also fixes-up the RM endpoint so that it actually Works.
2017-04-11 15:01:39 +01:00
lukebarnard e263c26690 Initial commit of RM server-side impl
(See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u)
2017-04-11 11:55:30 +01:00
Erik Johnston 142b6b4abf Merge pull request #2011 from matrix-org/matthew/turn_allow_guests
add setting (on by default) to support TURN for guests
2017-03-31 14:37:09 +01:00
Erik Johnston 2a06b44be2 Merge pull request #1986 from matrix-org/matthew/enable_guest_3p
enable guest access for the 3pl/3pid APIs
2017-03-31 14:36:03 +01:00
Matthew Hodgson 9ee397b440 switch to allow_guest=True for authing 3Ps as per PR feedback 2017-03-31 13:54:26 +01:00
Richard van der Hoff 01fc847f7f Merge pull request #2057 from matrix-org/rav/missing_yield_2
Add another missing yield on check_device_registered
2017-03-24 12:46:43 +00:00
Richard van der Hoff 13c8749ac9 Add another missing yield on check_device_registered 2017-03-23 22:18:53 +00:00
David Baker a2dfab12c5 Fix token request for addition of phone numbers 2017-03-23 18:46:17 +00:00
Erik Johnston 6957bfdca6 Don't recreate so many sets 2017-03-16 11:54:26 +00:00
Erik Johnston a297155a97 Remove unused import 2017-03-15 14:49:25 +00:00
Erik Johnston 6c82de5100 Format presence events on the edges instead of reformatting them multiple times 2017-03-15 14:27:34 +00:00
David Baker 0ad44acb5a Merge pull request #1997 from matrix-org/dbkr/cas_partialdownload
Handle PartialDownloadError in CAS login
2017-03-15 13:52:34 +00:00
Matthew Hodgson e0ff66251f add setting (on by default) to support TURN for guests 2017-03-15 12:22:18 +00:00
Erik Johnston 7827251daf Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2
Phone number registration / login support v2
2017-03-15 09:59:54 +00:00
Richard van der Hoff f2ed64eaaf Merge pull request #1992 from matrix-org/rav/fix_media_loop
Fix routing loop when fetching remote media
2017-03-14 23:40:35 +00:00
Richard van der Hoff 1d09586599 Address review comments
- don't blindly proxy all HTTPRequestExceptions
- log unexpected exceptions at error
- avoid `isinstance`
- improve docs on `from_http_response_exception`
2017-03-14 14:15:37 +00:00
David Baker 1ece06273e Handle PartialDownloadError in CAS login 2017-03-14 13:37:36 +00:00
David Baker 7b6ed9871e Use extend instead of += 2017-03-14 10:49:55 +00:00
David Baker d79a687d85 Oops, remove print 2017-03-14 10:40:20 +00:00
Luke Barnard bbeeb97f75 Implement _simple_delete_many_txn, use it to delete devices
(But this doesn't implement the same for deleting access tokens or e2e keys.

Also respond to code review.
2017-03-13 17:53:23 +00:00
David Baker 0a9945220e Fix registration for broken clients
Only offer msisdn flows if the x_show_msisdn option is given.
2017-03-13 17:29:38 +00:00
David Baker 73a5f06652 Support registration / login with phone number
Changes from https://github.com/matrix-org/synapse/pull/1971
2017-03-13 17:27:51 +00:00
Luke Barnard c077c3277b Flake 2017-03-13 16:45:38 +00:00
Luke Barnard c81f33f73d Implement delete_devices API
This implements the proposal here https://docs.google.com/document/d/1C-25Gqz3TXy2jIAoeOKxpNtmme0jI4g3yFGqv5GlAAk for deleting multiple devices at once in a single request.
2017-03-13 16:33:51 +00:00
Richard van der Hoff 170ccc9de5 Fix routing loop when fetching remote media
When we proxy a media request to a remote server, add a query-param, which will
tell the remote server to 404 if it doesn't recognise the server_name.

This should fix a routing loop where the server keeps forwarding back to
itself.

Also improves the error handling on remote media fetches, so that we don't
always return a rather obscure 502.
2017-03-13 16:30:36 +00:00
Erik Johnston 7eae6eaa2f Revert "Support registration & login with phone number" 2017-03-13 09:59:33 +00:00
Matthew Hodgson a61dd408ed enable guest access for the 3pl/3pid APIs 2017-03-12 19:30:45 +00:00
David Baker ece7e00048 Comment when our 3pids would be incomplete 2017-03-08 19:07:18 +00:00
David Baker 727124a762 Not any more, it doesn't 2017-03-08 19:00:23 +00:00
David Baker d4d3629aaf Better error message 2017-03-08 17:01:26 +00:00
David Baker 1c99934b28 pep8 2017-03-08 11:58:20 +00:00
David Baker 85bb322333 Pull out datastore in initialiser 2017-03-08 11:51:25 +00:00
David Baker 65d43f3ca5 Minor fixes from PR feedback 2017-03-08 11:48:43 +00:00
David Baker 82c5e7de25 Typos 2017-03-08 11:42:44 +00:00
David Baker 2e27339add Refector out assert_params_in_request
and replace requestEmailToken where we meant requestMsisdnToken
2017-03-08 11:37:34 +00:00
David Baker 88df6c0c9a Factor out msisdn canonicalisation
Plus a couple of other minor fixes
2017-03-08 11:03:39 +00:00
David Baker 402a7bf63d Fix pep8 2017-03-08 09:33:40 +00:00
David Baker 00466e2feb Support new login format
https://docs.google.com/document/d/1-6ZSSW5YvCGhVFDyD2QExAUAdpCWjccvJT5xiyTTG2Y/edit#
2017-03-07 16:37:23 +00:00
David Baker b0effa2160 Add msisdns as 3pids during registration
and support binding them with the bind_msisdn param
2017-03-03 18:34:39 +00:00
David Baker 3557cf34dc Merge remote-tracking branch 'origin/develop' into dbkr/msisdn_signin 2017-03-01 17:20:37 +00:00
Erik Johnston acf6d4d2e3 Merge pull request #1945 from jkolo/fix_dynamic_thumbnails_aspect
Fix #1677 (dynamic thumbnails aspect)
2017-02-27 09:51:52 +00:00
Jurek aea5461488 Fix dynamic thumbnails aspect 2017-02-24 22:43:27 +01:00
Erik Johnston 17673404fb Remove unused param 2017-02-20 15:02:01 +00:00
David Baker fa467e62a9 Merge pull request #1917 from matrix-org/dbkr/make_ban_reasons_work
Make kick & ban reasons work
2017-02-14 16:10:06 +00:00
David Baker 355d62c499 Make kick & ban reasons work
We somehow specced APIs with reason strings, preserve the content
in the events  and even have the clients display them, but failed
to actually pass the parameter through to the event content.
2017-02-14 15:10:55 +00:00
David Baker ce3e583d94 WIP support for msisdn 3pid proxy methods 2017-02-14 15:05:55 +00:00
Kegan Dougal d0497425f8 Ordering is important on errbacks so add the cleanup func before creating an ObservableDeferred 2017-02-13 13:49:44 +00:00
Kegan Dougal 808ddf0ae7 Pop the txn from the map in case it has already been deleted somehow 2017-02-13 13:36:15 +00:00
Kegan Dougal feb15dc99f Don't cache errors at all 2017-02-13 13:33:12 +00:00
Kegan Dougal ecd7e36047 http txns: Do not cache error responses
Previously we did. This meant that, amongst other errors, rate-limiting errors
would be cached and prevent messages with that txn ID being sent.
2017-02-13 13:16:48 +00:00
Erik Johnston af6da6db2d Merge pull request #1784 from morteza-araby/user-admin
Administration functionalities
2017-02-06 16:21:10 +01:00
Erik Johnston 6826593b81 sets aren't JSON serializable 2017-02-02 14:55:54 +00:00
Morteza Araby 2849d3f29d admin,storage: added more administrator functionalities
administrators can now:
 - Set displayname of users
 - Update user avatars
 - Search for users by user_id
 - Browse all users in a paginated API
 - Reset user passwords
 - Deactivate users

Helpers for doing paginated queries has also been added to storage

Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>
2017-02-02 14:02:26 +01:00
Erik Johnston 51adaac953 Fix email push in pusher worker
This was broken when device list updates were implemented, as Mailer
could no longer instantiate an AuthHandler due to a dependency on
federation sending.
2017-02-02 10:53:36 +00:00
Erik Johnston 7e919bdbd0 Include newly joined users in /keys/changes API 2017-02-01 17:33:16 +00:00
Erik Johnston 73d676dc8b Comment 2017-02-01 13:17:17 +00:00
Erik Johnston acb501c46d Comment 2017-02-01 10:32:49 +00:00
Erik Johnston 97479d0c54 Implement /keys/changes 2017-02-01 10:30:03 +00:00
Erik Johnston 2367c5568c Add basic implementation of local device list changes 2017-01-25 14:27:27 +00:00
Marvin Steadfast 1e38be3a7a Added username and password for turn server
It makes it possible to use a turn server that needs a username and
password instead of a token.
2017-01-19 14:08:20 +01:00
David Baker 8c5009b628 Lowercase all email addresses before querying db
Since we store all emails in the DB in lowercase
(https://github.com/matrix-org/synapse/pull/1170)
2017-01-18 13:25:56 +00:00
Erik Johnston ebf94aff8d Fix spurious Unhandled Error log lines 2017-01-12 17:19:47 +00:00
Mark Haines 32019c9897 Log which files we saved attachments to in the media_repository 2017-01-10 14:19:50 +00:00
Erik Johnston 7d0981b312 Merge pull request #1787 from matrix-org/erikj/linearize_member
Linearize updates to membership via PUT /state/
2017-01-10 14:04:54 +00:00
Erik Johnston 6823fe5241 Linearize updates to membership via PUT /state/ 2017-01-09 18:25:13 +00:00
Erik Johnston f7085ac84f Name linearizer's for better logs 2017-01-09 17:17:10 +00:00
David Baker 0c88ab1844 Add /account/3pid/delete endpoint
Also fix a typo in a comment
2016-12-20 18:27:30 +00:00
Marcin Bachry 24c16fc349 Fix crash in url preview when html tag has no text
Signed-off-by: Marcin Bachry <hegel666@gmail.com>
2016-12-14 22:38:18 +01:00
Erik Johnston 1574b839e0 Merge pull request #1676 from matrix-org/erikj/room_list
Add new API appservice specific public room list
2016-12-12 17:00:10 +00:00
Richard van der Hoff abed247182 Remove unspecced GET endpoints for e2e keys
GET /keys/claim is a terrible idea, since it isn't idempotent; also it throws
500 errors if you call it without all the right params.

GET /keys/query is arguable, but it's unspecced, so let's get rid of it too to
stop people relying on unspecced APIs.
2016-12-12 12:31:40 +00:00
Erik Johnston 52d12ca782 Add /room/<room_id>/joined_members API
This returns the currently joined members in the room with their display
names and avatar urls. This is more efficient than /members for large
rooms where you don't need the full events.
2016-12-08 13:32:07 +00:00
Erik Johnston de796f27e6 Add joined_rooms servlet 2016-12-08 11:39:03 +00:00
Erik Johnston f32fb65552 Add new API appservice specific public room list 2016-12-06 16:12:27 +00:00
Johannes Löthberg 32c8b5507c preview_url_resource: Ellipsis must be in unicode string
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-01 13:12:13 +01:00
Richard van der Hoff 6841d8ff55 Fix doc-string
Remove refresh_token reference
2016-12-01 11:42:17 +00:00
Richard van der Hoff aa09d6b8f0 Rip out more refresh_token code
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.

Still TODO: removing the table from the schema.
2016-11-30 17:40:18 +00:00
Richard van der Hoff dc4b23e1a1 Merge branch 'develop' into rav/no_more_refresh_tokens 2016-11-30 17:10:04 +00:00
Richard van der Hoff 321fe5c44c Merge pull request #1656 from matrix-org/rav/remove_time_caveat
Stop putting a time caveat on access tokens
2016-11-30 16:53:20 +00:00
Richard van der Hoff 77eca2487c Merge pull request #1653 from matrix-org/rav/guest_e2e
Implement E2E for guests
2016-11-29 17:41:35 +00:00
Richard van der Hoff 1c4f05db41 Stop putting a time caveat on access tokens
The 'time' caveat on the access tokens was something of a lie, since we weren't
enforcing it; more pertinently its presence stops us ever adding useful time
caveats.

Let's move in the right direction by not lying in our caveats.
2016-11-29 16:49:41 +00:00
Richard van der Hoff 5c4edc83b5 Stop generating refresh tokens
Since we're not doing refresh tokens any more, we should start killing off the
dead code paths. /tokenrefresh itself is a bit of a thornier subject, since
there might be apps out there using it, but we can at least not generate
refresh tokens on new logins.
2016-11-28 10:13:01 +00:00
Richard van der Hoff f62b69e32a Allow guest access to endpoints for E2E
Expose /devices, /keys, and /sendToDevice to guest users, so that they can use
E2E.
2016-11-25 15:26:34 +00:00
Richard van der Hoff 7f02e4d008 Give guest users a device_id
We need to create devices for guests so that they can use e2e, but we don't
have anywhere to store it, so just use a fixed one.
2016-11-25 15:25:30 +00:00
Erik Johnston feec718265 Shuffle receipt handler around so that worker apps don't need to load it 2016-11-23 15:14:24 +00:00
Kegsay d4a459f7cb Merge pull request #1638 from matrix-org/kegan/sync-event-fields
Implement "event_fields" in filters
2016-11-22 14:02:38 +00:00
Kegan Dougal cea4e4e7b2 Glue only_event_fields into the sync rest servlet 2016-11-22 10:14:05 +00:00
Mark Haines a289150943 Fix flake8 2016-11-18 17:15:02 +00:00
David Baker 544722bad2 Work around client replacing reg params
Works around https://github.com/vector-im/vector-android/issues/715
and equivalent for iOS
2016-11-18 17:07:35 +00:00
Kegan Dougal 3991b4cbdb Clean transactions based on time. Add HttpTransactionCache tests. 2016-11-14 11:19:24 +00:00
Kegan Dougal af4a1bac50 Move .observe() up to the cache to make things neater 2016-11-14 09:52:41 +00:00
Kegan Dougal 8ecaff51a1 Review comments 2016-11-11 17:47:03 +00:00
Kegan Dougal f6c48802f5 More flake8 2016-11-11 15:08:24 +00:00
Kegan Dougal a88bc67f88 Flake8 and fix whoopsie 2016-11-11 15:02:29 +00:00
Kegan Dougal 42c43cfafd Use ObservableDeferreds instead of Deferreds as they behave as intended 2016-11-11 14:54:10 +00:00
Kegan Dougal c7daf3136c Use observable deferreds because they are sane 2016-11-11 14:13:32 +00:00
Kegan Dougal 8a8ad46f48 Flake8 2016-11-10 15:22:11 +00:00
Kegan Dougal 2771447c29 Store Promise<Response> instead of Response for HTTP API transactions
This fixes a race whereby:
 - User hits an endpoint.
 - No cached transaction so executes main code.
 - User hits same endpoint.
 - No cache transaction so executes main code.
 - Main code finishes executing and caches response and returns.
 - Main code finishes executing and caches response and returns.

 This race is common in the wild when Synapse is struggling under load.
 This commit fixes the race by:
  - User hits an endpoint.
  - Caches the promise to execute the main code and executes main code.
  - User hits same endpoint.
  - Yields on the same promise as the first request.
  - Main code finishes executing and returns, unblocking both requests.
2016-11-10 14:49:26 +00:00
Mark Haines b1c27975d0 Set CORs headers on responses from the media repo 2016-11-02 11:29:25 +00:00
Erik Johnston 78c083f159 Merge pull request #1164 from pik/error-codes
Clarify Error codes for GET /filter/
2016-10-19 14:26:17 +01:00
pik e8b1d2a452 Refactor test_filter to use real DataStore
* add tests for filter api errors
2016-10-18 12:17:38 -05:00
Alexander Maznev d9350b0db8 Error codes for filters
* add tests

Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>
2016-10-14 10:18:28 -05:00
Richard van der Hoff f7aed3d7a2 Merge pull request #1168 from matrix-org/rav/ui_auth_on_device_delete
User-interactive auth on delete device
2016-10-13 09:38:41 +01:00
Richard van der Hoff 9009143fb9 Handle delete device requests with no body
We should probably return a 401 rather than a 400 for existing clients that
don't know they have to do the UIA dance to delete a device.
2016-10-12 18:47:28 +01:00
Richard van der Hoff fbd3866bc6 User-interactive auth on delete device 2016-10-12 16:16:31 +01:00
Mark Haines 9e18e0b1cb Merge pull request #1167 from matrix-org/markjh/fingerprints
Add config option for adding additional TLS fingerprints
2016-10-12 15:27:44 +01:00
Mark Haines 0af6213019 Improve comment formatting 2016-10-12 14:45:13 +01:00
Mark Haines 6e9f3ab415 Add config option for adding additional TLS fingerprints 2016-10-11 19:14:46 +01:00
Erik Johnston a2f2516199 Merge pull request #1157 from Rugvip/nolimit
Remove rate limiting from app service senders and fix get_or_create_user requester
2016-10-11 11:20:54 +01:00
Patrik Oldsberg 7b5546d077 rest/client/v1/register: use the correct requester in createUser
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 22:12:32 +02:00
Richard van der Hoff f382117852 window.postmessage for Interactive Auth fallback
If you're a webapp running the fallback in an iframe, you can't set set a
window.onAuthDone function. Let's post a message back to window.opener instead.
2016-10-06 18:16:59 +01:00
Patrik Oldsberg 9bfc617791 storage/appservice: make appservice methods only relying on the cache synchronous 2016-10-06 15:24:59 +02:00
Erik Johnston 22578545a0 Time out typing over federation 2016-09-23 14:00:52 +01:00
Erik Johnston 87528f0756 Support /initialSync in synchrotron worker 2016-09-21 11:46:28 +01:00
Erik Johnston ddc89df89d Enable guest access to POST /publicRooms 2016-09-17 15:55:24 +01:00
Erik Johnston 2e67cabd7f Make POST /publicRooms require auth 2016-09-16 11:32:51 +01:00
Erik Johnston c33b25fd8d Change the way we calculate new_limit in /publicRooms and add POST API 2016-09-15 17:35:20 +01:00
Erik Johnston 68c1ed4d1a Remove default public rooms limit 2016-09-15 13:56:20 +01:00
Erik Johnston 1d98cf26be By default limit /publicRooms to 100 entries 2016-09-15 13:18:35 +01:00
Erik Johnston 5810cffd33 Pass since/from parameters over federation 2016-09-15 10:36:19 +01:00
Erik Johnston f3eead0660 Allow paginating both forwards and backwards 2016-09-15 10:15:37 +01:00
Erik Johnston 4131381123 Remove support for aggregate room lists 2016-09-15 09:28:15 +01:00
Mark Haines 18ab019a4a Move the E2E key handling into the e2e handler 2016-09-13 11:35:35 +01:00
Mark Haines 949c2c5435 Add a timeout parameter for end2end key queries.
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.

Fetch keys from different servers in parallel rather than in series.

Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
Shell Turner 29205e9596 Conform better to the CAS protocol specification
Redirect to CAS's /login endpoint properly, and
don't require an <attributes> element.

Signed-off-by: Shell Turner <cam.turn@gmail.com>
2016-09-09 21:20:14 +01:00
Mark Haines dbff7e9436 Merge pull request #1096 from matrix-org/markjh/get_access_token
Add helper function for getting access_tokens from requests
2016-09-09 17:09:27 +01:00
Mark Haines 8aee5aa068 Add helper function for getting access_tokens from requests
Rather than reimplementing the token parsing in the various places.
This will make it easier to change the token parsing to allow access_tokens
in HTTP headers.
2016-09-09 16:33:15 +01:00
Paul Evans 56f38d1776 Merge pull request #1091 from matrix-org/paul/third-party-lookup
Improvements to 3PE lookup API
2016-09-09 15:43:11 +01:00
Paul "LeoNerd" Evans d271383e63 Filter returned events for client-facing format 2016-09-09 14:40:15 +01:00
Paul "LeoNerd" Evans 0fc0a3bdff Allow clients to specify the format a room state event is returned in 2016-09-09 14:34:29 +01:00
Paul "LeoNerd" Evans bdbcfc2a80 appease pep8 2016-09-09 13:31:39 +01:00
Paul "LeoNerd" Evans 6eb0c8a2e4 Python isn't JavaScript; have to quote dict keys 2016-09-09 13:31:17 +01:00
Paul "LeoNerd" Evans 25eb769b26 Efficiency fix for lookups of a single protocol 2016-09-09 13:25:02 +01:00
Paul "LeoNerd" Evans 3328428d05 Allow lookup of a single 3PE protocol query metadata 2016-09-09 13:19:04 +01:00
Matthew Hodgson b8f84f99ff Merge pull request #1081 from matrix-org/dbkr/notifications_only_highlight
Implement `only=highlight` on `/notifications`
2016-09-09 00:09:51 +01:00
Erik Johnston 257025ac89 Merge pull request #1082 from matrix-org/erikj/remote_public_rooms
Add server param to /publicRooms
2016-09-08 16:04:22 +01:00
Erik Johnston 3f9889bfd6 Use parse_string 2016-09-08 15:51:10 +01:00
Mark Haines fa9d36e050 Merge branch 'develop' into markjh/direct_to_device_federation 2016-09-08 13:43:43 +01:00
David Baker 4ef222ab61 Implement `only=highlight` on `/notifications` 2016-09-08 13:43:35 +01:00
Erik Johnston 791658b576 Add server param to /publicRooms 2016-09-08 11:53:05 +01:00
Mark Haines d4a35ada28 Send device messages over federation 2016-09-06 18:16:20 +01:00
Erik Johnston d51b8a1674 Add quotes and be explicity about script-src 2016-09-05 17:35:01 +01:00
Erik Johnston 662b031a30 Allow PDF to be rendered from media repo 2016-09-05 17:25:26 +01:00
Mark Haines 7ed5acacf4 Fix up the calls to the notifier for device messages 2016-09-01 18:08:40 +01:00
Mark Haines 1aa3e1d287 Add a replication stream for direct to device messages 2016-08-31 10:38:58 +01:00
Mark Haines 4bbef62124 Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device 2016-08-26 14:35:31 +01:00
Paul "LeoNerd" Evans 1294d4a329 Move ThirdPartyEntityKind into api.constants so the expectation becomes that the value is significant 2016-08-25 18:34:47 +01:00
Mark Haines ab34fdecb7 Merge branch 'develop' into markjh/direct_to_device 2016-08-25 18:34:46 +01:00
Mark Haines b162cb2e41 Add some TODOs 2016-08-25 18:18:53 +01:00
Mark Haines 641efb6a39 Fix the deduplication of incoming direct-to-device messages 2016-08-25 18:14:02 +01:00
Mark Haines e993925279 Add store-and-forward direct-to-device messaging 2016-08-25 17:35:37 +01:00
Paul Evans d9dcb2ba3a Merge pull request #1041 from matrix-org/paul/third-party-lookup
Extend 3PE lookup APIs for metadata query
2016-08-25 17:06:53 +01:00
Paul "LeoNerd" Evans 8e1ed09dff Move static knowledge of protocol metadata into AS handler; cache the result 2016-08-24 13:01:53 +01:00
Paul "LeoNerd" Evans 965f33c901 Declare 'gitter' known protocol, with user lookup 2016-08-24 12:34:03 +01:00
Paul "LeoNerd" Evans 9899824b85 Initial hack at the 3PN protocols metadata lookup API 2016-08-24 12:33:01 +01:00
Erik Johnston 9219139351 Preserve some logcontexts 2016-08-24 11:58:40 +01:00
Paul "LeoNerd" Evans 63c19e1df9 Move 3PU/3PL lookup APIs into /thirdparty containing entity 2016-08-24 11:55:57 +01:00
Kegan Dougal 4b31426a02 Pass through user-supplied content in /join/$room_id
It was always intended to allow custom keys on the join event, but this has
at some point been lost. Restore it.

If the user specifies keys like "avatar_url" then they will be clobbered.
2016-08-23 16:32:04 +01:00
Matthew Hodgson 6e80c03d45 Merge branch 'develop' into dbkr/notifications_api 2016-08-20 00:16:18 +01:00
David Baker 0acdd0f1ea Use tuple comparison
Hopefully easier to read
2016-08-18 17:51:08 +01:00
Paul "LeoNerd" Evans b515f844ee Avoid so much copypasta between 3PU and 3PL query by unifying around a ThirdPartyEntityKind enumeration 2016-08-18 17:19:55 +01:00
David Baker 602c84cd9c Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api 2016-08-18 17:15:26 +01:00
Paul "LeoNerd" Evans 105ff162d4 Authenticate 3PE lookup requests 2016-08-18 16:19:23 +01:00
Paul "LeoNerd" Evans 06964c4a0a Copypasta the 3PU support code to also do 3PL 2016-08-18 16:09:50 +01:00
Paul "LeoNerd" Evans f3afd6ef1a Remove TODO note about request fields being strings - they're always strings 2016-08-18 15:53:01 +01:00
Paul "LeoNerd" Evans d5bf7a4a99 Merge remote-tracking branch 'origin/develop' into paul/thirdpartylookup 2016-08-18 14:21:01 +01:00
Paul "LeoNerd" Evans 3856582741 Ensure that 3PU lookup request fields actually get passed in 2016-08-18 14:06:02 +01:00
Erik Johnston 0af9e1a637 Set `Content-Security-Policy` on media repo
This is to inform browsers that they should sandbox the returned
media. This is particularly cruical for javascript/HTML files.
2016-08-17 16:27:39 +01:00
Paul "LeoNerd" Evans fa87c981e1 Thread 3PU lookup through as far as the AS API object; which currently noöps it 2016-08-17 16:17:28 +01:00
Erik Johnston f90b3d83a3 Add None check to _iterate_over_text 2016-08-17 15:17:17 +01:00
Paul "LeoNerd" Evans e3e3fbc23a Initial empty implementation that just registers an API endpoint handler 2016-08-17 12:46:49 +01:00
Erik Johnston 109a560905 Flake8 2016-08-16 14:57:21 +01:00
Erik Johnston 48b5829aea Fix up preview URL API. Add tests.
This includes:

- Splitting out methods of a class into stand alone functions, to make
  them easier to test.
- Adding unit tests to split out functions, testing HTML -> preview.
- Handle the fact that elements in lxml may have tail text.
2016-08-16 14:53:24 +01:00
Erik Johnston 4e1cebd56f Make synchrotron accept /events 2016-08-12 15:31:44 +01:00
Erik Johnston 866a5320de Dont invoke get_handlers fromClientV1RestServlet
hs.get_handlers() can not be invoked from split out processes. Moving
the invocations down a level means that we can slowly split out
individual servlets.
2016-08-12 10:03:19 +01:00
David Baker b4ecf0b886 Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api 2016-08-11 14:09:13 +01:00
Will Hunt 2510db3e76 Don't change status_msg on /sync 2016-08-10 12:59:59 +01:00
Erik Johnston fa1ce4d8ad Don't print stack traces when failing to get remote keys 2016-08-10 10:44:37 +01:00
David Baker cd41c6ece2 Merge pull request #995 from matrix-org/rav/clean_up_cas_login
Clean up CAS login code
2016-08-09 10:21:56 +01:00
Richard van der Hoff 65666fedd5 Clean up CAS login code
Remove some apparently unused code.

Clean up parse_cas_response, mostly to catch the exception if the CAS response
isn't valid XML.
2016-08-08 17:17:25 +01:00
Richard van der Hoff 0682ca04b3 Fix CAS login
Attempting to log in with CAS was giving a 500 error.
2016-08-08 17:01:30 +01:00
Erik Johnston 5bcccfde6c Don't include html comments in description 2016-08-05 14:45:11 +01:00
Erik Johnston b5525c76d1 Typo 2016-08-04 16:10:08 +01:00
Erik Johnston e97648c4e2 Test summarization 2016-08-04 16:09:09 +01:00
Erik Johnston b3682df2ca Merge branch 'develop' of github.com:matrix-org/synapse into erikj/xpath_fix 2016-08-04 15:29:45 +01:00
Richard van der Hoff 530bc862dc Merge branch 'rav/null_default_device_displayname' into develop 2016-08-03 14:30:32 +01:00
Richard van der Hoff 4fec5e57be Default device_display_name to null
It turns out that it's more useful to return a null device display name (and
let clients decide how to handle it: eg, falling back to device_id) than using
a constant string like "unknown device".
2016-08-03 11:53:00 +01:00
Mark Haines 921f17f938 Merge branch 'develop' into rav/refactor_device_query 2016-08-03 11:12:47 +01:00
Erik Johnston 58c9653c6b Don't infer paragrahs from newlines 2016-08-02 18:50:24 +01:00
Erik Johnston 6b58ade2f0 Comment on why we clone 2016-08-02 18:41:22 +01:00
Erik Johnston 9e66c58ceb Spelling. 2016-08-02 18:37:31 +01:00
Erik Johnston f83f5fbce8 Make it actually compile 2016-08-02 18:32:42 +01:00
Erik Johnston aecaec3e10 Change the way we summarize URLs
Using XPath is slow on some machines (for unknown reasons), so use a
different approach to get a list of text nodes.

Try to generate a summary that respect paragraph and then word
boundaries, adding ellipses when appropriate.
2016-08-02 18:25:53 +01:00
Richard van der Hoff 986615b0b2 Move e2e query logic into a handler 2016-08-01 18:02:07 +01:00
David Baker 271d3e7865 Fix adding emails on registration
Synapse was not adding email addresses to accounts registered with an email address, due to too many different variables called 'result'. Rename both of them. Also remove the defer.returnValue() with no params because that's not a thing.
2016-07-29 15:25:24 +01:00
Richard van der Hoff f6f8f81a48 Add r0.1.0 to the "supported versions" list 2016-07-28 10:14:07 +01:00
Richard van der Hoff fda078f995 Add r0.2.0 to the "supported versions" list 2016-07-28 09:14:21 +01:00
Richard van der Hoff ccec25e2c6 key upload tweaks
1. Add v2_alpha URL back in, since things seem to be using it.

2. Don't reject the request if the device_id in the upload request fails to
   match that in the access_token.
2016-07-27 16:41:06 +01:00
Richard van der Hoff d47115ff8b Delete e2e keys on device delete 2016-07-27 12:24:52 +01:00
Richard van der Hoff 2e3d90d67c Make the device id on e2e key upload optional
We should now be able to get our device_id from the access_token, so the
device_id on the upload request is optional. Where it is supplied, we should
check that it matches.

For active access_tokens without an associated device_id, we ought to register
the device in the devices table.

Also update the table on upgrade so that all of the existing e2e keys are
associated with real devices.
2016-07-26 23:38:12 +01:00
Richard van der Hoff eb359eced4 Add `create_requester` function
Wrap the `Requester` constructor with a function which provides sensible
defaults, and use it throughout
2016-07-26 16:46:53 +01:00
Richard van der Hoff 012b4c1913 Implement updating devices
You can update the displayname of devices now.
2016-07-26 07:35:48 +01:00
Richard van der Hoff 436bffd15f Implement deleting devices 2016-07-26 07:35:48 +01:00
David Baker 2c28e25bda Merge pull request #943 from matrix-org/rav/get_device_api
Implement GET /device/{deviceId}
2016-07-21 13:41:42 +01:00
David Baker 1e6e370b76 Merge pull request #942 from matrix-org/rav/fix_register_deviceid
Preserve device_id from first call to /register
2016-07-21 13:16:31 +01:00
Richard van der Hoff 406f7aa0f6 Implement GET /device/{deviceId} 2016-07-21 12:00:29 +01:00
Richard van der Hoff 1a64dffb00 Preserve device_id from first call to /register
device_id may only be passed in the first call to /register, so make sure we
fish it out of the register `params` rather than the body of the final call.
2016-07-21 11:34:16 +01:00
Richard van der Hoff 7314bf4682 Merge branch 'develop' into rav/get_devices_api
(pick up PR #938 in the hope of fixing the UTs)
2016-07-20 17:40:00 +01:00
Richard van der Hoff bc8f265f0a GET /devices endpoint
implement a GET /devices endpoint which lists all of the user's devices.

It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
Richard van der Hoff b97a1356b1 Register a device_id in the /v2/register flow.
This doesn't cover *all* of the registration flows, but it does cover the most
common ones: in particular: shared_secret registration, appservice
registration, and normal user/pass registration.

Pull device_id from the registration parameters. Register the device in the
devices table. Associate the device with the returned access and refresh
tokens. Profit.
2016-07-20 16:38:27 +01:00
Richard van der Hoff c68518dfbb Merge pull request #933 from matrix-org/rav/type_annotations
Type annotations
2016-07-20 12:26:32 +01:00
David Baker e967bc86e7 Merge pull request #932 from matrix-org/rav/register_refactor
Further registration refactoring
2016-07-20 11:03:33 +01:00
Erik Johnston 1e2a7f18a1 Merge pull request #922 from matrix-org/erikj/file_api2
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
Richard van der Hoff 3413f1e284 Type annotations
Add some type annotations to help PyCharm (in particular) to figure out the
types of a bunch of things.
2016-07-19 18:56:16 +01:00
Richard van der Hoff 40cbffb2d2 Further registration refactoring
* `RegistrationHandler.appservice_register` no longer issues an access token:
  instead it is left for the caller to do it. (There are two of these, one in
  `synapse/rest/client/v1/register.py`, which now simply calls
  `AuthHandler.issue_access_token`, and the other in
  `synapse/rest/client/v2_alpha/register.py`, which is covered below).

* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
  access_tokens into `_create_registration_details`. This means that the normal
  flow no longer needs to call `AuthHandler.issue_access_token`; the
  shared-secret flow can tell `RegistrationHandler.register` not to generate a
  token; and the appservice flow continues to work despite the above change.
2016-07-19 18:46:19 +01:00
David Baker b9e997f561 Merge pull request #931 from matrix-org/rav/refactor_register
rest/client/v2_alpha/register.py: Refactor flow somewhat.
2016-07-19 16:13:45 +01:00
Richard van der Hoff 8f6281ab0c Don't bind email unless threepid contains expected fields 2016-07-19 15:50:01 +01:00
Richard van der Hoff 0da0d0a29d rest/client/v2_alpha/register.py: Refactor flow somewhat.
This is meant to be an *almost* non-functional change, with the exception that
it fixes what looks a lot like a bug in that it only calls
`auth_handler.add_threepid` and `add_pusher` once instead of three times.

The idea is to move the generation of the `access_token` out of
`registration_handler.register`, because `access_token`s now require a
device_id, and we only want to generate a device_id once registration has been
successful.
2016-07-19 13:12:22 +01:00
Richard van der Hoff f863a52cea Add device_id support to /login
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.

Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.

Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
Richard van der Hoff 93efcb8526 Merge pull request #928 from matrix-org/rav/refactor_login
Refactor login flow
2016-07-18 16:12:35 +01:00
Richard van der Hoff dcfd71aa4c Refactor login flow
Make sure that we have the canonical user_id *before* calling
get_login_tuple_for_user_id.

Replace login_with_password with a method which just validates the password,
and have the caller call get_login_tuple_for_user_id. This brings the password
flow into line with the other flows, and will give us a place to register the
device_id if necessary.
2016-07-18 15:23:54 +01:00
Will Hunt 511a52afc8 Use body.get to check for 'user' 2016-07-16 18:44:08 +01:00
Will Hunt e885e2a623 Fall back to 'username' if 'user' is not given for appservice reg. 2016-07-16 18:33:48 +01:00
Erik Johnston a3036ac37e Merge pull request #921 from matrix-org/erikj/account_deactivate
Feature: Add an /account/deactivate endpoint
2016-07-14 17:25:15 +01:00
Erik Johnston a98d215204 Add filter param to /messages API 2016-07-14 16:30:56 +01:00
Erik Johnston 209e04fa11 Merge pull request #918 from negzi/bugfix_for_token_expiry
Bug fix: expire invalid access tokens
2016-07-14 15:51:52 +01:00
Erik Johnston 848d3bf2e1 Add hs object 2016-07-14 10:25:52 +01:00
Erik Johnston b55c770271 Only accept password auth 2016-07-14 10:00:38 +01:00
Erik Johnston d543b72562 Add an /account/deactivate endpoint 2016-07-14 09:56:53 +01:00
Negar Fazeli 0136a522b1 Bug fix: expire invalid access tokens 2016-07-13 15:00:37 +02:00
David Baker c55ad2e375 be more pythonic 2016-07-12 14:15:10 +01:00
David Baker aaa9d9f0e1 on_OPTIONS isn't neccessary 2016-07-12 14:13:14 +01:00
David Baker 75fa7f6b3c Remove other debug logging 2016-07-12 14:08:57 +01:00
David Baker a5db0026ed Separate out requestTokens to separate handlers 2016-07-11 09:57:07 +01:00
David Baker 9c491366c5 Oops, remove debug logging 2016-07-11 09:07:40 +01:00
David Baker 385aec4010 Implement https://github.com/matrix-org/matrix-doc/pull/346/files 2016-07-08 17:42:48 +01:00
Erik Johnston 67f2c901ea Add rest servlet. Fix SQL. 2016-07-06 15:56:59 +01:00
Erik Johnston a17e7caeb7 Merge branch 'erikj/shared_secret' into erikj/test2 2016-07-06 14:46:31 +01:00
Erik Johnston 76b18df3d9 Check that there are no null bytes in user and passsword 2016-07-06 11:17:53 +01:00
Erik Johnston 0da24cac8b Add null separator to hmac 2016-07-06 11:05:16 +01:00
Erik Johnston 651faee698 Add an admin option to shared secret registration 2016-07-05 17:30:22 +01:00
Erik Johnston caf33b2d9b Protect password when registering using shared secret 2016-07-05 17:18:19 +01:00
Erik Johnston 2d21d43c34 Add purge_history API 2016-07-05 10:28:51 +01:00
Kent Shikama bb069079bb
Fix style violations
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-04 22:07:11 +09:00
Kent Shikama 2e5a31f197 Use .get() instead of [] to access password_hash 2016-07-04 22:00:13 +09:00
Kent Shikama fc8007dbec
Optionally include password hash in createUser endpoint
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-03 15:08:15 +09:00
Erik Johnston f328d95cef Feature: Add deactivate account admin API
Allows server admins to "deactivate" accounts, which:

- Revokes all access tokens
- Removes all threepids
- Removes password

The API is a POST to `/admin/deactivate/<user_id>`
2016-06-30 15:40:58 +01:00
Erik Johnston f52cb4cd78 Remove race 2016-06-29 15:24:50 +01:00
Erik Johnston a70688445d Implement purge_media_cache admin API 2016-06-29 14:57:59 +01:00
Erik Johnston 314b146b2e Track approximate last access time for remote media 2016-06-29 11:41:20 +01:00
Mark Haines 13e334506c Remove the legacy v0 content upload API.
The existing content can still be downloaded. The last upload to the
matrix.org server was in January 2015, so it is probably safe to remove
the upload API.
2016-06-21 11:47:39 +01:00
Erik Johnston 09a17f965c Line lengths 2016-06-15 16:58:12 +01:00
Erik Johnston 1e9026e484 Handle floats as img widths 2016-06-15 16:58:05 +01:00
Erik Johnston a60169ea09 Handle og props with not content 2016-06-15 16:57:48 +01:00
Erik Johnston eba4ff1bcb 502 on /thumbnail when can't contact remote server 2016-06-09 11:29:43 +01:00
Erik Johnston 95f305c35a Remove redundant exception log in /events 2016-06-09 11:15:04 +01:00
Erik Johnston 690029d1a3 Don't make rooms visibile by default 2016-06-08 14:47:42 +01:00
Erik Johnston efeabd3180 Log user that is making /publicRooms calls 2016-06-08 14:23:15 +01:00
Erik Johnston 6a0afa582a Load push rules in storage layer, so that they get cached 2016-06-03 11:10:00 +01:00
David Baker 1f31cc37f8 Working unsubscribe links going straight to the HS
and authed by macaroons that let you delete pushers and nothing else
2016-06-02 17:21:31 +01:00
David Baker 3a3fb2f6f9 Merge branch 'dbkr/split_out_auth_handler' into dbkr/email_unsubscribe 2016-06-02 13:35:25 +01:00
David Baker 4a10510cd5 Split out the auth handler 2016-06-02 13:31:45 +01:00
David Baker 991af8b0d6 WIP on unsubscribing email notifs without logging in 2016-06-01 17:40:52 +01:00
David Baker d240796ded Basic, un-cached support for secondary_directory_servers 2016-05-31 17:20:07 +01:00
David Baker 887c6e6f05 Split out the room list handler
So I can use it from federation bits without pulling in all the handlers.
2016-05-31 11:05:16 +01:00
David Baker 37b7e84620 Include the ts the notif was received at 2016-05-24 11:33:32 +01:00
David Baker b791a530da Actually make the 'read' flag correct 2016-05-23 18:48:02 +01:00
David Baker a24bc5b2dc Add GET /notifications API 2016-05-23 18:33:51 +01:00
Kegan Dougal 332d7e9b97 Allow clients to specify a server_name to avoid 'No known servers'
Multiple server_names are supported via ?server_name=foo&server_name=bar
2016-05-19 13:50:52 +01:00
Mark Haines 0cb441fedd Move typing handler out of the Handlers object 2016-05-17 15:58:46 +01:00
Mark Haines f68eea808a Move SyncHandler out of the Handlers object 2016-05-16 20:19:26 +01:00
Mark Haines 3b86ecfa79 Move the presence handler out of the Handlers object 2016-05-16 18:56:37 +01:00
Mark Haines eb79110beb Clean up the blacklist/whitelist handling.
Always set the config key with an empty list, even if a list isn't specified.
This means that the codepaths are the same for both the empty list and
for a missing key. Since the behaviour is the same for both cases this
makes the code somewhat easier to reason about.
2016-05-16 13:03:59 +01:00
Negi Fazeli 40aa6e8349 Create user with expiry
- Add unittests for client, api and handler

Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>
2016-05-13 15:34:15 +02:00
David Baker 997db04648 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-05-10 14:40:19 +02:00
David Baker c00b484eff More consistent config naming 2016-05-10 14:39:16 +02:00
David Baker 94040b0798 Add config option to not send email notifs for new users 2016-05-10 14:34:53 +02:00
Mark Haines 573ef3f1c9 Rename openid/token to openid/request_token 2016-05-05 15:15:00 +01:00
Mark Haines 9c272da05f Add an openidish mechanism for proving to third parties that you own a given user_id 2016-05-05 13:42:44 +01:00
Erik Johnston 8e6a163f27 Add timestamp and auto incrementing ID 2016-05-04 15:19:12 +01:00
Erik Johnston 5650e38e7d Move event_id to path 2016-05-04 13:19:39 +01:00
Erik Johnston 984d4a2c0f Add /report endpoint 2016-05-04 11:28:10 +01:00
David Baker 2df75de505 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-04-29 20:28:47 +01:00
David Baker dc2c527ce9 Fix password reset
Default requester to None, otherwise it isn't defined when resetting using email auth
2016-04-29 12:07:54 +01:00
David Baker b2c04da8dc Add an email pusher for new users
If they registered with an email address and email notifs are enabled on the HS
2016-04-29 11:43:57 +01:00
Mark Haines 8d7ad44331 Report per request metrics for all of the things using request_handler 2016-04-28 10:57:49 +01:00
Erik Johnston 52ecbc2843 Make pyjwt dependency optional 2016-04-25 14:30:15 +01:00
Erik Johnston b9675ef6e6 Merge pull request #687 from nikriek/jwt-fix
Fix issues with JWT login
2016-04-21 17:42:25 +01:00
Niklas Riekenbrauck 565c2edb0a Fix issues with JWT login 2016-04-21 18:10:48 +02:00
Erik Johnston e8884e5e9c Add self.media_repo to PreviewUrlResource 2016-04-19 14:51:34 +01:00