Commit Graph

1310 Commits (0bc4627a731d0edd437905a5b07e85421c7553d8)

Author SHA1 Message Date
Richard van der Hoff 669400e22f Enable auto-escaping for the consent templates
... to reduce the risk of somebody introducing an html injection attack...
2018-05-22 14:58:34 +01:00
Richard van der Hoff 8810685df9 Stub out ServerNoticesSender on the workers
... and have the sync endpoints call it directly rather than obsure indirection
via PresenceHandler
2018-05-22 11:54:51 +01:00
Richard van der Hoff d5dca9a04f Move consent config parsing into ConsentConfig
turns out we need to reuse this, so it's better in the config class.
2018-05-22 11:54:51 +01:00
Richard van der Hoff d14d7b8fdc Rename 'version' param on user consent config
we're going to use it for the version we require too.
2018-05-22 11:54:51 +01:00
Richard van der Hoff 6e1cb54a05 Fix logcontext leak in HttpTransactionCache
ONE DAY I WILL PURGE THE WORLD OF THIS EVIL
2018-05-21 16:58:20 +01:00
Richard van der Hoff 6d6e7288fe Stop the transaction cache caching failures
The transaction cache has some code which tries to stop it caching failures,
but if the callback function failed straight away, then things would happen
backwards and we'd end up with the failure stuck in the cache.
2018-05-21 16:49:59 +01:00
Adrian Tschira 933bf2dd35 replace some iteritems with six
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:59:26 +02:00
Adrian Tschira d9fe2b2d9d Replace some more comparisons with six
plus a bonus b"" string I missed last time

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:56:31 +02:00
Adrian Tschira aafb0f6b0d py3-ize url preview 2018-05-19 17:35:20 +02:00
Erik Johnston fa30ac38cc
Merge pull request #3221 from matrix-org/erikj/purge_token
Make purge_history operate on tokens
2018-05-18 10:35:23 +01:00
Richard van der Hoff c46367d0d7 Move RoomCreationHandler out of synapse.handlers.Handlers
Handlers is deprecated nowadays, so let's move this out before I add a new
dependency on it.

Also fix the docstrings on create_room.
2018-05-17 09:08:42 +01:00
Richard van der Hoff 8030a825c8
Merge pull request #3213 from matrix-org/rav/consent_handler
ConsentResource to gather policy consent from users
2018-05-16 07:19:18 +01:00
Erik Johnston 5f27ed75ad Make purge_history operate on tokens
As we're soon going to change how topological_ordering works
2018-05-15 16:23:50 +01:00
Richard van der Hoff 47815edcfa ConsentResource to gather policy consent from users
Hopefully there are enough comments and docs in this that it makes sense on its
own.
2018-05-15 15:11:59 +01:00
Richard van der Hoff 318711e139 Set Server header in SynapseRequest
(instead of everywhere that writes a response. Or rather, the subset of places
which write responses where we haven't forgotten it).

This also means that we don't have to have the mysterious version_string
attribute in anything with a request handler.

Unfortunately it does mean that we have to pass the version string wherever we
instantiate a SynapseSite, which has been c&ped 150 times, but that is code
that ought to be cleaned up anyway really.
2018-05-10 18:50:27 +01:00
Richard van der Hoff 645cb4bf06 Remove redundant request_handler decorator
This is needless complexity; we might as well use the wrapper directly.

Also rename wrap_request_handler->wrap_json_request_handler.
2018-05-10 12:19:53 +01:00
Konstantinos Sideris 88868b2839 notifications: Convert next_token to string according to the spec
Currently the parameter is serialized as an integer.

Signed-off-by: Konstantinos Sideris <sideris.konstantin@gmail.com>
2018-05-05 12:55:02 +03:00
Richard van der Hoff 902673e356
Merge pull request #3161 from NotAFile/remove-v1auth
Make Client-Server API return 403 for invalid token
2018-05-03 10:10:57 +01:00
Richard van der Hoff be31adb036 Fix logcontext leak in media repo
Make FileResponder.write_to_consumer uphold the logcontext contract
2018-05-02 16:14:50 +01:00
Adrian Tschira 6495dbb326 Burminate v1auth
This closes #2602

v1auth was created to account for the differences in status code between
the v1 and v2_alpha revisions of the protocol (401 vs 403 for invalid
tokens). However since those protocols were merged, this makes the r0
version/endpoint internally inconsistent, and violates the
specification for the r0 endpoint.

This might break clients that rely on this inconsistency with the
specification. This is said to affect the legacy angular reference
client. However, I feel that restoring parity with the spec is more
important. Either way, it is critical to inform developers about this
change, in case they rely on the illegal behaviour.

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-30 22:20:43 +02:00
Krombel 576b71dd3d add guard for None on purge_history api 2018-04-30 14:29:48 +02:00
Richard van der Hoff dbf6f28d64
Merge pull request #3155 from NotAFile/py3-bytes-1
more bytes strings
2018-04-30 00:38:21 +01:00
Richard van der Hoff aab2e4da60
Merge pull request #3140 from matrix-org/rav/use_run_in_background
Use run_in_background in preference to preserve_fn
2018-04-30 00:34:28 +01:00
Richard van der Hoff 1315d374cc
Merge pull request #3156 from NotAFile/py3-hmac-bytes
Construct HMAC as bytes on py3
2018-04-30 00:33:20 +01:00
Richard van der Hoff 9e2601f830
Merge pull request #3108 from NotAFile/py3-six-urlparse
Use six.moves.urlparse
2018-04-30 00:33:05 +01:00
Adrian Tschira 122593265b Construct HMAC as bytes on py3
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-29 00:19:41 +02:00
Adrian Tschira e9143b6593 more bytes strings
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-29 00:13:57 +02:00
Richard van der Hoff fc149b4eeb Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background 2018-04-27 14:31:23 +01:00
Richard van der Hoff 2a13af23bc Use run_in_background in preference to preserve_fn
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
2018-04-27 12:55:51 +01:00
Richard van der Hoff 9255a6cb17 Improve exception handling for background processes
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.

This is unsatisfactory for a number of reasons:
 - logging on garbage collection is best-effort and may happen some time after
   the error, if at all
 - it can be hard to figure out where the error actually happened.
 - it is logged as a scary CRITICAL error which (a) I always forget to grep for
   and (b) it's not really CRITICAL if a background process we don't care about
   fails.

So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
Adrian Tschira 2a3c33ff03 Use six.moves.urlparse
The imports were shuffled around a bunch in py3

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 21:22:43 +02:00
Erik Johnston eaa2ebf20b
Merge pull request #3079 from matrix-org/erikj/limit_concurrent_sends
Limit concurrent event sends for a room
2018-04-10 16:43:58 +01:00
Richard van der Hoff a3599dda97
Merge pull request #2996 from krombel/allow_auto_join_rooms
move handling of auto_join_rooms to RegisterHandler
2018-04-10 01:11:00 +01:00
Richard van der Hoff 87478c5a60
Merge pull request #3061 from NotAFile/add-some-byte-strings
Add b prefixes to some strings that are bytes in py3
2018-04-09 23:54:05 +01:00
Richard van der Hoff c508b2f2f0
Merge pull request #3073 from NotAFile/use-six-reraise
Replace old-style raise with six.reraise
2018-04-09 23:53:40 +01:00
Richard van der Hoff 37354b55c9
Merge pull request #2938 from dklug/develop
Return 401 for invalid access_token on logout
2018-04-09 23:52:56 +01:00
Erik Johnston 56b0589865 Use create_and_send_nonmember_event everywhere 2018-04-09 12:04:18 +01:00
Adrian Tschira 4f40d058cc Replace old-style raise with six.reraise
The old style raise is invalid syntax in python3. As noted in the docs,
this adds one more frame in the traceback, but I think this is
acceptable:

    <ipython-input-7-bcc5cba3de3f> in <module>()
         16     except:
         17         pass
    ---> 18     six.reraise(*x)

    /usr/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
        691             if value.__traceback__ is not tb:
        692                 raise value.with_traceback(tb)
    --> 693             raise value
        694         finally:
        695             value = None

    <ipython-input-7-bcc5cba3de3f> in <module>()
          9
         10 try:
    ---> 11     x()
         12 except:
         13     x = sys.exc_info()

Also note that this uses six, which is not formally a dependency yet,
but is included indirectly since most packages depend on it.

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-06 23:06:24 +02:00
Richard van der Hoff c7f0969731
Merge pull request #2986 from jplatte/join_reponse_room_id
Add room_id to the response of `rooms/{roomId}/join`
2018-04-05 17:29:06 +01:00
Luke Barnard 104c0bc1d5 Use "/settings/" (plural) 2018-04-05 14:07:16 +01:00
Adrian Tschira 6168351877 Add b prefixes to some strings that are bytes in py3
This has no effect on python2

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-04 13:48:51 +02:00
Luke Barnard eb8d8d6f57 Use join_policy API instead of joinable
The API is now under
 /groups/$group_id/setting/m.join_policy

and expects a JSON blob of the shape

```json
{
  "m.join_policy": {
    "type": "invite"
  }
}
```

where "invite" could alternatively be "open".
2018-04-03 16:16:40 +01:00
David Baker c5de6987c2 This should probably be a PUT 2018-03-28 16:44:11 +01:00
David Baker 79452edeee Add joinability for groups
Adds API to set the 'joinable' flag, and corresponding flag in the
table.
2018-03-28 14:03:37 +01:00
Krombel 6152e253d8 Merge branch 'develop' of into allow_auto_join_rooms 2018-03-28 14:45:28 +02:00
Erik Johnston fa72803490 Merge branch 'master' of github.com:matrix-org/synapse into develop 2018-03-19 11:41:01 +00:00
Erik Johnston a8ce159be4 Replace some ujson with simplejson to make it work 2018-03-16 00:27:09 +00:00
Erik Johnston 926ba76e23 Replace ujson with simplejson 2018-03-15 23:43:31 +00:00
Krombel 91ea0202e6 move handling of auto_join_rooms to RegisterHandler
Currently the handling of auto_join_rooms only works when a user
registers itself via public register api. Registrations via
registration_shared_secret and ModuleApi do not work

This auto_joins the users in the registration handler which enables
the auto join feature for all 3 registration paths.

This is related to issue #2725

Signed-Off-by: Matthias Kesler <krombel@krombel.de>
2018-03-14 16:45:37 +01:00
Jonas Platte 47ce527f45 Add room_id to the response of `rooms/{roomId}/join`
Fixes #2349
2018-03-13 14:48:12 +01:00
Erik Johnston f5160d4a3e RoomMembershipRestServlet doesn't handle /forget
Due to the order we register the REST handlers `/forget` was handled by
the correct handler.
2018-03-13 12:12:55 +00:00
Richard van der Hoff e48c7aac4d Add transactional API to history purge
Make the purge request return quickly, and allow scripts to poll for updates.
2018-03-12 16:22:55 +00:00
Richard van der Hoff f8bfcd7e0d Provide a means to pass a timestamp to purge_history 2018-03-05 14:37:23 +00:00
dklug af7ed8e1ef Return 401 for invalid access_token on logout
Signed-off-by: Duncan Klug <dklug@ucmerced.edu>
2018-03-02 22:01:27 -08:00
Erik Johnston 784f036306 Move RoomMemberHandler out of Handlers 2018-03-01 14:36:50 +00:00
Erik Johnston 92c52df702 Make store_file use store_into_file 2018-02-14 17:55:18 +00:00
Erik Johnston c0c9327fe0
Merge pull request #2854 from matrix-org/erikj/event_create_worker
Create a worker for event creation
2018-02-13 18:07:10 +00:00
Richard van der Hoff 8fd1a32456 Fix typos in purge api & doc
* It's supposed to be purge_local_events, not ..._history
* Fix the doc to have valid json
2018-02-13 13:09:39 +00:00
Erik Johnston 1026690cd2
Merge pull request #2857 from matrix-org/erikj/upload_store
Tell storage providers about new file so they can upload
2018-02-12 13:52:58 +00:00
Richard van der Hoff 10b34dbb9a
Merge pull request #2858 from matrix-org/rav/purge_updates
delete_local_events for purge_room_history
2018-02-09 14:11:00 +00:00
Richard van der Hoff 74fcbf741b delete_local_events for purge_history
Add a flag which makes the purger delete local events
2018-02-09 13:07:41 +00:00
Erik Johnston 5fa571a91b Tell storage providers about new file so they can upload 2018-02-07 13:35:08 +00:00
Erik Johnston 8ec2e638be Add event_creator worker 2018-02-07 10:32:32 +00:00
Erik Johnston 3e1e69ccaf Update copyright 2018-02-06 16:40:38 +00:00
Erik Johnston 3fa362502c Update places where we create events 2018-02-05 16:01:48 +00:00
Erik Johnston 1f881e0746
Merge pull request #2791 from matrix-org/erikj/media_storage_refactor
Ensure media is in local cache before thumbnailing
2018-02-05 11:28:52 +00:00
Richard van der Hoff 9c9356512e
Merge pull request #2845 from matrix-org/rav/urlcache_error_handling
Handle url_previews with no content-type
2018-02-02 15:27:52 +01:00
Travis Ralston 6e87b34f7b
Merge branch 'develop' into travis/admin-list-media 2018-02-01 18:05:47 -07:00
Richard van der Hoff d5352cbba8 Handle url_previews with no content-type
avoid failing with an exception if the remote server doesn't give us a
Content-Type header.

Also, clean up the exception handling a bit.
2018-02-02 00:53:46 +00:00
Matthew Hodgson 9a72b70630 fix thinko on 3pid whitelisting 2018-01-24 11:07:47 +01:00
Matthew Hodgson d32385336f
add ?ts massaging for ASes (#2754)
blindly implement ?ts for AS. untested
2018-01-23 09:59:06 +01:00
Matthew Hodgson ab9f844aaf
Add federation_domain_whitelist option (#2820)
Add federation_domain_whitelist

gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
Matthew Hodgson d84f65255e
Merge pull request #2813 from matrix-org/matthew/registrations_require_3pid
add registrations_require_3pid and allow_local_3pids
2018-01-22 13:57:22 +00:00
Travis Ralston 5552ed9a7f Add an admin route to get all the media in a room
This is intended to be used by administrators to monitor the media that is passing through their server, if they wish.

Signed-off-by: Travis Ralston <travpc@gmail.com>
2018-01-20 22:37:53 -07:00
Matthew Hodgson 49fce04624 fix typo (thanks sytest) 2018-01-19 19:55:38 +00:00
Richard van der Hoff b0d9e633ee
Merge pull request #2814 from matrix-org/rav/fix_urlcache_thumbs
Use the right path for url_preview thumbnails
2018-01-19 18:57:15 +00:00
Richard van der Hoff ad7ec63d08 Use the right path for url_preview thumbnails
This was introduced by #2627: we were overwriting the original media for url
previews with the thumbnails :/

(fixes https://github.com/vector-im/riot-web/issues/6012, hopefully)
2018-01-19 18:29:39 +00:00
Matthew Hodgson 62d7d66ae5 oops, check all login types 2018-01-19 18:23:56 +00:00
Matthew Hodgson 293380bef7 trailing commas 2018-01-19 15:38:53 +00:00
Matthew Hodgson 447f4f0d5f rewrite based on PR feedback:
* [ ] split config options into allowed_local_3pids and registrations_require_3pid
 * [ ] simplify and comment logic for picking registration flows
 * [ ] fix docstring and move check_3pid_allowed into a new util module
 * [ ] use check_3pid_allowed everywhere

@erikjohnston PTAL
2018-01-19 15:33:55 +00:00
Matthew Hodgson 9d332e0f79 fix up v1, and improve errors 2018-01-19 00:53:58 +00:00
Matthew Hodgson 0af58f14ee fix pep8 2018-01-19 00:33:51 +00:00
Matthew Hodgson 28a6ccb49c add registrations_require_3pid
lets homeservers specify a whitelist for 3PIDs that users are allowed to associate with.
Typically useful for stopping people from registering with non-work emails
2018-01-19 00:19:58 +00:00
Erik Johnston cd871a3057 Fix storage provider bug introduced when renamed to store_local 2018-01-18 18:37:59 +00:00
Erik Johnston 8ff6726c0d
Merge pull request #2812 from matrix-org/erikj/media_storage_provider_config
Make storage providers configurable
2018-01-18 18:33:57 +00:00
Erik Johnston 3fe2bae857 Missing staticmethod 2018-01-18 17:11:45 +00:00
Erik Johnston aae77da73f Fixup comments 2018-01-18 17:11:29 +00:00
Erik Johnston 9a89dae8c5 Fix typo in thumbnail resource causing access times to be incorrect 2018-01-18 15:06:24 +00:00
Erik Johnston 0af5dc63a8 Make storage providers more configurable 2018-01-18 14:07:21 +00:00
Erik Johnston 2cf6a7bc20 Use better file consumer 2018-01-18 12:00:46 +00:00
Erik Johnston 4a53f3a3e8 Ensure media is in local cache before thumbnailing 2018-01-18 12:00:46 +00:00
Erik Johnston 300edc2348 Update last access time when thumbnails are viewed 2018-01-17 10:24:43 +00:00
Erik Johnston 05f98a2224 Keep track of last access time for local media 2018-01-17 10:24:43 +00:00
Erik Johnston d728c47142 Add docstring 2018-01-17 10:06:14 +00:00
Erik Johnston d863f68cab Use local vars 2018-01-16 16:24:15 +00:00
Erik Johnston 6368e5c0ab Change _generate_thumbnails to take media_type 2018-01-16 16:17:38 +00:00
Erik Johnston 0a90d9ede4 Move setting of file_id up to caller 2018-01-16 16:03:05 +00:00
Erik Johnston 5dfc83704b Fix typo 2018-01-16 14:32:56 +00:00
Erik Johnston 307f88dfb6 Fix up log lines 2018-01-16 13:53:52 +00:00
Erik Johnston 9795b9ebb1 Correctly use server_name/file_id when generating/fetching remote thumbnails 2018-01-16 12:02:06 +00:00