Commit Graph

44 Commits (ad3f43be9a597dd4fdf59e0a95e4630e7b9502fe)

Author SHA1 Message Date
Patrick Cloke 4fc8875876
Refactor media modules. (#15146)
* Removes the `v1` directory from `test.rest.media.v1`.
* Moves the non-REST code from `synapse.rest.media.v1` to `synapse.media`.
* Flatten the `v1` directory from `synapse.rest.media`,  but leave compatiblity
  with 3rd party media repositories and spam checkers.
2023-02-27 08:26:05 -05:00
David Robertson 2dff93099b
Typecheck tests.rest.media.v1.test_media_storage (#15008)
* Fix MediaStorage type hint

* Typecheck tests.rest.media.v1.test_media_storage

* Changelog

* Remove assert and make the comment succinct

* Fix syntax for olddeps
2023-02-07 15:24:44 +00:00
David Teller 11f811470f
Uniformize spam-checker API, part 5: expand other spam-checker callbacks to return `Tuple[Codes, dict]` (#13044)
Signed-off-by: David Teller <davidt@element.io>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-07-11 16:52:10 +00:00
David Teller a164a46038
Uniformize spam-checker API, part 4: port other spam-checker callbacks to return `Union[Allow, Codes]`. (#12857)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-06-13 18:16:16 +00:00
David Robertson a2b00a4486
Bump `black` and `click` versions (#12320) 2022-03-29 10:41:19 +00:00
Patrick Cloke 10a88ba91c
Use auto_attribs/native type hints for attrs classes. (#11692) 2022-01-13 13:49:28 +00:00
Sean Quah 6c83c27107
Fix race conditions when creating media store and config directories (#10913) 2021-09-27 11:29:23 +01:00
Patrick Cloke b93259082c
Add missing type hints to non-client REST servlets. (#10817)
Including admin, consent, key, synapse, and media. All REST servlets
(the synapse.rest module) now require typed method definitions.
2021-09-15 08:45:32 -04:00
Patrick Cloke b996782df5
Convert media repo's FileInfo to attrs. (#10785)
This is mostly an internal change, but improves type hints in the
media code.
2021-09-14 07:09:38 -04:00
Jonathan de Jong 98aec1cc9d
Use inline type hints in `handlers/` and `rest/`. (#10382) 2021-07-16 18:22:36 +01:00
Jonathan de Jong 4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Erik Johnston 3d2acc930f Return a 404 if we don't have the original file 2021-02-19 10:46:18 +00:00
Eric Eastwood 0a00b7ff14
Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Erik Johnston 7e8083eb48 Add check_media_file_for_spam spam checker hook 2021-02-04 17:01:30 +00:00
Patrick Cloke d34c6e1279
Add type hints to media rest resources. (#9093) 2021-01-15 10:57:37 -05:00
Erik Johnston 46f4be94b4
Fix race for concurrent downloads of remote media. (#8682)
Fixes #6755
2020-10-30 10:55:24 +00:00
Richard van der Hoff 73d93039ff
Fix bug in remote thumbnail search (#8438)
#7124 changed the behaviour of remote thumbnails so that the thumbnailing method was included in the filename of the thumbnail. To support existing files, it included a fallback so that we would check the old filename if the new filename didn't exist.

Unfortunately, it didn't apply this logic to storage providers, so any thumbnails stored on such a storage provider was broken.
2020-10-02 12:29:29 +01:00
DeepBlueV7.X 560f3b8609
Include method in thumbnail media name (#7124)
This fixes an issue where different methods (crop/scale) overwrite each other.

This first tries the new path. If that fails and we are looking for a
remote thumbnail, it tries the old path. If that still isn't found, it
continues as normal.

This should probably be removed in the future, after some of the newer
thumbnails were generated with the new path on most deployments. Then
the overhead should be minimal if the other thumbnails need to be
regenerated.

Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
2020-09-08 17:19:50 +01:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Patrick Cloke 8ff2deda72
Fix async/await calls for broken media providers. (#8027) 2020-08-04 09:44:25 -04:00
Patrick Cloke 68626ff8e9
Convert the remaining media repo code to async / await. (#7947) 2020-07-27 14:40:11 -04:00
Patrick Cloke 5ea29d7f85
Convert more of the media code to async/await (#7873) 2020-07-24 09:39:02 -04:00
Dagfinn Ilmari Mannsåker a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Richard van der Hoff 6b7462a13f
a bit of debugging for media storage providers (#6757)
* a bit of debugging for media storage providers

* changelog
2020-01-23 12:11:44 +00:00
Amber Brown 4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Amber Brown 463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Richard van der Hoff 5c445114d3
Correctly account for cpu usage by background threads (#4074)
Wrap calls to deferToThread() in a thing which uses a child logcontext to
attribute CPU usage to the right request.

While we're in the area, remove the logcontext_tracer stuff, which is never
used, and afaik doesn't work.

Fixes #4064
2018-10-23 13:12:32 +01:00
Amber Brown da7785147d
Python 3: Convert some unicode/bytes uses (#3569) 2018-08-02 00:54:06 +10:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
Amber Brown 77ac14b960
Pass around the reactor explicitly (#3385) 2018-06-22 09:37:10 +01:00
Richard van der Hoff be31adb036 Fix logcontext leak in media repo
Make FileResponder.write_to_consumer uphold the logcontext contract
2018-05-02 16:14:50 +01:00
Adrian Tschira 4f40d058cc Replace old-style raise with six.reraise
The old style raise is invalid syntax in python3. As noted in the docs,
this adds one more frame in the traceback, but I think this is
acceptable:

    <ipython-input-7-bcc5cba3de3f> in <module>()
         16     except:
         17         pass
    ---> 18     six.reraise(*x)

    /usr/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
        691             if value.__traceback__ is not tb:
        692                 raise value.with_traceback(tb)
    --> 693             raise value
        694         finally:
        695             value = None

    <ipython-input-7-bcc5cba3de3f> in <module>()
          9
         10 try:
    ---> 11     x()
         12 except:
         13     x = sys.exc_info()

Also note that this uses six, which is not formally a dependency yet,
but is included indirectly since most packages depend on it.

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-06 23:06:24 +02:00
Erik Johnston 92c52df702 Make store_file use store_into_file 2018-02-14 17:55:18 +00:00
Erik Johnston 5fa571a91b Tell storage providers about new file so they can upload 2018-02-07 13:35:08 +00:00
Erik Johnston 1f881e0746
Merge pull request #2791 from matrix-org/erikj/media_storage_refactor
Ensure media is in local cache before thumbnailing
2018-02-05 11:28:52 +00:00
Richard van der Hoff ad7ec63d08 Use the right path for url_preview thumbnails
This was introduced by #2627: we were overwriting the original media for url
previews with the thumbnails :/

(fixes https://github.com/vector-im/riot-web/issues/6012, hopefully)
2018-01-19 18:29:39 +00:00
Erik Johnston 2cf6a7bc20 Use better file consumer 2018-01-18 12:00:46 +00:00
Erik Johnston 4a53f3a3e8 Ensure media is in local cache before thumbnailing 2018-01-18 12:00:46 +00:00
Erik Johnston e21370ba54 Correctly reraise exception 2018-01-12 14:44:02 +00:00
Erik Johnston 85a4d78213 Make Responder a context manager 2018-01-12 13:32:03 +00:00
Erik Johnston 227c491510 Comments 2018-01-12 11:22:41 +00:00
Erik Johnston 8f03aa9f61 Add StorageProvider concept 2018-01-09 16:16:12 +00:00
Erik Johnston ada470bccb Add MediaStorage class 2018-01-09 16:15:07 +00:00