Commit Graph

57 Commits (c00100b607ab97dfea690880434657b7ee11466d)

Author SHA1 Message Date
kontrollanten f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz 213e30ef90 Add banner tests 2021-04-08 10:07:53 +02:00
Chocobozzz 2cb03dc1f4 Add banners support 2021-04-08 10:07:53 +02:00
Chocobozzz f43db2f46e
Refactor auth flow
Reimplement some node-oauth2-server methods to remove hacky code needed by our external
login workflow
2021-03-24 18:18:41 +01:00
Chocobozzz fbd51e69f2
Upgrade server dependencies 2021-03-03 16:14:32 +01:00
Chocobozzz c6c0fa6cd8 Live streaming implementation first step 2020-11-09 15:33:04 +01:00
Chocobozzz 668b7f096e Add openapi doc 2020-07-31 11:35:19 +02:00
Chocobozzz 94148c9028 Add abuse messages management in my account 2020-07-31 11:35:19 +02:00
Chocobozzz edbc932546 Add server API to abuse messages 2020-07-31 11:35:19 +02:00
Chocobozzz 310b5219b3 Add new abuses tests 2020-07-10 14:02:41 +02:00
Chocobozzz 57f6896f67 Implement abuses check params 2020-07-10 14:02:41 +02:00
Chocobozzz d95d155988 Use 3 tables to represent abuses 2020-07-10 14:02:41 +02:00
Chocobozzz 67ed6552b8 Reorganize client shared modules 2020-06-23 16:00:49 +02:00
Chocobozzz c24aac6bc7
Fix tests 2020-06-18 13:39:13 +02:00
Chocobozzz 26d6bf6533
Split types and typings 2020-06-18 10:46:27 +02:00
Chocobozzz faa9d434b4
Update server dependencies 2020-06-17 11:42:50 +02:00
Chocobozzz 80fdaf0645
Add moderation helpers to plugins 2020-05-07 15:07:01 +02:00
Chocobozzz a4995eb7ac Add ability to unregister plugin auths 2020-05-04 16:21:39 +02:00
Chocobozzz 5a7eecdd56 Add config helper for plugins 2020-05-04 16:21:39 +02:00
Chocobozzz 4a8d113b9b Begin support for external auths 2020-05-04 16:21:39 +02:00
Chocobozzz e307e4fce3 Add ability for auth plugins to hook tokens validity 2020-05-04 16:21:39 +02:00
Chocobozzz e1c5503114 Support logout and add id and pass tests 2020-05-04 16:21:39 +02:00
Chocobozzz 7fed637506 Begin auth plugin support 2020-05-04 16:21:39 +02:00
Rigel Kent d1261d9a3c Add links from user-edit 2020-05-01 16:41:02 +02:00
Rigel Kent 86521a67b2 Add video channel and video thumbnail, rework video appearance in row 2020-05-01 16:41:02 +02:00
Rigel Kent 68d19a0ace Make sure a report doesn't get deleted upon the deletion of its video 2020-05-01 16:41:02 +02:00
Chocobozzz b3af2601da
Add ability to remove privacies using plugins 2020-04-20 15:42:27 +02:00
Chocobozzz 5e2b2e2775
Add ability for plugins to add custom routes 2020-04-10 15:23:25 +02:00
Chocobozzz ab3ead3a6f
Add ability to remove a video from a plugin 2020-04-09 11:37:44 +02:00
Chocobozzz 1b05d82d86
Add SQL query support in plugins 2020-04-09 11:00:30 +02:00
Chocobozzz bc0d801bb7
Refactor plugin helpers factory 2020-04-09 09:57:32 +02:00
Chocobozzz 943e519390
Don't refresh videos when processing views
It allows us to use a cache
2020-02-04 15:58:59 +01:00
Chocobozzz 7eba5e1fa8
Add model cache for video
When fetching only immutable attributes
2020-02-04 15:00:47 +01:00
Chocobozzz a15871560f
Move to eslint 2020-02-03 08:31:02 +01:00
Chocobozzz ca6d36227a
Add url field in caption and use it for thumbnails 2020-01-30 11:53:38 +01:00
Chocobozzz 0374b6b5cd
Cleanup server fixme 2020-01-28 14:45:17 +01:00
Chocobozzz b764380ac2 Add ability to list redundancies 2020-01-28 11:35:26 +01:00
Chocobozzz ac0868bcc0
Improve SQL query for my special playlists 2020-01-03 14:17:57 +01:00
Chocobozzz d7a25329f9
Add ability to disable webtorrent
In favour of HLS
2019-11-25 10:59:43 +01:00
Chocobozzz 47581df073
Fix federation with some actors
That don't have a shared inbox, or a URL
2019-10-23 11:33:53 +02:00
Chocobozzz 10a105f0c8 Add notification on auto follow index 2019-09-04 16:24:58 +02:00
Chocobozzz 8424c4026a Add auto follow back support for instances 2019-09-04 16:24:58 +02:00
Chocobozzz d636ab58d0
Fix html tag with blacklisted video 2019-08-22 10:46:54 +02:00
Chocobozzz b5fecbf441
Type toActivityPubObject functions 2019-08-21 15:23:04 +02:00
Chocobozzz f92e7f76d4
Fix ownership change 2019-08-21 08:57:00 +02:00
Chocobozzz 1ca9f7c3f7
Type toFormattedJSON 2019-08-20 19:23:10 +02:00
Chocobozzz 0283eaac2a
Cleanup model types 2019-08-20 14:53:42 +02:00
Chocobozzz 453e83ea5d
Stronger model typings 2019-08-19 17:26:35 +02:00
Chocobozzz 5224c394b3
Stronger actor association typing in AP functions 2019-08-09 08:17:16 +02:00
Chocobozzz 1198edf4bb
Fix user notifications on new follow 2019-08-02 10:53:36 +02:00