Commit Graph

655 Commits (1fa23d6f5e487f3c149e1f0001beadd919ee82fc)

Author SHA1 Message Date
kontrollanten f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz e024fd6a74
Update channel updatedAt when uploading a video 2021-05-10 09:38:11 +02:00
Chocobozzz 1ff9f1cda3
Fix ffmpeg version parsing 2021-05-07 14:23:43 +02:00
Chocobozzz a66c2e3252
Fix remote actor creation date 2021-05-07 09:00:09 +02:00
Chocobozzz 60f1f61579
Fix ffmpeg version checker 2021-05-06 16:39:17 +02:00
Chocobozzz 1fd61899ea
Add ability to filter my videos by live 2021-05-03 16:42:15 +02:00
Chocobozzz c9ae74d6bc
Try to fix bad timestamps in .srt 2021-04-27 09:00:16 +02:00
Chocobozzz 08370f6244
Fix h265 video import using CLI 2021-04-21 15:18:22 +02:00
Chocobozzz 46460547ca
Fix exception in markdown parser 2021-04-16 07:41:35 +02:00
Rigel Kent a073c91270 modify tests to support current behaviour regarding plaintext description 2021-04-13 15:45:09 +02:00
Rigel Kent 84bced652c render markdown and plainify descriptions on previews 2021-04-13 15:45:09 +02:00
Chocobozzz a60696ab18 Add scale filter to documentation 2021-04-09 15:53:18 +02:00
Chocobozzz 3e03b961b8 Add ability for plugins to specify scale filter 2021-04-09 15:53:18 +02:00
Théo Le Calvar 43f7a43ca4 add option for transcode plugins to add video filters and make all options optional 2021-04-09 15:53:18 +02:00
Théo Le Calvar 5fb7cfbac5 add support for inputOptions in trancode plugins 2021-04-09 15:53:18 +02:00
Chocobozzz 22820226e5
Add server plugin helpers 2021-04-09 15:17:43 +02:00
Chocobozzz dc48fdbe68
Merge branch 'release/3.1.0' into develop 2021-04-08 15:16:01 +02:00
Chocobozzz 4c7449d2ad
Fix subtitles import 2021-04-08 14:29:03 +02:00
Chocobozzz 84531547bc
Add size info in db for actor images 2021-04-08 13:38:04 +02:00
Chocobozzz 20373985c0
Fix static ffmpeg version check 2021-04-08 10:08:11 +02:00
Chocobozzz 213e30ef90 Add banner tests 2021-04-08 10:07:53 +02:00
Chocobozzz 2cb03dc1f4 Add banners support 2021-04-08 10:07:53 +02:00
Thavarasa Prasanth 4097c6d66c
fix missing title attribute on <iframe> tag suggested for embedding (#3901)
* title attribute is missing on <iframe> tag suggested for embedding #3861

* fix #3901

* fix: escapeHTML #3901

* fix: playlist title instead of video title #3901

* fix #3901

* assign title directly #3901
2021-03-31 08:32:05 +02:00
Chocobozzz 32a18cbf33
Add new plugin/peertube version notifs 2021-03-24 18:18:41 +01:00
Chocobozzz 829523cfa5
Reduce bundle size using a const enum 2021-03-24 18:18:40 +01:00
Chocobozzz ae71acca14
Check ffmepg version on startup 2021-03-24 18:18:40 +01:00
Chocobozzz e7053b1d9d
Fix AP security tests 2021-03-24 18:18:40 +01:00
Chocobozzz 43b513660b
Fix AP flag validator 2021-03-24 18:18:40 +01:00
Chocobozzz b5c361089f
Fix 404 AP status codes 2021-03-24 18:18:40 +01:00
Chocobozzz b329abc2f0
Fix request body limit 2021-03-24 18:18:40 +01:00
Chocobozzz db4b15f21f
Use got instead of request 2021-03-24 18:18:40 +01:00
Chocobozzz 67f87b66fe
Refactor a little bit AP validators 2021-03-24 18:18:40 +01:00
Chocobozzz 452b3bea08
Introduce tags to the logger
That could help to filter unneeded logs and/or select particular
components
2021-03-24 18:18:40 +01:00
Chocobozzz cd2c3dcdc4
Remove unnecessary transcoding job error 2021-03-03 10:23:05 +01:00
Chocobozzz 74d249bc13
Add ability to cleanup remote AP interactions 2021-03-03 10:08:08 +01:00
Chocobozzz 8efc27bf14 Cleanup 2021-02-18 13:38:09 +01:00
Chocobozzz d9a2a03196 Don't guess remote tracker URL 2021-02-18 13:38:09 +01:00
Chocobozzz 90a8bd305d Dissociate video file names and video uuid 2021-02-18 13:38:09 +01:00
Chocobozzz 1664bc60eb Optimize remote image processing 2021-02-16 10:36:44 +01:00
Chocobozzz 6302d599cd Generate a name for caption files 2021-02-16 10:36:44 +01:00
Chocobozzz ba5a8d89bb
Update server dependencies 2021-02-03 09:45:08 +01:00
Chocobozzz d8ba492119
Fix broken HLS playback with some audio streams 2021-02-02 11:19:46 +01:00
Chocobozzz 789951460b
Try to support other codecs 2021-01-29 14:05:41 +01:00
Chocobozzz ffd970faf9
Don't quick transcode if using a custom profile 2021-01-29 14:05:41 +01:00
Chocobozzz 1896bca09e
Support transcoding options/encoders by plugins 2021-01-28 15:55:39 +01:00
Chocobozzz 529b37527c
Use a profile manager for transcoding 2021-01-28 15:55:39 +01:00
Rigel Kent ba5d4a849c move from trending routes to alg param 2021-01-28 15:55:34 +01:00
Rigel Kent 3da68f0a78 add default trending page choice, revert comments count for hot strategy 2021-01-28 15:55:34 +01:00
Chocobozzz a4a8cd3971
Add missing niceness to ffmpeg thumbnail processes 2021-01-27 09:09:59 +01:00
Chocobozzz d223dca0cd
Reformat sql console output 2021-01-26 10:03:41 +01:00