OpenCloud
/
PeerTube
mirror of https://github.com/Chocobozzz/PeerTube
1
0
Fork 0
Code Issues Releases Wiki Activity
8,730 Commits
4 Branches
139 Tags
523 MiB
Commit Graph

757 Commits (81d02aac56d76227c094d40b17b29875d88417f4)

Author SHA1 Message Date
Chocobozzz 1bcb03a100
Use a class for youtube-dl 2021-05-11 11:32:31 +02:00
kontrollanten f6d6e7f861
Resumable video uploads (#3933) 
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
 2021-05-10 11:13:41 +02:00
Chocobozzz e024fd6a74
Update channel updatedAt when uploading a video 2021-05-10 09:38:11 +02:00
Chocobozzz 1ff9f1cda3
Fix ffmpeg version parsing 2021-05-07 14:23:43 +02:00
Chocobozzz a66c2e3252
Fix remote actor creation date 2021-05-07 09:00:09 +02:00
Chocobozzz 60f1f61579
Fix ffmpeg version checker 2021-05-06 16:39:17 +02:00
Kimsible 1e37d32f4b Add server API actors route 2021-05-05 11:47:03 +02:00
Chocobozzz 1fd61899ea
Add ability to filter my videos by live 2021-05-03 16:42:15 +02:00
Chocobozzz c9ae74d6bc
Try to fix bad timestamps in .srt 2021-04-27 09:00:16 +02:00
Chocobozzz 08370f6244
Fix h265 video import using CLI 2021-04-21 15:18:22 +02:00
Chocobozzz 46460547ca
Fix exception in markdown parser 2021-04-16 07:41:35 +02:00
Rigel Kent a073c91270 modify tests to support current behaviour regarding plaintext description 2021-04-13 15:45:09 +02:00
Rigel Kent 84bced652c render markdown and plainify descriptions on previews 2021-04-13 15:45:09 +02:00
Chocobozzz a60696ab18 Add scale filter to documentation 2021-04-09 15:53:18 +02:00
Chocobozzz 3e03b961b8 Add ability for plugins to specify scale filter 2021-04-09 15:53:18 +02:00
Théo Le Calvar 43f7a43ca4 add option for transcode plugins to add video filters and make all options optional 2021-04-09 15:53:18 +02:00
Théo Le Calvar 5fb7cfbac5 add support for inputOptions in trancode plugins 2021-04-09 15:53:18 +02:00
Chocobozzz 22820226e5
Add server plugin helpers 2021-04-09 15:17:43 +02:00
Chocobozzz dc48fdbe68
Merge branch 'release/3.1.0' into develop 2021-04-08 15:16:01 +02:00
Chocobozzz 4c7449d2ad
Fix subtitles import 2021-04-08 14:29:03 +02:00
Chocobozzz 84531547bc
Add size info in db for actor images 2021-04-08 13:38:04 +02:00
Chocobozzz 20373985c0
Fix static ffmpeg version check 2021-04-08 10:08:11 +02:00
Chocobozzz 213e30ef90 Add banner tests 2021-04-08 10:07:53 +02:00
Chocobozzz 2cb03dc1f4 Add banners support 2021-04-08 10:07:53 +02:00
Thavarasa Prasanth 4097c6d66c
fix missing title attribute on <iframe> tag suggested for embedding (#3901) 
* title attribute is missing on <iframe> tag suggested for embedding #3861

* fix #3901

* fix: escapeHTML #3901

* fix: playlist title instead of video title #3901

* fix #3901

* assign title directly #3901
 2021-03-31 08:32:05 +02:00
Chocobozzz 32a18cbf33
Add new plugin/peertube version notifs 2021-03-24 18:18:41 +01:00
Chocobozzz 829523cfa5
Reduce bundle size using a const enum 2021-03-24 18:18:40 +01:00
Chocobozzz ae71acca14
Check ffmepg version on startup 2021-03-24 18:18:40 +01:00
Chocobozzz e7053b1d9d
Fix AP security tests 2021-03-24 18:18:40 +01:00
Chocobozzz 43b513660b
Fix AP flag validator 2021-03-24 18:18:40 +01:00
Chocobozzz b5c361089f
Fix 404 AP status codes 2021-03-24 18:18:40 +01:00
Chocobozzz b329abc2f0
Fix request body limit 2021-03-24 18:18:40 +01:00
Chocobozzz db4b15f21f
Use got instead of request 2021-03-24 18:18:40 +01:00
Chocobozzz 67f87b66fe
Refactor a little bit AP validators 2021-03-24 18:18:40 +01:00
Chocobozzz 452b3bea08
Introduce tags to the logger 
That could help to filter unneeded logs and/or select particular
components
 2021-03-24 18:18:40 +01:00
Chocobozzz cd2c3dcdc4
Remove unnecessary transcoding job error 2021-03-03 10:23:05 +01:00
Chocobozzz 74d249bc13
Add ability to cleanup remote AP interactions 2021-03-03 10:08:08 +01:00
Chocobozzz 8efc27bf14 Cleanup 2021-02-18 13:38:09 +01:00
Chocobozzz d9a2a03196 Don't guess remote tracker URL 2021-02-18 13:38:09 +01:00
Chocobozzz 90a8bd305d Dissociate video file names and video uuid 2021-02-18 13:38:09 +01:00
Chocobozzz 1664bc60eb Optimize remote image processing 2021-02-16 10:36:44 +01:00
Chocobozzz 6302d599cd Generate a name for caption files 2021-02-16 10:36:44 +01:00
Chocobozzz ba5a8d89bb
Update server dependencies 2021-02-03 09:45:08 +01:00
Chocobozzz d8ba492119
Fix broken HLS playback with some audio streams 2021-02-02 11:19:46 +01:00
Chocobozzz 789951460b
Try to support other codecs 2021-01-29 14:05:41 +01:00
Chocobozzz ffd970faf9
Don't quick transcode if using a custom profile 2021-01-29 14:05:41 +01:00
Chocobozzz 1896bca09e
Support transcoding options/encoders by plugins 2021-01-28 15:55:39 +01:00
Chocobozzz 529b37527c
Use a profile manager for transcoding 2021-01-28 15:55:39 +01:00
Rigel Kent ba5d4a849c move from trending routes to alg param 2021-01-28 15:55:34 +01:00
Rigel Kent 3da68f0a78 add default trending page choice, revert comments count for hot strategy 2021-01-28 15:55:34 +01:00
Chocobozzz a4a8cd3971
Add missing niceness to ffmpeg thumbnail processes 2021-01-27 09:09:59 +01:00
Chocobozzz d223dca0cd
Reformat sql console output 2021-01-26 10:03:41 +01:00
Chocobozzz 1e743faafe
Disable sql prettifier by default 
It adds too much lines, leading to difficulties when reading dev logs
 2021-01-26 09:54:32 +01:00
Rigel Kent 2a6cf69cff
prettify SQL queries during debug (#3635) 
* prettify SQL queries during debug

* Use sql-formatter

Co-authored-by: Chocobozzz <me@florianbigard.com>
 2021-01-25 15:37:41 +01:00
Chocobozzz 77d7e851dc Add priority to transcoding jobs 
(1 = highest priority)

100 for new resolutions
10 for original file optimization

Add a malus for transcoding jobs depending on how many uploads the user did in the
last 7 days
 2021-01-25 14:38:52 +01:00
Chocobozzz 24516aa26a
Refactor transcoding job handlers 2021-01-21 15:58:17 +01:00
Chocobozzz 3b01f4c0ac
Support progress for ffmpeg tasks 2021-01-21 14:42:43 +01:00
Chocobozzz 86f553fbdf
Fix raw file import 2021-01-20 08:45:21 +01:00
Chocobozzz 805b8619c1
Fix (again) youtube import 2021-01-19 16:36:21 +01:00
Chocobozzz 969e59d17d
Fix youtube import 2021-01-18 14:29:16 +01:00
Chocobozzz a8a63b1ffe
Fix youtube-dl format fallback 2021-01-18 11:40:31 +01:00
Rigel Kent 454c20fa7c
stricter youtubedl format selectors (#3516) 
* stricter youtubedl format selectors

make sure selectors avoid av1, and otherwise match as closely to the
maximum resolution enabled for transcoding

* add support for merge formats in youtubedl

* avoid vp9.2 in youtubedl to avoid any HDR

* move getEnabledResolutions, safer replace of imported extension

* add test for youtube-dl selectors
 2021-01-15 15:56:56 +01:00
Chocobozzz dd9c7929e6
Display debug ffmpeg stdout/stderr 2021-01-15 10:23:40 +01:00
Rigel Kent 7abb6060a9 specific niceness for live transcoding 2021-01-13 09:16:44 +01:00
Rigel Kent b7085c7132 add support for 1440p (Quad HD/QHD/WQHD) videos 2021-01-13 09:14:09 +01:00
Rigel Kent 0984960345 add script printing command to generate a resolution for a given file 2021-01-13 09:13:31 +01:00
Chocobozzz b3fa96eea0
Don't reject accounts with empty description 2021-01-11 11:06:11 +01:00
Chocobozzz 027e308099
Don't crash on youtube-dl update write error 2021-01-04 10:05:47 +01:00
Chocobozzz 402145b863
Refactor jobs state 2020-12-14 12:00:35 +01:00
Rigel Kent 040d6896a3
add display of logs matching any state 2020-12-13 19:27:25 +01:00
Chocobozzz b49f22d8f9 Upgrade sequelize to v6 2020-12-09 11:41:22 +01:00
Rigel Kent f2eb23cd87
emit more specific status codes on video upload (#3423) 
- reduce http status codes list to potentially useful codes
- convert more codes to typed ones
- factorize html generator for error responses
 2020-12-08 21:16:10 +01:00
Rigel Kent 2d53be0267
replace numbers with typed http status codes (#3409) 2020-12-07 14:32:36 +01:00
Chocobozzz e772bdf14c
Load video in permanent live after last one ended 2020-12-04 15:38:32 +01:00
Chocobozzz 3851e732c4
Fix audio issues with live replay 2020-12-04 15:10:13 +01:00
Chocobozzz 49bcdb0d66
Try to fix live freeze 2020-12-04 10:40:53 +01:00
Chocobozzz 55223d6574
Fix live transcoding threads config 2020-12-04 10:02:09 +01:00
Chocobozzz bb4ba6d94c
Add permanent live support 2020-12-03 15:21:16 +01:00
Chocobozzz 4a54a93941
Fix live invalid save replay duration 2020-12-03 10:05:10 +01:00
Chocobozzz 2650d6d489
Fix live replay duration glitch 2020-12-02 10:18:15 +01:00
Chocobozzz 937581b8f6
Fix high CPU with long live when save replay is true 2020-11-30 15:59:22 +01:00
Chocobozzz ce4a50b99b
Fix "Too many packets buffered for output stream" 
And move encoder global options in a global wrapper
 2020-11-26 14:45:48 +01:00
Chocobozzz 884d2c39ae
Fix live FPS limit 2020-11-26 11:30:18 +01:00
Chocobozzz a9d4c3c8ca
Avoid too large cert error messages 2020-11-25 14:43:43 +01:00
Chocobozzz 18490b0765 Fix migration and test 2020-11-25 11:07:56 +01:00
Rigel Kent 5beb89f223 refactor scoped token service 2020-11-25 11:07:56 +01:00
Rigel Kent afff310e50 allow private syndication feeds via a user feedToken 2020-11-25 11:07:56 +01:00
Chocobozzz f619de0e43 Add tests to gif resizer 2020-11-25 10:07:51 +01:00
Chocobozzz 6b67897e2e Add transcoding module comments 2020-11-25 10:07:51 +01:00
Chocobozzz 33ff70baa6 Fix audio encoding params 2020-11-25 10:07:51 +01:00
Chocobozzz 5a547f69d5 Support encoding profiles 2020-11-25 10:07:51 +01:00
Chocobozzz 9252a33d11 Export encoders options in a dedicated struct 2020-11-25 10:07:51 +01:00
Chocobozzz daf6e48010 Split ffmpeg utils with ffprobe utils 2020-11-25 10:07:51 +01:00
Kimsible 123f619336
Add pixel size to tooltip and gif support with FFmpeg for avatar upload (#3329) 
* Add avatar pixel size upload in tooltip

* Add gif support for avatar

* Add ffmpeg GIF process

Co-authored-by: kimsible <kimsible@users.noreply.github.com>
 2020-11-25 09:26:31 +01:00
Chocobozzz d218e7de94
Do not transcode to an higher bitrate 
Thanks bkil https://github.com/bkil
 2020-11-20 15:11:31 +01:00
Chocobozzz 0aa52e1707
Add ability to display all channel/account videos 2020-11-18 15:29:38 +01:00
Chocobozzz e0409585a7
Fix youtube-dl ipv4 PR 2020-11-18 14:11:15 +01:00
Mark Van den Borre 677012b4ee
default youtube-dl to use ipv4 (#3311) 
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
 2020-11-18 14:06:20 +01:00
Chocobozzz 2122606318
Increase live segment file name 2020-11-12 16:56:37 +01:00
Chocobozzz 797d05bdd9
Force signed headers in http signatures 
Thanks Roger
 2020-11-12 16:29:32 +01:00
Chocobozzz 7a4ea93246
Remove deprecated abuse api 2020-11-10 14:50:16 +01:00
Chocobozzz 9e2b2e76ba Don't add live videos in redundancy 2020-11-09 15:33:04 +01:00
Chocobozzz 053aed43fb Regenerate miniature on live save 2020-11-09 15:33:04 +01:00
Chocobozzz 3bc68dfd61 Fix audio sync after saving replay 
hls.js seems to not correctly handle audio gaps with fragmented mp4
(but can with a ts playlist)
 2020-11-09 15:33:04 +01:00
Chocobozzz 68e70a745b Add save replay live tests 2020-11-09 15:33:04 +01:00
Chocobozzz af4ae64f6f Begin live tests 2020-11-09 15:33:04 +01:00
Chocobozzz 77e9f859c6 Add check params live tests 2020-11-09 15:33:04 +01:00
Chocobozzz d988e9a2f8 Fix live infohash block 2020-11-09 15:33:04 +01:00
Chocobozzz 31c82cd914 Fix replay saving 2020-11-09 15:33:04 +01:00
Chocobozzz b5b687550d Add ability to save live replay 2020-11-09 15:33:04 +01:00
Chocobozzz fb7194043d Check live duration and size 2020-11-09 15:33:04 +01:00
Chocobozzz de6310b2fc Handle live federation 2020-11-09 15:33:04 +01:00
Chocobozzz c6c0fa6cd8 Live streaming implementation first step 2020-11-09 15:33:04 +01:00
Rigel Kent 44fb529740
add node-youtube-dl download host env var in auto-updater 2020-10-24 12:42:06 +02:00
Girish Ramakrishnan 313921b50f Fix transcoding errors in readonly docker containers 
ffmpeg seems to create some temporary files in the cwd. When PeerTube
is run in a read-only docker container, this causes all transcoding
to fail. As a workaround, we set the cwd to the configured tmp dir.
 2020-10-23 14:51:44 +02:00
Chocobozzz 30bc55c88b Refactor video extensions logic in server 2020-08-11 07:57:20 +02:00
Chocobozzz bd45d503e5 Reorganize shared models 2020-08-07 08:28:14 +02:00
Chocobozzz dca0fe12ec
Fix resolution for portrait videos 2020-08-03 16:03:52 +02:00
Chocobozzz 94148c9028 Add abuse messages management in my account 2020-07-31 11:35:19 +02:00
Chocobozzz edbc932546 Add server API to abuse messages 2020-07-31 11:35:19 +02:00
Chocobozzz 17aa80ed01
Merge branch 'release/2.3.0' into develop 2020-07-10 15:23:31 +02:00
Chocobozzz 1878224214
Handle webp images from youtube-dl 2020-07-10 14:54:11 +02:00
Chocobozzz 310b5219b3 Add new abuses tests 2020-07-10 14:02:41 +02:00
Chocobozzz 57f6896f67 Implement abuses check params 2020-07-10 14:02:41 +02:00
Chocobozzz d95d155988 Use 3 tables to represent abuses 2020-07-10 14:02:41 +02:00
Rigel Kent 1ebddadd07
predefined report reasons & improved reporter UI (#2842) 
- added `startAt` and `endAt` optional timestamps to help pin down reported sections of a video
- added predefined report reasons
- added video player with report modal
 2020-06-22 13:00:39 +02:00
Chocobozzz 26d6bf6533
Split types and typings 2020-06-18 10:46:27 +02:00
Chocobozzz 982f2fc9b4
Correctly fix subtitles import 2020-06-17 09:42:00 +02:00
Chocobozzz c7763eddd1
Fix getSubs import 2020-06-16 11:26:46 +02:00
Chocobozzz 3b0bd70aa0 Add search target check params 2020-06-10 14:02:41 +02:00
Rigel Kent e6dfa58689 space optimizations for `node_modules` and client stats removal 
- replace sharp with lighter jimp alternative
- remove stats in builds

fixes #2807
 2020-06-10 14:01:49 +02:00
Levi Bard 3092e9bbb0
Make federation of unlisted videos an instance-level server preference (#2802) 
* Add preference for federating unlisted videos

* Connect unlisted video federation with new preference

* Apply pull request feedback

* Fix lint issues

* Remove preference for federating unlisted videos from web admin interface
 2020-06-03 09:42:07 +02:00
Chocobozzz 444c0a0e01 Add ability to bulk delete comments 2020-05-29 09:21:26 +02:00
Chocobozzz 9934b6f31e
Add missing playlist and playlistElement types in AP context 2020-05-13 10:53:18 +02:00
Chocobozzz feb34f6b6b Use video abuse filters on client side 2020-05-07 08:33:34 +02:00
Chocobozzz 1c32067367
Fix HLS audio only transcoding 2020-05-05 16:27:46 +02:00
Chocobozzz 5644f9b035 Accept numbers in plugin names 2020-05-04 16:21:39 +02:00
Chocobozzz e307e4fce3 Add ability for auth plugins to hook tokens validity 2020-05-04 16:21:39 +02:00
Chocobozzz 8dc8a34ee8 Avoir some circular dependencies 2020-05-04 16:21:39 +02:00
Rigel Kent 0251197e24 Factorize rest-table and fix/simplify SQL 2020-05-01 16:41:02 +02:00
Rigel Kent 68d19a0ace Make sure a report doesn't get deleted upon the deletion of its video 2020-05-01 16:41:02 +02:00
Chocobozzz 86ad0cde9e
Add licence and language support to y-dl imports 2020-04-20 10:41:43 +02:00
Chocobozzz 652c64165b
Fix import captions test 2020-04-15 14:15:44 +02:00
kimsible 50ad0a1c16 Add getSubs to YoutubeDL video import 2020-04-14 14:39:30 +02:00
Chocobozzz 1b05d82d86
Add SQL query support in plugins 2020-04-09 11:00:30 +02:00
Chocobozzz bc0d801bb7
Refactor plugin helpers factory 2020-04-09 09:57:32 +02:00
Chocobozzz edaf5b862a
Fix torrent import on windows 2020-04-07 10:40:52 +02:00
Chocobozzz d57d1d83c6
Support audio files import 2020-04-03 15:41:39 +02:00
Chocobozzz a2b6ec7c3f
Fix video codec in HLS playlist 2020-04-03 14:07:00 +02:00
Chocobozzz 7b81edc854
Video file metadata PR cleanup 2020-03-10 16:18:29 +01:00