Commit Graph

505 Commits (b111cd8c288c76b7d2216e14f4bf95248ea5af14)

Author SHA1 Message Date
kontrollanten f6d6e7f861
Resumable video uploads (#3933)
* WIP: resumable video uploads

relates to #324

* fix review comments

* video upload: error handling

* fix audio upload

* fixes after self review

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/middlewares/validators/videos/videos.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* Update server/controllers/api/videos/index.ts

Co-authored-by: Rigel Kent <par@rigelk.eu>

* update after code review

* refactor upload route

- restore multipart upload route
- move resumable to dedicated upload-resumable route
- move checks to middleware
- do not leak internal fs structure in response

* fix yarn.lock upon rebase

* factorize addVideo for reuse in both endpoints

* add resumable upload API to openapi spec

* add initial test and test helper for resumable upload

* typings for videoAddResumable middleware

* avoid including aws and google packages via node-uploadx, by only including uploadx/core

* rename ex-isAudioBg to more explicit name mentioning it is a preview file for audio

* add video-upload-tmp-folder-cleaner job

* stronger typing of video upload middleware

* reduce dependency to @uploadx/core

* add audio upload test

* refactor resumable uploads cleanup from job to scheduler

* refactor resumable uploads scheduler to compare to last execution time

* make resumable upload validator to always cleanup on failure

* move legacy upload request building outside of uploadVideo test helper

* filter upload-resumable middlewares down to POST, PUT, DELETE

also begin to type metadata

* merge add duration functions

* stronger typings and documentation for uploadx behaviour, move init validator up

* refactor(client/video-edit): options > uploadxOptions

* refactor(client/video-edit): remove obsolete else

* scheduler/remove-dangling-resum: rename tag

* refactor(server/video): add UploadVideoFiles type

* refactor(mw/validators): restructure eslint disable

* refactor(mw/validators/videos): rename import

* refactor(client/vid-upload): rename html elem id

* refactor(sched/remove-dangl): move fn to method

* refactor(mw/async): add method typing

* refactor(mw/vali/video): double quote > single

* refactor(server/upload-resum): express use > all

* proper http methud enum server/middlewares/async.ts

* properly type http methods

* factorize common video upload validation steps

* add check for maximum partially uploaded file size

* fix audioBg use

* fix extname(filename) in addVideo

* document parameters for uploadx's resumable protocol

* clear META files in scheduler

* last audio refactor before cramming preview in the initial POST form data

* refactor as mulitpart/form-data initial post request

this allows preview/thumbnail uploads alongside the initial request,
and cleans up the upload form

* Add more tests for resumable uploads

* Refactor remove dangling resumable uploads

* Prepare changelog

* Add more resumable upload tests

* Remove user quota check for resumable uploads

* Fix upload error handler

* Update nginx template for upload-resumable

* Cleanup comment

* Remove unused express methods

* Prefer to use got instead of raw http

* Don't retry on error 500

Co-authored-by: Rigel Kent <par@rigelk.eu>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-05-10 11:13:41 +02:00
Chocobozzz d29ced1a85
Move engines outside package.json
Because of invalid warnings displayed by yarn that could confuse admins
when they install peertube
2021-05-10 09:55:46 +02:00
Chocobozzz 6b6e1d5d02
Drop NodeJS 10 support 2021-05-10 09:41:21 +02:00
Chocobozzz c9783c7b72
Update server dependencies 2021-05-05 11:08:02 +02:00
Chocobozzz f463e2dfc0
Use bash for scripts 2021-05-05 10:56:49 +02:00
Chocobozzz a0e69167ae
Remove scripty
It breaks our tests with the latest npm release
2021-05-05 10:51:32 +02:00
Chocobozzz 931d343018
Move to stylelint 2021-04-28 17:53:57 +02:00
Chocobozzz 5182473faf
Upgrade server dep' 2021-04-12 16:35:04 +02:00
John Livingston 09d535ef98
Fix #3940: unload all children from the plugin module on updates. 2021-04-09 09:32:16 +02:00
Chocobozzz b70025bfdc
Remove libxmljs in favour of pure JS implem 2021-04-06 10:25:21 +02:00
Chocobozzz a786d8a08b
Merge branch 'release/3.1.0' into develop 2021-03-25 16:25:28 +01:00
Chocobozzz f6160062f8
Fix postgresql version 2021-03-25 16:25:14 +01:00
Chocobozzz c2bd7a6fcf
Add regenrate thumbnails scripts 2021-03-24 18:18:41 +01:00
Chocobozzz a784bd7fde
Upgrade to latest oauth2-server 2021-03-24 18:18:41 +01:00
Chocobozzz f43db2f46e
Refactor auth flow
Reimplement some node-oauth2-server methods to remove hacky code needed by our external
login workflow
2021-03-24 18:18:41 +01:00
Chocobozzz db4b15f21f
Use got instead of request 2021-03-24 18:18:40 +01:00
Chocobozzz d007175238
Bumped to version v3.1.0 2021-03-24 18:18:40 +01:00
Chocobozzz 29553847dc
Bumped to version v3.1.0 2021-03-24 17:47:59 +01:00
Chocobozzz 4468e1e4f7
Bumped to version v3.1.0-rc.1 2021-03-04 10:54:29 +01:00
Chocobozzz fbd51e69f2
Upgrade server dependencies 2021-03-03 16:14:32 +01:00
Chocobozzz 4abbeff575
Add benchmark script 2021-02-22 15:54:15 +01:00
Chocobozzz fd0c11551c
Remove es5 module support
Avoid using differential loading. Break compat with IE11, Opera Mini, UC
Browser for Android

See https://caniuse.com/?search=es6-module for more information
2021-02-11 11:23:19 +01:00
Chocobozzz 9162476fe9
Support NodeJS 14 and 15, deprecate Node 10 2021-02-04 14:31:18 +01:00
Chocobozzz ba5a8d89bb
Update server dependencies 2021-02-03 09:45:08 +01:00
Rigel Kent 2a6cf69cff
prettify SQL queries during debug (#3635)
* prettify SQL queries during debug

* Use sql-formatter

Co-authored-by: Chocobozzz <me@florianbigard.com>
2021-01-25 15:37:41 +01:00
Chocobozzz 51de2c7ff3
Update project description 2021-01-14 09:43:06 +01:00
Rigel Kent 0984960345 add script printing command to generate a resolution for a given file 2021-01-13 09:13:31 +01:00
Chocobozzz 69e0e678be
Bumped to version v3.0.1 2021-01-12 08:25:29 +01:00
Chocobozzz e33f385dd5
Bumped to version v3.0.0 2021-01-07 08:26:11 +01:00
Chocobozzz 505816fa10
Bumped to version v3.0.0-rc.1 2020-12-16 09:46:27 +01:00
Chocobozzz b49f22d8f9 Upgrade sequelize to v6 2020-12-09 11:41:22 +01:00
Chocobozzz fce7fe04ee
Update server dependencies 2020-11-19 09:07:20 +01:00
Chocobozzz 374e419738
Update my email 2020-11-12 16:29:32 +01:00
Rigel Kent 867ed6514d
move dependencies versions list to engines in package.json 2020-11-10 15:02:26 +01:00
Chocobozzz c2ecfe82b7
Fix prod deploy 2020-11-10 10:09:11 +01:00
Chocobozzz bd54ad1953 Add live notification tests 2020-11-09 15:33:04 +01:00
Chocobozzz c6c0fa6cd8 Live streaming implementation first step 2020-11-09 15:33:04 +01:00
Kimsible 98b9464312
render html/markdown for new comment notification email (#3255)
Co-authored-by: kimsible <kimsible@users.noreply.github.com>
Co-authored-by: Rigel Kent <sendmemail@rigelk.eu>
2020-11-07 22:59:58 +01:00
Girish Ramakrishnan 313921b50f Fix transcoding errors in readonly docker containers
ffmpeg seems to create some temporary files in the cwd. When PeerTube
is run in a read-only docker container, this causes all transcoding
to fail. As a workaround, we set the cwd to the configured tmp dir.
2020-10-23 14:51:44 +02:00
Chocobozzz b43d19cf44
Bumped to version v2.4.0 2020-09-08 11:12:05 +02:00
Chocobozzz 5520f6658e
Bumped to version v2.4.0-rc.1 2020-08-27 08:46:36 +02:00
Chocobozzz 50fcdebdb1
Update server dependencies 2020-08-25 13:54:59 +02:00
Chocobozzz 49a56c7203
Use update i18n command instead of generate 2020-08-24 09:35:27 +02:00
kimsible 0672dc769b Add unicode emoji to markdown 2020-08-14 15:03:38 +02:00
Chocobozzz a3b5e78af3 Analyze embed too with bundlewatch 2020-08-06 10:58:36 +02:00
Chocobozzz d632a14729 Test bundlewatch 2020-08-06 09:42:33 +02:00
Chocobozzz 97aaeadad5
Bumped to version v2.3.0 2020-07-21 10:03:58 +02:00
Chocobozzz 27647da17f
Bumped to version v2.3.0-rc.1 2020-07-06 11:09:12 +02:00
Rigel Kent 0c4bacbff5
provide specific engine boundaries for nodejs and yarn 2020-06-22 14:18:45 +02:00
Chocobozzz faa9d434b4
Update server dependencies 2020-06-17 11:42:50 +02:00
Rigel Kent e6dfa58689 space optimizations for `node_modules` and client stats removal
- replace sharp with lighter jimp alternative
- remove stats in builds

fixes #2807
2020-06-10 14:01:49 +02:00
Chocobozzz e0ef8dab28
Use tslib in server 2020-06-09 14:52:55 +02:00
Chocobozzz a55c98031b
Bumped to version v2.2.0 2020-06-02 10:04:05 +02:00
Chocobozzz d9cb522780
Bumped to version v2.2.0-rc.1 2020-05-18 08:13:34 +02:00
Rigel Kent df4c603dea Switch emails to pug templates and provide richer html/text-only versions 2020-05-08 15:31:51 +02:00
Chocobozzz d96994289b
Update server dependencies 2020-04-01 14:46:53 +02:00
Chocobozzz 84ebcf345c
Fix federation with pleroma 2020-03-12 14:14:11 +01:00
Chocobozzz 3637106d7a
Upgrade oauth server dependency 2020-02-28 16:32:46 +01:00
Chocobozzz bdd428a6d9
Update dependencies 2020-02-28 16:21:34 +01:00
Chocobozzz 93468b0889
Bumped to version v2.1.1 2020-02-28 13:45:42 +01:00
Average Dude 002df3813d OpenAPI clients generation 2020-02-14 14:14:14 +01:00
Chocobozzz 7024e9120b
Merge branch 'release/2.1.0' into develop 2020-02-11 09:44:59 +01:00
Chocobozzz 6b7e246ac7
Bumped to version v2.1.0 2020-02-11 08:35:39 +01:00
Chocobozzz 9a11f73392
Upgrade express-rate-limit 2020-02-04 10:52:05 +01:00
Chocobozzz a15871560f
Move to eslint 2020-02-03 08:31:02 +01:00
Chocobozzz a22046d166
Upgrade server dep 2020-02-03 08:31:02 +01:00
Chocobozzz 835053fc8b
Remove xliff package 2020-02-03 08:30:55 +01:00
Chocobozzz 005b51b977
Force node 10 usage 2020-01-28 14:03:12 +01:00
Chocobozzz b764380ac2 Add ability to list redundancies 2020-01-28 11:35:26 +01:00
Chocobozzz 5aafc6f373
Bumped to version v2.1.0-rc.1 2020-01-27 08:44:04 +01:00
Chocobozzz 36619ac8f5
Reduce dev commands RAM usage 2020-01-22 10:18:57 +01:00
Chocobozzz 5e0171daaa
Fix missing dependency 2020-01-21 14:30:53 +01:00
Chocobozzz a3705089fa
Remove husky 2020-01-21 14:28:28 +01:00
Chocobozzz e92269053e
Update http signature 2020-01-07 15:56:09 +01:00
Chocobozzz 7cde3b9c2e
Update validator dependency 2020-01-07 15:56:07 +01:00
Chocobozzz 60815a7921
Update server dependencies 2020-01-07 15:56:05 +01:00
Chocobozzz 03d641a0d7
Add embed api build 2019-12-18 10:14:25 +01:00
Chocobozzz c026a2e673
Remove watch scripts 2019-12-18 10:14:24 +01:00
Chocobozzz 478924a044
Add package.json for embed api 2019-12-18 10:14:24 +01:00
Rigel Kent 69c7f7525d
(rss) quickfix for pfeed when category is null 2019-12-04 09:16:51 +01:00
Chocobozzz 3e101afb90
Fix tests 2019-12-03 20:19:42 +01:00
Rigel Kent 16d9224a1c
(rss) add mrss properties and update pfeed
resolves #2298
2019-12-03 16:11:10 +01:00
Chocobozzz a5a254c4b8
Upgrade server deps 2019-12-03 14:14:30 +01:00
Chocobozzz d7a25329f9
Add ability to disable webtorrent
In favour of HLS
2019-11-25 10:59:43 +01:00
Chocobozzz e8e09e27ff
Bumped to version v2.0.0 2019-11-12 08:04:15 +01:00
Chocobozzz 16fd7c4b86
Bumped to version v2.0.0-rc.1 2019-10-29 08:26:43 +01:00
Chocobozzz 2be68ab979
Update jsonld library 2019-10-21 16:05:08 +02:00
Chocobozzz 464687bb96
Fix bug with tsconfig paths 2019-10-21 16:02:15 +02:00
Chocobozzz d5d9b6d7bf
Update server dependencies 2019-10-21 15:03:28 +02:00
Chocobozzz 8c559fad1e
Merge branch 'release/1.4.0' into develop 2019-09-11 11:09:18 +02:00
Chocobozzz 22af8aac79
Bumped to version v1.4.1 2019-09-09 08:15:24 +02:00
Chocobozzz 8424c4026a Add auto follow back support for instances 2019-09-04 16:24:58 +02:00
Chocobozzz ad513607a3
Remove old JSON LD signature implementation
Only PeerTube was compatible with it, and the library has moved on
RsaSignature2018 and removed RsaSignature2017 support. We had to create
a dirty fork of the RsaSignature2017 branch, which is not ideal.

Now we use the Mastodon implementation, that most other AP
implementations that support JSONLD signatures use.
2019-08-29 16:15:41 +02:00
Chocobozzz 076d3ace1e
Bumped to version v1.4.0 2019-08-26 11:56:10 +02:00
Chocobozzz f01dc977ae
Bumped to version v1.4.0 2019-08-26 08:49:46 +02:00
Chocobozzz 54beb819f0
Bumped to version v1.4.0-rc.1 2019-08-13 15:34:39 +02:00
Chocobozzz 322adef183
Upgrade sequelize 2019-08-12 08:52:44 +02:00
Chocobozzz 557b13ae24
Lazy load avatars 2019-08-09 11:32:40 +02:00
Chocobozzz 2284f20207
Add gitlab ci support 2019-07-29 13:40:39 +02:00
Chocobozzz 523990dbe6
Upgrade server dependencies 2019-07-25 11:00:41 +02:00
Chocobozzz b5f919ac8e WIP plugins: update plugin 2019-07-24 10:58:16 +02:00
Chocobozzz 8d2be0ed7b WIP plugins: move plugin CLI in peertube script
Install/uninstall/list plugins remotely
2019-07-24 10:58:16 +02:00
Chocobozzz f023a19c3e WIP plugins: install/uninstall 2019-07-24 10:58:16 +02:00
Chocobozzz b6a1dd4d1b
Merge branch 'release/v1.3.0' into develop 2019-06-17 09:48:41 +02:00
Chocobozzz 0109c4921c
Bumped to version v1.3.1 2019-06-17 08:38:32 +02:00
Chocobozzz 1a12f66d63
Add more CLI tests 2019-06-13 11:09:38 +02:00
Chocobozzz 4c7f776136
Add lint command 2019-06-11 10:05:47 +02:00
Chocobozzz 6091983127
Upgrade server dependencies 2019-06-06 15:39:11 +02:00
Chocobozzz a41b9da1a9
Merge branch 'release/v1.3.0' into develop 2019-06-03 09:32:33 +02:00
Chocobozzz 88ebb43310
Bumped to version v1.3.0 2019-06-03 08:47:25 +02:00
Chocobozzz 4c72c1cd41
Move concurrently in dev package 2019-05-31 11:48:28 +02:00
Chocobozzz 7e9d3f259d Move CLI dependencies in their own package.json 2019-05-27 14:56:06 +02:00
Chocobozzz faebb42355
Bumped to version v1.3.0-rc.2 2019-05-22 09:37:13 +02:00
Chocobozzz ea0877b405
Bumped to version v1.3.0-rc.2 2019-05-22 08:22:14 +02:00
Chocobozzz da3a3ab682
Try parallel check params tests 2019-05-14 15:37:42 +02:00
Chocobozzz 3fbbadabea
Bumped to version v1.3.0-rc.1 2019-05-14 14:00:03 +02:00
Chocobozzz 47f6409bb8
Use apicache instead of our broken implementation 2019-04-25 11:27:13 +02:00
Chocobozzz 3acc508440
Upgrade sequelize 2019-04-24 16:26:21 +02:00
Chocobozzz 1735c82572
Update sequelize 2019-04-24 16:26:20 +02:00
Chocobozzz a1a216d9cb
Upgrade server dep 2019-04-02 16:42:13 +02:00
Rigel Kent 590924284c
add NOCLIENT option to install only server dependencies 2019-02-25 22:30:47 +01:00
Chocobozzz 9b39106d57
findById -> findByPk 2019-02-21 14:40:10 +01:00
Chocobozzz 374c1db98c
Upgrade server dependencies 2019-02-21 14:22:39 +01:00
Chocobozzz 301fc49c17
Bumped to version v1.2.1 2019-02-15 09:43:39 +01:00
Chocobozzz 4c280004ce Use a single file instead of segments for HLS 2019-02-11 09:13:02 +01:00
Chocobozzz 0920929696 Add hls support on server 2019-02-11 09:13:02 +01:00
Chocobozzz f5305c04aa
Bumped to version v1.2.0 2019-02-06 11:57:57 +01:00
Chocobozzz cfeae291c0
Bumped to version v1.2.0-rc.1 2019-01-17 16:15:20 +01:00
Chocobozzz cef534ed53 Add user notification base code 2019-01-09 11:15:15 +01:00
Chocobozzz a4255d8efc
Remove unused dependencies 2018-12-06 13:35:27 +01:00
Chocobozzz 2feebf3e6a
Add sitemap 2018-12-05 17:44:34 +01:00
Chocobozzz f9a971c671
Update dependencies 2018-12-04 10:08:45 +01:00
Chocobozzz b6ff69719d
Bumped to version v1.1.0 2018-12-04 08:29:12 +01:00
Rigel Kent d7ea359d36
quickfix back to recent nodemon that fixes event-stream vulnerability 2018-11-26 21:23:10 +01:00
Rigel Kent 0218a46c1a
quickfix back to older nodemon 2018-11-26 18:43:47 +01:00
Rigel Kent 81658ae047
quickfix back to safe nodemon 2018-11-26 18:31:03 +01:00
Chocobozzz 6cc98dfff3
Add open api bump version in release script 2018-11-22 15:04:14 +01:00
Chocobozzz 326b44070b
Bumped to version v1.1.0-rc.1 2018-11-22 14:27:46 +01:00
Rigel Kent 3e9e6f2f14
migrate Swagger 2.0 spec to OpenAPI 3.0.0 2018-11-15 14:10:16 +01:00
Chocobozzz f7454ca60a
Upgrade server dep 2018-11-15 10:54:37 +01:00
Rigel Kent b83b8dd5ae add cli option to run without client 2018-11-14 15:59:56 +01:00
Rigel Kent 1d859b5afa
sync API version with PeerTube version and auto-update it via git hooks 2018-11-13 14:31:48 +01:00
Rigel Kent 7cf3e57376 (doc) add manpage to peertube cli 2018-10-20 19:16:56 +02:00
Chocobozzz 6df8130efc
Bumped to version v1.1.0-alpha.2 2018-10-19 15:52:15 +02:00
Chocobozzz 41f2ebae4f
Add HTTP signature check before linked signature
It's faster, and will allow us to use RSA signature 2018 (with upstream
jsonld-signature module) without too much incompatibilities in the
peertube federation
2018-10-19 12:26:37 +02:00
Chocobozzz cdf4cb9eaf
Fix transcoding 2018-10-18 10:44:16 +02:00
Chocobozzz 7cdc3ab63b
Bumped to version v1.1.0-alpha.1 2018-10-17 08:43:16 +02:00
Chocobozzz 71e318b4fe
Merge branch 'release/v1.0.0' into develop 2018-10-11 09:52:16 +02:00
Chocobozzz 333210d862
Bumped to version v1.0.0 2018-10-11 09:06:16 +02:00
Felix Ableitner edb4ffc7e0 Set bitrate limits for transcoding (fixes #638) (#1135)
* Set bitrate limits for transcoding (fixes #638)

* added optimization script and test, changed stuff

* fix test, improve docs

* re-add optimize-old-videos script

* added documentation

* Don't optimize videos without valid UUID, or redundancy videos

* move getUUIDFromFilename

* fix tests?

* update torrent and file size, some more fixes/improvements

* use higher bitrate for high fps video, adjust bitrates

* add test video

* don't throw error if resolution is undefined

* generate test fixture on the fly

* use random noise video for bitrate test, add promise

* shorten test video to avoid timeout

* use existing function to optimize video

* various fixes

* increase test timeout

* limit test fixture size, add link

* test fixes

* add await

* more test fixes, add -b:v parameter

* replace ffmpeg wiki link

* fix ffmpeg params

* fix unit test

* add test fixture to .gitgnore

* add video transcoding fps model

* add missing file
2018-10-08 16:26:04 +02:00