Commit Graph

9969 Commits (d33b3610002d7e6c6fe8341a43622064b7cf9131)

Author SHA1 Message Date
Claire d33b361000 Fix addressing of remote groups' followers (#16700)
Fixes #16699
2022-01-28 22:52:42 +01:00
Claire 5b07f4e90e Fix some link previews being incorrectly generated from other prior links (#16885)
* Add tests

* Fix some link previews being incorrectly generated from different prior links

PR #12403 added a cache to avoid redundant queries when the OEmbed endpoint can
be guessed from the URL. This caching mechanism is not perfectly correct as
there is no guarantee that all pages from a given domain share the same
OEmbed provider endpoint.

This PR prevents the FetchOEmbedService from caching OEmbed endpoint that
cannot be generalized by replacing a fully-qualified URL from the endpoint's
parameters, greatly reducing the number of incorrect cached generalizations.
2022-01-28 22:52:42 +01:00
Claire 0994c4b11a Fix "bundle exec rails mastodon:setup" crashing in some circumstances (#16976)
Fix regression from #16896
2022-01-28 22:52:42 +01:00
Claire d2cdfe92ed Fix mastodon:setup to take dotenv/docker-compose differences into account (#16896)
In order to work around https://github.com/mastodon/mastodon/issues/16895,
add a warning to .env.production.sample, and change the mastodon:setup rake
task to:
- output a warning if a variable will be interpreted differently by dotenv
  and docker-compose
- ensure the printed config is compatible with docker-compose
2022-01-28 22:52:42 +01:00
Claire 6c344d90c7 Fix tootctl self-destruct not sending Delete activities for recently-suspended accounts (#16688)
* Do not block existing users' emails on self-destruct

That is wasteful and unintuitive

* Do not close registrations when running tootctl self-destruct with --dry-run

* Close registrations on self-destruct regardless of known remote accounts

* Fix tootctl self-destruct not sending Deletes for recently-suspended accounts

* Suspend local users even if no remote account is known

* Do not show scary confirmation text if ran with --dry-run
2022-01-28 22:52:42 +01:00
Claire c0b2c2c166 Fix serialization of followers/following counts when user hides their network (#16418)
* Add tests

* Fix serialization of followers/following counts when user hides their network

Fixes #16382

Signed-off-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:52:42 +01:00
Claire 19edb7a3f4 Fix followers synchronization mechanism not working when URI has empty path (#16744)
Follow-up to #16510, forgot the controller exposing the actual followers…
2022-01-28 22:52:42 +01:00
Claire 53e4efd07d Fix followers synchronization mechanism not working when URI has empty path (#16510)
* Fix followers synchronization mechanism not working when URI has empty path

To my knowledge, there is no current implementation on the fediverse
that can use bare domains (e.g., actor is at https://example.org instead of
something like https://example.org/actor) that also plans to support the
followers synchronization mechanism. However, Mastodon's current implementation
would exclude such accounts from followers list.

Also adds tests and rename them to reflect the proper method names.

* Move url prefix regexp to its own constant
2022-01-28 22:52:41 +01:00
Claire 3f882c2c17 Fix scheduled statuses decreasing statuses counts (#16791)
* Add tests

* Fix scheduled statuses decreasing statuses counts

Fixes #16774
2022-01-28 22:39:48 +01:00
Holger 1cfa2bdb03 use relative path for `scope` (#16714)
Use relative path for `scope` in web manifest to allow users use PWA correctly via alternate domains.
2022-01-28 22:39:48 +01:00
Claire 1b32c001bc Fix migration script not being able to run if it fails midway (#16312)
* Fix migration script not being able to run if it fails midway

* Fix old migration script

* Fix old migration script

* Refactor CorruptionError
2022-01-28 22:39:48 +01:00
Claire 31d9aa8ed0 Fix media proxy RedisLocks auto-releasing too fast (#16291)
Follow-up to #16276
2022-01-28 22:39:48 +01:00
Claire 4d41c91335 Fix some RedisLocks auto-releasing too fast (#16276)
* Fix Delete and Create-related locks expiring too fast

Fixes #16238

By default, RedisLock expires after 10 seconds, which may not be enough to
process statuses, especially when those have attached media files.

This commit extends those 10 seconds to 15 minutes, which should be plenty
enough to handle any status, while being short enough to not waste many
sidekiq job retries in the exceedingly rare case in which a sidekiq process
would crash when processing a `Create` or `Delete`.

* Fix other RedisLock autorelease durations

Fixes #15645

- things that only perform a few simple database queries (e.g. finding and
  saving a record) have been left unchanged, so they'll still use the default
  10s duration
- things that perform significantly more complex database queries have been
  changed to a 5 minutes timeout
- things that perform multiple HTTP queries have been changed to a 15 minutes
  timeout
2022-01-28 22:39:48 +01:00
Jeong Arm 678e0ad419 Remove set-cookie header on custom.css (#16314)
* Remove set-cookie header on custom.css

* Additional fix for set-cookie
2022-01-28 22:39:48 +01:00
Claire c89809afc5 Fix some IDs in instance actor outbox (#16343) 2022-01-28 22:39:48 +01:00
Eugen Rochko a319fd3cc4 Fix app name, website and redirect URIs not having a maximum length (#16042)
Fix app scopes not being validated
2022-01-28 22:39:48 +01:00
Claire 24dee67d32 Create instance actor if it hasn't been properly seeded (#15693)
An uncommon but somewhat difficult to digagnose issue is dealing with
improperly-seeded databases. In such cases, instance-signed fetches will
fail with a ActiveRecord::RecordNotFound error, usually caught and handled
as generic 404, leading people to think the remote resource itself has not
been found, while it's the local instance actor that does not exist.

This commit changes the code so that failure to find the instance actor
automatically creates a new one, so that improperly-seeded databases do
not cause any issue.
2022-01-28 22:39:48 +01:00
kaiyou 07042a0913 Support clock drift in Omniauth SAML provider (#15511)
The setting is not well documented by the provider, but allows for
clock skew between SP and IDP, see:
https://github.com/omniauth/omniauth-saml/blob/master/spec/omniauth/strategies/saml_spec.rb

Co-authored-by: kaiyou <dev@kaiyou.fr>
2022-01-28 22:39:48 +01:00
Eugen Rochko 4978d387ee WIP (#15222) 2022-01-28 22:39:48 +01:00
Stanislas 0951c691ff tootctl emoji import: case insensitive duplicate check (#15738) 2022-01-28 22:39:48 +01:00
Sophie Parker fc4b9856f8 Improve Emoji import (fix #15429) (#15430)
* Improve Emoji import

Skip macOS '._' shadow files in tar archive to speed up import

* Fix codeclimate format issue with whitespace

* Update lib/mastodon/emoji_cli.rb

suggestions from Gargron to improve comment

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

* Update emoji_cli.rb

Remove extraneous comment (macOS-specific comment now with correct line)

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-01-28 22:39:48 +01:00
ThibG add7b9f82e Fix “tootctl accounts unfollow” (#15639)
Fixes #15635

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
Claire 9437e9f0b9 Fix custom CSS when CDN_HOST is set (#15927) 2022-01-28 22:39:48 +01:00
Levi Bard 0fe57a9140 Fix muting users with duration via the REST api (#15516) 2022-01-28 22:39:48 +01:00
Claire 3a4d9f1f2d Fix not being able to change world filter expiration back to “Never” (#15858)
Fixes #15849
2022-01-28 22:39:48 +01:00
ThibG 78d5bda973 Fix race conditions on account migration creation (#15597)
* Atomically check for processing lock in Move handler

* Prevent race condition when creating account migrations

Fixes #15595

* Add tests

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
ThibG 4b025cf7e6 Fix sign-up restrictions based on IP addresses not being enforced (#15607)
Fixes #15606

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
Eugen Rochko 4bd8dc09d8 Fix reports of already suspended accounts being recorded (#16047) 2022-01-28 22:39:48 +01:00
Claire 3799fd17ba Fix edge case where accepted follow cannot be processed because of follow limit (#16098) 2022-01-28 22:39:48 +01:00
Claire 53814b2b31 Fix blocking someone not clearing up list feeds (#16205) 2022-01-28 22:39:48 +01:00
ThibG 2012c5ae50 Fix maintenance script not re-indexing some indexes on textual values (#15515)
* Fix maintenance script not re-indexing some indexes on textual values

Fixes #15475

* Refresh instance view at the end of the maintenance script run

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
Claire 9aa7286c8f Change deduplication order of tootctl maintenance fix-duplicates (#15923)
Hopefully fixes #15922

Also update support up to latest database schema version
2022-01-28 22:39:48 +01:00
Eugen Rochko 4b9a0cfe5e Fix media processing getting stuck on too much stdin/stderr (#16136)
* Fix media processing getting stuck on too much stdin/stderr

See thoughtbot/terrapin#5

* Remove dependency on paperclip-av-transcoder gem

* Remove dependency on streamio-ffmpeg gem

* Disable stdin on ffmpeg process
2022-01-28 22:39:48 +01:00
Eugen Rochko b593a7da8c Fix database serialization failure returning HTTP 500 (#16101)
Database serialization failure occurs when a read-replica is used
and a query takes long enough that rows on the primary database
become unavailable. It should return HTTP 503 as it is temporary.

Re-order rescue definitions according to their status codes
2022-01-28 22:39:48 +01:00
ThibG 48b25e457d Fix /activity endpoint not require signature in authorized fetch mode (#15592)
Fixes #15589

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
Claire da14725a96 Fix crash on receiving requests with missing Digest header (#15782)
* Fix crash on receiving requests with missing Digest header

Return an error pointing out that Digest is missing, instead of crashing.

Fixes #15743

* Fix from review feedback
2022-01-28 22:39:48 +01:00
Claire cc21670b3c Fix URI of repeat follow requests not being recorded (#15662)
* Fix URI of repeat follow requests not being recorded

In case we receive a “repeat” or “duplicate” follow request, we automatically
fast-forward the accept with the latest received Activity `id`, but we don't
record it.

In general, a “repeat” or “duplicate” follow request may happen if for some
reason (e.g. inconsistent handling of Block or Undo Accept activities, an
instance being brought back up from the dead, etc.) the local instance thought
the remote actor were following them while the remote actor thought otherwise.

In those cases, the remote instance does not know about the older Follow
activity `id`, so keeping that record serves no purpose, but knowing the most
recent one is useful if the remote implementation at some point refers to it
by `id` without inlining it.

* Add tests
2022-01-28 22:39:47 +01:00
ThibG 685cde55cb Skip processing Update activities on unknown accounts (#15514)
This also skips fetching the actor completely.

This will be useful if we end up distributing Update activities linked to
account suspensions more widely (they are currently only delivered to
the suspended account's followers), as currently, instances not knowing
about the suspended account would fetch it to then process the suspension.

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:47 +01:00
ThibG a2dc4e583b Fix processing of incoming Block activities (#15546)
Unlike locally-issued blocks, they weren't clearing follow
relationships in both directions, follow requests or notifications.

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:47 +01:00
Claire 13d1111a92 Fix processing of remote Delete activities (#16084)
* Add tests

* Ensure deleted statuses are marked as such

* Save some redis memory by not storing URIs in delete_upon_arrival values

* Avoid possible race condition when processing incoming Deletes

* Avoid potential duplicate Delete forwards

* Lower lock durations to reduce issues in case of hard crash of the Rails process

* Check for `lock.aquired?` and improve comment

* Refactor RedisLock usage in app/lib/activitypub

* Fix using incorrect or non-existent sender for relaying Deletes
2022-01-28 22:39:47 +01:00
ThibG 6386421d1a Fix profile update not being sent on profile/header picture deletion (#15461)
Fixes #15460

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:47 +01:00
Claire 4063bbe04e Fix Mastodon not understanding as:Public and Public (#15948)
Fixes #5551
2022-01-28 22:39:47 +01:00
Eugen Rochko 4ea2c1da95 Fix remote reporters not receiving suspend/unsuspend activities (#16050) 2022-01-28 22:39:47 +01:00
abcang 47cab05003 Fix N+1 query when rendering with StatusSerializer (#15641) 2022-01-28 22:39:47 +01:00
Eugen Rochko 02d195809b Fix thread resolve worker retrying when status no longer exists (#16109) 2022-01-28 22:39:47 +01:00
Eugen Rochko 64d9b84f1d Fix media redownload worker retrying on unexpected response codes (#16111) 2022-01-28 22:39:47 +01:00
Claire a6e9c41ed4 Bump dependencies so that 3.3.x can be installed on current systems
New system requirement: shared-mime-info
2022-01-28 22:39:47 +01:00
Eugen Rochko 633d175146
Bump version to 3.3.0 (#15433) 2020-12-27 06:00:55 +01:00
Yamagishi Kazutoshi e43532c9cc
Normalize locale files (#15434) 2020-12-27 05:30:56 +01:00
ThibG f1f96ebf02
Fix being able to import more than allowed number of follows (#15384)
* Fix being able to import more than allowed number of follows

Without this commit, if someone tries importing a second list of accounts to
follow before the first one has been processed, this will queue imports for
the two whole lists, even if they exceed the account's allowed number of
outgoing follows.

This commit changes it so the individual queued imports aren't exempt from
the follow limit check (they remain exempt from the rate-limiting check
though).

* Catch validation errors to not re-queue failed follows

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-26 23:52:46 +01:00