Commit Graph

24 Commits (4ef8d8b77c596680be13b26400bc2a09038928de)

Author SHA1 Message Date
Eugen Rochko 5d3feed191
Refactor fetching of remote resources (#11251) 2019-07-10 18:59:28 +02:00
Eugen Rochko 23aeef52cc
Remove Salmon and PubSubHubbub (#11205)
* Remove Salmon and PubSubHubbub endpoints

* Add error when trying to follow OStatus accounts

* Fix new accounts not being created in ResolveAccountService
2019-07-06 23:26:16 +02:00
Renato "Lond" Cerqueira fe180f18ff Change conditional to avoid nil into string error in sidekiq (#5987)
* Change conditional to avoid nil into string error in sidekiq
When obtaining information about users with mastodon in a different
subdomain, sidekiq was giving out a 'no implicit conversion of nil into String'

* Use presence instead of blank? with ternary.
Following suggestion on PR
2017-12-12 15:11:13 +01:00
ThibG 95fe20b78a Fix remote status fetching for “pure” ActivityPub WEB_ACCOUNT users (#5372)
Remote ActivityPub users that have never been known as OStatus users
(whether or not they support it) will not have a “remote_url” attribute
set. In case they reside on an instance with WEB_DOMAIN ≠ LOCAL_DOMAIN,
the current check did rely on “remote_url” to verify the user's domain.
2017-10-13 12:58:13 +02:00
Akihiko Odaki 63f0979799 Validate id of ActivityPub representations (#5114)
Additionally, ActivityPub::FetchRemoteStatusService no longer parses
activities.
OStatus::Activity::Creation no longer delegates to ActivityPub because
the provided ActivityPub representations are not signed while OStatus
representations are.
2017-10-04 01:13:48 +02:00
Akihiko Odaki bb4d005a83 Introduce OStatus::TagManager (#5008) 2017-09-19 18:08:08 +02:00
Yamagishi Kazutoshi 5f22c0189d Add support for searching AP users (#4599)
* Add support for searching AP users

* use JsonLdHelper
2017-08-14 14:08:34 +02:00
Eugen Rochko 4e75f0d889 Hook up URL-based resource look-up to ActivityPub (#4589) 2017-08-14 02:29:36 +02:00
Eugen Rochko 1fcdaafa6f Fix webfinger retries (#4275)
* Do not raise unretryable exceptions in ResolveRemoteAccountService

* Removed fatal exceptions from ResolveRemoteAccountService

Exceptions that cannot be retried should not be raised. New exception
class for those that can be retried (Mastodon::UnexpectedResponseError)
2017-07-20 01:59:07 +02:00
ThibG 05cd37097c Optimize uri normalization (#4212)
* Add dependency on idn-ruby to speed up URI normalization

* Use normalized_host instead of normalize.host when applicable

When we are only interested in the normalized host, calling normalized_host
avoids normalizing the other components of the URI as well as creating a
new object
2017-07-15 17:24:35 +02:00
abcang 617208053c Rescue exceptions related to Goldfinger at FetchRemoteStatusService (#4138) 2017-07-10 13:59:29 +02:00
Eugen Rochko bafd22ecf4 Fix #2706 - Always respond with 200 to PuSH payloads (#2733)
Fix #2196 - Respond with 201 when Salmon accepted, 400 when unverified
Fix #2629 - Correctly handle confirm_domain? for local accounts
Unify rules for extracting author acct from XML, prefer <email>, fall back
to <name> + <uri> (see also #2017, #2172)
2017-05-03 17:02:18 +02:00
Eugen Rochko 2af4f3c4e2 Improve shared status verification (#2525)
* Instead of parsing shared status contents verbatim, make roundtrip
to purported original URL. Confirm that the "original" URL is from the
same domain as the author it claims to be from.

* Fix obvious typo, add comment

* Use URI look-up first

* Add test, update Goldfinger dependency to make less useless HTTP requests per Webfinger lookup
2017-04-27 17:06:47 +02:00
Eugen 17c591ffba Punycode URI normalization (#2370)
* Fix #2119 - Whenever about to send a HTTP request, normalize the URI

* Add test for IDN request in FetchLinkCardService

* Perform IDN normalization on domains before they are stored in the DB
2017-04-25 02:47:31 +02:00
Eugen Rochko 5aa3df017b Fix full-text search query quotation, improve tag search performance with an index,
add ability to open status by URL from search (fix #53)
2017-03-22 17:36:34 +01:00
Eugen Rochko fdc17bea58 Fix rubocop issues, introduce usage of frozen literal to improve performance 2016-11-15 16:56:29 +01:00
Eugen Rochko 7ce4670164 Force utf-8 encoding when processing XML 2016-11-13 19:12:40 +01:00
Eugen Rochko 93212bc2c4 Add test for FanOutOnWriteService 2016-11-06 15:56:34 +01:00
Eugen Rochko 852c82435d Fix method return when rescuing 2016-10-20 18:36:12 +02:00
Eugen Rochko 2febc6ed65 Fix typo 2016-10-05 13:40:14 +02:00
Eugen Rochko fe77921e47 Catching more exceptions that slipped through, removing AR logging from
production as it's very verbose and not very useful
2016-10-05 13:26:44 +02:00
Eugen Rochko 927333f4f8 Improve code style 2016-09-29 21:28:21 +02:00
Eugen Rochko c6b0311b86 Fix #54 - Fetch remote accounts by URL from mentions
Fetching atom extracted from FetchRemoteAccountService and FetchRemoteStatusService
into FetchAtomService. Mentions of the constant "http://activityschema.org/collection/public"
skipped as it's not a real URL/user.
2016-09-26 16:44:40 +02:00
Eugen Rochko 4bec613897 Fix #24 - Thread resolving for remote statuses
This is a big one, so let me enumerate:

Accounts as well as stream entry pages now contain Link headers that
reference the Atom feed and Webfinger URL for the former and Atom entry
for the latter. So you only need to HEAD those resources to get that
information, no need to download and parse HTML <link>s.

ProcessFeedService will now queue ThreadResolveWorker for each remote
status that it cannot find otherwise. Furthermore, entries are now
processed in reverse order (from bottom to top) in case a newer entry
references a chronologically previous one.

ThreadResolveWorker uses FetchRemoteStatusService to obtain a status
and attach the child status it was queued for to it.

FetchRemoteStatusService looks up the URL, first with a HEAD, tests
if it's an Atom feed, in which case it processes it directly. Next
for Link headers to the Atom feed, in which case that is fetched
and processed. Lastly if it's HTML, it is checked for <link>s to the Atom
feed, and if such is found, that is fetched and processed. The account for
the status is derived from author/name attribute in the XML and the hostname
in the URL (domain). FollowRemoteAccountService and ProcessFeedService
are used.

This means that potentially threads are resolved recursively until a dead-end
is encountered, however it is performed asynchronously over background jobs,
so it should be ok.
2016-09-21 01:50:31 +02:00