<buttonid="sidebar-toggle"class="icon-button"type="button"title="Toggle Table of Contents"aria-label="Toggle Table of Contents"aria-controls="sidebar">
<ahref="https://github.com/matrix-org/synapse/edit/develop/docs/url_previews.md"title="Suggest an edit"aria-label="Suggest an edit">
<iid="git-edit-button"class="fa fa-edit"></i>
</a>
</div>
</div>
<divid="search-wrapper"class="hidden">
<formid="searchbar-outer"class="searchbar-outer">
<inputtype="search"id="searchbar"name="searchbar"placeholder="Search this book ..."aria-controls="searchresults-outer"aria-describedby="searchresults-header">
<p>Design notes on a URL previewing service for Matrix:</p>
<p>Options are:</p>
<ol>
<li>Have an AS which listens for URLs, downloads them, and inserts an event that describes their metadata.</li>
</ol>
<ul>
<li>Pros:
<ul>
<li>Decouples the implementation entirely from Synapse.</li>
<li>Uses existing Matrix events & content repo to store the metadata.</li>
</ul>
</li>
<li>Cons:
<ul>
<li>Which AS should provide this service for a room, and why should you trust it?</li>
<li>Doesn't work well with E2E; you'd have to cut the AS into every room</li>
<li>the AS would end up subscribing to every room anyway.</li>
</ul>
</li>
</ul>
<olstart="2">
<li>Have a generic preview API (nothing to do with Matrix) that provides a previewing service:</li>
</ol>
<ul>
<li>Pros:
<ul>
<li>Simple and flexible; can be used by any clients at any point</li>
</ul>
</li>
<li>Cons:
<ul>
<li>If each HS provides one of these independently, all the HSes in a room may needlessly DoS the target URI</li>
<li>We need somewhere to store the URL metadata rather than just using Matrix itself</li>
<li>We can't piggyback on matrix to distribute the metadata between HSes.</li>
</ul>
</li>
</ul>
<olstart="3">
<li>Make the synapse of the sending user responsible for spidering the URL and inserting an event asynchronously which describes the metadata.</li>
</ol>
<ul>
<li>Pros:
<ul>
<li>Works transparently for all clients</li>
<li>Piggy-backs nicely on using Matrix for distributing the metadata.</li>
<li>No confusion as to which AS</li>
</ul>
</li>
<li>Cons:
<ul>
<li>Doesn't work with E2E</li>
<li>We might want to decouple the implementation of the spider from the HS, given spider behaviour can be quite complicated and evolve much more rapidly than the HS. It's more like a bot than a core part of the server.</li>
</ul>
</li>
</ul>
<olstart="4">
<li>Make the sending client use the preview API and insert the event itself when successful.</li>
</ol>
<ul>
<li>Pros:
<ul>
<li>Works well with E2E</li>
<li>No custom server functionality</li>
<li>Lets the client customise the preview that they send (like on FB)</li>
</ul>
</li>
<li>Cons:
<ul>
<li>Entirely specific to the sending client, whereas it'd be nice if /any/ URL was correctly previewed if clients support it.</li>
</ul>
</li>
</ul>
<olstart="5">
<li>Have the option of specifying a shared (centralised) previewing service used by a room, to avoid all the different HSes in the room DoSing the target.</li>
</ol>
<p>Best solution is probably a combination of both 2 and 4.</p>
<ul>
<li>Sending clients do their best to create and send a preview at the point of sending the message, perhaps delaying the message until the preview is computed? (This also lets the user validate the preview before sending)</li>
<li>Receiving clients have the option of going and creating their own preview if one doesn't arrive soon enough (or if the original sender didn't create one)</li>
</ul>
<p>This is a bit magical though in that the preview could come from two entirely different sources - the sending HS or your local one. However, this can always be exposed to users: "Generate your own URL previews if none are available?"</p>
<p>This is tantamount also to senders calculating their own thumbnails for sending in advance of the main content - we are trusting the sender not to lie about the content in the thumbnail. Whereas currently thumbnails are calculated by the receiving homeserver to avoid this attack.</p>
<p>However, this kind of phishing attack does exist whether we let senders pick their thumbnails or not, in that a malicious sender can send normal text messages around the attachment claiming it to be legitimate. We could rely on (future) reputation/abuse management to punish users who phish (be it with bogus metadata or bogus descriptions). Bogus metadata is particularly bad though, especially if it's avoidable.</p>
<p>As a first cut, let's do #2 and have the receiver hit the API to calculate its own previews (as it does currently for image thumbnails). We can then extend/optimise this to option 4 as a special extra if needed.</p>
<li>If HTML, just stores it in RAM and parses it for OG meta tags
<ul>
<li>Download any media OG meta tags to the media repo, and refer to them in the OG via mxc:// URIs.</li>
</ul>
</li>
<li>If a media filetype we know we can thumbnail: store it on disk, and hand it to the thumbnailer. Generate OG meta tags from the thumbnailer contents.</li>
