Imageboard federation protocol

[DangerOnTheRanger]

I've been thinking about this concept for a little while now, and I thought it might be good to make an issue to discuss it a bit further. I think a logical next step from imageboards that allow users to create their own boards is to have a federated imageboard setup that allows users to make posts in the same shared boards from across different sites.

For example, say there are two imageboard sites, and they both have an /a/ board on them. I think it should be possible for a user on one site to make a thread on /a/, and for a different user on the other site to see that thread and reply to it with their own post. And this wouldn't be restricted to two sites, but perhaps users from five or ten could all participate. This sort of thing is already done with software like Mastodon, and I see no reason why it couldn't be extended to imageboards.

The rationale for wanting this kind of thing mostly stems from seeing how hard it is to grow a site. It's always a chicken-and-egg problem, but I think having a larger network of readily-available content will help provide an extra push. Sites could still have boards that they didn't share/federate with others if they wanted to provide unique content, too - it wouldn't have to be an all-or-nothing affair.

There's a couple caveats, of course. If we're to keep the traditional way of replying to posts (>>[post ID]), then post IDs will need to be consistent across sites, at least for federated boards. Then the de facto tactic of making each new post ID equal to (`last posted ID + 1) will be prone to cross-site race conditions, so something else will need to be done, maybe randomly generating an ID.

There's also the problem of agreeing on what imageboard features to support. I'd like to make the protocol generic so that imageboard engines other than Maniwani could benefit, but then there needs to be some discussion on what markup to support in post bodies (if any), a base set of supported MIME types for attachments, and other minor things, such as maximum post length and so on. That's going to involve a lot of discussion, and possibly some hacks - for instance, Maniwani has globally-unique post IDs such that knowing what board a post is on isn't necessary to link to it, which isn't true for most other imageboards - but I think it can be done.

Maybe a good concrete end-to-end example would be helpful. Say on maniwani-site.org, someone makes a new thread on /a/, and hits the post button. The server at maniwani-site.org then broadcasts this to a list of all imageboard sites that it's affiliated with, maybe by making a POST request to an /api/federation endpoint. The content of that request could look something like this:

{
	"post-id": 123456,
	"thread-id": 789,
	"posted-at": "2020-10-05T14:48:00.000Z",
	"board": "a",
	"subject": null,
	"content": "Hello /a/ World!",
	"author": "43E7",
	"attachment-mimetype": "image/jpg",
	"attachment-url": "cdn.maniwani-site.org/attachment/11223344.jpg"
}

One of these federated sites - we'll call it other-imageboard.net - processes this request, copies the attachment to its own servers, and displays the post on its /a/ board. A short time later, a user on other-imageboard.net sees the thread and makes a reply. The site broadcasts the reply to all sites in the federation just as maniwani-site.org did, and hits the same /api/federation endpoint on the other sites. The request body looks a little similar:

{
	"post-id": 314159,
	"thread-id": 789,
	"posted-at": "2020-10-05T14:50:00.000Z",
	"board": "a",
	"subject": "hello there",
	"content": ">>123456\nright back at you",
	"author": "FZ78",
	"attachment-mimetype": "video/webm",
	"attachment-url": "other-imageboard.net/attach/fz78-314159.webm"
}

maniwani-site.org will then add the post and copy the attachment back onto its CDN, and the end result is a thread with two posts and two attachments, made by two separate anonymous members of the same imageboard federation, posting from two completely separate sites that might not even be running the same software. I think that sort of use case is worth a closer look.