Feature Support: MSPFA

[GiovanH]

Describe the solution you'd like
Native support for MSPFAs, provided by mods.

Needs to include all MSPFA features including themes

Design Considerations
How are spoilers handled? New reader mode? We can't assume MSPFAs are even relatively linear.

[GiovanH]

Some additional thoughts on this current design:

Packaging MSPFAs is currently going to be A Step. There are a few reasons for this:

• UHC is not meant to just be a wrapper for the MSPFA site that uses their service and skims off their ad revenue
• UHC is meant for archives of content that work offline and when services go down, and just fetching content from servers allows links to break
• Having a human involved in the packaging step lets a person make manual corrections, like SWF replacements and and style corrections, that would not be feasible with a fully-automatic system

This means for any panel updates to a fan adventure, there would need to be a full mod update. This isn’t desirable but seems like a natural consequence of the design.

[noirscape]

Some comments from my poking at MSPFA over the past years (out of sheer curiosity; the site was what got me interested into archiving in the first place after the Dropbox public link nuke):

From what I can tell, MSPFA is conceptually not very complicated from a programming perspective; it doesn't do anything like image hosting or CSS hosting. It just provides the text, a default layout reminiscent of "classic" MSPA (not homestuck.com) and the ability for the user to load custom stylesheets and javascript on a "per adventure" basis. These stylesheets and javascript are loaded fully in-line; that is to say, they aren't stored in any files.

There's some other small options like a comment box (probably just irrelevant), upstream links, mirrorer credit and probably most notably, the option to link to an icon that is displayed under a foldout.

I should stress; no images or custom stylesheets are hosted on MSPFA.

The advantage of this is that from a technical perspective, for a "laziest possible" you don't have to do that much and most of what is needed is already provided by the modding API.

The disadvantage is that any sort of scraping of MSPFA will be near impossible (probably not a negative however); you'd have to do some nasty HTML parsing to get content and it's practically impossible to get any meaningful data out of it. It'd always be a human involved in the step; see below for more.

In terms of what this would mean for the bare minimum solution:

• Make it easy to just make a new page with the regular collection page template and apply a path-wide CSS/Javascript override; out of the eye of archival, I'd encourage forcing these sorts of overrides to be fully local but that might not be feasible since [insert ethereal screams about parsing arbitrary CSS/Javascript for stuff].
• Add BBCode and arbitrary HTML insertion; MSPFAs syntax for editors is just BBCode, but arbitrary HTML is nice in cases where the author has vanished.
• Everything else is pretty much already possible from what I can tell; things like logos and other things are pretty much given with the archive.tweaks.modHomeRowItems feature. Maybe add a way to nicely format the usernames of the authors and mirrorers and offer the option to provide links to their sites.
• Add the option to easily generate a chronological sitemap for a tree of pages.

---

The cleanest and "fanciest" solution would probably be to make, and allow the designation of, specific fanventure "mods". Basically, make it a folder that the mod loader will as a whole map to a unique tree in the collection with a simple json file in the folder that describes the "overall" details (name/description/authors/editors/links for each/path to css/path to js file/path to icon image/where to start/default next page text).

Every other file in that folder can then be mapped and parsed to a default collection page with the extension stripped if it's a simple text file (ie 1.txt becomes /fanventure name/1. Easiest file format from an archival perspective is probably just a text file with a yaml header that specifies the link to the next page, the command to display and the date at which the page was originally published on MSPFA. Possibly add a yaml header flag to make the fanventure not parse stuff through BBCode (see arbitrary HTML comment).

The alternative is to make people write a lot of JSON which is probably rather archivist-unfriendly.

[GiovanH]

Some of that is right, some is not.

MSPFA does not host images, which is why images break so easily. Many fan adventures have images hosted externally, or on dropbox and photobucket, and are nonfunctional. See The Felt.

However MSPFA is absolutely scrapable and provides most data about adventures, including the full page content and custom css styles, in an API.

I have a full implementation of a lot of this here: https://github.com/GiovanH/unofficial-homestuck-collection/blob/mspfa/tools/mspfa/mspfa.py I'll edit it into the top description

[thecnoNSMB]

As a fanventure reader and author I would also like the following features:

• Spoiler gate adventures, in part or in whole, by Problem Sleuth or Homestuck progress (if this isn't already a feature)
• The ability to quickly create a template mod from the site's own JSON adventure export.

[nope1game]

I think the best plan of action if to get in contact withe the devs of the site.

[Epsylon42]

A little shameless plug, but I'm currently working on a tool that creates mods out of fanventures. It still needs some polish, including documentation, but as far as I can tell it Basically Works for most cases already

It seems to be more feature-complete than the current implementation in mspfa.py. For example it recursively handles CSS imports, downloads youtube embeds with yt-dlp, and some other things. Also uses parsers for bbcodes/css/html instead of regexes (incuding doing the abovementioned nasty html parsing).

I did not take into account #311 when making it, so the mods are currently self-contained. If that would be useful though, I can try to look into adapting its output for the code in that PR

Meanwhile, it can be used as is - https://github.com/Epsylon42/mspfa-archive-helper