Implementing ActivityPub and Zot support in SSB clients #202
Comments
|
@strypey there isn't a 32 bit linux release of patchwork, but patchfoo is just a simple local web api (without even using javascript) https://github.com/ssbc/patchfoo you should definitely be able to run that, if you can install scuttlebot. |
|
@dominictarr Yes, but you can't install the latest @strypey I think here can be the right place to discuss this issue! There are two primary reasons why dev conversation happens on ssb:
Would you be up for trying to clone scuttlebot down and compiling from source? From there we can discuss lighter client options. |
Thanks for the friendly invite, but see points 1) and 2) ;) Plus, a new LTS version of Trisquel (the GNU-Linux distro I use) has finally been released and I'm about to start transitioning over, which will chew up some time. I'm also trying to carve out some time to do test installs of a few server packages (eg some teachers I know want a community-hosted Moodle instance). I'm keen to try out Scuttlebutt at some point, but I really don't have time to be compiling apps from source right now, so it probably won't happen until there's a graphical client with a 32-bit AppImage, a PPA, or its in the Debian repos ;)
Great! Let's do that :) BTW @dominictarr I believe we know each other from when I helped you move some materials from a storage room into the Maker Space you were running at Creative Cities in Welly? Been following your work on SSB from a distance with some interest, and just added some comments to the HN thread about Scuttlebutt as a potential FB replacement. |
That sounds quite likely but was a very long time ago... @evbogue what is the difference between the patchfoo you can npm install and the latest (tbh, imo it should just be npm installable, or something like that)
@strypey what about npm install? |
|
On 2018-04-30 22:53, Dominic Tarr wrote:
OH, CLICK, now I remember you - you are _Stripey_
That's me :)
@strypey what about npm install?
I've never npm'd before, but I'd be willing to give it a go.
If you folks don't mind though, I'd prefer to talk about the potential
for fediverse protocol bridging (see OP) in this Issue on GH. Just so
other fediverse folks can easily follow the discussion, and chime in
when I get important technical details wrong ;-P
|
|
@strypey talking about AP, I think there is a lot of work needed to map from scuttlebutt to ActivityPub since activities in that protocol are updatable as can be seen on the spec. Right now, posts on ssb are not editable. This could of course be implemented by creating new ActivityPub message type and the necessary submessages for CRUD. If such collection of messages is implemented, then reliable pubservers could run a plugin to be able to listen and respond to ActivityPub requests thus becoming Fediverse Instances in the eyes of the rest of the fediverse but without breaking the decentralized nature of ssb. So in my humble and shallow view, this requires two large steps: Implement ActivityPub messagesThese are a collection of messages that enable CRUD of ActivityPub activities. It is not unlike how Create plugin for pubservers to exchange data with fediverseOnce ssb can store AP data, pubservers can become gateways to the fediverse by replicating (and converting while doing that) messages/activities from ssb into AP. So if someone from Mastodon follows an SSB user (with an AP account such as When the contact follows back, the pub server upon seeing that message, converts it into the appropriate AP HTTP Request for the fediverse and sends it to that user instance. What are your thoughts on that @dominictarr @staltz @strypey @evbogue? I'd be game to work on this after I finish my current ssb grant. |
|
@soapdog sounds fantastic to me. Are you familiar with Bridgy-Fed? They aim to do a similar thing for the IndieWeb network that your proposal would do for the SSB network. Maybe there's some way to work in with them, borrow code from them, or just get some ideas from the way they handle the various fediverse protocols? |
|
@soapdog are you still working on your current grant? Is there any footwork that would help you in this mission that a non dev (but still quite technically minded person) might be able to do? Would really love to help get the ball rolling with bridging these two communities. |
|
But what if the pubserver malforms posts over the AP endpoint? |
|
@wakest I am still working in my grant, quite focused on it actually. It is now divided into two repos, scuttle-shell, for the host app and patchfox, for the add-on. Unfortunatelly, I don't have the time to devote to this bridge at the moment. I want to see it done as much as you but I can't commit to it. |
|
@Serkan-devel that can happen... ssb clients can create broken posts and AP clients can also create broken posts. It sucks but it do happens. My own SSB feed has a share of broken posts already. |
|
@soapdog how do you get broken posts already and is there a way to mitigate it? |
|
Oh and could the Matrix protocol also be implemented? |
In theory, sure. Will anybody take it on as a project? Don't know. Probably worth creating a separate issue proposing a Matrix protocol, describing the benefits you see in it, and sketching how the UX might work, given that most Matrix implementations so far work as realtime chat. |
|
Some news and ... well ... scuttlebutt from the world of the fediverse:
There are estimated to be well over 2 million users across all the apps listed on the instance monitoring site at the-federation.info. So if and when all the existing apps federate via AP, and all the newer alpha and beta apps take off, that's quite a network effect. Having a bridge between the AP fediverse and the Scuttle-verse would massively increase the number of people you could communicate with from an SSB client (increasing your network effect), as well as further bolstering the network effective of the fediverse with the addition of SSB users. A classic win-win. EDIT: typos |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
I'm do not think that ActivityPub posts are actually mutable, at least not how they are used presently. Mastodon has a "Delete and redraft" option, but that's an immutable set of actions. |
No, Friendica also runs on Zot protocol, among others. |
The native protocol of Friendica is DFRN. They added support for OStatus and Diaspora some time back, and recently rolled out support for ActivityPub.AFAIK they do not support Zot. If you have any references that show otherwise please share them, and I will make sure their protocol list on fediverse.party gets updated accordingly. |
|
Yes, my bad, you're right, they had some initail plans to implement it in 2013, but currently do not.
Intended for the recipient's use only.
3 Mar 2019, 14:03 by [email protected]:
…
@marek-lach <https://github.com/marek-lach>
>
> Friendica also runs on Zot protocol
>
>
The native protocol of > Friendica is DFRN <https://github.com/friendica/friendica/wiki/Protocol>> . They added support for OStatus and Diaspora some time back, and recently rolled out support for ActivityPub.AFAIK they do not support Zot. If you have any references that show otherwise please share them, and I will make sure their protocol list on fediverse.party gets updated accordingly.
—
You are receiving this because you were mentioned.
Reply to this email directly, > view it on GitHub <#202 (comment)>> , or > mute the thread <https://github.com/notifications/unsubscribe-auth/AryXeROCIoGDQSDp5LNpQbWa3Xz32WNQks5vS8g-gaJpZM4TshUK>> .
|
|
Thanks for the clarification.
|
|
Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward? |
|
I've written up a js Plugin using sbot that currently just provides functionality to save a given AP activity and store it in the ssb log. I will be working on the actual communications with AP servers next. Constructive criticism is always welcome! |
|
Great stuff @DanielMowitz ! I've added a link to the [watchlist of AP
implementations](https://gitlab.com/fediverse/fediverse.gitlab.io/wikis/watchlist-for-activitypub-apps)
I help with for fediverse.party
|
|
Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward? |
|
@DanielMowitz Can you explain in a bit more detail why you worry that allowing an SSB message to go through a pub to an AP instance, or vice-versa, could allow a MitM? If the nuts and bolts of the issue are a bit clearer, I may be able to find someone with the AP knowledge to help you solve it. |
|
Well the problem I see goes as follows: Alice uses SSB and publishes a message. Bob uses an AP service like Mastodon and has followed Alice through a bridging pub. For the message to reach Bob, the message is signed with Alice's key, sent to the pub and somehow converted and finally sent into Bobs inbox. I currently see three Options for said conversion:
All of these points apply to a message send in the other direction, you just have to swap SSB and AP. |
|
Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward? |
|
I recently thought about the issue I outlined above again. One way forward that seems feasible is a combination of 1. and 2.; I think this still isn't a very elegant solution, but it seems to be the best one I'm able to come up with. |
|
@DanielMowitz:
I appreciate you continuing to think about ways of bridging the SSB
Scuttleverse and the ActivityPub Fediverse. But it seems unlikely to me
that most fediverse app developers could cope with adding this to their
workload. Is there a way such a plug-in could instead be included in SSB
pubs and/or AP servers?
This only works for public posts of course
The fediverse doesn't yet have a solution for federation of encrypted
private messages. Does it make the problem any simpler to put that aside
and focus on making public SSB posts available to fediverse users and
vice-versa? Or even to start with making public fediverse posts
available to SSB users?
|
|
Well the message signing is there because we don't want to have to trust the SSB pubs and/or AP servers. In my mind a post with this new approach would look something like this:
or something similar. |
|
AP users already have to trust the AP servers to present AP messages without tampering, so why couldn't the same users trust the same servers with conveying SSB messages without tampering? If you don't want to have to trust a server then AP is the wrong protocol (not saying that's inherently bad) in my understanding. |
|
That is true. I have indeed misremembered how the signing in AP happens (done by the server) so the point about not trusting the server is void. I still see a difference between AP users and SSB bridged accounts: The SSB users might not know that the server is relaying their messages (perhaps maliciously) so where all AP users have the option of no longer posting to a server they no longer trust, bridged SSB messages might still be relayed there by a third party. |
It's theoretically possible for ActivityPub so post content that has been signed by the browser's private key, and not a server's private key, and it would be a fairly small tweak to the spec. We can still let the server do the digital signatures it currently does to without any change to that but additionally tack on a new property for the signature created by the client/browser. |
|
A couple of questions. One, is anyone still working on this? Two, is anyone still doing serious work on SSB itself, or is gradually fading away in favour of Nost? |
|
hi @strypey, i can't speak to SSB <-> ActivityPub, sorry. is a good idea! i can say people are still doing things with SSB:
why would Nostr be considered the alternative to SSB? personally, i'm more interested in projects that sprouted from SSB: 🌱 |
|
Thanks for the info @ahdinosaur.
But for how long? Patchwork, the original flagship app, is abandoned, The Planetary folks are pivoting to Nostr. Now ManyVerse - developed by a passionate champion of SSB and the last usable client I was aware of - are pivoting away to their own protocol. It seems likely to me that these other projects will eventually hit the same brick walls they did.
Two sets of reasons. One, all the reasons @rabble gave for choosing Nostr. Two, critical mass; that's where most of the people interested in P2P social networks seem to be converging.
I agree : ) This has already been done for Nostr via Mostr, while this issue has been open since 2018. I think that speaks volumes about the relative usability of (and interest in) the two protocols from a dev POV. |
|
@ahdinosaur Even a one way bridge that allows people to follow fediverse accounts using SSB clients would be a useful feature. I believe most (if not all) fediverse software publishes feeds as RSS, so code that can ingest RSS feeds and present them as SSB feeds could be a place to start? |
|
My $0.02, as an inconsistent user and occasional bug fixer: Nostr looks to be much simpler than SSB. I don't have any experience actually using it, and that simplicity is part of the reason. Allow me to explain. The appeal of SSB, for me, was for the durability it provides. Every client has the ability to exchange data with every other client (theoretically). There's no distinction between clients and relays. For me, this is a big deal, because in the event of a total communications system collapse (or even just insufficient IP infrastructure such as in remote areas and developing countries) having every node be equal as far as data exchange is huge. As a ham radio operator, I see SSB's greatest strength in a similar way to how I see RACES and our local search-and-rescue group - a way to ensure that communications stay up and are open and free in the face of disruption. And I don't see that philosophy embodied in Nostr nearly as well. The security of it is huge as well. The chain-signing feature, while it does make accounts rather fragile due to feed forking, is amazing for ensuring authentication and data integrity. It's mathematically impossible (okay, technically absurdly heat-death-of-the-universe improbable without quantum computing) to censor or forge without detection, which in the wake of what we've seen from governments around the world over the last few years, is increasingly important in my mind. This is something I don't see implemented in Nostr, and it is a concerning step backward for the utility of a P2P social network system in my mind. It's not as protective of its users or their ability to communicate. The downside to the simplicity vs. utility argument is that SSB as it is right now is really complicated. Between SHS, the way message hashes rely on JS-specific serialization of messages, rooms protocols, etc. it makes it very difficult to build an interoperable system on something other than the original JS codebase. I've considered it and decided it wasn't worth it. Which is I think part of the reason it loses steam so fast with anyone trying to work on the codebase. For me, I consider SSB something to work on when I know I will have a ton of time to devote to it, because even figuring out where you need to modify something to fix a bug might involve going between 16 different projects and filing a pull request to 4 of them that all need to be implemented to fix a bug. There are so many quirks and edge cases that it makes it nearly impenetrable if you don't already have deep knowledge of the design of everything. The modular system's a great idea, but it's really complicated, and even though I've worked on various chunks of the codebase a fair bit, I don't think I could write an SSB stack module without starting by copying someone's else's code as a framework. If I can't do that in the same language, imagine trying to reimplement in another language. There is so much structural code that trying to implement anything functional feels like an exercise in copy-pasting boilerplate. Additionally, I've never been able to put an SSB stack together that works without copying someone else's exact configuration. Very minimal changes to the stack, like which order modules are instantiated in, can have dramatic results and can make or break whether a configuration works at all. It's also quite prone to corruption of local data, forcing a redownload from somewhere else. As I alluded to before, having critically-important modules scattered across tons of different projects and even different development platforms (GitHub vs. GitLab, ssbc vs. critically-important modules housed in personal users' accounts, etc.), with some projects being completely abandoned by their original authors, makes developing really painful. All-in-all, I can't say I blame any developers from taking a break from or walking away from the codebase entirely. Of all the codebases I've worked on, this is one of the most difficult. Another problem I've noticed is less of a development problem and more of a community problem. When you first join the network and start looking through other peoples' posts, the main content of the existing SSB network's community comes off as an incredibly insular echo chamber. There's very little diversity of opinion or thought. For me, it feels like a private club, and if you don't constantly walk on eggshells and spout the party line you're ostracized. For a protocol that was original designed for off-the-grid living and occasional connectivity, there's very little focus anymore on off-grid or developing nations' use cases, with the focus instead being pushing for social causes in fully developed nations with good working internet infrastructure. For some further reading, take a look at ssbc/rooms2#19 which was a very pleasant and civil discussion but shows how quickly the network could devolve into being unusable and pointless for newcomers through "minor" implementation details. And this mentality is pushed forward by things like the disdain for open pubs to try to keep the network invite-only and the work toward TrustNet to make sure you never have to hear an opinion that disagrees with you. Large parts of the community come off as openly-hostile toward taking on new users. That may have been a viable strategy when the internet first started and no one had an already-formed idea of how a social network is supposed to work, but the disconnect between new users' perceptions of actual and expected behavior doesn't make it easy to grow a modern social network's popularity, nor does the closed community encourage open (and civil) discourse with dissenting opinions. Diversity of opinion and thought, and thus escaping the confines of echo chambers and learning to understand each other, is critically important for peace and reducing the division between people. How can you even be so arrogant as to think you could debate someone and possibly convince them to join you, if all you see them as is an unwashed enemy unworthy of your time and consideration and only an obstacle to be blocked to protect your mental utopia? So, is all hope lost? No, I don't think so. The design concepts behind SSB make it an amazing technology, and I'm a huge proponent of it and talk it up to friends and acquaintances a lot. But SSB definitely needs some introspection if it wants to compete. And that seems to be happening with the SSB2/minibutt stuff over at https://github.com/ssbc/ssb2-discussion-forum/issues. And that seems to be the same conclusion that I see expressed in the Manyverse March 2023 update - that it's time to fix the hull of the ship rather than keep rearranging the deck chairs. There are a lot of systemic design flaws that are being worked through over there. The logistics of multi-key users and partial replication are being considered as well as private message deficiencies in the current protocol among other things. There's discussion about logged editing and deletion, which still contain history if asked but don't need to be fully replicated. The ability to have separation between replicating private messages and replicating public ones so data can still be exchanged in encryption-banned environments (like ham radio). The limpaalinks stuff in ssbc/ssb2-discussion-forum#13 is one of the more intriguing ideas I've seen in a while for doing proper validation while only doing partial replication. So to a certain extent, I think the pause you're seeing is actually a sign of health of the community rather than a flaw. It's thinking outside the box, outside the echo chambers, and doing root cause analysis of some of the problems. Learning from the lessons of the existing network and coming back with something even stronger. All in all, IMHO this is an incredibly good thing. |
|
Kia ora @KyleMaas, I had a number of attempts at scaling this wall of text. I copied it to a text file and mucked around with the size and column width. I even started trying to chunk it into smaller paragraphs so I could parse it. I tried every way I could think of to free solo this one, but I fell off every single time. I have some suggestions that might add some safety ropes to the wall;
If you're willing to do this, maybe add a new comment with the TL;DR. That way I'll get an email, and I'll get out my climbing harness and have another go :) |
|
TL;DR: |
|
Thanks @KyleMaas. I'm very aware that we're doing that thing where a GH Issues drift into general discussion. So let's bring it back to the topic of this Issue, which is how to bridge the Scuttleverse and the fediverse. Do you have any thoughts on how we might achieve that, for example any responses to the ideas I sketched here? |
|
@strypey To be honest, I think you'd have an easier time with a bridging pub - namely, a pub plugin like @soapdog mentioned in #202 (comment) There's too much core functionality that relies on very strict semantics of the SSB protocol and breaks down entirely if it isn't exactly as it expects. It's not flexible enough to ingest, for example, editable/deletable messages, and without a strict signed always-incrementing integer sequence number. |
So it seems that Nostr is ...
... while SSB is neither as simple nor as feature-complete as ActivityPub. Will SSB-2 be better in this regard? Would it be more viable to focus on a bridge between that and the AP network? |
|
@strypey : SSB, Nostr, and ActivityPub all have different trade-offs and affordances, the features between these protocols is not as simple as one has more than the other, both SSB and ActivityPub have features the other doesn't. for example, SSB is designed to work even if the internet collapses (or if we have 20-minute latency to Mars), the messages can gossip through any means available (e.g. carrier pigeon, roaming vagabonds). (but no question ActivityPub has features that are more useful to the average person, ActivityPub is the best of existing centralized social networks, Scuttlebutt is mad science of what's possible with peer-to-peer networks and cryptography.) at the core, these protocols have different "messages" to represent the same activity, different "envelopes" that contain the "messages" that represent the same activity, different "transports" to send/receive the "envelopes" that contain the "messages" that represent the same activity. the main barrier for SSB being interoperable with ActivityPub is the "envelope" (that contains the message that represents the activity). in SSB, each "envelope" (SSB message) is part of an append-only linked list of signed messages, references are based on the hash of the content, and content is inline. there's many side effects of this design, some good some bad. this is why deletes are impossible in SSB, to delete a message would break your ability to validate the linked list, and deleting only the inline content would break the hash of the content. ActivityPub is not designed this way. so trying to do a 1-to-1 mapping of SSB to ActivityPub is not possible (as others have mentioned), it's not possible to put a square peg in a round hole, at best you can do an ActivityPub-compatible mirror of SSB feeds. Scuttlebutt is almost 10-years-old now, so since then we've learned (and other projects have learned from Scuttlebutt) that this is a mistake! (and many other such mistakes) so successors to Scuttlebutt (like "PPPPP" or p2panda) don't inline the content in the message, and instead use a hash reference to the content, so you're able to delete the content without affecting the linked list. these successors also have fancy ways of representing chains of messages that are verifiable up to the very first message, without being a linear linked list, so you can do "partial replication" (securely download new messages without downloading all messages). so yes, successors to SSB will be more viable to bridge with ActivityPub. but there will still be things that p2p protocols have to worry about that federated-server protocols like ActivityPub don't have to worry about. so interoperability will never be a direct 1-to-1 mapping. at best, i think p2p protocols should re-use the "message" formats (the way of representing social network activity) as ActivityPub, even if the "envelopes" and "transports" are very different in p2p protocols. but at the same time, whatever motivates unpaid volunteer open source developers to work on their passions, that's what they're going to focus on. i think the motivation for ActivityPub is different than the motivation for SSB (and successors), so i'm not surprised if their focus will be different. (e.g. anyways, i hope that helps. 💜 |
|
FWIW...
I don't think that word, "centralized", means what you think it means. In the classic Paul Baran network topology, a "centralized" network is one where all nodes connect directly to one central node. AP, like matrix, is "decentralized", in contrast with SSB and Nostr, which are "distributed". I find it more useful to say that anything that isn't centralised is decentralised, and break that down into federated (eg email, XMPP, AP, matrix), distributed or peer-to-peer (SSB, Jami, Tox, Nostr), and hybrid (eg BitTorrent depends on tracker and search servers, P2P matrix etc). |
Excellent. Is there a better repo than this to start an issue in, in case anyone is keen to have a go?
Sounds sensible. Would that mean P2P social networks using the message types in the ActivityStreams standard ? |
yes i know, sorry words. i meant to emphasize the similarity between centralized social networks like Twitter and federated social networks like Mastodon. you still have a web page or app that connects to remote server (in my case cloudisland.nz), the client asks for data from the remote server (and assumes it to be true), and the remote server stores data in a mutable database. i think the gap between Twitter and Mastodon is less than the gap between Mastodon and Scuttlebutt, which leads to confusion because "how hard could it be".
https://github.com/ssbc/ssb2-discussion-forum
yes. been brought up before:
(just to name a few) but Scuttlebutt is a do-ocracy. having ideas is easy, implementing things is hard, and maintaining things is even harder. ⛵ |
Ah, OK. In which case you're looking for a phrase like "web-based" or "server-based" : )
Understood. Sometimes a developer is willing to work on something, but wants to be sure someone will use it. Being a squeaky wheel is something that's within my skills to do. Sharing my limited knowledge of AP and fediverse HX (Human eXperience) too. By commenting here, I'm also implicitly volunteering to do alpha and beta testing of any attempted implementations. |
|
I think re-using ActivityPub data structures is a good idea, that would definitely help with interop. Also, ActivityPub protocol can be extended to be more P2P-like. I wrote a technical proposal that describes client-side activity signing: https://codeberg.org/fediverse/fep/src/branch/main/fep/ae97/fep-ae97.md. The proposal also suggests a way to remove DNS/HTTPS dependency (by switching to server-independent identifiers). |
First thing, this may be a totally inappropriate place to discuss this. If so, please feel free to tell me so, and direct me to a more appropriate forum for discussion. Please don't ask me to discuss it on Scuttlebutt though, for three reasons;
I have a 32-bit PC (running GNU-Linux) so AFAIK there isn't an SSB client I can use yet. At least not without compiling from source and potentially spending hours in dependency hell every time there is an update. No thanks ;-P
I think it's worth having a dedicated channel for dev-related discussion accessible to folks who aren't yet SSB users, as this may be a way to resolve issues (or clear up misunderstandings) that are preventing them using SSB (eg there is a client with a 32-bit release, I just don't know about it)
I think too much meta-discussion about SSB dev on the SSB network is off-putting to non-dev users. I've been told as much in the fediverse by former SSB users.
Anyway, to biz. I'd love to see the network effect of the Scuttle-verse combine forces with those of the fediverse. If I could move to Scuttlebutt, and still keep up contact with my followees/ followers on sites running GNU Social, Mastodon, Friendica etc, without having to maintain two social network presences, that would make using an SSB client much more attractive to me. It would also open up a much larger range of potential friend connection to people already using SSB.
The quickest way to do that would be to create some kind of integration with ActivityPub. AP is far from perfect, but except for Diaspora, all of the apps currently making up the OStatus fediverse and the Diaspora federation are going to be rolling it out (or have already). Plus a range of other (micro-)blogging, file-storage, media-sharing, and code forge apps have plans to implement it too. AP has both a server-to-server and server-to-client protocol suites in the standard, so the fact that SSB is P2P not server-based isn't necessarily a show-stopper.
I also suggest taking a look at whether you could inter-operate with Zot somehow. It's a ground-breaking protocol that allows cool things like Nomadic Identity and secure, decentralized, private messaging. Hubzilla is currently the only app that implements it, but the process of creating a separate implementation, especially one that isn't a GLAMP stack (Hubzilla is PHP and hubs usually use MySQL), could really help other devs get their heads around it.
The text was updated successfully, but these errors were encountered: