GwenStarlight

Producing lesbian demons since 1993

  • She/her They/them

TMA, multiply neurodivergent, ancient by internet standards, poylam and happily married.

I was on Cohost from 11/04/2022 until its last day (10/01/2024)


DecayWTF
@DecayWTF

So what would a good federated protocol look like?


To start with, the open model the fediverse uses is garbage and we could have learned that from the early days of IRC, when they went through a whole lot of pain and had to develop a whole system (Q lines) to finally lock the network down to authorized servers. EFnet was the product of a solution to the problem of malicious nodes and uncontrolled peering, a problem the Fediverse can't solve (in fact, or is considered a feature) because there's no structure to the network and so malicious nodes need to be defederated at the individual node level, one at a time, in an unending game of wack-a-mole.

Beyond that, we find that the structureless nature of the Fediverse also means it is necessarily noisy. There's no mechanism to control delivery or create backbone sites capable of carrying larger amounts of traffic, so every single node is at constant risk of being DDOSed by the rest of the Fediverse; all it takes is one person on a small node to become popular (or targeted by bad actors) and that node will be blasted with up to the entire global traffic of the network. The protocol spec doesn't even admit this is a potential problem, in spite of being designed years after bounce attacks were regularly used in this exact manner to attack mail servers, as it doesn't specify any mechanism for batching messages to users even on the same node (which was eventually added as an extension).

There are a lot of benefits to a relay (IRC) or store-and-forward (email, Usenet) model that just cannot be realized in any way in a purely many-to-many model like the Fediverse. The problem of nick authentication: Solved decades ago on Dalnet, courtesy of NICKSERV. A similar model with human-in-the-loop identity verification could solve the blue checkmark problem. The problem of broadcast blasts: Leaf nodes only ever need to send one copy of any sent message upstream, with a list of nodes - not individual users! - that should be the final recipients of the message. Each node needs to know which users their users follow (to properly route received messages) and which follow their users (to properly route outbound messages). Intermediate relay servers don't need either piece of information.

Let's look at a pathological case, a user on a leaf node who is followed by every user on the network. On the current Fediverse, the user's node would have to send at minimum one message to every single fediverse server every time they posted. In our relay system, that node only ever has to transmit a single message, marked for upstream with a list of all destination servers. This is still far from ideal but is a vast improvement on the current situation; IRC uses channels to avoid having to slop around massive lists of users and we could do something similar where intermediate servers update their routing tables for messages to given groups. Each relay then only has to transmit to their connected routes leading to the destination servers for the messages they receive. The inverse problem (someone who follows everyone on the network, let's call them Ollie) is not improved against the current state for that particular node, but is for most other nodes on the network, who don't have to add additional server connections to Ollie's server just for them. Furthermore, with a little thought we could implement a message batching mechanism, where upstream servers combine messages for a designated endpoint into a single delivery, that would alleviate that situation as well, and would also keep Ollie from being as much of a burden on their own server because there would be fewer connections.

It should be obvious at this point that many of the technical deficiencies of ActivityPub are resolved (or at least resolvable) on a structured relay network in a way they just aren't in the current regime. What doesn't it solve and what new problems would it introduce?

Well, it doesn't solve the problem of multiple accounts or needing to choose a home server. I am not convinced that these are, in the main, real problems. People regularly deal with qualified names (Decay#1234 vs Decay#42069 on Discord for instance, or email addresses, or, well, people with physical addresses) and multi server systems like email; Discord, again, even pretends to a federated server model deliberately. The biggest issue there is not choosing a server but migration. A structured peering model introduces options to handle that transparently, by returning forwarding records that the other servers can process directly, and a purely backend mechanism for migration based on mutual confirmation (Mastodon has implemented a mechanism along these lines but it's awkward and frictional, and because it's a protocol extension it's only supported between Mastodon servers, no other implementations).

The other problem, such as it is, is governance. The major nominal "feature" of the Fediverse is the structurelessness; anyone can set up a server and talk to anyone else. This is clearly a misfeature at best, and large chunks of the Fediverse (the Mastodon section) largely have a very ad hoc shared governance anyway mostly revolving around pretending social and governance issues are technical ones and trying to deal with them as github issues, or are dealt with by general consent because everyone who isn't a nazi agrees that eg. Gab should be defederated, which then still has to be done via wack-a-mole. A relay or store and forward network does give some amount of potential control to the operators of the backbone links. Currently, however, the same thing applies except that is the operators of the largest servers, and while peering can be changed and backbone servers routed around the same can't be said of large nodes, because they still have the users, so this at very least doesn't worsen that particular issue but makes some related ones easier to solve, and with migration much easier there should be less weight on the larger servers regardless.


You must log in to comment.

in reply to @DecayWTF's post:

the informal DDOS aspects here are especially interesting. a known problem we have is that we don’t have a fast path for opengraph scrapers (legitimately difficult to get the list of crawler useragents) and while all good implementations cache, the distributed nature of mastodon means that if someone shares a cohost link on mastodon, we immediately get DDOS’d by every single instance that saw that post.

during heavy load periods, we have to block traffic from mastodon user agents to prevent even further load. making opegraph faster is obvs a good solution here, the actual design of activitypub creates an issue for sites that don’t even use it!

Good god, I hadn't thought about that but yeah, that's a nightmare too. I can... understand why that case wasn't considered in the design of the protocol but it's more evidence that ActivityPub really just isn't (in @atomicthumbs's words) fit for purpose at all because like the bounce attacks large networks DDOSing smaller ones by automated requests has been a known issue for ages. Opengraph requests should never have been implemented without shared caching in the first place and it's active malpractice that they were.

yep!! it's kind of a mess!! although it did occur to me while i was writing the first comment that we could have a small list of user agents we actually cache dynamic pages for, which was easy to implement this morning and seems to be working, which is good! rubber duck debugging via comments.