Twitter Federation

Dan Wineman lights up the federation crew by asking for a federated Twitter.  Dan outlines the benefits as well as the problems when it comes to a decentralized, federated, real-time, social communications app.

"I’d like to see App.net move toward a federated architecture. Broadly, what that means is that instead of being a central service that each client connects to directly, it would become a loosely organized mesh of independently controlled nodes. Users and devices would connect to whatever node they liked best — you can run your own if you want — and the nodes would talk to each other in some clever way to collectively maintain the appearance of a single unified social network.

The advantages are numerous and comparable to those of the web itself: no single point of failure, no concentration of power, no risk that the entire network will be sold to Facebook.

But does this work for a service like Twitter? Can the behavior we’ve come to expect from social networks be reproduced in this model?

Here’s a list of three constraints we’ve come to expect of our social timelines:

• Immediacy: if a post has been made by someone I follow, I can see it in my timeline right away (or close enough that I don’t notice the difference).
• Chronology: posts always appear in order by time posted.
• Monotonicity: timelines grow only from the top; older posts are never retroactively inserted.

The problem appears to be that no federated architecture can simultaneously satisfy all three of these conditions. You can have any two: for example, if you let go of immediacy, your node can just wait until it’s received the latest content from every other node before displaying anything. But that’s not very scalable, and it makes real-time conversation impossible, so let’s keep immediacy. Now we have to decide what to do when content from a far-away node arrives late: if we’ve already displayed newer posts, we have to violate either chronology (by posting the older content above the newer) or monotonicity (by inserting it chronologically into the timeline).

The moral of the story is that the qualities that make Twitter interesting — its mix of conversation, discovery, and one-to-many communication — are direct consequences of its centralized architecture. Without the centralization you can still have something interesting, but it’s a different thing.

I’d love to be proven wrong."

A few of the comments related to Dan's post include:

  Kevin Marks

You're assuming a single conversational thread, but the key value of twitter is the semi-overlapping publics that mean we don't have to read all responses to a post in order, just those from people we choose to follow. Chronology is violated by Twitter, explictly by retweets and quoting. They favour monotonicity instead, but this does lead to repetition. 

Monotonicity is a display choice at the client.
See http://j.mp/twittertheory for more on semi-overlapping publics.
See SWAT0 for discussions fo ow to test and achieve this: http://www.w3.org/2005/Incubat...

• Josh Triplett

  First of all, it seems odd to have a post about federated Twitter without mentioning OStatus, the concrete implementation of federated Twitter that already exists and works.  It implements pretty much everything about Twitter that you'd care about, except for the userbase, which seems like a given for any federated microblogging implementation given that Twitter doesn't allow or support federation themselves.

  Second, these criteria are not binary.  With a decently functioning network, you can have enough immediacy to expect messages within a minute or so, which means you can expect enough monotonicity to only look for new messages within the most recent minute or so of your timeline, a region that normally fits on your screen without scrolling unless you're insanely popular (and thus probably not reading every message anyway).

  • dwineman

    I probably should have mentioned OStatus, but it seems to have different goals. Does it satisfy the constraints I listed, or make them irrelevant somehow?

    And you're right, the constraints have tolerances, and in the absence of multi-party conversations (or to someone catching up a few minutes later) they don't really matter at all.

Chris Saad

So tired of this conversation. Blogging type software with 140 character constraint + PubSubHub + An aggregator = Distributed Microblogging/Twitter service. It's easy.