Matt Tucker, CTO at Jive Software
"Since the beginning of the Internet, if you wanted to sync services between two servers the most common solution was to have the client ping the host at regular intervals, which his known as polling. Polling is how most of us check our email. We ping our email server every few minutes to see if we got new mail. It's also how nearly all web services APIs work.
Take, for example, Twitter. High Scalability recently covered the load stats on Twitter reporting that they average 200-300 connections per second with spikes that climb to 800 connections per second. Their MySQL server handles 2,400 requests *per second*! Recently, the Macworld keynote became the most recent culprit for causing Twitter to cut off its API, which has 10x the load of their website. While Twitter is not a cloud service, nor the largest demand service on the internet (with a paltry 350,000ish users, which pales in comparison to a MySpace or Yahoo!), they do illustrate the kind of frustration a user experiences with polling based services. And, that's just Twitter! Imagine the impact on overall Internet traffic congestion polling creates worldwide.
Interestingly, the recent Twitter outage lead some influencers, like Dave Winer, to suggest that Twitter move to XMPP which we've already begun experimenting with".XMPP was invented for instant messaging and presence, and is the dominant open protocol in that space. Instant messaging? Yep, it turns out that all of the problems that had to be solved for instant messaging make the protocol perfect for cloud computing:
- It allows for easy two-way communication, so bye bye polling. It even has rich pub-sub functionality built-in.
- It's XML-based and easily extensible, perfect for both new instant messaging features and custom cloud services.
- It's efficient and proven to scale to millions of concurrent users on a single service (such as Google's GTalk). It also has a built-in worldwide federation model.
I'm not the only one to notice that XMPP is a great fit for cloud computing. Tivo is switching to XMPP as a more efficient alternative to their old architecture:
"Today each TiVo polls TiVo’s severs roughly every 15 minutes to check for new scheduled recordings, TiVoCast downloads, Unbox downloads, etc. That’s highly inefficient - nearly all of those polling calls are for nothing. There is nothing waiting to be done. And it introduces a lag when you want to start a download - up to 15 minutes. And it doesn’t scale well as TiVo’s user base keeps growing.
So what’s changed? The polling system is gone. TiVo is using XMPP now instead. (...) Yep, TiVo is basically using instant messaging for real- time communication. Now when the TiVo server has a new recording to schedule, it will IM the TiVo to tell it. Or if there is a download to pull, it will IM the TiVo to tell it to do so. This is a much more efficient system and it eliminates latency. It is really a clever idea."
Marshall Kirkpatrick on the evolution of XMPP
"What Could a Future Built on Jabber Look Like?
Ask yourself what a decentralized, open source infrastructure for real time communication could offer. A lot. As an RSS-head, I'd love to see XMPP let my various RSS clients do more faster and get bogged down in fewer unnecessary activities. RSS is all about speed for me but clients can only do so much so often when they have to pester someone else's server every time they want to check for new information. Those delays can be of real consequence.
The Tivo example is just one of many possibilities in the realm of machine-to-machine XMPP communication. Automated alerts are useful in all kinds of more serious use-cases and why use polling to monitor changed conditions when you can use an open source protocol that can make presence and real time communication scalable?"
Comments