All Things Distributed

Wired on the Scalability of Feed Aggregators

It seems that people finally start to 'get it' (see wired article); that the current approach of continuous feed polling, will not scale. I have explained this in detail before, see for example 'On the Scalability of Feeds and Aggregators', or as early as two years ago (see Wes Felter's report from the Internet2 meeting).

It is cool to see that the solutions people are now willing to discuss (p2p or bittorrent style) are the ones that I have been advocating for a number of years, and that have found their implementation in Newswire. We have had some legal issues with the deployment of the newswire beta, and a new core communication component is being developed. But I would be happy to break open the ideas in the system and work with others to see whether this can be a good experimentation platform.