A Proposal for an RSS Based Simple Web Services Architecture for the Future

A Proposal for an RSS Based Simple Web Services Architecture for the Future

I recently came in much closer contact with the world of RSS (Real Simple Syndication) as a result of establishing my investment board at the TheMarketTraders.

I had used RSS news feeds for a while as readers like Omea allowed me to effectively filter large bodies of news into smaller, manageable sets of items that I could analyze and use in my investment pursuits.

The TMT message board system supported RSS (Phorum based) supports RSS in some rudimentary form. However, it didn’t sport a number of desired features the most prominent being that it lacked topic replies in its feeds (tough to read boards if responses are omitted).

As I didn’t control the software nor the server, I had find creative ways around this limitation and in my quest came across Feedity, RapidFeeds, FeedBurner, many others and finally a public infrastructure enabling Yahoo Pipes which had just entered beta. By combining Yahoo Pipes and Feedity I was able to achieve a mashup pipe that contained replies (though only headers) as well as the original posts.

 
However, I now had gotten excited and started to really comprehend the full potential of RSS in a connected world.  While debugging and working around bugs in Yahoo Pipes, I came across a post talking about RSSBus.  I went to the site and read their architecture white paper.  I loved their approach and architecture though I felt it needed expansion to accommodate mission critical applications and the full inclusion of feed discovery and classification.

 
What I would like to see is an Open Source, fully scriptable implementation of this concept.  I believe that it could lead to an exceptionally fast evolving web due to its simplicity and extensibility.

In the process of evaluating various Web based RSS readers, I came across WizzRSS, a Firefox based reader and started to communicate with mikek123.  The conversation forced me to formalize my thoughts and this post is the result. 

Any comments are greatly appreciated.

Here is a simple diagram of what I envision (click on image to expand):

<>

Edited 7 time(s). Last edit at 03/11/2007 12:36PM by regli.

I had some more communications via email on the subject and I decided to just copy some of that into responses.  It may help clarify some of the complexities/confusion when looking at the diagram. :) 

--------------------------------------------------------------------------------------
I apologize for not elaborating much on the diagram.  I know it looks cluttered and isn't very clear.  However, it really helped me clarify my thoughts.

I didn't mean to imply to one should rely on Pipes or RSSBus.  These implementations brought home to me the larger picture of what could and should be done with RSS as a new Simple Web Services Architecture.

It was just a quicky attempt to describe a framework that would allow for an easy and organic growth of web services.

The license issues IMO are not a big concern as I am not advocating even the usage of RSSBus though I wanted to give them credit for helping structure my thoughts.  As mentioned in my initial note, I hope that the framework would be provided and expanded on by the Open Source community. 

Even while developing an RSS reader, I believe that I would follow this architectural concept. Note that I added a routing function after the operation box (6.0).  I will upload the new diagram sometime today.

Even from purely a reader context, IMO there still should be a separation of processes issue,  i.e. the Feed Connection Processor (3.0) is invoked as a result of an RSS subscription action to determine if the feed is a browser redenderable RSS stream and if so then the Feed/Event Router (5.0) should route the connected feed to the Feed Dependent Processor (4.0).  In the case of a reader IMO this should be a background database update/maintenance process which does all the feed indexing and categorizing (RegEx?), etc.

An important aspect is to think of the stream not only as data stream but just as much as a stream of events.

If one takes as an example an "active and self directed" reader then the "renderer", i.e. reader could be started when there is something to read (a new Firefox renderable item) arrives in a feed that is so designated (a calendar item or an important finance news story?).  Think of a RegEx scan for particular stock ticker or a scan for words like "emergency" AND "event", etc.

I hope this sheds a little more light on the ideas well hidden in the diagram... :)

Here is another email:

The thing to me that is somewhat revolutionary is that RSS is a high-level protocol that could potentially allow user determined merging of streams or feeds.  Presently it is primarily limited to  offline feed  merges (mashups).  However, I believe that real-time merging will really revolutionize the Web.  

I should elaborate in the post and add an example to make it clearer:

Imagine a scenario where you have a collaboration session watching a football game.

You first subscribe to each voice feed of your friends.

All participants then subscribe to the video of the football game.

Each of the participants can also present their strategy either overlaid directly on the game video or in a separate window as the game proceeds.  The strategy feed is time synced with the video.

A user subscribes to a renderer by identifying how he/she wants to view that particular feed as it may be rendered in different forms. Note that the renderer router should allow the process output to be rendered  in multiple forms and to multiple end points.

As a consumer of feeds, one should be able to view these feeds in any way one desires.  Take as an example a simple voice conversation. Let's call it a feed from you to the other person and another feed from the other person to you, VoIP packets encapsulated in a feed.

Let's say the other person is deaf.  He/she would subscribe to the audio feed which through a voice recognition operation would be converted to text and then rendered as a text feed in a Firefox window.

If the person were deaf and blind, this text feed would enter another operation which would convert it into Braille and render it to a Braille display device.

By using RSS feeds as generalized communication streams, users could with the aid of an array of loosely coupled simple web processes/services and some standardized feed/stream rules start to really control the way we communicate and collaborate on the Web.

I hope this illuminates a bit what I was trying to portray in the diagram.

Edited 2 time(s). Last edit at 03/13/2007 02:07AM by regli.