apache apachecon apacheroller app apple asf atom atomprotocol atompub barcamprdu blogapps blogging businessblogging conferences family feeds foss general glassfish google humor ibm java javaone links linux mac microsoft movies music netbeans opensocial opensource photos politics rest roller rome rss socialnetworking socialsite socialsoftware sun triangle trianglebloggers vacation webdev webservices wiki


Main | Next day (Aug 9, 2004) »

Sunday Aug 08, 2004

How Rome works.

Comments[6] by Dave in Blogging

I spent some time exploring the new Rome feed parser for Java and trying to understand how it works. Along the way, I put together the following class diagram and notes on the parsing process. I provide some pointers into the Rome 0.3 Javadocs, but, because this summary is based on the latest Rome codebase from CVS you will notice that some class and interface names have changed.

You don't need to know this stuff to use Rome, but it you are interested in internals you might find it interesting.

Notes on the Rome parsing process

Rome is based around an idealized and abstract model of a Newsfeed or "Syndication Feed." Rome can parse any format of Newsfeed, including RSS variants and Atom, into this model. Rome can convert from model representation to any of the same Newfeed output formats.

Internally, Rome defines intermediate object models for specific Newsfeed formats, or "Wire Feed" formats, including both Atom and all RSS variants. For each format, there is a separate JDOM based parser class that parses XML into an intermediate model. Rome provides "converters" to convert between the intermediate Wire Feed models and the idealized Syndication Feed model.

Rome makes no attempt at Pilgrim-style liberal XML parsing. If a Newsfeed is not valid XML, then Rome will fail. Perhaps, as Kevin Burton suggests, parsing errors in Newsfeeds can and should be corrected. Kevin suggests that, when the parse fails, you can correct the problem and parse again. (BTW, I have some sample code that shows how to do this, but it only works with Xerces - Crimsom's SAXParserException does not have reliable error line and column numbers.)

Here is what happens during Rome Newsfeed parsing:

diagram of Rome classes and the parsing process.
  1. Your code calls SyndFeedInput to parse a Newsfeed, for example (see also Using Rome to read a syndication feed): 
    2. URL feedUrl = new URL("file:blogging-roller.rss");
SyndFeedInput input = new SyndFeedInput();
SyndFeed feed = input.build(new InputStreamReader(feedUrl.openStream()));
  2. SyndFeedInput delegates to WireFeedInput to do the actual parsing.
  3. WireFeedInput uses a PluginManager of class FeedParsers to pick the right parser to use to parse the feed and then calls that parser to parse the Newsfeed.
  4. The appropriate parser parses the Newsfeed parses the feed, using JDom, into a WireFeed. If the Newsfeed is in an RSS format, the the WireFeed is of class Channel and contains Items, Clouds, and other RSS things from the com.sun.syndication.feed.rss package. Or, on the other hand, if the Newsfeed is in Atom format, then the WireFeed is of class Feed from the com.sun.syndication.atom package. In the end, WireFeedInput returns a WireFeed.
  5. SyndFeedInput uses the returned WireFeedInput to create a SyndFeedImpl. Which implements SyndFeed. SyndFeed is an interface, the root of an abstraction that represents a format independent Newsfeed.
  6. SyndFeedImpl uses a Converter to convert between the format specific WireFeed representation and a format-independent SyndFeed.
  7. SyndFeedInput returns to you a SyndFeed containing the parsed Newsfeed.
Other Rome features

Rome supports Newsfeed extension modules for all formats that also support modules: RSS 1.0, RSS 2.0, and Atom. Standard modules such as Dublic Core and Syndication are supported and you can define your own custom modules too.

Rome also supports Newsfeed output and for each Newsfeed format provides a "generator" class that can take a Syndication Feed model and produce from it Newsfeed XML.

Learning more

I've linked to a number of the Rome 0.3 Tutorials, here is the full list from the Rome Wiki:

  1. Using Rome to read a syndication feed
  2. Using Rome to convert a syndication feed from one type to another
  3. Using Rome to aggregate many syndication feeds into a single one
  4. Using Rome to create and write a feed
  5. Defining a Custom Module bean, parser and generator
Conclusion

Overall, Rome looks really good. It is obvious that a lot of thought has gone into design and a lot of work has been done on implementation (and docs). Rome is well on the way to "ending syndication feed confusion by supporting all of 'em" for us Java heads.

Please leave a comment if I have gotten something wrong.

This work is licensed under a Creative Commons License.
Copyright 2002-2007, David M Johnson (dave.johnson at rollerweblogger.org)

This is a personal weblog, I do not speak for my employer.