Parsing a Flickr feed with SimpleXML

Parsing a Flickr RSS feed requires some preparatory steps. First of all, I tried to download a static copy of my feeds using wget just to study the file structure. Wrong choice! In fact, the format returned was Atom, not RSS. So I used the following approach:

$file = file_get_contents('http://api.flickr.com/services/feeds/photos_public.gne?id=31968388@N02&lang=it-it&format=rss_200');
echo $file;

So I got the correct format. Parsing the feed is quite a simple task:

$rss = simplexml_load_file('http://api.flickr.com/services/feeds/photos_public.gne?id=31968388@N02&lang=it-it&format=rss_200');
    
    foreach($rss->channel->item as $entry) {
    
    
    
        $title = $entry->title;
        $raw_published = $entry->pubDate;
        $published = str_replace('-0700', '', $raw_published);
        $raw_content = $entry->description;
        $content = html_entity_decode($raw_content, UTF-8);
        
        
        echo '<li>' . "\n" . '<h2>' . $title . '</h2>' . "\n" . '<p class="pubdate">' . $published . "</p>\n" . 
        $content . "</li>\n";
    
    
    
    
    }

Two notes here:

  1. you need to start parsing from the root element, not from the SimpleXMLObject itself
  2. the description element contains markup that needs to be expanded using the html_entity_decode() function

You can see this test here.

Parsing a FeedBurner feed with jQuery

Parsing a FeedBurner feed with jQuery requires two components:

  1. a proxy to fetch the feed
  2. Ajax to parse the feed

Here's the proxy, written in PHP:

<?php
header('Content-Type: text/xml');

$feed = file_get_contents('http://feeds.feedburner.com/blogspot/onwebdev/');

echo $feed;

?>

Now we can add jQuery:

$(document).ready(function() {
    
        $('<ul></ul>').insertAfter('h1');
    
    
        $.get('proxy.php', function(data) {
        
        
            var $items = $(data).find('item');
            
            $items.each(function() {
            
                var html = '';
            
                var $item = $(this);
                
                var $title = $item.find('title').text();
                var raw_author = $item.find('author').text();
                
                var $author = raw_author.replace('gabriele.romanato@gmail.com', '').
                              replace('(', '').replace(')', '');
                              
                var $link = $item.find('author').next().next().text();
                
                var raw_date = $item.find('pubDate').text();
                var $pubdate = raw_date.replace('+0000', '');
                
                
                html += '<li><a href="' + $link + '">' + $title + '</a>' + '<div class="author">' + $author + '</div>' + 
        '<div class="pubdate">' + $pubdate + '</div>' + "</li>";

                $(html).appendTo('ul');
            
            });
        
        
        
        });
    
    
    });

There's only a difficult part here: selecting the feedburner:origLink element which has a namespace. To accomplish this task, we've used the next() method which returns the adjacent sibling of a given element. You can see the final result here.

Parsing a FeedBurner feed with SimpleXML

Parsing a FeedBurner feed with SimpleXML requires only a single gotcha: the actual parsing starts from the root element, not from the whole object created with the simplexml_load_file() function. For example, the following code returns nothing:

$feed = simplexml_load_file('http://feeds.feedburner.com/blogspot/onwebdev/');

foreach($feed->item as $item) {

    //... nothing here

}

Instead, the following code works as expected:

 $feed = simplexml_load_file('http://feeds.feedburner.com/blogspot/onwebdev/');
    
    foreach($feed->channel->item as $item) {
    
        $title = $item->title;
        $raw_author = $item->author;
        $author = str_replace('gabriele.romanato@gmail.com', '', $raw_author);
        $author = str_replace('(', '', $author);
        $author = str_replace(')', '', $author);
        $links = $item->children('http://rssnamespace.org/feedburner/ext/1.0');
        $link = $links->origLink;
        $raw_date = $item->pubDate;
        $pubdate = str_replace('+0000', '', $raw_date);
        
        echo '<li><a href="' . $link . '">' . $title . '</a>' . "\n" . '<div class="author">' . $author . '</div>' . "\n" .
        '<div class="pubdate">' . $pubdate . '</div>' . "</li>\n";
    
    
    }

You can notice that now the parsing starts from the channel element. You can see the final result here.

Parsing CSS with jQuery and JSS

JSS is a powerful jQuery plugin written by Andy Kent that serves as a really handy CSS parser in many contexts. Since I'm still looking for a good JavaScript CSS parser that honors the Flex syntax used by the W3C specifications, I've found this plugin really useful for learning something new. For example, all parsing is handled by four methods, sanitize(), parse(), parseSelectorName() and parseAttributes(). They are as follows:

    // ---
 // ultra lightweight CSS parser, only works with 100% valid css files, no support for hacks etc.
 // ---
 
 sanitize: function(content) {
  if(!content) return '';
  var c = content.replace(/[\n\r]/gi,''); // remove newlines
  c = c.replace(/\/\*.+?\*\//gi,''); // remove comments
  return c;
 },
 
 parse: function(content){
  var c = this.sanitize(content);
  var tree = []; // this is the css tree that is built up
  c = c.match(/.+?\{.+?\}/gi); // seperate out selectors
  if(!c) return [];
  for(var i=0;i<c.length;i++) // loop through the selectors & parse the attributes
   if(c[i]) 
    tree.push( { selector: this.parseSelectorName(c[i]),  attributes: this.parseAttributes(c[i]) } );
  return tree;
 },
 
 parseSelectorName: function(content){
  return $.trim(content.match(/^.+?\{/)[0].replace('{','')); // extract the selector
 },
 
 parseAttributes: function(content){
  var attributes = {};
  c = content.match(/\{.+?\}/)[0].replace(/[\{\}]/g,'').split(';').slice(0,-1);
  for(var i=0;i<c.length; i++){
   if(c[i]){
    c[i] = c[i].split(':');
    attributes[$.trim(c[i][0])] = $.trim(c[i][1]);
   }; 
  };
  return attributes;
 }
  1. sanitize() - all the work here is done by the two regular expressions contained in the c variable; this method removes new lines and CSS comments
  2. parse() - this method first calls sanitize() to get a cleaned CSS input stream, then separates out selectors with a regular expression, and eventually loop through selectors and parses attributes
  3. parseSelectorName() - extracts the selector from a CSS rule by using a regular expression with the match function which returns an array of matches
  4. parseAttributes() - returns an array containing all attributes found in a CSS rule.

This is a string-based parsing with regular expressions, an approach that I followed myself a couple of months ago. Honestly, I didn't find the correct regular expressions, though. I wonder if it's possible to use a different approach, for example a stream-based approach like the JSON parsing used by Douglas Crockford in its json.js library. When I have more spare time, I'll surely make some tests.

Parsing RSS feeds with the DOM and JavaScript

Parsing RSS feeds with the traditional DOM approach is not the simplest way to perform this task. However, if you want to get a finer control over the whole process, this may be a feasible way. Here's the basic code to achieve this goal:

function XMLDoc() {
 var me = this;
 var req = null;
 if (window.XMLHttpRequest) {
  req = new XMLHttpRequest();
 }
 
 else if (window.ActiveXObject) {
  try {
   req = new ActiveXObject("MSXML2.XMLHttp.6.0");
  }
  catch(e) {
   try {
    req = new ActiveXObject("MSXML2.XMLHttp.3.0");
   }
  catch(e) {
   req = null;
  }
  
  }
 }
 
 
 this.request = req;
 this.loadXMLDoc = function (url, handler) {
  if (this.request) {
   this.request.open ("GET", url, true);
   this.request.onreadystatechange = function () {
    handler(me);
   };
   this.request.setRequestHeader("Content-Type", "text/xml");
   this.request.send(null);
  }
 };
}

function initXML () {
 var newrequest = new XMLDoc();
 newrequest.loadXMLDoc("rss.xml", getRSS);
}

function getRSS (req) {
 req = req.request;
 var content = document.getElementById("content");
 var div = document.createElement("div");
 div.className = "entries";
 var h3 = document.createElement("h3");
 h3.innerHTML = "Recent entries";
 div.appendChild(h3);
 var ol = document.createElement("ol");
 div.appendChild(ol);
 
 
 if (req.readyState == 4 && req.status == 200) {
  var root = req.responseXML.documentElement;
  var items = root.getElementsByTagName("item");
  
  
  for (var i=0, len=items.length; i<len; i++) {
  
  
   var title = items[i].getElementsByTagName("title")[0].firstChild.nodeValue;
   var link = items[i].getElementsByTagName("link")[0].firstChild.nodeValue;
   var li = document.createElement("li");
   li.innerHTML = "<a href='" + link + "'>" + title + "</a>";
   
   
   
   ol.appendChild(li);
   
   
   
  
  
  
  }
  
  content.appendChild(div);
  
  
  
  
 }
}


window.onload = initXML;

The first object, XMLDoc, creates and returns an XHMHttpRequest object. It also opens the given resource and sets the content type for the Ajax request (in this case, it uses the HTTP verb GET) through the loadXMLDoc() function, which also accepts a function reference to handle the request via the onreadystatechange event. The function initXML() uses an instance of XMLDoc to set the event handler to the function getRSS().

The getRSS() function simply retrieves a DOMDocument instance from the XMLDoc object, loops through all item elements by starting at the root of the XML document (using responseXML.documentElement as reference) and then extracts the values of the link and title elements to create XHTML links.

Although this approach actually allows you to see all the details of an Ajax connection, it's better to use the Ajax functionality of some JavaScript library (such as jQuery or Prototype) in order to simplify the code and avoid redundance. Futher, if you want to use this DOM approach, you should cache your data and use local variable reference instead of global reference. For example, instead of writing:

var content = document.getElementById("content");
var div = document.createElement("div");

you should write:

var doc = document;
var content = doc.getElementById("content");
var div = doc.createElement("div");

By doing so, you avoid global lookups and improve the performance of your script.

Parsing XML with jQuery

There are currently two feasible approaches to XML parsing with JavaScript and Ajax: a DOM approach, which makes use of common XML DOM methods (such as getElementsByTagNameNS()) or just simple DOM methods, and a library-based one, which uses an underlying framework in order to perform some common tasks.

With the first approach, you have to:

  1. create an instance of the XMLHttpRequest object
  2. set the target content type (usually text/xml)
  3. open the given resource
  4. create a reference to a DOMDocument object
  5. use DOM methods to retrieve data

A rough example, just to give you an idea:

function getXMLData(xmlDoc) {
  xmlDoc = xmlDoc || documentElement;
  var items = xmlDoc.getElementsByTagName('item');
  var len = items.length;
  for (var i=0; i<len; i++) {
    // do something with each item
  }
}

With the second approach, instead, all the details of the request are simply automated, so with jQuery you have only to write a few lines of code, like the following:

$.get('rss.xml', function(data) {
  var items = $(data).find('item');
  items.each(function() {
    // do something with each item
  });
});

This approach is actually much simpler than the former. Using jQuery we don't need to understand the nuts and bolts of the current process. What we need is simply parse the XML file and all its elements and contents. That's all. Just remember a few gotchas:

  • be careful with resource URIs; sometimes a relative URI may trigger some errors during parsing, simply because browsers don't have the necessary server permissions to access the given resource (if so, use an absolute URI)
  • cache data in local variables for a better performance; remember that local is faster than global.

How Firefox parses CSS URLs

Here's a quotation taken from the nsCSSScanner.cpp of Firefox:

Process a url lexical token. A CSS1 url token can contain characters beyond identifier characters (e.g. '/', ':', etc.) Because of this the normal rules for tokenizing the input don't apply very well. To simplify the parser and relax some of the requirements on the scanner we parse url's here. If we find a malformed URL then we emit a token of type "InvalidURL" so that the CSS1 parser can ignore the invalid input. We attempt to eat the right amount of input data when an invalid URL is presented.

Basically, CSS URLs take form within the url() function, that can be applied to background properties, list properties and generated content properties. Everything inside the parentheses is considered a URL. Anyway, as stated above, parsing CSS URIs can be quite challenging, because the tokenization must include new types of characters which are not included by default in the Flex notation for IDENTs of the CSS specifications. Further, URLs may either be contained within quotes or not, which adds an additional level of complexity (e.g. checking if quotes occur in matching pairs, like "..."). Here's how Firefox copes with this:

aToken.mType = eCSSToken_InvalidURL;
    nsString& ident = aToken.mIdent;
    ident.SetLength(0);

    if (ch == ')') {
      Pushback(ch);
      // empty url spec; just get out of here
      aToken.mType = eCSSToken_URL;
    } else {
      // start of a non-quoted url
      Pushback(ch);
      PRBool ok = PR_TRUE;
      for (;;) {
        ch = Read(aErrorCode);
        if (ch < 0) break;
        if (ch == CSS_ESCAPE) {
          ch = ParseEscape(aErrorCode);
          if (0 < ch) {
            ident.Append(PRUnichar(ch));
          }
        } else if ((ch == '"') || (ch == '\'') || (ch == '(')) {
          // This is an invalid URL spec
          ok = PR_FALSE;
        } else if ((256 > ch) && ((gLexTable[ch] & IS_WHITESPACE) != 0)) {
          // Whitespace is allowed at the end of the URL
          (void) EatWhiteSpace(aErrorCode);
          if (LookAhead(aErrorCode, ')')) {
            Pushback(')');  // leave the closing symbol
            // done!
            break;
          }
          // Whitespace is followed by something other than a
          // ")". This is an invalid url spec.
          ok = PR_FALSE;
        } else if (ch == ')') {
          Unread();
          // All done
          break;
        } else {
          // A regular url character.
          ident.Append(PRUnichar(ch));
        }
      }

      // If the result of the above scanning is ok then change the token
      // type to a useful one.
      if (ok) {
        aToken.mType = eCSSToken_URL;
      }
    }
  }
  return PR_TRUE;
}

Parsing starts with a '(' token and ends with a ')' token. Firefox checks if:

  • the url() function is empty
  • the url() function contains a quoted or a non-quoted URL
  • the url() function contains invalid tokens (e.g. a '(' token)
  • the url() function contains whitespace, which is allowed at the end of the URL (Firefox removes/eats it, however)
  • the url() function is followed but other tokens after the whitespace (this is invalid)

Safari has a problem with nested matching quotes within the url() function, i.e. it accepts them instead of marking them as invalid. This is not the case of Firefox, as you can see.

Parsing XHTML with PHP SimpleXML

Again, XHTML is really powerful when served as application/xhtml+xml. We can even parse it with the PHP SimpleXML library. For example, given the following markup:

<body>
  <p>Test</p>
</body>

we can write the following PHP code:

$xhtml_file = 'test.xhtml';
$xhtml_doc = simplexml_load_file($xhtml_file);

$p = $xhtml_doc->body->p;
echo $p; // 'Test'

Very simple, isn't it? The fact is that XHTML is treated exactly as XML when served with its proper content type.

Parsing XHTML with the PHP DOM extension

When served as application/xhtml+xml, XHTML is simply treated as XML so we can parse an XHTML document as if it was an XML file. Here's how it can be done using the PHP DOM extension: given the following XHTML fragment

<div id="test">
  <p>Test</p>
</div>

we can use the following PHP code:

$document = new DOMDocument();
$document->load('test.xhtml');

$test = $document->getElementById('test');
$p = $test->getElementsByTagName('p')->item(0)->firstChild->nodeValue;

echo $p; // 'Test'

Obviously we can parse this way even more complex XHTML documents, because the DOM extension allows us to access every single node within an XHTML document.