In reply to a tweet about wanting PUSH-based construct for web applications, @prasnation pointed me to Comet, which Wikipedia defines as a “WWW architecture in which a web server sends data to a client program (normally a web browser) asynchronously without any need for the client to explicitly request it”. I’ve been reading about it for the past couple of days, and it’s beginning to look like the next thing to hit web-applications. Standardised adoption in browsers will allow mass adoption in applications. That, will make hosting companies work to offer more powerful servers (dedicated/VPS) at cheaper costs, so that they can handle “always alive” connections for more people.

How does Comet work?

Simply put, it is a PUSH based alternative to current methods. The way HTTP works right now, the connection between the browser and the server is severed the moment the transfer of the page data is complete. Any changes happening on the server post-initial transfer, are not reflected on the already loaded page until the user explicitly refreshes. This has absolutely nothing to do with AJ (more on it later), and is purely about the transfer of data between a browser and the server.

Comet works by keeping the connection alive, so as to allow the server to PUSH data to the page as and when it changes. This allows for a more efficient method of serving dynamic pages, one that doesn’t require constant GET calls using AJAX or fancy IFRAMEs. Comet (as proposed) plans to work natively (without client side scripting) in a page, with certain blocks marked out as Comet elements, so that the browser knows that those elements will be updated in the future of the session.

Examples of places where the Comet architecture can be most beneficial are web-chat clients, notification systems — like Facebook’s Beacon or news tickers, and services of the same form. Any page that needs to stay in touch with an ever updating server for “fresh” data, should use the Comet model.

Unfortunately, no browser supports it completely (Opera 8.5+ supports it sparingly), because this is a part of the HTML5 specification¹. HTML5 is still in the decision-making phase, so the entire specification is not complete, which means it will take further time to finally get this out of the door, after which the browsers will have to implement it properly, and back-end servers will have to become powerful enough to handle such connections. It’s safe to say there’s a long wait before we see it peeking out of the hood. That’s why people have already come up with techniques to make up for it’s absence.

Alternatives to Comet

There are many ways of imitating a PUSH-based model, but I prefer only 3 — keeping efficiency in mind. I’ve tried out the following on my shared host, and have discovered a few things which might help you if you want to create an application that uses these alternatives.

XHR polling: The loaded page contains a recursive XHR call to the server that checks for updates every ‘n’ seconds (assuming data changes quickly, like in a chat client). The next XHR is fired the moment the previous is closed, which can be achieved by either polling the server again onSuccess, or making the server send a callback, in which case the second poll is only called in case the previous one was a success. This is a very intensive method, and is the fastest way to bring down weak-shared hosts (I’ve done it twice), so I wouldn’t recommend it unless you’re on dedicated servers. If you do use this, make sure you poll at decent time intervals; 5 seconds is too low, while 30 seconds makes your application seem unresponsive.

XHR streaming: A big improvement on the previous method, this uses the concept of streaming — letting data trickle in in bits and pieces — to make the page dynamic. Of course, this involves keeping the connection open all the time so that the server can continue sending data through. It uses the onreadystatechange (state 3) of XHR to take action every time new data is available. The problem is that existing shared hosts put a maximum execution time on scripts (usually 30), and we need infinite loops to keep the connection alive. This connection will automatically be closed when you hit that execution time limit. So, if you have access to your PHP configuration (which is highly unlikely), you can change the values and fix the issue. Otherwise, this method is pretty useless.

“Forever” IFRAME: This is a spin off the XHR streaming method. An invisible IFRAME is pointed to the server script, while the server spits out callbacks to functions on the main document (by using the top object). The IFRAME will continue to load, which might make the browser behave as if the page is still loading, but that’s a minor setback. Again, for shared hosts, the maximum execution time will limit how long the IFRAME can continue loading the script, but this is where things branch out from the streaming method. You can simple refresh the IFRAME to reset the execution timer, and the user won’t know the difference. The refresh can be a setInterval every ‘x’ seconds, where ‘x’ is the same number as the maximum execution time limit. This is cleaner, and is a much lesser headache.

---

All the above methods require server power, so dedicated hosts are your best bet at the moment. A shared host is alright if you want to play around with this and get the hang of it, but a full blown application will not hold.

Comet is a new way of doing an old thing. Most importantly, it is the correct way of doing it. I want it to get implemented quickly, even if alternatives exist. As the web comes closer and closer to the desktop in terms of applications and their usability, it is much better to work with the correct model than buggier, harder to maintain alternatives. Really.