Hacks, and RSS HTTP


I’ve put a page of hacks up; just somewhere for code snippets and minimal programs, hopefully of use to others. Hacks so far: one. download-files.pl is a relatively smart file downloader, intended to watch for occasionally–changing files, that also gives me a nice excuse to rant about the state of RSS readers’ HTTP implementations.

One obvious culprit is NewsGator/2.0, which is getting conditional requests right, but totally ignoring Expires and overoptimistically checking twelve times as frequently as necessary. Dave Winer had some interesting ideas about dynamic choice of ttl (RSS’s equivalent of Expires), but obviously this stuff doesn’t work if clients ignore it. Yes, there’s also room for smartness on the client side, but that doesn’t let you ignore the server. (If I was MLPing, I’d suggest that this interplay of declarative/predictive timings somehow linked in to the whole Fluidtime thing.)

Worse by far is Opera 7.50, which apparently supports RSS now. It’s not making conditional requests at all, and is fetching a fresh new copy of my feed every time it checks. Okay, it’s only a preview release, but you’d expect browser authors to have a better grasp of HTTP than this. If we’re being pedantic (we are), it’s also getting its User-Agent string wrong. RFC 2616, section 3.8: it’s token ["/" product-version], not the ‘Opera 7.50’ that’s being sent (full disclosure: getting this right is a shibboleth of pedantry. And there’s something delightfully archaelogical about Opera’s quirk of announcing itself as “Netscape Navigator’s development codename, but with a later version because they couldn’t change the token (actually, really Internet Explorer pretending to be Netscape), except really it was Opera after all.” One day someone’s going to need to spoof Opera.).

Pithy moral: the web’s an ecosystem, and its ongoing success depends on the steady correction of even minor problems like these. Mechanisms that foster this (automated patches and updates, open source and public bug tracking systems) are a Good Thing.

Real moral: I get angry when I see broken things. (Time, again. Could be significant.)

(Update 2004-05-24: It looks like this was fixed by the 7.50 release, presumably included in “Fixed multiple update issues with RSS newsfeeds,” although it’s hard to be certain without a public bug database. I’m certainly serving a few 304s to clients calling themselves Opera/7.50 now. Hallvord R. M. Steen mailed me to explain that Opera’s User-Agent is necessary to work around the broken browser detection on some sites, although it is settable as a preference - thanks. Also, apparently, “some users of Konqueror do spoof Opera.” Okay. But nobody spoofs Konqueror....)

(Music: R.E.M., “Crush With Eyeliner”)
(More from this year, or the front page? [K])