Syndicated sidebars

Rumours of the demise of RSS seem to come in waves. We’re in the midst of another one just now, with moves by Twitter and Google (in the form of Feedburner) seemingly calling its future into question.
Before I began to bore everyone with the wonders of WordPress, I used to bore people with the wonders of RSS. In fact, it was WordPress’s handling of RSS feeds which initially won my heart. Feeds are now so ingrained in my daily life, I don’t even think about it any more. But the reality is, RSS reading (per se) didn’t catch on.
That isn’t so say that it’s a dead technology though, far from it. Five years ago, I suggested Facebook ‘might actually be the app which brings RSS to the masses’. Not too far off, as it turned out. I wonder how many people receive Facebook updates, or automated tweets, or email newsletters, which are really just renderings of an RSS feed?
And so to a little project I’ve been working on, in conjunction with leading Lib Dem blogger Mark Pack.
Mark wanted to recreate an application he used to manage whilst a full-time party employee: a javascript ‘widget’ to display political campaign buttons on other people’s web pages. A simple ‘ad server’, if you will. Nothing too clever technically, but a nice idea well implemented. Sadly though, since Mark had moved on, it had ‘fallen out of favour‘.
Its successor is the new Lib Dem Widget, a curated collection of current political content and links, which – like its predecessor – can be added to any website with a single line of code.
<script type="text/javascript" src="http://libdemwidget.markpack.org.uk/"></script>
We pull in details of the latest party news item, the latest YouTube video, plus a collection of other interesting bits from around the web, and render it in a nicely style-guide-compliant ‘skyscraper’, which should drop seamlessly into any website sidebar. And yes, of course, it’s (mostly) driven by RSS feeds.
You won’t be surprised to learn we built it with WordPress: but it’s a rather unusual use case. The widget runs as a theme, giving us easy access to:

  • WP’s built-in feed-fetching;
  • its various caching options – including, but not limited to WP Super Cache; and
  • its oEmbed calls, for easy inclusion of the YouTube clip.

In effect, it’s the ‘homepage’ of a one-page site… but a site with no actual content of its own.
To give things a visual lift, we also ‘code scrape’ the various linked pages, looking for details of thumbnail images. Facebook’s OpenGraph meta tags are now in fairly widespread use, so there’s a healthy chance that we’ll see an og:image in the HTML header – and if so, we’ll use that.
Writing code which will work on any website – regardless of sidebar width, existing CSS styles etc – has been quite a challenge. The markup ends up pretty ugly, with everything included ‘inline’, just to be sure. And even then, I’m not sure we’ve sorted out every possible use case… but that’s what beta testing is there for.

Paul Waugh takes his audience with him to PoliticsHome


Evening Standard deputy political editor Paul Waugh starts his new job this morning, as editor of (increasingly paywalled) website PoliticsHome.com. Mildly interesting in itself, as evidence of the still-growing influence of online in the political space, although far from the first time a ‘proper’ journalist has gone over to the blogs’ side.
What’s quite interesting is the mechanics of the move itself. His final post on the Standard’s (Typepad-powered) blog gave full details of his new job, and where you’d be able to follow him – including direct links to his new home page. I find it very hard to imagine any other media outlet being so relaxed about a star reporter or columnist ‘taking his readers / audience with him’.
Equally intriguing is the fact that his (personal) Twitter account has just kept going as it always did.
[blackbirdpie url=”http://twitter.com/paulwaugh/status/593819410112512″]
Despite the on-page linking and the background wallpaper – Standard last week, PolHome this morning – Waugh ‘owns’ this particular channel of communication… and its almost 10,000 followers. He isn’t dependent on his employer’s infrastructure, or brand, to talk to his audience.
Former BBC man James Cridland, now a ‘radio futurologist’ (?), wrote an excellent piece about this issue 18 months ago, in the context of radio presenters moving jobs. His rather draconian-sounding conclusion was this – although it’s worth noting the dissent, some from known names in the industry, in the ensuing comments:

Give your presenters official Twitter feeds for your station, and make it clear that they can only promote these. XFM is doing the right thing here, since it has a set of them – @daveberry_xfm is Dave Berry, for example – but this is clearly part of the station’s output. Ensure that -you- retain the password, and ensure that you actively monitor what they say (just like you monitor what they say on-air.) That way, when you part company with that presenter, you can communicate this fact to their followers your way – and, crucially, you stay in control.

[Over the next couple of days, James also offered opinions on promoting personal websites (in short: no) and email addresses (likewise), stirring similar levels of controversy.]
So whether he realises it or not, Paul is offering an interesting case study in what constitutes ‘brand’ in the world of third-party online services. When communications infrastructure was difficult, employers could keep control. When we’re all just a few seconds away from creating our own Twitter / Facebook accounts, the employer is left with little more than guidelines. And perhaps a rather weak argument about using company resources for personal purposes.
I really enjoy Paul’s stuff: and I’d happily be subscribing to his new blog right now… except that somehow, the website – running on a bespoke platform which happily ‘ingests’ other people’s RSS feeds –  can’t offer an RSS feed of its own, although one is promised ‘soon’. (FYI: it’s two months since prominent blogger Waugh’s move was announced.)
Oh, and by the way, PoliticsHome – disabling the ability to right-click on your pages… really?

DFID's blogs feed new Guardian site


With the kind assistance of the Bill & Melinda Gates Foundation (I think he used to be big in computing or something), the Guardian has launched a new (sub)site dedicated to global development. And quite remarkably, it features regular contributions by UK civil servants.
In fact, it’s feeding (literally) off the DFID Bloggers site, built by Puffbox nearly two years ago (!). It pulls in the latest handful of stories from the DFID site’s RSS feed, and displays them in a cute little animated box. Pretty much what the DFID homepage itself does…
… except that the DFID homepage does something a little bit cleverer. As you’ll see, it carries not only the title and description – typical of any RSS feed; but it also shows the author’s face and job title, neither of which are standard RSS elements. It also turns the blogger’s name into a link to their personal blogging archive. Cool, eh? – dead easy, actually.
We do this with the aid of a little WordPress magic. The author photo is uploaded into WP using a plugin called User Photo; but Simon Wheatley and I worked on a ‘meta plugin’ to ensure these photos would be square, and hence more predictable to work with. And before you ask, yes, the meta plugin was indeed made available at wordpress.org… and has been downloaded over 3,700 times as I write this.
(The job title is an additional field added to the user profile; at the time, we did that with a custom plugin, but now I’d probably use this code by WP guru Peter Westwood.)
We then call this extra info – where available – into the RSS feed with a custom function, using the rss_item hook. (It’s all formatted using the standard MediaRSS extension, originally by Yahoo.) And so, each time a new post is added to the DFID Bloggers site, the DFID homepage can extract all the data it needs from the RSS feed, and slot them into the appropriate box.
… which is a very roundabout way of demonstrating that, contrary to what you may have read lately, there’s plenty of life in RSS yet.

RSS usage on Whitehall's websites

How many central government websites offer RSS feeds these days? The good news is that of the 20 departments represented in the Cabinet, I could only find one that didn’t. But it was a bit of a surprise to see how few offered ‘full text’ feeds, as opposed to ‘summary only’.
I visited each of the 20 departments listed on the Parliament web page – the top result in Google for ‘UK cabinet ministers’, looking for a main RSS news feed. Here’s what I found:

  • There are explicit references to RSS feeds on 18 of the 20 sites: the exceptions are the Scotland Office and Defra. There is a Defra feed if you know where to look (namely COI); but how many would know to look there? That leaves the Scotland Office as the only department completely lacking an RSS feed for departmental news. (Its Secretary of State, Jim Murphy does have a blog, but I’m not counting that here.)
  • Five of the 20 fall back on the feeds produced by COI’s News Distribution Service. That leaves 14 of the 20 producing their own feeds – in most cases, in addition to the feeds at COI.
  • Only one, FCO, directs people through Google’s Feedburner service.
  • Only 3 of the 20 provide ‘full text’ RSS feeds – allowing people to read the full press release (etc) instantly, and opening up the possibilities for easy information re-use (ie ‘mashups’). The rest require people to ‘click through’ to a page on the originating website. This is common in commercial publishing, where on-page advertising is a key driver.
  • Of the 3 offering ‘full text’, 2 are running on WordPress: Number10 and the Wales Office, both of which I admittedly had some involvement in. The other one is DECC.
  • The Department of Health’s RSS feeds aren’t valid: the ‘link’ element quoted in the feed doesn’t include www.dh.gov.uk. A curious problem to have caused yourself, and a trivial one to fix. I’ve mentioned this before, in the context of Directgov; and of course, the two share a publishing platform. A broken one, in this case.
  • It was a pleasant surprise to see the majority of sites have ‘autodiscovery tags‘ in the header of their homepages – a behind-the-scenes way of indicating that a site has an RSS feed, which can (for example) light up an icon in the browser interface. But 8 don’t. I’m looking at you FCO, Home Office, Defra, DFID, Cabinet Office, Defence, Transport, and DCMS. Some of them have the appropriate tags deeper into the site, to be fair… but it’s a free and instant win those sites are missing out on.

The thing is, it’s so easy to get RSS right. Ask any blogger: when executed properly, RSS feeds should be an automatic, never-even-think-about-it thing. Each time a new item becomes available on a site, it should just drop into the RSS feed, notifying people – and importantly, mechanical services – of its availability.
And the easiest way to get RSS right is to build your news website on WordPress. Out of the box, you get valid RSS feeds for virtually any view of your site’s news content. Feeds by category / press office desk / minister? By keyword tag… or combinations of keyword tags? How about infinitely customisable feeds, based on search queries? Yes, to all of those. Probably within a couple of days, if you get the right people in. (Hint hint.)
A lot of government websites are going to need a rethink following the next election. It’s the ideal opportunity to upgrade the news area, by moving to a system that’s been explicitly designed around the timely publication of short text articles, generally presented in chronological order. By which I mean, a blogging system. And specifically, WordPress.

Don't get a feed, get a blog

I didn’t write about Mash The State when I first heard about it, because the ambitions seemed embarrassingly modest: getting each council in the country to offer an RSS feed by Christmas. In 2009? – seriously?
And then I note that, of the three e-government super-sites – Directgov, Businesslink, NHS Choices, annual budget approx £30m each – only the NHS site offers RSS feeds (and even then, only a few). Directgov has recently started offering its first RSS feed, but if you look at the source code, you’ll note that the URLs all begin with slashes. In other words, they aren’t valid RSS. Or in other less diplomatic words, they’re useless. If a guid isn’t globally unique, then it isn’t a guid. Still, at least they’re trying. Businesslink doesn’t seem to have anything in RSS. At all.
Meanwhile, the rest of the web is racing ahead. I’m especially proud of the DFID Bloggers site in that regard: helpful as ever, WordPress offers pretty much every list available through the site as an RSS feed, if you know the right URL to call. Each category has an RSS feed. Each tag has an RSS feed. Each individual author has an RSS feed. Heck, you can even get search queries as RSS feeds: meaning, in effect, you can have a customised RSS feed of ‘every time that WordPress site mentions X’. All out of the box; at zero charge and zero effort. They just happen.
RSS continues to delight me as a website designer and builder. Recent WordPress releases have added some extra – undocumented? – tricks: for example, if you can construct the right URL query string, you can get an RSS feeds of all items except those from a certain category. (Clue: ‘cat=-1’.) And it’s going to get even better imminently, with the inclusion of the brilliant SimplePie, for consuming RSS, into the next WordPress release.
I’ve built entire sites like Real Help Now and onepolitics powered solely by RSS feeds from third-party sites. I’m even building a couple of WordPress sites now which will use their own internal RSS feeds to surface content, rather than me coding ‘proper’ PHP/SQL queries. It’s just easier. And when you’re doing something as an outsider because it’s easier than the ‘proper’ internal method, you know we’ve reached somewhere significant.
The truth is, if your website still isn’t offering an RSS feed, you’re falling further and further behind the rest of the web, and you’re depriving yourself of the magic which eager geeks might bring to your content. But before you go spending money adding an RSS feed to, say, your press release pages – don’t. There’s a content management solution which is optimised for delivering text documents on a rolling basis, presented chronologically. You’re looking at it.

Civil Service jobs API: five years in the making

Five years ago – to the very minute, as it happens! – I was working on a proposal to put to someone at the Cabinet Office. I was still working at ONS, and was trying to think of a clever way to handle our job adverts. We were obliged to post details of all vacancies into the (very recently departed) Civil Service Recruitment Gateway website. So I thought, what if that site could feed our vacancies back to us?
I approached the Cabinet Office with a proposal to not only help them spec up the work, but to pay for it. I’ve still got the PowerPoint slides I produced for the ‘pitch’.
Click to view slide-by-slide
Five years ago, this was truly visionary stuff – in effect, an open API on all government jobs, way beyond anything that had been done before. And even though I’d documented the whole thing, even though I was putting up the money myself for them to do it, to build a function for everyone to use freely… it never happened. An all too familiar story. So it’s especially amusing to see Steph’s news, exactly five years on, of Civil Service jobs, your way.

Given the enduring popularity of job search online, this is an exciting development for a major government data set. It should provide something which third party developers can use to derive valuable commercial services to their customers, as well as helping to ensure Government broadens the reach of its recruitment at lower cost, facilitating the creation of innovative new services based on public data. With luck, it’s the business case for APIs to government data that we’ve been looking for.

Now admittedly, my proposal was a modest affair based on a straight-down-the-line RSS feed. There were few specific references to XML, never mind API, and certainly not RDFa. But reading Steph’s piece, and the ensuing comments, I can see a direct line between my 2004 proposal – which, let’s be honest, is ancient history in online terms – and today’s unveiling. If you ever wanted a precise metric for how slowly government moves, there it is.
Regardless of the history, it’s an excellent piece of work by the Cabinet Office team; and – I hope – having done the donkey work to set it up, someone is ready to take it to market, and make people aware of what the service can do for them. Some relatively straightforward PHP or ASP would be enough to put an automated list of all current vacancies on each department’s own homepage; perhaps the Cabinet Office team could go a step further, and deliver it via a Javascript-to-PHP call (as the LibDems do for their ‘campaign buttons‘), making it child’s-play for the recipient site. The requirement to obtain an API key doesn’t help their cause, though.

Web 2.0 is just so 2008

I’ve never been one for New Year’s resolutions or predictions; I’ll leave those to other people.
Suffice to say, I’m increasingly of the opinion that web 2.0, as a phase in the web’s development, is over. I’m using almost exactly the same tools now that I was this time last year. It’s ages since any new technology (in and of itself) blew me away. And I’m very close to unsubscribing from TechCrunch, the trade journal of web 2.0. I just can’t think of the last thing I read in it which really excited me.
So here we are. It really feels like we have our winners in most of the ‘web 2.0’ categories: WordPress, Facebook, Flickr, YouTube, Twitter, Basecamp, Google in various guises. And it feels like we have all the tools we need, most of them free of charge, to make things happen. Let 2009 be the year where we really start to make use of them.
What excites me?

  • Optimised presentation on mobile devices, specifically the iPod Touch / iPhone (for now)… and as I’ve mentioned before, on games consoles like the Wii. By September last year, there were 3.6m Wiis in UK hands; and it’s been another big Christmas for Wiis. That’s a lot of internet-ready devices in a lot of UK living rooms.
  • High definition video. We’re starting to see HD camcorders coming in at affordable prices, and YouTube is starting to deliver very high quality versions of uploaded clips. The quality is at least as good as TV, and sometimes I’d say better. I particularly like the look of the Canon SX1 (stills) camera – digital SLR functionality in a consumer product, with the ability to shoot HD video to SD card. £400’s a lot of money, but I’m still tempted.
  • RSS. No, seriously. As an industry, we seem to have given up on mass consumer adoption. Instead, attention seems to be heading into how we can use it to create new sites in their own right, like Steph’s digitalgovuk catalogue or my own OnePolitics; or connect sites seamlessly across different CMSes. I’m even planning to build one site whose homepage will be powered primarily by its own RSS feed (too boring to explain). We need it more than ever, and it still isn’t letting us down.
  • JQuery. I’ve always resisted learning Javascript; it felt like too much of a step towards becoming a full-time developer. But JQuery makes it so ridiculously easy, it’s a crime not to use it. So far, most of my work with it has been invisible, behind-the-scenes stuff. But I’m looking for excuses to go JQuery-crazy.
  • WordPress. Or more accurately, me using WordPress. I’m thinking up more and more clever ways to use it, and it’s almost a case of finding projects where I can squeeze in my new ideas – with or without the client knowing. 🙂 More details as things emerge, naturally.
  • Cost-cutting. Having too much money is almost as bad as not having enough. It’s a magnificent opportunity for open source generally, and for people like myself (forgive me) who can whip up impressive solutions with it. The business cases for buying Big Ugly CMSes and hiring Big Ugly Consultancies will need to be very, very good.
  • But most of all… good, substantial material going online. We’ve done enough trials and experiments to see what works and what doesn’t. Specifically, we’ve got enough examples to show that it won’t work unless you really make an effort. So let’s hope the ascent of Obama, and the prospect (however slim) of a UK general election inspire politicians in government and outside to really get stuck in.

We have all the tools we need, and it’s even easier than before. Let’s start delivering.
PS: Coincidentally – and yes, not a little ironically given the above – TechCrunch has today posted its list of ‘products I can’t live without‘. Most notable, to my half-awake eye, is the similarity with last year, and the gradual pruning of the more obscure names in favour of the Old Favourites. (Slightly surprised to see FriendFeed making the 2009 list, by the way.)

COI's instant response to RSS request – blimey!


I’m pinching myself. Wednesday, 08:30am: Justin Kerr-Stevens makes a request via OPSI’s Public Sector Information Unlocking Service. A couple of dozen people sign up to say ‘good idea’. A few people (me included) add some more substantial comments. Fast forward two days to Friday, 12:32pm: COI publishes details of RSS feeds for (virtually) every Cabinet-level government department.
I say ‘virtually’ every Cabinet-level department: the exceptions are the Scotland, Wales and Northern Ireland Offices. I can’t see an RSS feed on the Scotland Office site; the NIO site’s feed is curious to say the least, showing just one update since Christmas (which clearly isn’t right). But the Wales Office is happily pumping out beautiful RSS. Puffbox and WordPress may have had something to do with that.
Gripes? The URLs are truly revolting. But that’s the biggest nit I can pick. Justin, COI, OPSI, Mr Watson – thank you all. Did that really just happen?
Update: I’m told they were on the site somewhere, just not obvious. Even knowing that now, I still can’t find where they were. And if you can’t find them, they might as well not be there. Next steps, please: an entry in the site’s primary navigation, nicer URLs, and auto-discovery tags on each department’s homepage.

'Linking here' lists with Google feed API

Time for some tech talk. A few weeks back, I wrote about Google’s new AJAX Feed API. Having played with it last week on behalf of a client, and having liked what I saw, I decided to implement it myself.
If you’re reading this on the puffbox.com website itself, you might see a list in the sidebar headed ‘Who’s linking here?’. (If not, see here for an example.) It’s something I introduced a while back, powered by feeds from Google’s Blogsearch engine, and processed using the excellent SimplePie. But I’ve now switched over to doing it client-side, through the Google API.
Once the blog post finishes loading, the Javascript calls in the RSS feed (actually, it’s Atom format) from Google. If it finds any blogs linking to that specific post, it writes a title into one previously empty DIV, and a disclaimer into another. In between, it generates up to 10 <LI> list items, each active as a link to the linking blog.
If you want to see how it’s done, and copy the idea for yourself, a quick glance at the page’s source code will reveal how straightforward it was. (You’ll need your own API key, obviously.)
Why go client-side? It’s less effort for the server to process; and it doesn’t build up a mass of cached feeds. It should also be marginally more secure on paper, which is important to some clients. And whilst the function is dependent on Javascript being available, it’s dead easy to offer a ‘noscript’ alternative – a link to a pre-formatted Google search query. It’s nowhere near as slick, but it ensures the information is still available to those without Javascript, so it passes accessibility requirements (W3C guidelines, checkpoints 6.3 and 11.4).

Another Downing St travel-blog

Gordon Brown’s off to Japan for at the weekend, to hang out with the other G8 heads of government. So it’s time to crank out another Puffbox production for 10 Downing Street: the now-familiar mash-up of a travel-blog, Twitter stream and Flickr photo set.
As with previous trips to the US and Brussels, it’s based primarily around WordPress, with data pulled in from the third-party services via RSS. Once again I’m using SimplePie to handle the RSS processing; although if I’d had more time this week, I’d probably have tried out Google’s new AJAX Feed API, to do it all on the client-side. I’ve done some preliminary trials with the service, and it seems very useful indeed.
This will probably be the last such travel-blog to be done in this form, for reasons which will very soon become apparent. But these set-piece microsites have served their purpose – providing an excellent excuse to expose the Number10 team to WordPress, and the possibilities of lightweight content management.
Update – the site got a nice mention in Monday’s Washington Post. ‘British Prime Minister Gordon Brown is stealing the G-8 show online. No. 10 Downing Street is blogging the summit in Toyako, throwing up an integrated social media site to follow “our man in Japan,” with Twitter and Flickr feeds to boot. Downingstreet, “the official twitter channel for the Prime Minister’s Office based at 10 Downing Street,” has more than 3,000 followers, and is part of the prime minister’s ongoing Web-savvy operation.’ Best press No10 has had in ages… 🙂