Puffbox

In late November 2009, I noted the launch of Code of Everand, a multi-player online game commissioned by the Department for Transport, aimed at teaching kids road safety in an innovative way. I found it ‘a bit tedious and mechanical’, but I acknowledged that I wasn’t exactly its target demographic.

But all sorts of alarm bells were ringing. A site like this couldn’t possibly have come cheap – but that was only the start of my concerns. It had apparently taken more than two years to build. It was built by a US-based agency. It had apparently ‘been on the brink of cancellation twice.’ And by the time it finally launched, none of the Transport staff who had originally commissioned it were still in post.

I signed off with the words ‘I imagine an opposition PQ is already in draft.’ – but as if to demonstrate just how influential this blog really is, no such PQ was ever forthcoming.

When the site’s first birthday came and went without fanfare, I could no longer suppress my curiosity… and drafted a FOI request via MySociety’s What Do They Know service. Whether the site was a success or a failure, cheap or costly, it was going to be an interesting response.

I got the response yesterday. So here we go.

That’s a grand total of £2,785,695 from project inception until the end of the current financial year. By any standards, that’s a staggering amount of money to spend on a single campaign website. Was it money well spent?

I also asked for data on user registrations and site traffic. This is a chart of new user registrations per month:

That’s a dramatic drop from a peak of 54,500 new registrations in March, to just 6,500 in April; and it’s basically been flat-lining since June. A total of 170,000 registered users, 91.8% of whom were signed up before the end of the last financial year – and the spending of a further £700k.

There’s a similar pattern to the numbers of monthly Absolute Unique Visitors, as measured by Google Analytics – although it’s curious to note the differing extents of the peaks.

The Google data regarding average time spent on the site is worth noting:

From an understandably low base, as the site built its following, the average has hovered around the 10 minute mark. That’s much more than most websites could boast, and stayed remarkably constant even as the monthly visitor numbers crashed – before suddenly jumping to nearly 18 minutes in November. Similarly, in terms of page views, it has typically recorded around 16 per visit, already very high in comparison with most sites – before shooting up to 28 in November.

I think it’s also worth mentioning that the project has social media presences in all the obvious places. And as I’m writing this, in January 2011, Code of Everand has 568 friends on Facebook – and 82 (let me confirm that: eighty-two) followers on Twitter. It’s even worse on the more child-friendly networks, where you might expect it to perform better with its notional target audience: Habbo (18 friends) and Bebo (10 – yes, ten).

Whatever interest it might have generated in those first few months, and however well it has engaged its hardcore following, it’s surely impossible to say the site has been a sustained success. Since April, its audience has all but disappeared. And yet believe it or not, just as I was writing this, there was an update on its Facebook account:

So it’s an outrageously expensive failure, right? Believe it or not, I’m still withholding judgment. Yes of course, it’s a huge amount of money: £16.33 per registered user. But it’s conceivable, just about, that it’s had its desired effect in terms of educating the youngsters it’s aimed at.

We’ll find out in due course. Transport confirmed in their response to me that:

A contract to evaluate the game was let to the Transport Research Laboratory (TRL) in March 2010. The evaluation team consists of TRL, the Serious Games Institute based in Coventry University, and Simon Christmas Ltd, an independent social researcher. We are currently in the planning stage and do not yet have a date for publishing the work and final report.

Given the collapse in visitor numbers, and the continued spending of public money on a project which would seem to have had its day – if it ever had it at all, they might want to get a move on.

Data on road casualties during 2010 are scheduled to be published in June 2011.

• If you fancy crunching the numbers for yourself, you can access the source data at What Do They Know. Thanks to the MySociety crew for making it all so easy.
• Note: I’ve since found info from the Serious Games Institute regarding their ‘five strand’ evaluation research, being led by Dr Adam Qureshi (contact details here). Sadly no publication date is quoted for their report.