puffbox

Set the Census data free

One particularly difficult phase of my career was my time with National Statistics, in the aftermath of the 2001 Census. I tried, and ultimately failed, to persuade the organisation to recognise the tremendous asset they held in Census data, and to make wide public access a priority. I’m proud of some of the (relatively modest) things we managed to put out, but overall I’m disappointed at the many opportunities that were missed.

I remember my frustration at how everything was driven by very narrow ’stakeholder consultation’, which ultimately resulted in the same old people asking for the same old things. The potential for civic engagement ranked well down the list of organisational priorities; the possibilities for data mashing didn’t even register. Despite the huge sums of money spent on countless consultancies, the end product was - ahem - somewhat underwhelming.

So when I discover that the 2011 Census outputs are the subject of the latest blog-based consultation, part of the Hansard Society’s Digital Dialogues programme, of course I’m interested. And I think we all should be.

Two dates to bear in mind here. It’s nearly a year since the publication of the Mayo-Steinberg Power Of Information report, which called for ‘a strategy in which government … supplies innovators that are re-using government-held information with the information they need, when they need it, in a way that maximises the long-term benefits for all citizens.’ And just as importantly, we’re probably five years away from the first publication of census data.

This must be the first Census to take a truly web-first, and arguably even an API-first, approach to publication. Several reasons:

• Because it’s a one-off event, for which we have several years to prepare.
• Because if you think the world is web-first in 2008, just you wait and see what 2013 looks like.
• Because outsiders - from Experian to MySociety - will almost certainly do a better job than the Civil Service (sorry).
• Because it doesn’t actually prevent government doing the ‘old school’ thing itself, if it wants. In fact, if you think ‘API first’, it’ll probably result in the ‘old school’ outputs coming together easier and quicker too. Be your own client.
• Because to have any validity, the Census requires the goodwill and engagement of every person in the country. It’s one of the rare occasions where every resident puts something into a national kitty. Even if it’s only symbolic, this should be the prime example of the state giving something back to them in return.

This is one government consultation where the geek community (by which I mean us, sadly) should bring its influence to bear. We all know it’s the right thing to do; but they won’t do it unless there’s a sizeable, quantifiable demand. This would be a huge symbolic victory for openness and democratisation. This is our chance.

• Jonathan at 2:21 pm on 25 March 2008

  Completely agree. The last census was a massive missed opportunity. For the wealth of data and information collected very little meaningful ‘new’ output was generated. And certainly access seemed almost kept to a minimum rather than freely available on every street corner.

  Admittedly the general public will view the publication of a census with a hint of interest, be delighted, amused (Jedi?) and outraged (Daily Mail) for about 20 mins then forget about it for another decade.

  The more ‘we’ (the public) can access and make sense of this information the more we can come to our own conclusions, ask our own questions and get a realistic view of the country we live in. Rather than what we’re fed by all the interested parties and bias. When we’re fed a stat, we can check it instantly, and evaluate the validity of the source and the motivation behind it. Stats will cease to be ’selective’. Open access will compel honesty

  I’m expecting that by 2013 I expect data to be prevalent, fast and everywhere. That Census data should be at our fingertips and available to any question or decision that we may wish to reference it against. Maybe by then some private concern with be disillusioned enough to use the very same technology to run a rival census in parallel?

Got something to say? Say it.