In which Simon makes the case for the ‘government machine’ (in the diagram above), for government departments to publish fairly basic written information about their work, to be built on something which already exists, instead of being built ‘from the ground up’. If you haven’t already, do read Neil’s piece… then Stephen Hale’s piece about the Department of Health’s new approach… then read on. And do please note the line about ‘Seriously, this isn’t about WordPress.’
It won’t entirely surprise you to learn that, when Neil Williams’s blog post about government web publishing in the world of a Single Domain popped up in my feeds, the first thing I did was press control-F, and search for ‘wordpress’. And hooray, multiple mentions! Well, yes, but.
Some background, for those who need it: Neil is head of digital comms at BIS, currently ‘on loan’ to the Government Digital Service team, to lead the work exploring how government departments fit into the grand Lane Fox / Loosemore / Alphagov vision. A ‘hidden gem‘, Tom Loosemore calls him – which seems a bit harsh, as Neil has been quite the trailblazer in his work at BIS, not least with his own web consolidation project. It’s hard to think of anyone better placed to take up this role.
With that track record, it’s happily predictable to see Neil reserving a specific place for WordPress (and the like). More generally, the vision – as illustrated in the diagram reproduced above – is sound, with the right things in the right places. There’s so much to welcome in it. But there’s one line, describing an ‘irreducible core‘, which stopped me in my tracks:
a bespoke box of tricks we’ll be building from the ground up to meet the publishing needs most government organisations have in common, and the information needs ‘specialist’ audiences most commonly have of government.
Here’s my question for the GDS team: why bespoke? why ‘from the ground up’?
It’s a decision which requires justification. ‘Bespoke’ invariably costs more and takes longer. It will increase the risks, and reduce the potential rewards. It would also seem to be directly in breach of the commitment Francis Maude made in June 2010, to build departmental websites ‘wherever possible using open source software’. Were all the various open source publishing platforms given due consideration? Was it found to be literally impossible to use any of them, even as a basis for development?
Let’s assume WordPress and Drupal, the two most obvious open source candidates, were properly considered, as required by the Minister. Let’s assume contributions were sought from people familiar with the products in question, and just as importantly, the well-established communities around them. Both are perfectly capable of delivering the multi-view, multi-post type, common taxonomy-based output described in the ‘multistorey’ diagram. Both are widely used and widely understood. So why might they have been rejected?
Did the team spot security or performance issues? If so, wouldn’t the more responsible, more open-source-minded approach be to fix those issues? Then we’d all see direct benefits – on our own personal or company websites – from the expert insight of those hired by our government. If things are wrong with such widely-used technologies, whether inside or outside government, it’s already government’s problem.
Or were there particular functions which weren’t available ‘out of the box’? If so, is it conceivable that someone else might have needed the same functions? Local government, perhaps – for which the GDS team has ‘no plans or remit‘. We’re seeing plenty of take-up of WordPress and Drupal in local government land too. They have very similar needs and obligations as regards news and policy publication, consultations, documents, data, petitions, biographies of elected representatives, cross-cutting themes, and so on. Why not make it easy, and cheap, for them to share in the fruits of your labours?
But I think it goes wider than different tiers of government. Government is under a moral obligation to think about how its spending of our taxes could benefit not just itself, but all of us too.
Even if this project’s bespoke code is eventually open-sourced, the level of knowledge required to unpick the useful bits will be well beyond most potential users. Given that it’ll probably be in Ruby or Python, whose combined market share is below 1%, it won’t be much use ‘out of the box’ to most websites. A plugin uploaded to the WordPress repository, or a module added to Drupal’s library, would be instantly available to millions, and infinitely easier to find, install and maintain. (Well, certainly in the former case anyway.)
I can’t help thinking of the example of the BBC’s custom Glow javascript library, which does simplified DOM manipulation (a bit like jQuery), event handling (a bit like jQuery), animations (a bit like jQuery), etc, proudly open-sourced two years ago. It appears to have attracted a grand total of 3 non-BBC contributors. Its second version, incompatible with the first, remains stuck at the beta-1 release of June 2010. Its Twitter account died about the same time; and its mailing list isn’t exactly high-traffic. I’m not convinced it ever ‘unlock[ed] extraordinary value out there in the network‘. Proof, surely, that open-sourcing your own stuff isn’t the same as pitching in with everyone else.
Seriously, this isn’t about WordPress – although that’s unquestionably where the Whitehall web teams’ desire path leads. It’s not really even about open source software. It’s about government’s obligation to the citizens and businesses which fund it. It’s about engaging with existing communities, instead of trying to create your own. Acknowledging people’s right to access and make use of the data – erm, sorry, the code – whose creation they funded. Any of that sound familiar?
There’s so much right about the picture Neil paints. And maybe I’m reading too much into a single line. But the idea of building yet another bespoke CMS to meet Whitehall’s supposedly-unique requirements seems to be three to five years out of date. And it didn’t work too well, three to five years ago. Or three to five years before that. Or…
Responses
Blimey, that’s a bit of a diatribe, seemingly over your (over?) interpretation of one word! (‘bespoke’)
I’ll set the record straight: we’ve not yet decided what technology to use for the core of the corporate platform, other than they will be open source.
Obviously, we’ll use the best tech to meet the needs neil is busy identifying, but I would worry if the GDS tech archs and developers who will make the call started their thinking with the question ‘which cms should we use?’
In my experience (and I’ve got deep cms scars – webobjects, anyone?) starting a new product development with a debate about which cms to use is a category error.
Some thoughts, from my perch by the pool;
1) We are developing layers of a digital platform, not rolling out a cms. We well doubtless end up using different tech for different layers.
2) For the core layers of the ‘corporate’ platform, the challenge is around modelling, creating, managing and querying structured data, of which ‘content’ is but one subset of datatypes.
3) The closest analogue for this irreducible core is probably the awe-inspiring http://www.legislation.gov.uk You can rest assured we’ll be asking Jenny Tennyson for her thoughts.
4) I adore WordPress. And by volume of publishing, I’d hazard a guess that the majority of gov.uk pages will probably be best delivered via a simple shared cms of its ilk. But even I know one size rarely fits all, and as the mySociety tech lead gently said to me as he revoked my cvs access, ‘ Tom, you know just enough to be dangerous’.
– now, where are my flip flops
If the only tool you’ve got is a CMS everything looks like a page.
Preposterous. Sorry.
Let’s stick with WordPress for the sake of argument:
Every single type of information ever that’s not a page would need a custom content type. Every single type of information that needs to relate to any other piece of custom information will also need a custom content type, and a new piece of admin interface to handle those relationships in both backward and forward directions.
Every. Single. Scrap. Of atomic data fragment. Needs an entry in a single monolothic WordPress table called (counterintuitively) ‘posts’, which it shares with every other piece and type of information of any and every sort throughout the entire system. Not one of them can be retrieved, edited or displayed without an SQL JOIN between two tables of now-arbitrarily-enormous size.
Essentially you’ve just described a custom system — every little plugin and content type would need to be created from scratch — but one which requires a single mandated root CMS that’s not designed for this purpose.
In what meaningful way is that remotely different from rolling your own system from scratch anyway?
Except for the built-in requirement for database inefficiency that’s required as part of the process, that is. Oh, and the need to roll up a security update for WordPress, which any regular user knows happens seemingly every few hours. Any one of which may alter the underlying fit-for-a-different-purpose CMS in such a way as to cause some or all of the massive custom additions to stop working.
And then eventually the entire WordPress underlayer falls out of security coverage entirely, loads of APIs change and the entire project may have to be done again. As custom work, again. At a similar cost, again.
Sounds like a recipe for a hugely-bloated government IT catastrophe in the medium term to me.
Simon, I think there’s a difference between a CMS performing one function very well and a whole range of line-of-business apps that need to support workflow, complex data structures, BI and other needs. Some of these needs can only be provided by bespoke apps. And they’re probably written in .NET. But that’s reality.
This is an interesting viewpoint, and it remind me that I really need to blog about the project I’m working on.
As a bit of background… I work for Technophobia, a smallish (80 employees) web dev agency in Sheffield. For the last 10 months I’ve been involved in the re-platforming of a major local council’s online platform – the website, forms, payments, social networking, online identity; basically any for of engagement that you could conceivably stick an “e-” or an “i-” in front of, if so inclined.
Based on the experience of this project, it is a *fundamental* mistake to start with a pre-conceived notion about the use of a CMS, let alone starting off with a single product in mind.
Assume for a second that the goal of a project is to provide a platform for collaboration, shared authoring and engagement with citizens (‘customers’ in LA speak) – and that’s the goal of *my* project, and I think @govuk; if you start from preconceptions about what your chosen platform is capable of, then you’re implicitly limiting your capability to shape the platform to the real needs of the business.
You have to start with a technology-neutral expression of user needs, based on:
– Requirements analysis of the “as-is”, so that you can ensure an equal level of functionality to what’s gone before (well, assuming it’s all being used – and that’s a big assumption)
– Requirements analysis of the “to-be”, from the perspective of the business (services, Ministries, departments, what-have-you)
– Iterative delivery, user testing cycles, feedback and cogent analysis of what’s important to *all* the stakeholders – customers, the business, IT, and ‘the community’.
That’s something that took a long time for us to get right. Many governmental organisations start with the immediate assumption (architectural principle, perhaps) that all software must be COTS, and vendors must fit into preconceived notions, rigid procurement cycles and archaic and inflexible project management methodologies. It’s about apportioning risk and blame, not getting a good result.
Getting the business to accede that responsiveness, agility and iteration – not “picking the right CMS” – were key to delivery is the biggest challenge we’ve faced; getting them to choose a community-supported platform (in our case it’s Magnolia that forms the content repository – not ‘CMS’ – at the core of the platform) was easy by comparison.
A good delivery team can work with anything – WordPress, Magnolia, .NET CMSes, bespoke PHP, Python, Java, Rails – spending time arguing about which software is better (vi? emacs? oh, wait, wrong flamewar) is a red herring, and only serves to mask the real goal here – enabling Government to cheaply, reliably deliver software that makes it easier to engage with people.
The only differentiators at a technical level should be “how does it support my requirements”, “how much (now and in future)”, “how fast”, “how maintainable” and “how easily can it be changed in response to shifting requirements”.
Despite my explicit statement above, people seem determined to interpret this as a proposal that everything should be built in WordPress. I didn’t say that, nor would I.
Of course there’s a need for some bespoke work; there always is. My point is that there’s a moral obligation to minimise it. Building from the ground up should be a last resort. And if we’re serious about open source, we need to be considering what the potential re-use is, and who the re-user might be.
This post doesn’t claim to offer an answer – I went out of my way not to. It questions what seemed like a rather odd assertion in Neil’s post. And I for one am keen to hear the answer.
I dispute this idea about “building from the ground up”.
Taken literally, that probably means you’re writing in assembler or building your own database engine. Everyone knows that’s not what’s implied.
So what’s the point at which you start building? What do you build upon?
Start too low and you’re wasting time writing generic functionality. You don’t need to write your own authentication or templating systems.
Start too high and you’re wasting time making square pegs fit into round holes. This is what happens when you try to make CMS (for example) do things that are outside its core use cases. You may be able to make it work but it’ll probably be brittle, inelegant and expensive to maintain.
Web application frameworks are pitched in the middle. They’re opinionated enough to know that you want to build a web application but not so opinionated that they mandate the type of application. Knitting together a set of core components for ORM, templating, authentication, routing etc. isn’t the same as trying to hack around someone else’s UI and workflow decisions, let alone their product roadmap.
Web frameworks are modular just like CMS. Even if no-one can re-use your bespoke application as a whole it’s likely that you’ll be able to create various plugins for parts of its functionality that could be used by others.
I think that’s exactly the question I’m asking, Adrian. Maybe I shouldn’t have phrased it in terms of ‘bespoke’ vs ‘not bespoke’: of course, it’s really a matter of ‘how bespoke’.
But we shouldn’t lose sight of exactly what was proposed in the diagram, and Neil’s brief summary of it. Neil’s ‘government machine’ isn’t about handling 70m NHS records, or processing the national accounts. It’s about publishing pretty basic information: news, policy, ministers, publications and consultations. Note, not handling the policy detail, or running the consultations… those are broken out separately (and rightly so).
Stephen Hale‘s post from a month ago, about the rethinking of the Dept of Health’s corporate web publishing strategy, is perfectly apposite here:
So ‘corporate web publishing’ boils down to news (posts), a few timeless information pages (pages), and date-stamped summaries of attached PDFs (attachments). Yes, it’s that simple. And yes, for the record, those are the three core WordPress content types.
Am I saying the entire complexity of running the country can be shoe-horned into WordPress? No. But I am saying that the information material a government department churns out can be. It already is. So if that particular ‘machine’ needs to be bespoke, and built from the ground up – I’m very keen to hear why.
SImon,
To re-hash the above, I think the point is that a lot of us fundamentally disagree with your suggestion that:
Instead, as Adrian’s point aludes, the Single Domain system would go far, far beyond a CMS to cover a huge amount of additional functionality. I’m sure that for the CMS-like tasks, the team will consider CMS-like tools and how they could be used.
But for automated data extraction, representation, manipulation and visualisation; transactional web forms built on top of high-performance APIs; and a whole heap of other necessary Web-delivered systems I’ve not even thought of, it doesn’t make sense to start from the perspective that this is an information-publishing platform. It’s so very much more than that.
I’m sure we all completely agree that we should be wary of re-inventing the wheel, and you’re right that too much bespoke code can lead to bit-rot and increase security/operational costs in the medium term – but so can trying to put “exotic” behaviour into a system that’s ultimately designed for something else.
Software can do anything, starting with anything. The art is knowing when it shouldn’t.
James.
Well done Simon for raising the red flag on this. As you say, Govt has ‘form’ trying to reinvent the wheel – and defaulting to over-thought, complex, owned solutions when workable, road-tested in the real world products exist aplenty and would be easier and cheaper. If ‘content is only one of many datasets’ at the publishing level (as TL says above) then I start to become worried that the single domain project will lose it original ‘streamlined’ mission. I should, sheepishly, also own up to being the former (and humble non-techy) civil servant who first proposed the single domain idea to Martha L-F and F Maude last September. What have I done? 🙂
Can I just say that its past Govt web procurement that has made ‘bespoke’ a dirty word. It’s possible to build bespoke and build relatively cheaply.
I’m part of a team doing it right now – one front end dev, one .NET dev, one prod dev (me). Annual cost is sub £200k all in and we’re getting a lot of bang for our buck.
I’m not disputing it’s possible; I’ve done it myself. And I stress, I’m not condemning the decision to go bespoke, merely questioning it.
You see, I think this project needs to do more than just deliver: it needs to stand for more than that. Such opportunities simply don’t come along very often. And (hopefully) I’m challenging the team to think wider than just what they’ve been tasked with, and what they’re planning to deliver against it.
If you can do awesome things, then great: I’ll praise you from the rooftops (and I hope my archives prove that). If you can do it in ways which are reproducible by others, elsewhere in the public sector or outside, even better. But if you can do it on a platform which is widely used, understood and trusted – and I’m talking beyond the uber-geek community here, as they aren’t the ones who need to be persuaded – then that’s the mother lode.
It’s a welcome challenge, Simon.
Don’t read anything into my silence on the issue. I was on holiday, and meanwhile all the best points have been made! I’m grateful to all the commenters here.
From my perspective as the product lead, I’m truly agnostic about the technology. But I’m passionate about making sure the dev team can bend it to my whim based on changes to, and evolving understanding of, user experience and business processes across a myriad of different organisations, from iteration to iteration. I find the view that starting with a product designed for one thing and making it do another is a constraint very persuasive, and in line with my experience of many a customised platform (including WP).
It’s a guiding principle in the team that we will use existing products wherever we can, and that potentially includes plugging them in to deliver parts of the core if it makes good sense to the devs and me when we get into the detail of it.
I would like to think that any ‘government machine’ actually lets you do things online, rather than just publishing reams of content.
That being the case, a CMS-based solution isn’t what’s required.
A thoughtful and detailed analysis of the data types and transactions involved is. No mean feat, considering they may change over time.
Only then is it worth thinking about how best to architect the system. Or systems – since the larger something becomes, the more difficult it becomes.
A single platform that lets you file a tax return, tax a car, cash in your Premium Bonds, choose a hospital… Can’t be that hard, can it? It’s not as if there are myriad bespoke legacy systems to interface with.
Sarcasm aside – the key thing about a single gov. domain is that it appears so to the user. Behind the scenes, you can either choose to build a big, do-it-all platform, or promote a modular approach where you work with each department and help them plug in.
The departments must be in control of their content, processes, and system interfaces. The rest can be achieved with open, documented frameworks.
Chris – yes indeed. Although I must admit to being mystified about the advantages of one entry point. I’m fairly happy going to HMRC for income tax, council website for local stuff and so on and so on.
I don’t carry around the map of sites/tasks in my head and generally get there in a few seconds thanks to google. Never had a problem with it really ! Do I need a shiny portaly thing to tell me where I live ? Not sure.