Where next for open data?

It's been a big year for the UK government's open data agenda. Chris Yiu reviews what's been achieved and asks what needs to happen next.

Image: CC-BY-SA Rubber Dragon

Way back in 2011, the government's consultations on Making Open Data Real and Data Policy for a Public Data Corporation left many people - including org  - wondering whether the government's heart was really in it. Now that the dust has settled around the government's recent open data white paper, it's worth pausing to reflect on where we are.

For the first time ever, the white paper was accompanied by a set of departmental open data strategies. The government's central data repository, data.gov.uk, has had a long-overdue overhaul. Following a raft of open data measures in the autumn statement, a new Data Strategy Board (DSB) has been established to advise Ministers on maximising the value of data from the Public Data Group (PDG) for long-term economic and social benefit. A new Open Data User Group will advise the DSB on priorities for open data releases. On the edges of government, a new Open Data Institute has funding to demonstrate and exploit the commercial potential of open data.

These are real steps forward. It's important to remember, however, that the big prize isn't really about getting more strategies published, extracting promises to review legislation, or breathless announcements about how many data sets are now online. The game is won only when open is the default position for public data – where openness is set out in the admirably clear Open Definition, and public data means everything bar that which would prejudice privacy, national security or sensitive advice to Ministers.

We are not there yet, but there is still hope. In his foreword to the white paper, Francis Maude writes "we are unflinching in our belief that data that can be published should be published". The white paper goes on to talk about "a culture that supports a presumption to publish... where data owners in the public sector look to release the data they hold... as part of business as usual".

As is so often the case when technology meets the public sector, the road ahead is more about people than it is about hardware, software or systems. Here then are three big challenges to overcome on the next phase of the UK's open data journey:

• Embedding the presumption to publish. The first pledge in the first chapter of the Conservative Technology Manifesto said "we will create a powerful new right to government data". Until this is achieved, we remain too reliant on forward thinking officials to get data out – fine while it lasts, but not the most sustainable approach if data really is the raw material for the 21st century. At present there is no strong incentive – carrot or stick – for public sector organisations to live by the presumption to publish. The government is grappling with a similar sort of cultural challenge as it tries to embed digital-by-default as the new operating model for the public sector.
  
  
• Unlocking core reference data in its entirety. Basic geographic data in particular is often the bridge that allows other datasets to be combined and put to imaginative use. Much more open data is coming out of the PDG (made up of Companies House, the Ordnance Survey, the Met Office and HM Land Registry) than ever before – but there are still questions to answer about whether the Shareholder Executive's commercial objectives can ever truly be squared with opening up all the public data they hold. The future of the Postcode Address File is another long-running saga that still needs to be resolved.
  
  
• Staying strategic. The recent surge of activity on open data has put an unprecedented number of bodies, boards, interactions and interdependencies in play (see page 18 in the DSB / PDG Terms of Reference for a simplified version of the landscape). These will need to find a way to work effectively together, and to avoid getting too distracted by micro demands at the expense of forcing a transformation in the way the public sector thinks about data. We also need to exercise caution whenever conversations start to edge toward sharing, linking or anonymising personal data. This is an important topic that merits more attention – but is well outside classic open data territory and the two should not be conflated.

None of these will be easy to achieve. Nevertheless, the open data revolution has the potential to deliver significant economic, social and political benefits. And the longer transparency beds in, and the further technology advances, the harder it will be for things to go backward. Sooner or later we will reach a world where public data is truly open. We just need to decide whether we are gutsy enough to head straight for it.

 

Chris Yiu is Head of the Digital Government Unit at the think tank Policy Exchange, where he authored the report A Right To Data. He is also a member of the new Coalition for a Public Interest Data Policy and sits on the Data Strategy Board. You can connect with him on Twitter: @PXDigitalGov