Maybe, we shouldn’t create new OpenData?

This week I received a mail from Ordnance Survey inviting me to attend a launch of some new OS OpenData

For the last few months, we have been very busy creating prototype OS OpenData products:
1. OS Gazetteer of Great Britain – consisting of settlement names, road names and numbers and postcodes and their locations.
2. OS Street View^® – an enhanced street-level product that includes a lot of new map data features and content with an improved cartographic style so that it fits much closer with our OS VectorMap^® family.
We want to build products that you will love to use and are looking for honest, sharp opinions from knowledgeable people like you in order to tell us what we are doing wrong and, hopefully, what we are doing right.

I wondered why OS were creating new open data products, surely the idea is to release what you already have as open data with minimum effort and cost not to create new products? But hey ho, life is too busy to worry about such things and anyway a bit more open data must be a good thing, mustn’t it? That would have been the end of the mail had I not spotted a tweet from Heather Savory, the chair of the Open Data User Group

@odugUK @CommonsPASC OS proto OpenData Gazetteer – GB settlement names, road names/#s, postcodes & locations showing @ GeoBiz2014! #opendata

— Heather Savory (@SaturnSA4) May 8, 2014

Heather seemed to be pleased that OS had released some more open data and I was puzzled why I wasn’t in there alongside her cheer leading for this new release of open data. We started a conversation on twitter but I couldn’t articulate my reservations in 140 character chunks so we took the conversation offline. What follows is based upon my mail to Heather (so the thoughts are mine not hers).

The original case for open data as articulated by the Free Our Data campaign and others was that public bodies which created data in the course of delivering their public tasks should by default make this data available as open data free of costs and restriction regarding use and re-use, assuming it was not personal data. This was (and remains) unarguable in my opinion. Advocates claim that the release of government open data will encourage Transparency, Accountability and Innovation – I am a passionate supporter of the Transparency bit. I have some reservations on the Accountability part but I think there are signs that it may be working. I think that at best, we would have to say that the jury remains out on the Innovation part, particularly with regard to the incredible economic benefits that were asserted which were in my opinion without any realistic basis.

Where is the ramble going you may be wondering? The problem case for me is Ordnance Survey, which was the prime target of the original FOD campaign. OS doesn’t ‘create data in the course of delivering their public task’ creating data is their sole task.

There was a protracted battle over opening up OS data which is the essential glue for making much of the rest of government open data work. The OS could not afford to give away a chunk of data that it previously sold while continuing to operate within a business model which requires them to charge users and recover the costs of creating and maintaining that data (I know it is not as simple as that one liner, I recognise some of the complexities of the current and alternative business models). When it was clear that government was going to insist on some OS data being released as open data, OS senior management fought a successful rearguard action to mitigate the consequences, which resulted in OS receiving payment from government to offset the lost revenues/costs of releasing previously chargeable products as open data (allegedly £20m in the first year).

Apparently the OS advised or determined what products would be released, they even went as far as to create a new product – VectorMap District, I am not sure why. In my opinion, OS released way more than the market was asking for or needed (postcodes, admin boundaries and a street centreline product were the important data sets, it’s debatable whether the remaining products with the exception of VMD are of any great use to people interested in open geo data).

OS now has a portion of its revenue dependent on convincing government to keep paying them to maintain these open data products. Several years on we have little publicly available information about the volumes of downloads (by product) or evidence of innovative use to enable us to judge whether this has been worthwhile.

Today, OS employs several people in roles to liaise with the ‘developer community’, run masterclasses to teach people how to load the data into a GIS and to run programs to ‘encourage’ innovation with OpenData through competitions and events. These activities are presumably funded by the open data release payments made by government. These may well be ‘good activities‘ but what does it say about OS OpenData that they need to spend money on such promotion and facilitation? Surely what was originally intended was releasing data as is and allowing the market to work out what to do with it for analysis, investigation and innovation?

Creating new products, specifically to release as open data does not seem to me to be in line with original motivation behind government open data, particularly in a period of austerity. What is the basis, market demand etc for these new OS OpenData(c) products, how many people will use them, what benefits will be generated? Who was consulted? Open data should be a costless or near costless by-product of other public sector activity rather than an activity needing funding to create new data products.

In my opinion, we need an open and robust evaluation of the downloads, usage (which may be different to downloads) and costs of maintenance and release of each OS open data product. It might identify which products the market is using and potentially could allow some low usage high cost data to be withdrawn with a reduction in the level of government subsidy for open data.  Perhaps we also need to assess and review the ancillary costs of advocates, marketeers, trainers and download facilities (I have never understood why OS OpenData couldn’t have been served from an FTP site or similar and catalogued on data.gov.uk).

Do we need more OpenData? Yes we do, but we don’t need to expend effort on creating new products, we need to release more of what already exists and is locked away due to resistance, IP and business models. If you asked most people in the geo arena what additional data they most want, they would probably respond “addresses”. So rather than creating some new raster products or a place name gazetteer, why not release a basic address product as OpenData? Something like the old AddressPoint would be a great starting point.