How complex can this Derived Data thingmy be?

Gosh we seem to have been talking about Derived Data for such a long time and still we don’t seem to making much progress. Every time government reviews Ordnance Survey it stumbles on the thorny subject of Derived Data.

Nearly 18 months ago when the short lived new business strategy for OS was launched by a now departed minister there were promises that the issue would be resolved soon. I consulted for OS on GeoVation last year and had several conversations with people across the organisation all of whom explained how complex the problem was and assured me that they were working on reducing the extent of the problem by providing clarity on what was and wasn’t Derived Data, this is the Derived Data Dilemma – which I may refer to as DDD for the rest of this post. October 2009 was a target date but that slipped, then the PM launched his Making Public Data Public initiative and whacked a great hole in the OS ship by freeing up quite a lot of OS data. Many people raised DD as a/the key issue in the consultation at the beginning of the year, some were even foolish enough to expect a resolution in the response to the consultation (come on guys this is complicated). What we actually got was what sounded like a fairly strong commitment in the government response:

“3.24 Ordnance Survey will also be proposing changes to the derived data policy for the commercial sector, including ‘Free To Use’ data, as part of its work on revised pricing and licensing. Further details of when these changes will be implemented will be communicated later in the year. In addition, Ordnance Survey will work with Cabinet Office and the Office of Public Sector Information (OPSI) to agree guidelines and processes which would permit certain other datasets created by public bodies and containing licensed paid-for Ordnance Survey data to be made available for unrestricted, including commercial, re-use. In agreeing these guidelines, the parties will need to balance a number of issues, including the significance of the Ordnance Survey data taken, the impact of release of any dataset on Ordnance Survey’s commercial business and any legal or regulatory consequences for Ordnance Survey. This reflects concerns expressed in the consultation responses about licensing of Ordnance Survey products, and in particular derived data restrictions.”

Now read quickly this may not have sounded too bad, a bit woolly but the sentiment and general thrust were encouraging. Now read my italics again, look who is doing the proposing and also note the mention of a DD policy – the core of the DDD is that there is no policy or not in the sense that most of us would understand a policy, a document that we can refer to that perhaps includes definitions and procedures. Much of the confusion that exists around DD is due to the lack of clear definition of what is and isn’t derived data and what you are and aren’t allowed to do with it if it is derived.

So cut to yesterday and after a bit of a PR warm up on Twitter the OS publish an announcement about DD on their blog. You can decide for yourself whether this constitutes a resolution of the DDD for public sector bodies within the PSMA and any commercial organisations that want to use data that has been created by a PSMA participant using OS licensed mapping as a background. Sure DEfRA can share some of their data with farmers but what about releasing it to a voluntary or commercial body who wishes to analyse the data and either hold DeFRA to account or sell services to farmers to help them get the maximum possible subsidies and grants from DeFRA?  In this era of Making Public Data Public and increased transparency one might have hoped for a simple statement enabling people to get on with using, sharing, reusing and even adding value and possible making profit from public data derived in part from an OS map. Something along the lines suggested by Ed Parsons a while ago (wildly paraphrased by me) “if it isn’t on the map then we make no claim to it” would have been good. Perhaps I am being a little harsh and there is much more clarity available to PSMA participants, if so please OS can you share with partners, consultants and others who need to understand and advise our clients what is and isn’t possible in this new liberalised DD policy – a plain English document maybe?

Mentioning Ed Parsons brings me to the second part of the OS blog post, despite claiming that the PSMA heralds a new era of data sharing and usage the same old roadblock remains if a council or police force wants to present its data to the public or internally using Google Maps. OS and Google have been unable to reach an agreement for several years now over the conflict between Google’s licensing and that of OS. I am not going to repeat the debate here if you don’t know what they are arguing about you can get a sense of it in the comments on the OS blog or on Ed’s blog. A substantial majority of people would prefer to use the Google Maps API to build their mashups (it’s free for public sites, has a lot of powerful functionality, is very scalable, has global coverage etc), they don’t want to be told that they can use another API which will cost them money or offer less capability (NB statement of interest – I have an involvement in a company that creates webmaps on both Google and Bing’s API’s). There must be a resolution to this, I can’t believe that Google wants to misappropriate any OS IPR and I suspect that the clause in the Google API that OS are so concerned about exists in other API’s but in a slightly different format. I agree with the sensible comment from Thierry Gregorius:

“Would OS and Google care/dare to draft a joint statement on the derived data matter? If it is purely about interpretation of facts I don’t quite see what the difficulty is, except that communication via keynotes or blog posts is probably not sufficient to make it happen.”

Please guys, there is a mass of public geodata and creativity that are just waiting for the opportunity to combine for the public benefit, help us to make it happen.

And now it is time for a cup of strong coffee ..