I picked up on the twitter conversation between my friend James Rutter and folk at CycleStreets about an application to Ordnance Survey’s GeoVation program for “a modern interface to UK wide planning application data“- note that OS is offering a share of £101,000 to the winners of this challenge, that’s quite a lot of taxpayer dosh. There was a tooing and froing about whether yet another initiative was needed to aggregate planning information. I have written before about innovation competitions and hackathons, questioning whether they represent a good use of taxpayer funds and whether they stimulate any sustainable innovation, is this another example that supports my sceptical point of view? Innovation or …
Sometimes you just know that someone else knows way, way more than you do about a subject and in this case I reckon that my friend James Rutter is that someone, James is the GI Manager for Surrey Heath Borough Council, an open source and open data evangelist and he just happened to be part of the team that built the Surrey Planning Hub with a bit of help from my friends at Astun.
What James has written here is in his own name and does not represent the views of his employers SHBC:-
There’s a lot of secret and weird stuff happening with respect to online local authority planning applications data at present. My recent experiences include having a comment that I left about UPRN numbers on an Ordnance Survey blog post removed, my Twitter account monitored (the merest whiff of a mention of UPRN numbers resulted in a phone call from the OS) through to calls from the PSMA Secretariat at BIS acknowledging the need to deal with my data exemption request with urgency, to the OS profoundly apologising to me for refusing to answer my emails which were chasing the outcome of said data exemption request.
The impetus for this post was catching sight of the Ordnance Survey Geovation Challenge finalists list . The irony of seeing ‘A modern interface to UK-wide planning application data’ is profound. Surrey authorities have built one already in close liaison with a similar effort from the chaps at the Hampshire Hub. Both these complementary projects received a not inconsiderable volume of taxpayers cash from the Data Strategy Breakthrough fund jointly administered by BIS and the LGA. Through a recent Twitter conversation it transpires that Northern Ireland has also built a very slick planning hub.
The premise of the Surrey planning hub revolves around Surrey districts publishing a daily XML feed of recent planning application data. Different authorities use different planning systems with different data schemas so key to this work was defining a common way to express planning information. To this end, a schema was devised at a workshop hosted by DCLG and attended by Hampshire and Surrey authorities, DCLG, the LGA (transparency and ESD staff) and chaired by the director of the Local e-Gov Standards Body. Usefully this work also coincided with the Local Open Data Incentive Scheme jointly run by the LGA and the Open Data User Group with Cabinet Office funding which recognised the work done on the planning schema and thus began to apply one-off payments to authorities to produce planning data to this standard (to date 21 authorities are publishing their planning data to this standard). With the recent spate of publication requirements on local government such as INSPIRE, FOI, the Transparency Code, all authorities should have tools (or if they haven’t they need to acquire them) to publish data to the web in different formats (including XML).
The Surrey hub takes these daily XML feeds, does some integrity checks, aggregates them and makes them available to the public through a simple API which can be queried in different ways. If developers want to put a map interface on top of the data they can do that too. The point is, that as local government we’re not interested in building expensive apps, but more interested in publishing our data in generic and ubiquitous web formats. The code which runs the Surrey hub is open source and is available for re-use by anyone.
So, the code is re-usable. But, no data publication discussion would be complete today without the usual cries for it to be ‘open data’ to facilitate its re-use. Issue one…the UPRN number. Most at the DCLG schema workshop were labouring under the misapprehension that the URPN would be open data. It’s not! The ‘open data movement’ as far as data that is remotely connected to the Ordnance Survey is caught in high level wranglings between the Shareholder Executive and the Treasury at one end (who are desperate to hold onto the income streams from heavily licensed data) and pressure from various bodies like the Open Data Institute, the Cabinet Office and the Open Data Users Group and some local government activists at the other. We awaited an announcement on the UPRN number during the Autumn Statement which arrived and passed. Now we wait for a new year announcement. My understanding is that government have told Ordnance Survey to say absolutely nothing to anyone until this announcement (hence the radio blackout on the formal submissions to the OS for data exemptions from Hampshire and Surrey!) It’s right that there should be funding discussions for organisations, after all as far as two key players with regard to UPRN numbers go (Ordnance Survey and Geoplace), they have to be paid for somehow.
All I will say is that it is morally wrong for anyone to claim a right or commercial value on the UPRN number. Having been around in the GIS and Gazetteer world pretty much since the inception of the UPRN I can tell you it was designed to join up different and various datasets, not for someone like Ordnance Survey to make commercial gain from it. The reason the UPRN is not open is because Geoplace claims a ‘database right’ on the number because it hands blocks of these numbers to authorities to ensure their uniqueness and no sequence overlaps between authorities. Ordnance Survey is contracted by Geoplace to commercialise the national gazetteer with it’s incorporation into it’s Addressbase products.
So to cut what could be a long ramble short, lets assess what we already have as far as a national planning hub goes…
- We have a data schema (it needs some tweaking but nonetheless it’s there)
- We have at least 3 planning hub technologies already
- We’re likely on the verge of some announcements regarding the ‘openness’ of what we want to publish with respect to planning.
To wrap this up, lets take advantage of all the work that’s already been done on planning data….consolidate it, improve it, build on it and most importantly, lets not spend more tax payers money by reinventing the wheel!
5 thoughts on “Ooooh – a round thing with spokes!”
Steven,
Thanks for this. As you know, I’m the person that submitted the proposal to the GeoVation challenge for some seed funding to get a UK-wide data feed system going.
Everything you’ve written is entirely sensible, and I agree with all of it. Except you have not addressed the key point, which is *UK-wide*.
What I propose the need for is that there should be a *single* endpoint feed of key planning application metadata, at a ultra-flexible API level, which forms a single endpoint that works for anywhere in the UK.
The use of the LGA schema is very sensible.
Initial data feeds from forward-thinking LAs like Surrey and Hampshire are very sensible and all work that does not need to be repeated.
All I am suggesting is aggregation of these, generalising the differences and multiple API URLs, and then (crucially) picking up the 95% of LAs that currently only publish in HTML (scrapeable format). No-one is going to build a web app or anything which has to go to hundreds of individual API endpoints. That is why an aggregator is needed. As more and more LAs switch to the schema (how long will that take?), the scrapers can be disabled and replaced with connecting the schema API output to the aggregator.
As far as I can see, that is not duplicating any work. Nor is it reinventing the wheel. At present it is not possible for someone to make even a simple app that will e-mail me whenever there is a planning application within 200m of my house (say), wherever that house is in the UK. Is there?
To be clear: I regard the Surrey and Hampshire work as massively helpful to aggregation. It eliminates a whole pile of annoying messy scraping stuff. None of it is money wasted, and any new system is not working at that layer level.
So we agree with each other then!
Granted, the “UK” bit is the difficult bit. Local gov is not like central gov and trying to get each local authority to do the same thing is like trying to herd cats. We have a schema (which, as I said, in my opinion needs tweaking but is essentially defined) and we have technology to publish planning data to this schema irrespective of underlying planning systems and now we also have scaleable technology to aggregate planning feeds into a single entity which could scale to incorporate all planning authorities in the UK. I agree with you that some sort of push is required to start joining this work up which is where the role of the LGA comes in. For information technology is already in place to enable you to receive alerts of planning applications within 200 yards of your house (in some authorities!) Take Spelthorne for example, you can sign up to exactly this type of alert here.. https://my.spelthorne.gov.uk/?tab=3 and this sort of thing will be available from Surrey Heath soon.
Surely if funding is available it could be used to extend the capability of Local Govt to deploy open source tech to publish their planning apps to a standard schema building on the work done at Surrey and Hampshire rather than scraping and aggregating etc?
Agree that an aggregator would be a useful feature, but that becomes much easier if all of the contributing feeds are standardised. Let’s get it right first time (well maybe 3rd or 4th time cos others have already tried)
S
> Surely if funding is available it could be used to extend the capability of Local Govt to deploy open source tech to publish their
> planning apps to a standard schema building on the work done at Surrey and Hampshire rather than scraping and aggregating etc?
Of course the ideal is that every LA to move to the Schema, and then an aggregator is a relatively trivial task.
But realistically, Steve, how far does £25k go in terms of getting literally hundreds of LAs all to amend their systems and business processes? Moreover, how long? Five years maybe? I don’t want to wait five years to get such data available.
As you say it’s been done before, with PlanningAlerts.com, but the key issue is sustainability. Scrapeing has the benefit of quickening where we actually want LAs to be (basically: “copy Surrey”) in that it demolishes the argument of any LA who may think it undesirable for planning application data to be widely available for whatever reason. Making it de facto pushes the problem up the agenda and gets something useable going, pragmatically.
> Granted, the “UK” bit is the difficult bit. Local gov is not like central gov and trying to get each local authority to do the same
> thing is like trying to herd cats.
Exactly.