Is the door on #OpenData opening or closing?

She left the door open by H Kopp Delaney http://www.flickr.com/photos/h-k-d/

On Tuesday evening I was “in conversation” with Jonathan Raper (a.k.a. @MadProf, very apposite IMHO) at the British Computer Society arguing, agreeing and venting on the topic “Open Data and the PDC: Whose data is it anyway and who pays for it?”

As most readers will or at least should know, the Cabinet Office are consulting at the moment for views on “Making Open Data Real” and “Data Policy for a Public Data Corporation“. I started out by setting out what I thought were the key points of these two interlinked consultations trying to be as neutral as I could, this short slide deck hopefully provides a good start for anyone who hasn’t read the full documents. Jonathan followed with his own perspectives of how the thinking re OpenData and the PDC was flawed by a lack of vision of the long term infrastructure investment needed around OpenData, his slides are here.

I think we both agreed that opening up as much non personal government data as possible is a good thing. Me, because of the potential democratic gain that comes from transparency and accountability and Jonathan, because he sees a great opportunity for new information services and businesses to flourish on the back of this flood of data. My skepticism on the size of the new economy generated via startups and Open Data is well known and does not need repeating here, suffice it to say that the Treasury appears to share my doubts and that grains of evidence to support Jonathan’s view have been difficult to find (I look forward to being proved wrong on this).

Jonathan wants to extend the free release of financial, performance, transport and other non personal departmental data derived from operations to include the data created by the Trading Funds that are being considered for inclusion in the Public Data Corporation. I am uncertain that the lost revenue from making the data free would be offset by either some significant further democratic gain or the new business opportunities.

It is a shame that the two consultations have been interlinked because the good stuff in the Open Data consultation may well be tarnished and confused by the flawed consultation on “data policy” (a euphemism for pricing and licensing) for the PDC. The PDC consultation looks to me as if the decisions have already been taken and it is going to be pretty much “same old same old”. OS, Land Registry and Met Office will be merged into a holding company with policies on pricing and licensing harmonised as much as possible across the three (which will be quite challenging because as businesses and data providers they have very different profiles). The big opportunities to consider radical change such as restricting them to collecting and supplying raw data only, moving towards a utility model, outsourcing major parts of their function or creating a custodianship franchise that is retendered every 7-10 years were all excluded from the consultation. Why? Because the PDC has to capable of “attracting external investment” (another euphemism for full or partial privatisation). On the plus side the PDC consultation does talk about a continuous process of releasing more data from the PDC businesses as Open Data, I wonder whether that will fall by the wayside in the name of attracting external investment.

In contrast, the Open Data consultation has some very good stuff on an enhanced right to access data, the use of machine readable formats and open standards and releasing dirty and early rather than spending time and money on trying to get perfect data out there. The bit that worries me is the section on innovation and economic growth

“Open Data can be a driver of economic growth.  A new market for public service information will thrive if data is freely available in a standardised format for use and re-use, particularly in the life sciences; population data mining and risk profiling; consumer technologies; and media sectors. At present the market for information on public services is highly underdeveloped. Open Data across government and public services would allow a market in comparative analytics, information presentation and service improvement to flourish.  This new market will attract talented entrepreneurs and skilled employees, creating high value-added services for citizens, communities, third sector organisations and public service providers, developing auxiliary jobs and driving demand for skills.”

This seems to be a triumph of optimism over evidence with only some rather loose examples of current best practice cited. In the case of the GI industry and the AGI report quoted, I don’t think that any of the growth in the market size that was predicted was dependent upon the release of OS OpenData or any other government Open Data. I am aware that OS recently commissioned a study on the economic benefits of releasing of OS OpenData, to date that study has not been published nor has it been referenced within the best practice section of the consultation. Hopefully we will get to see the results soon, after all under the principles of transparency espoused by the government we ought to be considering all of the available evidence in this process of consultation.

The consultation goes on to ask the question:

“Innovation with Open Data: to what extent is there a role for government to stimulate enterprise and market making in the use of Open Data?”

Elsewhere the consultation seeks opinions on whether government has a role in stimulating or supporting innovation. My answer to both of these questions is “No, No, No!”

Surely by now we have learnt that government does not do business or innovation very well. Who remembers the telephone service when it was run as part of the Post Office? That was government doing business in the technology sector. For all of its faults, BT is a vastly better business for being freed from the constraints and culture of the Civil Service. Surely a Tory government gets that? What are they thinking of?

“It is possible that investment will be required to equip PDC organisations with the infrastructure and resources to make accessing its data easy and cost-effective forusers. The business model for the PDC will need to consider the most effective way of meeting those investment needs, within the broader objectives for a PDC”

I am stunned to find myself, an old soft liberal lefty type, saying this but the Government needs to pay a bit more attention to its Tea Party wing on this topic not invest more money in building the infrastructure that the private sector will build if it really believes that the market exists for adding value to government information. Just release the data in the rawest form possible and at the lowest possible cost and let the market get on with it. Michael Cross wrote an excellent piece on this theme in yesterday’s Telegraph

“The consultation enthuses about the opportunities the corporation will have for developing “new, innovative ways of making data and information available”. Perhaps. But monolithic Whitehall-created corporations do not have a strong track record of innovation, and creating one seems an anomalous step by a Conservative government committed to localism.

If we’re serious about the potential of open data, let’s do it properly. Central government could enforce the new philosophy on existing public bodies without establishing new ones. The funding gap could be filled by central funds, user charges (for example property developers paying for changes to maps) and by doing fewer things to start with.

This means getting out of the commercial sector entirely. After all, the state no longer tries to earn revenues by building telephones and ships: what makes it think it can do so at the cutting edge of the knowledge economy?”

I think that the fundamental principles behind the Open Data policy are pretty good, even if this government may be doing the right things for the wrong reasons. We need to encourage government to open the door a bit further and to keep out of the added value chain that it believes will generate economic growth through innovation. We won’t do that by burning energy and enthusiasm trying to prise the Trading Funds’ data out of the PDC, leave that battle for another day.

Whatever your opinion, if you care about this stuff you should take time to read the docs and respond to the consultations.

#OpenData – who profits?

Pot of gold .... thanks to mockaitis http://www.flickr.com/photos/mockaitis/

This week I picked up some comment around the last in a series of Open Data Master Classes. I think a lot of the content of these classes was around the use of OS OpenData™ but the comments and following thoughts probably apply to all public sector data released as OpenData. The conversation was about the constraints on startups building businesses on Open Data. Jonathan Raper had recently written this blog post explaining the challenges that small businesses experienced in trying to work alongside/in partnership with government in this area. While I have some sympathy with his view, I am not sure that he really makes the case for the economic benefit and opportunity from OpenData. It seems to me that the promised innovation is slow or non-existent (particularly in a commercial sense), data.gov.uk has been open for nearly 2 years and the London Data Store has been around for about 18 months, surely we should be seing some shining examples by now?

Quoting from Jonathan’s list, the principle players are:

• OpenlyLocal promoting local government transparency
• Placr aggregating public transport information
• CycleStreets open access to cycle routes
• Level business opening Companies House information
• MySociety campaigning with open data on transport, housing and transparency
• Open Knowledge Foundation creating standards and technologies for data releases
• OpenStreetMap crowd sourcing a map of the world
• The Stationery Office government data publishers

No doubt you can point to some others that Jonathan has missed (itoWorld for example, maybe they are too competitive to placr?) but just for a moment let’s take stock of the organisations that are listed.

I am going to discount The Stationery office because I don’t think the innovation and new business theme of advocates of OpenData intended the new businesses to start up within government agencies or be funded by government until they could be privatised! Let’s have a look at the remaining 8 including itoWorld and categorise their business models. Incidentally 2 of the organisations (CycleStreets and OpenStreetMap), to my knowledge, are not using public sector OpenData, they are based on crowd sourced data which is rather different.

I have broken things down into a few broad brush categories and then assigned companies to one or more category as they may have a mixed revenue model.

• No apparent business model at this stage – 2 companies.
• These companies may not be seeking to build revenue, the sites could be a showcase for development skills and they may earn money from contracting.
• Alternatively they may be searching for a revenue model and it just isn’t yet clear to them (or at least to me from their sites) what that model is.
• Grants or other funding from the Public Sector – 4 companies
• In a time of austerity, relying on Public Sector to fund you with research grants or project based fees for manipulating OpenData may be a tad risky, funding may be curtailed or opened up to competitive procurement.
• Consulting – 2 companies
• Of course consulting is a viable business model but if the consulting is to help public sector bodies get to grips with the challenges of OpenData this is hardly the creation of the vibrant innovative high growth businesses that some advocates have predicted.
• Charity/Donation funding – 3 organisations
• Of course we want a 3rd sector but is the Open Knowledge Foundation really an example of an innovative startup that will create employment and new tax revenues?

No doubt my scepticism will be greeted with howls of “traitor”, “judas” etc and perhaps some of the organisations listed by Jonathan may want to follow Jonathan’s lead and set out the opportunities to build a business around OpenData in the way that he has done in his blog.

So what’s the point of this blog? While I have been a strong supporter of opening up data for 21st century democracy, supporting Transparency and even possibly Accountability (but see some reservations below and my previous posts and presentations on this subject – accountable to who?) I have always doubted the fantastic numbers (where on earth does that preposterous but much quoted £6bn come from?) that some advocates of OpenData have placed on the innovation gain.

With OS OpenData™ (can I say how much I dislike OS’ attempt to attach a trademark ™ to OpenData) the economic benefit seems to be largely down to the cost savings that some previous purchasers of the data have made and of course the important democratic gain of having some core reference data sets that can be used to visualise much of the other OpenData (the uncopyrighted or trademarked government type). Now if someone at BIS would give the Royal Mail a righteous kicking and get the PAF out of their grasping fingers we would have the missing part of Dr Bob Barr’s Core Reference Geography (yes I know NLPG/AddressPoint/AddressLayer/GeoPlace would be even better but PAF would be a start). I know an exercise has been commissioned to try and establish the economic benefit of opening up some OS data but the results have yet to be published, I look forward to considering the results if they ever see the light of day.

Personally I am very doubtful that there is a great deal of opportunity to generate profitable new information products and services on the back of public sector OpenData. I just can’t see colossal opportunities in bus departures, live train info, government expenditure data, planning, environment, health and crime data which are some of the topics that have attracted the most clamour from advocates. Of course there will be some small businesses that will survive or flourish but this is not the great information nirvana that we were promised by Rufus Pollock, Sir Tim Berners-Lee and Nigel Shadbolt. Maybe I am too impatient but in the fast moving world of digital, with all that suppressed entrepreneurship surely we should be seeing a bit more by now?

One last thought. OpenData is not without cost, infrastructure, API’s, web feeds, data maintenance, cleaning etc all cost money and require some level of human involvement at the moment. That is cost to the public sector that is making it available, possibly some innovative startups can do this more efficiently than government IT folk but there is still a cost to doing it.

Tony Hirst has done some great stuff visualising OpenData and would be considered by most to have impeccable credentials in this domain. In his blog post “So What’s Open Government Data Good For? Government and “Independent Advisers”, maybe?” he summarises some of the arguments for OpenData and looks at how businesses might monetise OpenData. He reviews the commercial models and suggests

“When all’s said and done, though, the biggest potential is surely within government itself? By making data from one department or agency available, other departments or agencies will have easier access to it.”

Tony manages to wrap up the accountability bit (remember that) with the commercial stuff with

“Maybe then, there is a route to commercialisation of public facing public data? By telling people the data’s there for you to make the informed choice, the lack of knowledge about how to use that information effectively will open up (?!) a whole new sector of “independent advisers”: want to know how to choose a good school? Ask your local independent education adviser; they can pay for training on how to use the monolithic, more-stats-than-you-can-throw-a-distribution-at one-stop education data portal and charge you to help you decide which school is best for your child. Want comforting when you have to opt for treatment in a hospital that the league tables say are failing? Set up an appointment with your statistical counsellor, who can explain to you that actually things may not be so bad as you fear. And so on…”

Maybe the PDC are right to be cautious about the economic benefit of opening up more public sector data.

Enough said, several hostages to fortune and the future left here. Perhaps someone will offer a convincing explanation of why I have got this completely wrong (sits back and waits)

Is the PDC slipping into the long grass or getting dressed up for a sale?

Thanks to siaronj http://www.flickr.com/photos/59489479@N08/

The Government published an update to it’s business plans today. The idea that a government has a business plan that pulls together all of its initiatives and targets and monitors progress seems quite refreshing and open. This document tracks changes in the plan which is very useful for monitoring progress. It looks as if there is quite a bit of slippage in the plans for the Public Data Corporation.

Have a look at 2.6.i and 2.6.ii in the Cabinet Office section. The establishment of the PDC has been put back from April to December 2011, while the lead responsibility has been transferred to the Department for Business, Innovation and Skills.

It could be pointless to speculate about the reasons behind these changes but since it’s a Friday afternoon, let’s have a bit of speculation.

Firstly the transfer of leadership from the Cabinet office to BIS.That looks as if the tussle over whether PDC was a vehicle for more efficiently exploiting government data assets or making more public data public on easy to understand and reuse terms has been resolved in favour of the Treasury and deficit reduction.

The delay till the end of the year could well indicate that someone has woken up to the fact that this is not a trivial exercise, particularly if you have privatisation or some other commercial model in mind

Alternatively it could just be that the PDC is little more than a an idea that has no substance yet and BIS will need a fair amount of time to work out what they are meant to be doing.

Conspiracy or cock up?

Whose data is it anyway?

Thanks to opensource.com http://www.flickr.com/photos/opensourceway/

Does anyone know how the Public Data Corporation is progressing? It seems that there is some form of conversation ongoing following the initial web based forum for comment and suggestions but I am not sure who is talking to who. The latest announcement from Francis Maude is pretty opaque and could be read as “everything is still under discussion” or as “we have pretty much made up our minds”

The minister also said that although most public sector information will be available under the UK Open Government Licence, launched in September 2010 by The National Archives to make it easier to re-use government data, there are some instances where this is not possible.

He said that examples include where third party intellectual property rights are present, or where charges are required to ensure the quality of the data.

Read that and try to work out whether OS data will be in or out. Is this about opening up public data for transparent government and possible innovation or creating a vehicle to capitalise on the commercial potential of public data asset?

Last week I was chatting with with someone who I thought should be in the know and he suggested that there was a need for more options for consideration. So in case the decisions have not yet been made here is my two penneth.

A while back I wrote a piece suggesting that the 10 year PSMA had made OS somewhat more privitisable than previously and that Sir Rob Margett’s comment that a free data model would cost £500m retrospectively made sense. Now I am not suggesting that there are plans to privatise the OS or that that is part of the PDC agenda, I have no inside information on this. But if anyone in the Treasury, the Shareholder Executive or the Cabinet Office is pondering the question of privatisation here is a thought or two for you.

We now have a de facto monopoly in the management and provision of large scale geographic information and addresses as a consequence of a 10 year public sector supply agreement and the unrealistic costs for a new entrant to build up comparable data. In 10 years time it is unlikely that any alternative sources of supply will have evolved – no doubt that bold prediction will prompt some spirited disagreement, bring it on. One might take the view that a lack of competition and opportunity will limit innovation and the delivery of best value to the tax payer, that seems to be thinking behind the proposed changes to the NHS, “what’s good for the goose is good for the gander” perhaps.

Perhaps there is an answer and  surprisingly (well at least for me) it can be found in the old LGIH/NLPG custodianship model. Perhaps the future for the PDC is to retain ownership of the National Map while letting a contract for the continuing collection, enhancement, processing and commercial exploitation (sale to the public and private sectors) for a fixed period. At the end of the contract period there could be a competitive bidding process for companies to take on the custodianship of the data for the next contract period while the data would always be retained as a national asset. This model would work regardless of whether the commercial model was to recover costs of data maintenance through sale or by a central government grant making the data free to use for all. Incidentally it could also be a model for PAF if government is planning to privatise Royal Mail.

Of course privatisation of any of the PDC assets may not be on the agenda, assuming the Cabinet Office, Treasury and Shareholder Executive have a common agenda. But if it is, custodianship may be a model worth considering.

Publishing Local Government geodata on Google Maps

I could have inserted an "F" word here. Thanks to Cudmore http://www.flickr.com/photos/cudmore/

Last year Geo.me were in discussions with a potential partner in Local Government and the dreaded derived data question came up, something along the lines of “we’d love to work with you but …”

There still seems to be quite a lot of confusion about what data Local Government (or for that matter any public sector user of Ordnance Survey base maps) can or cannot publish using the Google Maps API. I thought that I would try to produce a simple outline of what people could and couldn’t do. Given that OS had published some new guidelines on derived data I foolishly thought it would be a fairly simple task.

After several iterations (shared with OS and Google) and some helpful input from the licensing and legal folk at OS I finally handed over a finished version to Geo.me who have published it here. I must stress that this is my view of how PSMA members can publish data using Google Maps, OS have not officially sanctioned this view although I don’t believe that they materially disagree with it (watch the comments fill up on this post). Hopefully my version is a little easier to read and understand than this FAQ on the PSMA web site.

It is disappointing that we still cannot freely publish all public sector corporate geodata on top of Google Maps as part of routine business activity in the public sector. It costs the public sector and tax payers dearly in terms of usability, software license fees and infrastructure costs. Why does this continue to be a problem (particularly as OS are keen to point out that publishing on Bing Maps is OK, shame the API is less popular than Google’s and offers less usage for free)? Having spent over 3 months of wrangling, discussing etc, it seems to me that there is a lot of lawyer facing off going on here and somehow common sense is being suspended. I really don’t think Google wants to appropriate any OS IPR but OS lawyers remained stressed about their interpretation of Google’s T&C’s.

In their FAQ OS say

We have sought official clarification from Google on these points, and suggested alternative drafting that would resolve the issue from our perspective whilst, we hope, satisfying Google’s need to develop their service unencumbered for the benefit of their users. We understand that these proposals are receiving active consideration from their lawyers and we are hopeful that our recent positive engagement and experience with Google will result in mutually agreeable terms being adopted.

My response would be “Please get a move on”. There is business to be done, tax payers’ money to be saved and better public facing mapping experiences all waiting on a full resolution.

In the meantime there are loads of local government data sets that can be published using either the free or premier Maps API’s. Far be it for me to say to anyone interested in this topic “JFDI!”

Geo.me and Google are running an event for Local Government in June, you can register here. I imagine I will have a word or two to say on the subject

The pace of change

A new year - Thanks to Aquila Samson http://www.flickr.com/photos/quil/

On Tuesday I sat in on a CLG/OS event about the launch of the PSMA (Public Sector Mapping Agreement for anyone from outside of GB). Today I was speaking at an AGI North event and someone said to me “wow hasn’t a lot changed in the last year and a bit”, I choked on my sandwich and managed not to take the bait.

Let’s look back to 2009 and consider the change in UK GI over the last couple of years (apologies for conflating the fortunes of OS with the whole of the UK GI community)

May 2009 – a new business strategy for OS is launched with 5 goals:

1. Promote innovation for economic benefit and social engagement
2. Increase the use of Ordnance Survey data
3. Support the sharing of information across the whole of the public sector
4. Increase efficiency to develop a sustainable business for the future
5. Enhance value through the creation of an innovative trading entity

The Sir Rob Margett, new chairman suggests that the free data option (i.e government centrally funds data creation and makes it freely available to all users) has been rejected as it would have a net cost of “over £500m”. Charles Arthur of the Guardian persistently questioned this assertion but never got a meaningful response to my knowledge. Some people suggested that £500m could be the notional value that the Shareholder Executive had placed on OS in the event of a sell off or privatisation. Strangely the old link to the Strategy is broken and it doesn’t come up when searching the new OS website but the good old Way Back Machine has saved a copy for anyone who is keen to remind themselves of the strategy goals.

There were murmurs (or even howls) of protest from the Open Data community, partners and competitors and much sparring over the next few months. Not everyone seemed to be convinced by the new strategy but OS quietly got on with implementing it until Gordon’s Day in November (that’s the damascene moment when GB met Sir TBL and overnight became a convert not only to OpenData but committed to making a load of OS data free and left the folk at the Treasury gasping at the hole he had created).

A quick reprise from CLG and just 6 months after the strategy was announced we get a further consultation on the Policy Options for GI from Ordnance Survey. Several months of impact studies, a juicy contract for an economic consultancy, thousands of pages of submissions from the great the good (nearly 400 submissions) and much debate followed. In March 2010 CLG responded to the consultation slightly changing the scope of the data to be made freely available, announced the PSMA, promised simpler licensing and a resolution of the long running addressing wars and gave OS a leading role in implementing the UK INSPIRE infrastructure. I endeavoured to summarise the range of responses here, I concluded

Much to the surprise/disappointment of many who wished them ill, the OS seems to have come out of this as a big winner. It doesn’t look like they will be privatised, nor will they be broken up. They have won the funding for government for the free data sets and most of their core activities will continue.

With hindsight I think I underestimated how well OS came through the consultation process. Skip forward to April 2011.

• OS OpenData has been released and is fully funded by government
• PSMA signed between OS and public sector, a 10 year contract in place provides stability and financial certainty to both parties
• GeoPlace, a new addressing joint venture between Local Government and OS, is formed and acquires Intelligent Addressing creating a single source of addressing for GB. OS is appointed as sole distributor of the NAG.
• Ordnance Survey Ltd (remember that “innovative trading entity“) launches a new leisure portal including a print on demand service

One cannot but admire the way that OS has navigated through the storms of the last couple of years, weakened some of its competitors (there have certainly been losers outside of Southampton) and secured its future.

The pace of change? Seems more like business as usual.

Now when the PDC get’s it’s hands on that I wonder what value they will place on it? Maybe that £500m figure wasn’t so crazy after all.

Met goes funky and open

The Meteorological Office have come up with a real corker of a weather map with their Invent service about which they say

Met Office Invent will showcase some of our future plans for presenting web-based weather forecasts, products and information. It will allow you to become involved in the formation of new weather and climate change products, services or forecasts.

There are one heck of a lot of options for different data to be displayed, adjusting the time of the forecast, storing your favourite locations and the interface is pretty neat. To cap it all you can download the weather data under a pretty liberal license, shame they couldn’t just use the Open Government License which is so easy to understand.

Lies, damn lies and Crime Maps

Thanks to vistavision http://www.flickr.com/photos/vistavision/

When police.uk launched earlier this week I wanted to write about the debacle with it getting overloaded, the aggregation of the data, the approach to visualisation, the openness or otherwise of the data, misrepresentation and misunderstanding, the motivations behind it and some possible outcomes and what this all might mean for open data. But it has been a busy week and by now it seems that nearly everything that could be said has already been said.

Here is a selection of the blogs and news with a few supplementary thoughts from me.

The geofolk were understandably concerned about the poor presentation of the data and the misunderstandings that would be created due to the somewhat crude method of aggregation. Alex Singleton wrote

“…there appear to be some serious representational issues in this new mapping system which are not clearly documented and could be very misleading for the ill informed …

Outside of issues related to how you appropriately position a point for very long road, if the street is going to be the aggregating unit for the data, then this should also be used for the visualization. For example, roads could have been variably colored for different rates of crime (rates not counts… this is another representation issue entirely!!)…

The problem with this website as it stands is that crimes are easily misinterpreted as happening at a very specific locations…

These basic representational issues are typically covered in an undergraduate syllabus with a GIS component.”

Ken Field on the BCS blog added

“This sort of utterly inept mapping is precisely what enrages cartographers. Representing data effectively to report geography properly and communicate reality is what we do. It is also what any basic GIS course or cartography course would teach. The annoyance is borne of the frustration of seeing a site that will now be viewed widely be completely misinterpreted because of erroneous mapping techniques…

The use of totals and a single point is poor to say the least. If the desire is to build in fuzziness to deal with the privacy issue then use rates or proportions; use areas instead of points; or use some form of road-based linear symbol that varies by thickness or colour.”

That just about sums up the concerns of many who have an understanding of the data and the techniques available. I know that several people within the policing community expressed concerns about the aggregation methodology being used and the potential consequences but in the current climate of public service bashing no one wanted to stick their head above the parapet.

Several people have commented on the alleged £300k spent with Rock Kitchen Harris, the advertising agency who built the site including The Register and Freesteel. Incidentally, did anyone spot the ITT for this site? I wonder who else bid and what kind of approach they would have taken, or was this a stitch up?

£300k does sound like a lot of dosh for what should be a relatively simple backend aggregating data from 40 odd police forces that should be coming in a standard data model and then representing as clustered points on top of Google Maps. There are several companies out there that could have built this for a fraction of the cost including one I am associated with. The Register points at an alternative site based on the data API called Crimesearch, apparently the developer knocked it up in a day – to be honest it looks like it! This site shows even less than the official site and the points aren’t even in the right place, it is very very poor. If anything this shows why you do need to spend time and money in presenting this data properly. Hopefully someone will do something better with this open data to demonstrate what could have been achieved.

The other strand of comment has been about whether the publication of the aggregated crime data is a good thing or not. At a general level I think government has to be praised for making more data open to citizens and developers who may find innovative ways to use it. However in this case I think it is reasonable to question the motivation behind the high profile launch of the site (which lead to it overloading and crashing) and the questionable aggregation and misrepresentation underpinning it.

Simon Jenkins in the Guardian wrote

“Someone must have gone up to the home secretary in the corridor and murmured she could get some good publicity by announcing “the meanest street in Britain”, even if it did turn out to be a residential backwater in Preston…

I note that May and her officials censored more delicate information, such as of white-collar crime. … To the Home Office, theft and antisocial behaviour are what the poor do to the rich, not the rich to the poor. The map is seriously rightwing…

The truth is that there is information, useless information and Whitehall statistics. The crime map displays the same bureaucratic syndrome as has blighted Britain’s schools and hospitals since the 1990s.”

I like the concept of a right wing map

Adrian Short commented

Perhaps apocryphally, Stalin said that it’s not who votes that counts but who counts the votes. Likewise, we should be hugely cautious about giving too much weight to official visualisations of data. As the policing minister Nick Herbert wrote today (my emphasis):

We live in the age of accountability and transparency. The public deserve to know what is happening on their streets, and they want action. By opening up this information, and allowing the public to elect Police and Crime Commissioners, we are giving people real power – and strengthening the fight against crime.

So what we’re looking at here isn’t a value-neutral scientific exercise in helping people to live their daily lives a little more easily, it’s an explicitly political attempt to shape the terms of a debate around the most fundamental changes in British policing in our lifetimes.

Transparency isn’t wrong. It’s absolutely vital to make a meaningful contribution to public debate, but we need to distinguish pseudo-transparency from the real thing.

I think they both make some good points.

League tables may not in themselves help to reduce crime and deluging the police with conflicting requests to “put more resources into patrolling my neighbourhood” may not result in any overall net improvement (particularly with fron line resources coming under pressure while the HO spends money on sites like police.uk). However, there is a definite need for better publicly available information on crime and a map can be a helpful way to present the data.

Small area statistics normalised for population are a pretty good way of presenting a balanced view of crime, particularly if you could get some sense of comparatives with other areas, have a look at the Met Police site, not perfect by any means but a better overview than a mass of aggregated pins. If someone could mash that with the more granular aggregated “crime points” so people could drill in for more detail, maybe that would be a quick improvement.

Long term the presentation of crime statistics needs to be thought through by people who understand the data and how to present it in ways that will not cause a flurry of misleading news about the most dangerous street in Preston or Plymouth or wherever. I wonder how long will it be before a minister or friend of finds their house price blighted because it is mistakenly labelled as a crime blackspot? I might be a bit upset if I lived in Lancelot Place in Knightsbridge for example.

Postscript If you want to see the grand daddy of crime maps, have a look at Chicago Crime Maps and note the clear statement of what they are doing in the About

In praise of OS

It seems a while since OS got a “big up” from me, not because I am in an anti OS phase just because there hasn’t been much to say about our NMA recently except some slightly sceptical comment.

Aiming at the Address Management Unit, thanks to http://www.flickr.com/photos/colinwu/

My friend Bob Barr is a tireless campaigner to resolve the remaining bits of the Address Wars and he now has the Royal Mail Address Management Unit in his sights, good thing he is not a member of the Tea Party or some similar organisation or the phrase “in his sights” might have some uncomfortable connotations.

He copied me in on a mail which I thought was worth sharing:

Royal Mail AMU costs its customers £25 million per annum and maintains a single file of 27m postal addresses.

OS costs its customers £120 million per annum, maps billions of objects, has built and maintains the largest geospatial database in the world has dozens of products and an enormous range of licences and licensees.

Do you think these two figures stack up, or is one offering appalling value for its monopoly rent?

I know that he has perhaps simplified matters but it seems to me that there is some sense in his argument. What do you think?

If government is considering an effort to privatise/offload Royal Mail in this parliament, it might be an opportune moment to try and get the Postcode Address File released into the public domain as part of the process, after all it is only a rounding error in the overall finances of RM.

Oh yes, in case you were wondering about the title of this post – one might be inclined to agree that my favourite NMA does seem to be offering somewhat better value for money to the taxpayer for the national map than the people who still deliver the wrong letters repeatedly to our house, even though they are correctly addressed. Oops, I thought that was why RM are tasked with maintaining and licensing PAF?

It is that time of year

Yup it is the time of year for some reflection and some goodwill unto all men, particularly those in California, Southampton, Denver, Westminster, Haiti, Seattle to hint at just a few.

Thanks to Imelda http://www.flickr.com/photos/imelda/

Several people have or will shortly pronounce on what they consider to be the geo-events of the year, their favourite maps, the top tips for success in 2011 and a load of other stuff in that vein including OS and Google Maps Mania. So here in reverse order are my top 5 of the year just about to pass.

5. A GeoVation loser turns out to be a winner

Geo.me were one of the GeoVation finalists who although they caught my eye did not get the judges vote. I liked them enough to make an investment and join their board. Things have moved on pretty well since then and I have high hopes for the coming year as we move into new offices in January and start to ratchet things up. It is great to be back in the geogame.

4. Loads of mappy TV and Radio

There was a flurry of TV and radio programmes about maps including Magnificent Maps: Power, Propaganda and Art on TV and at the British Library (I went 3 times or was it 4?)

3. Open Source – a tipping point in the UK?

The announcement that the UK INSPIRE solution would be developed using Open Source components will perhaps be seen as the tipping point for Open Source Geo in the UK. I summed up my views on why this was important and possibly a game changer in this post, Ian Painter doesn’t agree with me on all of this and makes some good points in the comments.

2. Microsoft and MapQuest embrace OpenStreetMap

If I wasn’t so parochial I would have put this as number 1.

I wrote about the announcements from Bing and MapQuest at State of the Map 2010 and the tension I sensed amongst some of the core volunteers. I suggested that the participation of such big players would inevitably change the the game for OSM. Since that post Cloudmade announced that it had raised $12.3m in a funding round and soon after one of its founders, Steve Coast who also started OSM, jumped ship and shortly after resurfaced to join Microsoft.

If you look at the way MapQuest have integrated editing tools into their site using standard OSM tools you can get a feel for how these two giants could really drive uptake and coverage for OSM.

I expect there will be several more big announcements in 2011, maybe even a mid-sized one from not too far away.

1. OS OpenData (C)

Still think the copyright symbol is a bit ridiculous linked to OpenData, is it the lack of a space that qualifies it for the (c)?

This had to be the number 1 event of the year. After several years of Free Our Data campaigning a brief conversation between Gordon Brown and Sir Tim Berners-Lee unlocked this core data for the Open Data movement, I posted loads of assessment here so I am not going to recap. Suffice it to say that this has helped to unlock the derived data nightmare (still not completely resolved, need to post some thoughts on that soon) helped to get several new initiatives, apps underway and has provided some useful background to OSM which can aid filling in some of the missing GB bits.

I doubt that it will generate the tax revenue for the treasury that some FOD visionaries foresaw but we have a recession to help muddle that issue up so let’s enjoy the data and leave the cost to the treasury (who may cast an eagle eye over the costs of maintaining OS OpenData in the future)

2011

I have hinted at some of my views for 2011 above but let me draw out a couple more themes/predictions:

• Nearly half of UK GI spend is in the public sector, that market is going to be somewhere between not good and downright awful. It just has to be as the impact of massive cuts washes through to new GI investments and even continuing support and services contracts. On the positive side a squeeze of this scale opens up opportunities for businesses that offer radically different business models and big cost savings. Think shared services, cloud, open source and open data to name just a few.
• One of the big GI players will either “consolidate” into a specialist niche or be sold to an even bigger player, the field is getting too crowded with new entrants and they aren’t innovating sufficiently fast to buck market trends
• We will start to see less maps in mobile apps and more “ubiquitous” location that just works.
• Someone is going to mess up big time with location and privacy
• One of the big checkin players will check out

Several hostages to fortune there, no doubt some humble pie to be eaten this time next year. If you don’t blog, feel free to add your faves of 2010 and thoughts for 2011 in the comments.

I sometimes wonder whether I write this stuff for myself or for whoever reads it (a bit of both I guess). To all of you who have been kind enough to comment either on this blog or face to face, thanks for following me.

I hope you have a geotastic 2011 or as someone said “make maps not war”

That’s me done for 2010