Last year I commented on the AA’s Streetwatch survey of potholes questioning the methodology used.
I contacted the AA to see whether they would let me have a look at their data and see whether there were any other conclusions that could be drawn from this exercise in Citizen Science. After several mails and chasing for replies I eventually got a reply:
“This survey was never intended to be a scientific investigation of local road and footpath problems, their possible causes or reasons for improvement or deterioration. It was, as you allude to, a ‘big society’ approach to reviewing local issues around our AA Streetwatchers neighbourhoods. As you make clear there are a number of ways of carrying out such work but our approach was based on our AA Streetwatchers largely doing this themselves and us reporting their findings. The results merely provide a ‘snapshot’. Unlike the AA Streetwatch 1 survey this time we reported the observations by regions – hence the regional rather than postcode maps. This is only our third survey and our aim is to increase participation which we hope will produce a useful ‘dip-check’ of local conditions. For your information the numbers reported are just counts (with obvious error reports removed).”
Note – no offer to provide the data for alternative interpretation
Fair play to the AA, they accept that this was not a “scientific investigation”. But in their original press releases the AA talks about a “deluge” and “plague of potholes” and the summary of the report gives an indication of precision which they cannot back up with sample selection or size or methodology. That would seem to me to be claiming some kind of serious or scientific value to their “snapshot”, I doubt they will put out a further press release to correct any misinterpretation that may have occurred.
So next time the AA gathers a tiny sample of subjective data on road condition, I hope they will be clear that it is not intended to be a scientific investigation and that the results are a snapshot which may not provide a balanced or meaningful assessment of road condition or the effectiveness of road maintenance processes relative to weather and budget. But then the exercise wouldn’t get much publicity or serve whatever agenda the AA may be seeking to promote.
