I’m watching BB4’s The Joy of Stats, and it just described on the push for open data in US crime statistics. Today I was planning a conference session about using open content, open data and open access research papers. So the question of “what is open data for?” is at the forefront of my mind and I can’t resist responding to to Tony Hirst's open data skeptic challenge : "What public is intended to be the user of all this public data, and how are they going to use it exactly?"

The question brings to mind this post by Tom Steinberg on Open data how not to cock it up

“The first way we can avoiding cocking up Open Data is to ensure that we always advocate for it in the same way that scientists advocate for Blue Sky science research – we must argue for it as a numbers game – a calculated risk that is worth taking. So, we say loud and clear : we do not expect the majority of government datasets will contain massive wells of untapped value, just as we don't expect that most university research will  yield a new penicillin, or an atom bomb …”

So maybe much open data is long tail: niche. The chances of it being used might be low. But without it being available there is no chance it gets used.

So what if 99% of open data never gets used. Does this mean it’s pointless to advocate open data?

I wonder if there is an argument for open data that doesn't rely on it being used.

The data is already there somewhere, but it’s locked into proprietary systems, only published as visual images within reports, only exposed for different purposes.Forcing the data out of systems onto spreadsheets and websites at least makes it clearer that the data belongs to the whole council / department / agency, not to the head of finance, or the planning managers etc. Treating data as data has got to be better than treating data as reports.

But what’s most interesting to me is that I think all these opens: open educational resources, open data and open innovation are really about new ways of working. The things you have to do to make your work more open make your work better. Knowing people will see your work makes it better. Inviting other people to see your work as you do it makes the work better. Working openly is just better.

Transparency is ethically better than lack of transparency. Freedom of Information legislation led to publication schedules led to open data. Assuming that government data should be open gives people working inside government a reason to try to make the right things happen. The threat of bad decisions being exposed is a weapon that can be used to argue for better decisions.

Back to Steinberg:

“It may surprise you to hear that my vision of a perfect (but realistic) government, is one that would release nothing, not a jot of data, not a single row or column….. until someone asked for it. When they did ask, my perfect government would then instantly publish that data in a brilliant, helpful format, regularly updated, and running on a lovely webservice that fulfils every data-mashers dreams.”