At the end of last month, prime minister Gordon Brown, Treasury minister Liam Byrne and worldwide web inventor Sir Tim Berners-Lee announced that the public mapping agency Ordnance Survey (OS) would start giving away some of its data for free.

The decision might at first glance seem arcane – but it could completely transform how people find information such as when the next bus will arrive, how to get to their nearest recycling point, or what the crime rate is in their local area. This is because mapping data provides the backbone to all these services; access to and use of maps plays a key role in the daily lives of millions of people.

In recent years we have become an information nation, says Natalie Ceeney, head of the government’s Knowledge and Information Management (KIM) profession. “We are in a society where people work, live and breath access to information,” says Ceeney, who is also head of the National Archives. “We all go online to find out the weather, or how
we can travel to see a friend. We all need information; we all see it as a commodity and something we must have.”

This phenomenon began with Berners-Lee’s invention of the internet, and government has responded by steadily opening up public sector information (PSI) sources so that they can contribute to this new and information-rich society. The result is websites such as theyworkforyou.com, which uses information from sources such as Hansard to profile MPs’ beliefs and activities; another user-generated site shows accident black spots based on 2007 Department for Transport data.

The government’s efforts to release data were given a massive boost in June when Gordon Brown appointed Berners-Lee, a long-time advocate of freeing up public data, and Nigel Shadbolt, Professor of Artificial Intelligence at the University of Southampton, as his advisers on public sector information. While people such as Ceeney – as she puts it – “have been plugging away for years” on this agenda, Brown’s intervention put a spotlight on the issue. “From our perspective, it is really exciting that politicians as senior as the PM are interested,” she says.

Since Berners-Lee’s appointment, the data.gov.uk website has been launched for testing and is set to give the public access to hundreds of data sets from a range of Whitehall departments. Speaking last month, Brown said that Berners-Lee has picked out 1,100 data collections, and either released them or suggested that they should be open to the public. This, he said, is “information that should always have been theirs, but we had no mechanism to get it across to the people”.

The appointment of high-profile advisers does not signify a change of policy, says Carol Tullo, head of the Office for Public Sector Information – the regulator in this area, and the policy lead. The difference now “is that this open data policy is at the heart of the government’s strategy – for society’s benefit and for economic benefit, to drive innovation”.

The arguments for, and against

It is these social and economic benefits that are at the heart of the argument for why data should be as free as possible. Berners-Lee summed these arguments up last month: “When you publish how government and public services are working you enable the public to help put them back on track, you enable the public to point out where things could be better.”

“There will be new companies that spring up to use this data,” he added, and he claimed it would change people’s lives in the same way that the internet has: “People will think: ‘Before we had all that data, look at the things we had to do; the people we had to call; the notes we had to scribble to figure out what to do. Now these things are automatic’. There is just a huge benefit to industry and to the country in general. It will make the country a more efficient place and a better place to do business.”

However, it is not a case of access all areas. Some quasi-commercial public sector bodies pay their way by charging for use of the data sets they collect and maintain. So, for example, the not-for-profit website PlanningAlerts.com may provide a very useful service, emailing alerts about planning applications that affect your neighbourhood – but Royal Mail’s first concern is whether it is being paid for the postcode data the site relies on, not whether the site encourages people to get involved with local democracy (see box, right, for more details).

‘Trading funds’ – public sector bodies which must make at least 50 per cent of their income through selling the services or goods they produce – are particularly sensitive on this issue. Tullo says that six “data-rich” trading funds account for 70 per cent of the income made by all trading funds from selling public sector information: Ordnance Survey (OS), Met Office, UK Hydrographic Office, HM Land Registry, Driver and Vehicle Licensing Agency, and Companies House. Their approach to public sector information has been a matter of debate for several years, and while departments’ policies have been straightforward – release more and more – the path for trading funds has taken many twists and turns.

One step forward, two steps back

As with any such intractable or controversial issue, the government has commissioned a number of reports on the subject. With two academic colleagues Dr Rufus Pollock, Cambridge’s Mead Professor of Economics, wrote one that fed into the 2008 Budget. Dr Pollock says their report – the catchily titled Models of Public Sector Information Provision via Trading Funds – was necessary because an earlier Cross-Cutting Review of the Knowledge Economy, commissioned for the 2000 spending review, had not considered trading funds; that omission, he says, “was due to people horse-trading”.

His report, published with the Budget in March last year, recommended that trading funds selling digital information in bulk should only charge the cost of making a copy – which could be “so low it might as well be zero”. Pollock’s logic is that it is better for the government to fund the maintenance of these data sets, or for trading funds to charge those responsible for making updates necessary (for example, new firms, in the case of Companies House; house- or road-builders in the case of OS), than to risk stunting innovation by charging users.

Following hot on the heels of Pollock’s review came an announcement that there would be a review of the trading fund business model. The results were incorporated into this year’s Budget, and it appeared that further horse-trading had taken place. Pollock says the findings of his report “had been watered down”, adding that “some trading funds were not enthusiastic about the report” because they would no longer have had autonomy in setting charging policy, and would therefore lose some control over their finances.

Pollock was not the only advocate of free data who was feeling downcast earlier this year. Particularly vexing to many was the position of map-maker OS: in September someone felt strongly enough to leak a ministerial briefing by OS chairman Sir Rob Margetts to website Wikileaks. Margett’s paper set out the argument for the organisation’s preferred hybrid strategy, but free data campaigners argued that the figures did not add up. The response from OS was to point out that the hybrid strategy had been agreed months before with ministers.

A change of heart

As a result, many were surprised last month when Gordon Brown announced that OS data will be accessible for free. The announcement even took OS by surprise, says public sector union Prospect’s negotiator Ben Middleton, who represents many OS staff. He points out that the statement from OS chairman Sir Rob – describing himself as “delighted” – was a late addition that appeared on web versions of the government press release. “It is not clear to what extent OS were aware of what was being announced,” he says.

Ed Parsons, chief technology officer at OS until he joined Google three years ago, believes his former employers had the policy “very much forced upon them”. Like many commentators, he was struck by the absence of anyone from OS at the announcement’s press call, and the fact that OS “couldn’t come up with a comment other than something very bland”.

Parsons’ theory is that Berners-Lee was brought in to settle a long-standing battle between the Cabinet Office, which has been a stalwart champion of free data, and the Treasury, which is much happier to see trading funds bring in revenue for the public purse than give away goods for free. Because Berners-Lee was brought in by Brown, Parsons believes the OS decision is “coming from the very top”.

Having said all this, the announcement is in no way the end of the argument. The only details available so far – given at the bottom of a Communities and Local Government department press release – are that “data relating to electoral and local authority boundaries as well as postcode areas would be released for free re-use, including commercially. Mid-scale digital mapping information would also be released in the same way.” The specifics – questions such as what constitutes a ‘mid-scale’ map and how much central government funding will be provided to support its free distribution – are to be thrashed out between now and April next year. As well as internal negotiations, there will be a public consultation.

The twists and turns are a little exasperating for Prospect’s Middleton; a feeling that is probably shared by some in OS. “I think there is a degree of frustration over wanting to know quite which direction we’re going,” he says. After being “told to go away and define the hybrid strategy, this [announcement] seems to go completely in the face of this: don’t develop commercially, give it away for free. I don’t think OS would mind either way – provided someone makes a decision and sticks to it”.

Interestingly for free data campaigners who have been shouting ‘victory’ for the last few weeks, PSI policy lead Tullo does not believe OS’s hybrid strategy needs to be ripped up completely. The OS agrees: its press statement on the matter states that “much of this [existing business strategy] is still very relevant” – though OS declined to contribute to this piece. Negotiations over the exact shape of the new policy should certainly be interesting.

A map for others?

Beyond the OS debate, does Brown’s intervention in this area – by appointing Berners-Lee and implementing his first proposal – mean that other revenue-making public bodies, such as the Met Office or Royal Mail, will soon be subject to the same treatment?

While free data advocates such as Pollock and Parsons hope or believe so, those responsible for public sector information policy – Ceeney and Tullo – issue a word of warning.

“The mistake is that people take trading funds as one model. They’re not,” cautions Ceeney. The focus has been on OS, continues Tullo, “because OS mapping data is such a critical public asset. It really underpins so much activity: 80 per cent of all PSI is underpinned by place.” It was felt that OS hadn’t quite drawn the right line between free and charged-for data, says Ceeney; “hence the announcement”. But that announcement was about OS alone, and should be treated “on its own merit”, she warns.

Free data campaigners inside and outside government certainly won a major battle when Brown publicly backed the free distribution of mapping data, but they have not yet won the war.


The Meteorological Office

Dr Rufus Pollock, a Cambridge professor and author of a government report on trading funds and public sector information, describes the Met Office as a “crazy example” of official data control. The amount of money it gets from bulk-selling wholesale data (which his report recommended should be sold at negligible cost) is “really small – a couple of million in a £100m budget”.

In addition, he says the logistics of accessing Met Office data mean that climate-change researchers do not bother with UK data because it is “too much hassle”.

Met Office chief executive John Hirst says that its data policy is to add more value at the top end, increasing its revenues, and to “progressively give more data free of charge” at the bottom end.


Royal Mail

Visit sites such as PlanningAlerts.com, JobcentreProPlus.com, ‘HealthWhere’ or ‘LiveBus’, which are used to locate service providers, and you will find that they are “experiencing technical difficulties because of ongoing legal action by Royal Mail”.

This is because they cannot afford the £3,750 that Royal Mail charges for postcode data, and were instead using ernestmarples.com (named after the Postmaster General who introduced postcodes to the UK), which ‘scrapes’ the data from around the web. The site was set up by two free data campaigners, Richard Pope and Harry Metcalfe, who wanted to end the need for every site to do its own ‘scraping’ and highlight the problem that not-for-profit, citizen service sites faced.

That ended when Royal Mail sent a ‘cease and desist’ notice to the pair. Pope says it is ironic that PlanningAlerts.com, which he runs, has been praised in reports by one part of government and shut down by another. “Royal Mail doesn’t seem to be connected into the wider government policy,” he says; it is “not living in the modern world”.

The pair are in talks with Royal Mail, but in the meantime ernestmarples.com will stay offline because the pair are not willing to risk further legal action.

Royal Mail would not comment beyond the statement it issued at the time, which said that it had not forced the site to close, but simply asked it to stop allowing unauthorised access to Royal Mail data. The company has also recently reviewed its postcode reuse policy; the review should be published within weeks.