Skip to main content

Government Statistics

Volume 711: debated on Monday 1 June 2009

Question for Short Debate

Tabled By

To ask Her Majesty’s Government what steps they are taking to secure confidence in the use of government statistics.

My Lords, I am sure that we are all very grateful for those two bits of good news: that the Marine and Coastal Access Bill has finally ended after so long; and that noble Lords whose names are on the speakers list can now speak for six minutes rather than three.

I am afraid that I am rather a Johnny come lately to the world of statistics and I fully acknowledge that there are much greater experts than me in your Lordships’ House. I welcome many of them here today and the fact that they have an opportunity to speak. Through a mutual acquaintance, I met Professor Michael Chisholm, a retired Cambridge don, who became interested in the statistics that were used to promote unitary authorities. He had previously sat on a local government boundary commission. I was most grateful for his guidance because, as I said, I am a novice in the world of statistics.

Professor Chilsholm has been troubled by the statistics used to justify the creation of unitary authorities and wrote a book in collaboration with Professor Steve Leach entitled, Botched Business. I am pretty dispassionate about how local government is organised. Instinctively one worries when local government becomes less local, but if significant savings for council tax payers can be achieved, maybe the price of more distant local government is worth paying.

If the argument hinges on savings, the figures for savings have to be right. Professor Chisholm reveals that in a Parliamentary Answer on 12 May this year, the Minister stated that the new unitary authorities estimated restructuring costs at £138 million, with an outturn of £135 million. Not bad, on the face of it; quite an accurate forecast. Except that if you take the councils’ figures, the costs are £182 million. If you exclude Cornwall, where recurrent savings have not been very clearly divided, they are still £158 million. Durham County Council estimated that it will save £12.5 million in its bid to be a unitary authority, which includes £5.2 million savings on adult well-being and health plus children and young people's services, which it already runs, so it is difficult to see how they can be savings. Those are massive miscalculations and cast doubt on all other statistical forecasts in unitary authority bids.

Many similar discrepancies are to be found in widely different areas of government activity. I fear that, despite what has changed, many of the statistics are still extremely questionable in every way.

Your Lordships will be well acquainted with the debacle of the issue of the knife crime statistics. I will not dwell on that saga at length today, but suffice it to say that on 11 December, a special adviser at No. 10 prematurely published provisional statistics that had come from the National Health Service that showed that the number of teenagers admitted to hospital with knife wounds had fallen by 27 per cent. That was limited to 10 priority areas. In fact, the figures for knife crime since this Government took power show an increase of 34 per cent, and knife crime is at the moment at an all-time high.

Despite objections from the statistics authority, the Prime Minister insisted that those statistics should be released. So we have the story of a political adviser jumping on uncollaborated statistics and a Prime Minister keen to tell a good political story that knife crime was falling. What do the Government do? First, the Home Secretary apologises. Why was it not the Prime Minister? The figures were distorted by an official working for him in No. 10, and he insisted on their release. We have come to learn, have we not, that the Prime Minister does not do apologies.

It is what followed that is truly perturbing. The pit bull terriers were released. I can hear noble Lords asking whether pit bull terriers were not banned under the Dangerous Dogs Act introduced by my noble friend Lord Baker some years ago. No, pit bull terriers are still alive and well in Whitehall; they are now called special advisers. Those attack dogs went into action and articles appeared besmirching the reputation of Sir Michael Scholar, the chairman of the United Kingdom Statistics Authority.

Michael Scholar is public service at its best. He was permanent secretary in the Welsh Office and then at the DTI and he is now the president of St. John's College, Oxford. It was contemptible to suggest that the independent United Kingdom Statistics Authority was picking up on misleading statistics on knife crime because Sir Michael Scholar was seeking revenge because his son and other dedicated public servants had been moved out of the Prime Minister’s private office. That saga speaks volumes for the way that this Government do business.

There was an interesting letter dated 27 February from Kevin Brennan, Member of Parliament and Minister for the Third Sector, to Tony Wright, chairman of the Public Administration Select Committee. It stated that on 22 July last year, 73 special advisers were in Whitehall, of which 24 were in Downing Street. I worked in No. 10 as Parliamentary Private Secretary to my noble friend Lady Thatcher when she was Prime Minister. At that stage, there was a total of 65 people in all working in No. 10 Downing Street. One of them was reluctant to describe himself as a special adviser. First, Sir Stephen Sherbourne and then John Whittingdale, now Member of Parliament for Colchester, were the only political advisers who worked permanently there.

You have to ask: where do all these people go in Downing Street? I think that the answer is that they threw the Whips out of No. 12 Downing Street and packed many of them in there, where they are involved in spinning and campaigning endlessly as a sort of press unit.

Another incident of dispute occurred in February this year when the UK Statistics Authority published figures showing that one in nine UK residents working here were from abroad. This came after the ill advised and infamous speech by the Prime Minister about British jobs for British workers. What was the Government’s response? It was a rant from the Minister with responsibility for immigration, Mr Phil Woolas, that the Office for National Statistics had highlighted the figures because they were topical. That was at best naive and at worst sinister. I suppose we should be grateful that Phil Woolas was at least his own attack dog in this incident and did not rely on an unattributable briefing so dear to the heart of this Government. Phil Woolas’s attack on the independent UK Statistics Authority was typical of the reaction of this Government to statistics they do not like. The Government deserve congratulation on setting up the UK Statistics Authority, but you cannot create an independent body to monitor national statistics and then complain when it acts independently.

Another important aspect of dealing with statistics is the period for which departments can sit on statistics before they have to release them. It is sad that Scotland argues that it is a special case and must have five days for this; Wales is still consulting. For UK legislation and in Northern Ireland, it is 24 hours, and less for market-sensitive information. There may be some reason why government departments want a little time to prepare a case to go with an announcement on statistics, but not 24 hours. I should have thought that a maximum of three hours was enough, although the Royal Statistical Society, which has done so much to raise the profile of statistics and improve control of them, thinks that they should be released without any time at all, as does my honourable friend the shadow Home Secretary in another place.

The Government have made good advances with the introduction of impact assessments to demonstrate the financial costs and benefits of most primary legislation. Why can this information be found only on the internet? Should the impact assessment not be an addendum to any Bill and should it not subsequently be audited to find out how accurate the original calculations were? Almost more important, should impact assessments not be extended to secondary legislation? I am particularly mindful of orders enacting legislation from the EU, which always seem to be mindful of the benefits to individuals, but ignore the cost to taxpayers and employers.

In summary, the Government have made some important advances in the supervision of statistics. They have set up the UK Statistics Authority, but many would argue that they had to do that because the culture of spin and misinformation had become so pervasive that it was damaging the whole body politic. However, there is still a steep mountain to climb. As my noble friend Lord Jenkin of Roding reminded the Grand Committee on 16 March, the United Kingdom comes 27th out of 27 EU nations when it comes to trust in government statistics. To think that the Italians and the Bulgars have greater faith in their government statistics than we do is deeply shaming.

Let us take the independence of the UK Statistics Authority one step further; let us set up an independent national statistics commission and make it answerable to a Joint Committee of both Houses, in line with the recommendation of the Liaison Committee under the chairmanship of the noble Lord, Lord Turnbull. That Joint Committee would reflect the expertise on statistics that your Lordships' House contains and the other place does not. It would also recognise that the House of Commons Public Administration Committee has many other areas of responsibly. Statistics are at the heart of how the public can judge government. They should play a major role in deciding how we legislate and whether we legislate at all. This Government have proved that they cannot be trusted with statistics; statistics must be put in the hands of those beyond reproach.

My Lords, I resist the temptation to comment on the statistics that led to the splitting of Cheshire County Council by the Government, which has been a blot on the landscape of an otherwise very successful Government. It is a great honour to be a mere statistic between the noble Lord, Lord Hamilton, who introduced this important debate, and the noble Lord, Lord Moser, whose social survey method was my bible for so many years. Mentioning the Bible leads me to remind the House of my long debate, some years ago, on the use and funding of Britain’s historic places of worship. It was, in part, a response to the steep and precipitous decline in church attendance, now thought to be around 6 per cent, according to Christian Research’s English church census.

I am now appalled to learn that the ONS will keep the flawed 2001 question—“What is your religion?”—in the forthcoming 2011 census. This is a leading question. The census should ask first whether the respondent has a religion. It not only overrepresents the religious in our country, but underrepresents the non-religious. It also fails because it confuses and conflates the concepts of belief and ethnicity, as exemplified by our own Sikh and Jewish communities. It is, indeed, arguably discriminatory under the Human Rights Act 1998 and the Equality Act 2006.

The British Humanist Association, of which I am a member, sought information from the Office for National Statistics under the Freedom of Information Act on the result of its trialling of two alternative questions, including,

“Do you regard yourself as belonging to a religion?”.

The answers have been both tardy in coming and incomplete. I understand that face-to-face encounters between the ONS and the BHA indicate that results on the question of non-religion are more in line with many other surveys, which show a substantial rise in the number of the non-religious. Could the Minister comment on that? Incidentally, the alternative question and its trial were not even reported in the White Paper, Helping to Shape Tomorrow. Can my noble friend comment on this serious omission, which should have been included in the White Paper?

I therefore conclude that the Government should, first, ensure that there is complete disclosure of the internal data of the Office for National Statistics and the debate on the questions about this important issue that should be used in the 2011 census. Indeed, a further question might be added that separates data revealing ethnicity, to make it discrete from that gathered on religion. If that is not included, I suggest that the question on religion should be eliminated, especially in this flawed form. It was only introduced in 2001. If we do not have proper data to inform the debate and formulation of public policy, there will be inaccuracies and the direction of funds, resources and policies will be unhelpful to the nation as a whole. I include the example that I started with. We have a duty—and I speak as an atheist—to ensure that the wonderful churches that are part of our heritage are perhaps found new uses, or certainly sufficient funding to be retained as part of the stock of our culture. The debate is flawed and unsatisfactory. We must have good data and statistics to inform a proper public policy debate.

My Lords, I have spent much of my working life in official statistics, inevitably with issues of public trust never very far from my mind, so the initiative and comments of the noble Lord, Lord Hamilton, are very welcome, not least in these generally untrusting times.

I am rather optimistic. For one thing, we have a very good statistical service. Most people who know the international field, as I certainly do, would probably put Canada way ahead, with perhaps Australia not very far behind, but they have a centralised system that is much easier to run than our decentralised arrangements.

The other reason why I am pretty confident is the new structure; the noble Lord, Lord Hamilton, has already talked to us about that. We now have the UK Statistics Authority, which emerged as the central piece of the legislation in which your Lordships played such a crucial role. The existence of the authority gives me every reason for confidence. It has Sir Michael Scholar as its very strong chairman, a powerful board and very good staff. Above all, it has already shown us that it means business. That business is summarised in precise detail in the code of practice which the authority published earlier this year.

The code applies not just to the Office for National Statistics but to every government department. That is very important, because we are decentralised. It sets out the exact rules of practice that are to be followed, and, as has already been mentioned, we have seen in recent months that failure to comply with the code will be relentlessly pursued. That can apply in several directions. It can apply in the direction of the statisticians, as has already been experienced, but more likely it can be brought into action with politicians and, above all, Ministers. I have had much experience of Ministers failing to resist the temptation when dealing with statistics to release comments and so on.

The most blatant example has already been mentioned by the noble Lord, Lord Hamilton. From the Home Office and then No. 10—after all, the initiative for the new legislation came from the then Chancellor, Gordon Brown—came the totally inappropriate issue of data on knife crimes. What happened then was extraordinary and very impressive; Sir Michael immediately sprang into action, and No. 10, and the Home Secretary in the other place, had to apologise publicly. Moreover, the Cabinet Secretary followed up by instructing all the government departments that the code had to be obeyed. To my mind, that is the most encouraging thing that has happened in government statistics and is the reason for my confidence that public trust will be enhanced.

Finally, I refer to another piece of so-far unfinished business, which the noble Lord, Lord Hamilton, has already mentioned. The UK Statistics Authority reports to Parliament, and it has been assumed from the very beginning that this requires a Joint Committee of both Houses. It was debated in this House and strongly backed by your Lordships. Unfortunately, the Leader of the other place rejected the initiative, so we do not have a Joint Committee. To my mind, this unfinished business now has to be tackled. We must remember that hardly any government business or policy does not require official statistics. Surely, it is in the interests of both Houses, the general public, and of trust generally, that a Joint Committee should be set up without delay.

My Lords, my noble friend Lord Hamilton has launched a debate which so far has led us into some interesting byways—not only his speech about the costs of local government reorganisation but the fascinating remarks of the noble Lord, Lord Harrison, about the impact of the census, the questions that are asked, and the need to get those right. The business managers are wise to have originally given us three minutes, because it means that we have at least to start off with a very short speech.

I want to pick up just one point: the one with which the noble Lord, Lord Moser, who played such a notable part in the proceedings on the statistics Bill, finished his speech. As he said, the Act provides that the new UK Statistics Authority—which was originally called the Statistics Board but has wisely changed its name—should be accountable to Parliament. During the Bill’s passage many noble Lords argued that the authority should report to both Houses, ideally via a Joint Committee of both Houses. It may well be within the recollection of the House that I raised this issue again in a short debate on 29 November 2007. With the support of the noble Lord, Lord Turnbull, who has proved to be an immense source of wisdom on this issue, I tabled a paper to go to the Liaison Committee of this House calling for a Joint Committee. The Select Committee unanimously accepted the recommendation. On the same day it rejected three or four other proposals but it did accept mine, which was endorsed by the whole House. However, as the noble Lord, Lord Moser, said, Ministers’ preferred solution was adopted in the other place—namely that there should be accountability only to a committee of the other place: a House of Commons Select Committee. The other place subsequently recommended that it should be the Public Administration Committee.

Some of the recent controversies surrounding the issue of statistics have already been mentioned. The Public Administration Committee has therefore had to deal with two serious statistics issues. The Home Office’s release last December of certain statistics about knife crime has already been mentioned. I can assure the noble Lord, Lord Brett, that I do not intend to repeat what I said at some length in the debate on 16 March. The other issue, more recently, is the ONS’s controversial publication of certain statistics about the numbers of immigrants in the UK workforce. My noble friend Lord Hamilton mentioned that.

I have taken the trouble to read a lot of the evidence that was given by Mr Phil Woolas to the Select Committee at the other end, and the way in which the committee handled it was very impressive. Indeed, I have no complaints about or criticism of the Select Committee. It certainly does not pull its punches. It has quite rightly identified the two areas that it wanted to investigate. However, it has a very heavy agenda, most of which has nothing whatever to do with statistics. Moreover, it is a fact—I checked this today with the Clerk to the committee—that it has not yet issued the reports on these two matters of knife crime and immigration statistics.

If anything, that confirms my view that this House’s proposal was indeed the right and wise one. It would have been much better if this House’s view had been accepted and the proposal to establish a separate Joint Committee of both Houses, with the sole remit of holding the UK Statistics Authority to account, had been accepted. I know that that has the support of Sir Michael Scholar, for he has told me so in no uncertain terms. I can also tell the House—perhaps this is of even greater interest—that I have been assured more than once by the shadow Chancellor of the Exchequer, George Osborne, that the next Conservative Government will accept the view of this House that there should be a Joint Committee of both Houses to undertake this work. Let us hope that this House will not have to wait too long for its unanimous view to be implemented. That is something which I am sure we will all welcome.

My Lords, as the noble Lord, Lord Hamilton, illustrated in his excellent introduction to this debate, there are two ways in which government statistics can go wrong. One is that the statisticians get it wrong, which undermines public confidence. There was an example of that earlier this year when the forecast of what had happened to GDP was revised from minus 1.6 per cent to minus 1.9 per cent. As an economist, I have to say that that was a modest adjustment. However, it caused a tremendous kerfuffle in the press, which thoroughly enjoyed exploiting it. But that is a rare way for statistics to go wrong. What is much more common is when politicians bend statistics to suit their ends. I will not give examples of that if only because my party has been in power most recently and is probably the most serious offender. But if we are honest, they all do it: Conservative, Labour, Lib Dem, in opposition and in government. They do it year in and year out.

I applaud the efforts of Sir Michael Scholar and the UK Statistics Authority to act as referee in this Eton wall game of brutality, to no effect. It was very brave of Sir Michael to take on No. 10 so early in the life of his organisation. However, to be fair to Ministers, they did set up the statistics authority, which is a bit like burglars voting for more powers for the police. But they did it and they should get the credit.

I speak tonight to introduce to your Lordships’ House a new campaigning organisation called Straight Statistics, of which I have the honour to be the chair. It has been set up to combat the epidemic—I do not think that that is too strong a word—of statistical abuse that is sweeping through our national life. We launch formally on 17 June.

Of course politicians are not the only ones who bend the facts. Advertisers do it, companies do it, and PR companies do it in spades. The worst culprits, let us face it, are the newspapers, many of which never let statistical integrity stand in the way of a good story. But although there are many offenders, government statistics should be the gold standard by which the statistics of others and their use are judged. If government statistics are once distrusted, there is no basis in fact on which our national policy and political life can proceed.

Many in your Lordships’ House have supported Straight Statistics. I am delighted to say that two noble Lords who have already spoken, the noble Lords, Lord Moser and Lord Jenkin, both sit on our council. The noble Baroness, Lady O’Neill, in her role as chair of the Nuffield Foundation—I am sorry that she is not in her place tonight so that I could thank her publicly—has provided the funding, and we have had from the very start 100 per cent backing from Sir Michael and his vice-chair at the UK Statistics Authority, Sir Roger Jowell. They have done so not because they think that a campaigning organisation is a substitute for their work, but because it can act as a light-footed complement. I may say too that the Royal Statistical Society, which could well have regarded us as a rival, has instead embraced us as a member of our executive and is doing everything it can to help. All these statistical luminaries wish us well and I hope that the House will do so too.

My Lords, I should like to take the opportunity provided by the Question of the noble Lord, Lord Hamilton, to raise two topics, one general and one relating to the particular issue of inflation statistics.

My general comment is that I am appalled by the degree of statistical illiteracy abroad. Almost every time I read a newspaper I am aware that the journalists writing it have no knowledge of statistics. They simply pluck out things to create stories, as the noble Lord, Lord Lipsey, said, and therefore there is constant statistical abuse. Of course, it is very hard to be against more information but sometimes I think we would be better off with less. A good example of that kind of statistical abuse is the debate on climate change.

I do not know what the answer is. You could say that, as part of their training, journalists ought to have a compulsory course in statistics and should not be licensed to write anything unless they do. That, of course, is not feasible. However, a more sensible suggestion is that health warnings should come with official statistical information. I know that ONS guidelines require that statistics come with health warnings, but the kind of health warnings that they come with are totally incomprehensible to anyone but those who write them. For example:

“The variance of the IoP is fairly insensitive to the assumptions made about the variance of the EPD. This continues to be the case at 4-digit level. Thus the assumption made about the variance of the EPD when deriving formula (4) should be suitable”.

You can do better than that.

If you want any of these official statistics to have any impact on the public, then alongside the necessary technical blurb you must provide much more user-friendly health warnings. One of the most useful that you could provide is a list of a few unlikely but possible events which would render the forecast invalid, such as the collapse by 25 per cent of US house prices between 2006 and 2008. A list of those kinds of unlikely “black swans”, as they have been called recently, would be useful to have.

My second topic involves the battle of the indexes—that is, the pros and cons of the RPI and the CPI. The change in the index for the purposes of inflation forecasting was made in 2003. I remember it very well because I was a member of the Lords Select Committee on Economic Affairs. The assumption was—we were told so by the Chancellor and other witnesses—that the two indexes would converge. That reflected the efficient market hypothesis: after all, you cannot have two indexes which measure roughly the same based on roughly the same things that might diverge in the long term. But, of course, diverge is exactly what they have done.

This raises the question of what the purpose of the change was. As I understood it at the time, the purpose was to lower the headline rate of inflation in order to present the inflation record of the Government in a better light and therefore to decrease wage pressure. What has happened though is that the two indexes have diverged considerably. Which index inspires more confidence as a measure of the rate of inflation? In terms of confidence in the economy, the consumer prices index is better since it regularly grinds out lower rates of inflation than the retail prices index. In terms of confidence in the statistics, however, the RPI might be better because, it seems obvious to me, any credible inflation index should include mortgage interest payments, especially in a country such as Britain where housing is such a huge economic component.

RPI, then, is a better index than CPI, but mortgage payments are not an accurate measure of the flow of consumption in the economy, especially if asset prices are going up. I wish we could find a way of incorporating the increasing prices of housing stock in the retail prices index so that it more accurately reflected the trend of transactions in the economy. I hope that we can work towards that, and that this may be one of the lessons we learn from the present wreck of inflation targeting.

My Lords, it is a pleasure to follow the noble Lord, Lord Skidelsky, in my noble friend Lord Hamilton’s debate. The noble Lord, Lord Skidelsky, referred to the “overload” of information and gave some health warnings. My purpose is to talk about departmental statistics, particularly those produced by DCSF and DIUS. I can only hope that your Lordships are up to date with which departments those acronyms represent. They produce their own national and other statistics, and the confidence in those statistics is pretty low.

One of their aims is to produce data sets with the data and analysis to assess the progress being made in improving educational attainment and emotional and behavioural health—among, I assume, young people. I have three questions. First: some things have so many variables that it is difficult, or perhaps impossible, just to rely on statistical analysis. Can we really produce convincing data sets for young people’s emotional and behavioural health? Does the Minister agree that it is probably wise not to claim too much on this subject?

Secondly, there are certain to be many more ways than one of moving towards the objective of improving educational attainment, and many more ways of assessing progress. Those who set the statistical questions, however, as has already been referred to, have a need to simplify if they are to offer a national answer to the questions they pose. Which answer is comparable across the board? Education cannot be simplified any more than people can be; after all, when you walk down the street you never see two people who look exactly the same, and I would apply that comment even to my granddaughters, who are identical twins. In view of this complexity and the inevitable controversy, would it not be sensible to involve someone other than the two policy departments themselves in the production of statistics—the Office for National Statistics, for example?

Thirdly, there is a test of information, which is to ask: now that we have it, what can we do with it? I remember that when I was briefly in charge of a firm that was not doing terribly well, I set a test at lunch of who could produce the most interesting piece of useless information, and there was a prize for doing so. For example, the department produces statistical comparisons of educational attainment with other members of the OECD. At the moment, I cannot see what you would do with that information. However, if we produce information about so complex a subject and it is not clear to what that information will contribute, the rightly sceptical public will conclude that departments have simply come up with answers for purposes of their own.

Does the Minister agree that, with all departmental statistics, there must be an explanation of why they are collected and a description of the options for action to which they could lead?

My Lords, I, too, thank the noble Lord, Lord Hamilton of Epsom, for the opportunity to debate this important subject. As chairman of the All-Party Parliamentary Humanist Group, my concern, like that of the noble Lord, Lord Harrison, is with the approach taken by the Office for National Statistics to the question in the census on religious beliefs. I will try to reinforce the points so well made by my noble friend.

Following the 2001 census, there was criticism of the ONS for the distorted response it produced by deploying the plainly leading question, “What is your religion?”—to which only 15.5 per cent replied that they had no religion. That result contrasts with the ONS’s own Social Trends survey, which reported about the same proportion of people saying that they belonged to no religion as saying that they belonged to a Christian denomination. Recently, the British Social Attitudes Survey reported that 69 per cent of people either did not claim membership of a religion or said that they never attended a religious service.

Surely it could have been foreseen by statisticians that a leading question was bound to produce a misleadingly stark result. The ONS admits that it would not normally recommend the use of a leading question—so why ask one in 2001 and then plan to repeat it in the census of 2011? The explanation seems to be that the question about religion is also designed to fish for information about ethnicity. The ONS wants to identify by stealth as many members as it can of two ethnic groups protected under race legislation, by asking its leading question on religion, which, it claims,

“provides a reasonable proxy for Sikh and Jewish ethnic groups”.

That may be so, but surely it should not be at the cost of distorting the much larger issue of attitudes to religion in Britain today. To underline the oddness of the ONS report, in a survey of Jews in 2003, only 42 per cent said that they were religious or even “somewhat religious”.

Why does it matter? It matters because inaccurate data can lead to the misallocation of resources and public funds. It matters because misleading statistics can be used to argue the religious case for the expansion of faith schools, when some of the more divisive institutions discriminate against non-religious people in their staffing and admissions policy. It matters because more accurate statistics would offer reassurance to those who fear that their sceptical, tolerant, vaguely agnostic Britain is being defined and divided increasingly by religion. It matters because accurate statistics might have particular importance for the Equality Bill currently before Parliament, which would mandate public authorities to treat non-religious citizens equally and with the same respect as religious people. Finally, it undermines our confidence in the Office for National Statistics when it contradicts other authoritative surveys to declare that only 15 per cent of British people are non-religious.

As noble Lords have emphasised, the most precious attribute in our national statistics is public trust. Does the Minister agree that, in compiling the 2011 census, the Office for National Statistics would do better to ensure that it conforms to the highest professional standards in this sensitive and important matter?

My Lords, a visitor to your Lordships' House this evening could conclude that approximately 15 per cent of parliamentary time is spent debating matters of humanism, which just illustrates that too small a sample can sometimes be misleading.

It was only two years ago that this House was engaged in debating the Bill that set up the new statistics authority. I well remember the noble Lord, Lord Davies of Oldham, making strong points about the need for Ministers to be removed from the accountability structure of the organisation and for the appointment of an independent board. We have now seen the new authority up and running. However, there has been a long and undistinguished trend of this Government getting caught up in a problem, laying down the law as to how everyone should behave and then getting caught circumventing their own rules. One only has to think of the donations to political parties issue to reinforce that. No sooner had the Government brought the legislation through Parliament and ensured, to their great credit, that a powerful chairman was appointed, than they sought to get round the rules, even to the extent of attacking the authority for political bias.

In the short period since the arrangements have been put in place, there have been a number of clashes between the Government and the authority. It is in some ways encouraging that the authority is prepared to exert its independence and to take issue with the Executive. However, we have to look at the Government’s reaction and what they have done. We have heard of one important episode where statistics were released against the specific advice of the authority, as detailed by my noble friend Lord Hamilton and others. One has to ask why that was done. Was the importance of independence not accepted? Was the authority’s advice found to be unsound? Or had the Government’s commitment to independence wavered?

However, perhaps much more worrying was the row between the Minister in another place, Mr Phil Woolas, and the authority over immigration. The attack that the Minister launched against the authority was very serious. I should like to hear from the Minister this evening, the noble Lord, Lord Brett, whether Mr Woolas was speaking for the Government more broadly or whether they were just personal remarks—it is difficult to come to that conclusion when the Minister is speaking for the Government as a whole.

Have the Government therefore changed their mind on the independence of the statistics authority? Do they now accept that making an attack on it was, at best, extremely ill advised? The Government’s appetite for bad news is limited, notably around the No. 10 bunker, where the sound of messengers being shot is a fairly regular occurrence. In the run-up to the general election, we can expect much more use of statistics and much more controversy around them. However, we must give substantial credit to the statistics authority and in particular to its chairman, Sir Michael Scholar, for the courage that it has shown in bringing the Government to account.

My Lords, I, too, join in thanking the noble Lord, Lord Hamilton of Epsom, for bringing this matter to the attention of your Lordships today. I wish to declare a close and pecuniary interest as a deputy chairman of the UK Statistics Authority. Noble Lords will recall that this body came into being in April 2008 as a result of legislation contained in the Statistics and Registration Service Act 2007. As the authority has now been up and running for just over 12 months, I thought it might be helpful to inform noble Lords of progress that has been made by highlighting some of the events of the past year, as the authority unfortunately does not report to a Joint Committee of Parliament, which was the unanimous proposal of your Lordships' Liaison Committee in January 2008 in its first report of that Session.

One of the principal reasons for the Government's desire to create the authority was to restore public confidence in official statistics, a confidence which had been greatly eroded with the passage of time by either actual political manipulation and interpretation or the perception thereof. Therefore, from April 2008, the Office for National Statistics reported to this independent body rather than, as hitherto, to the Treasury. The authority itself reports to Parliament through the Public Administration Committee.

Since April 2008, the authority has initiated action to achieve four principal objectives: to improve statistical planning, to maintain high professional standards, to improve communication of statistics and related advice, and to build the trust of those who use official statistics in the statistical service and the authority. These objectives are being achieved by the authority assuming direct responsibility for the ONS, through a principal operating committee—the ONS board, which I chair—on the one hand, and on the other hand by the establishment of a monitoring and assessment team based in London, Newport and Edinburgh to carry out independent investigations and assessments of all official statistics against the code of practice that we published in January.

The code of practice was published after a 12-week public consultation period, which included the devolved Administrations. This activity is under the supervision of the authority’s second principal operating committee, the Committee for Official Statistics, chaired by Professor Sir Roger Jowell, the other deputy chairman of the authority. In the first months of our existence, we went live with UK National Statistics Publication Hub website, which separates statistical releases produced by professional statisticians from ministerial or political commentary. It has been noted that it has also been necessary in this first year for the authority to challenge undue political influence in regard to knife crime and, more recently, immigration statistics. As a result of the former instance, guidance was issued by the Cabinet Secretary to all non-statisticians working in government on how to use official statistics, how best to work with statisticians, and, above all, the importance of everyone respecting the code of practice.

These events, among others, demonstrate that the authority can, and will, act to challenge undue political influence, without fear or favour. The authority's mission, enshrined in the legislation, is to safeguard and promote official statistics for the public good. I can assure noble Lords that that is what we will continue to do. The authority operates transparently so that both Parliament and the public can see what it is doing to build the trust required. The authority's board papers are published; a public record is maintained of all issues that have been raised with us; and we publish our correspondence. These are but examples of the beginning of a determined work programme initiated by the authority with the support of Parliament, government and the Government Statistical Service. As an end-of-year report card, the Royal Statistical Society, in March 2009, included a statement in which it said the authority had passed,

“its first test with distinction”,

but recognising, as we all do, that there is still much to be done.

In conclusion, I am also pleased to report that the physical location of much of ONS’s operations to Newport, which was an issue strongly questioned during the passing of the Act in this House, is almost complete and will have been achieved through the excellent co-operation and understanding of staff without any diminution of quality of service.

My Lords, I, too, am grateful to the noble Lord, Lord Hamilton, for introducing the debate, which has demonstrated both the importance of statistics and the range of statistics that we consider important. Before the debate began, I had the chance to look up the publication hub on the web—one of the consequences of the statistics Act—which gives the dizzying array of statistics that government produce every day. I have some sympathy with the noble Viscount, Lord Eccles, who asked whether some of these statistics really serve any purpose. Just as we from time to time talk about the necessity for having a cull of quangos, I think that a cull or review of statistics should be on the ONS radar.

The debate is divided into two broad areas. The first is methodology—do we collect statistics in the right manner? The second is mechanics—do Ministers in particular use statistics, once collected, properly? The issues raised about methodology demonstrate the scope. The noble Lord, Lord Hamilton, began by talking about local authority statistics. The noble Lords, Lord Harrison and Lord Macdonald, made a powerful case about the questions asked in the census. My recollection is—although this was some time ago—that in the run-up to the last census there was a long debate in your Lordships’ House about the religious questions. It seems that we need to have that debate again as it is in nobody’s interest that the census or, indeed, any other set of statistics should give a misleading impression. Trust, as many noble Lords have said, is the core of what we are seeking to achieve from official statistics of various sorts. Whenever that is in question, we need to look again at how we do things.

The noble Lord, Lord Skidelsky, raised a couple of fascinating points. One was about forecasting and the suggestion that we might have a list of black swans. Recent events have demonstrated that nobody is very good at forecasting; certainly the well paid forecasters in the City have all got it wrong, big time. While it might have been a good thing if they had followed the advice of the noble Lord, Lord Skidelsky, and had a look at a number of black swans, I suspect that, given the nature of statistics and statisticians, it is quite hard to get forecasters to look at all the risks and come up with all the potential permutations. The one good example in that respect is the way in which the Monetary Policy Committee has a fan at the end of its forecast showing the probabilities of various outcomes. That good practice might be more generally adopted.

The noble Lord, Lord Skidelsky, also asked what the correct inflation measure is. I think that we can all agree that the CPI in its current form is not it. The acceptance over the past year or so by the Governor of the Bank of England that whatever measure you use has to take some cognisance of asset prices is a welcome change. We now wait to see how that change can best be implemented.

Recently, the ONS has also had to revise its methodology for collecting statistics on retail sales, which were out of kilter with the information that was coming from the retailers. That was another example where pressure needed to be brought to bear on a series of statistics that clearly did not have the confidence of the sector about which they pertained. It is a good thing that those statistics have now been recast.

On the role of Ministers, we have heard the arguments about how they have misbehaved. Nobody should be surprised that Mr Woolas behaved as he did. If ever anybody had form, it was him. Those of us who are veterans of the Littleborough and Saddleworth by-election would have found nothing surprising in his response. The question that noble Lords have raised is what further can be done to rein in Ministers who might wish either to misuse statistics or to abuse the statistics authorities. The only practical measure, which we have debated many times, is to have a Joint Committee. That would raise the status of the issue. A single committee focusing solely on this issue would make a big difference, both in how Ministers thought about dealing with statistics and in giving support to the professional statisticians, knowing that they had a parliamentary watchdog that was really focusing on what they were doing.

Finally, I was pleased to hear what the noble Lord, Lord Rowe-Beddoe, said about the move to Newport. We were concerned about it when the Bill was going through. It may be that the collapse of the financial services sector has meant that fewer statisticians have been thinking of leaving the service and going to greener pastures elsewhere. That may have had a marginal effect to the service’s benefit. It is good that that move is now largely and successfully completed. The noble Lord, Lord Moser, asked whether we should be more confident with the new arrangements. There is no doubt that, on balance, we should.

My Lords, I, too, thank my noble friend Lord Hamilton of Epsom for introducing this debate.

Sadly, the Government have not used statistics in a manner that inspires confidence. Figures have been used as a tool to spin rather than to provide accurate information. An example of that was the publication in 2008 of the Home Office knife crime figures, to which my noble friend Lord Hamilton referred and which I will not repeat. However, it is worth repeating Sir Michael Scholar’s words that provoked the attack to which my noble friend referred. Sir Michael described the figures as,

“premature, irregular and selective … corrosive of public trust”,

and as breaching,

“several parts of the Office for National Statistics code of practice”,

to which the noble Lord, Lord Moser, referred.

Another example is the publication in January 2009 showing a 10.2 per cent drop in first-time offenders in the criminal justice system. It later emerged that 19,000 young people had been left off the official release. If they had been included, the Government’s claim of a 10 per cent fall would have been wholly or largely wiped out. The former head of the Youth Justice Board, Rod Morgan, referred to those figures as “smoke and mirror ploys”. There is also the immigration example referred to by my noble friend Lord Jenkin.

There has been consistent criticism, both by statisticians and others, nearly all of whom have called for an end to the pre-release of statistics. The Royal Statistical Society commented that pre-release,

“leads to mistrust of the independence of official statistics”.

Figures should be released simultaneously to one and all. This, as well as the Joint Committee to which a number of noble Lords have referred this evening, is especially important when the narrow borderline between showing statistics in the most favourable light and the distortion of information has not always been observed. Pre-release to government departments enables them to hog the headlines with their spin. By the time others have analysed the information, the story has frequently gone cold and a corrected interpretation is unlikely to receive the same prominence—unless, as my noble friends Lord Hamilton and Lord Goschen pointed out, there is a requirement for anyone who has been criticising the Government to be thoroughly rubbished. In many major countries, there is no pre-release, or it is significantly restricted. In the USA, the President receives figures only half an hour before they are published. The Government claim that early release is in the public interest as it enables questions to be answered promptly, but the inability to distinguish the borderline between what is and is not acceptable destroys that argument. To rebuild confidence in statistics, I hope that the noble Lord will listen to what the noble Lord, Lord Rowe-Beddoe, said and will reconsider the early release of statistics.

My Lords, I join others in congratulating the noble Lord, Lord Hamilton of Epsom, on securing this debate on statistics—at least I do so in so far as it serves a common end; namely, to increase the public’s confidence in statistics and in the Government’s use of them. The Government recognise that prior to the events of the past two years, which have been described, there was a very low opinion of government statistics. We have sought to put that right through action that we have taken.

It is amazing to recall that the Statistics and Registration Service Act 2007 was the first major piece of statistical legislation for 60 years, and was designed to improve the quality of, and trust in, official statistics. It created a non-ministerial department: the UK Statistics Authority. This authority effectively replaced the Ministers at the Treasury as the top layer of governance for the ONS, the Office for National Statistics. The ONS, which is the biggest producer of official statistics in the UK, always produced its figures free from political interference, but with this change in governance arrangements, it can now be seen quite clearly to be at a distance from Ministers. This is intended to increase public trust in ONS statistics.

Of course, we are aware that a great many official statistics do not come from the ONS at all. To reassure the public of the high quality of the statistics produced throughout the statistical system, the Act made provision for the independent assessment of national statistics. As we have heard, that role is carried out by the head of assessment who reports to the authority publicly, who has published the new code of practice for official statistics, and who cannot be involved in the production of statistics, so he is able independently to assess statistics produced across government, including by the ONS. I am very grateful to the noble Lords, Lord Moser and Lord Rowe-Beddoe, for their expressions of confidence in the system that we are seeking to put in place, which will improve the performance of statistics. I might almost imagine that a general election was taking place because the non-partisan debates that we normally have on statistics in this House seemed to be slightly overtaken in this instance by the making of rather crude party political points. If I were to indulge in the same practice, I would point out that in my mid-career I seem to recall 29 changes to unemployment statistics between 1980 and 1997. I also seem to recall a reduction in staff of a fifth—from 250 to 200—in the then statistics collecting organisation. However, as we are not being party political, I will not do so. I express my gratitude to the noble Lord, Lord Jenkin, for not repeating—as he did when we debated this previously—his rather savage but elegant attack on the Government. I do so if only because the bruises have only just faded in the past two or three days.

The major point made in the debate—I do not think that many noble Lords would disagree with this—concerned the Liaison Committee’s recommendation that there should be a Joint Committee. Ultimately, that is for Parliament to decide. Although the Government have an influence in that matter, we follow the overriding principle that departmental scrutiny of public bodies falls to the committees responsible for that. The question arose of whether the Public Administration Committee could undertake this role. There seems to be no evidence—at least to date—that it cannot do so.

The committee has a long experience of oversight of cross-government activity by virtue of its experience in scrutinising the work of the Cabinet Office. The committee has already had four sessions looking into this area, which will be the subject of a report, and I have no doubt that your Lordships’ House and those within it who hold the very strong view that the situation would be improved by a Joint Committee will continue to press that point on every occasion. I should only point out to the noble Lord, Lord Jenkin, that I come from Cumbria, which has the finest chickens it is possible to eat—many of them raised on the estate of a former Member, Lord Lowther. However, the important thing with chickens is of course not to count them too early or before they are hatched.

My Lords, does the Minister have any information as to when the Select Committee will produce a report on either of these matters—knife crime or immigration statistics?

My Lords, the noble Lord knows that Select Committee reports are jealously guarded by Select Committees, and the production of their reports is in their hands. They will produce their reports when they have completed their work. I have no greater access to information on that than the noble Lord. I am sorry to disappoint him. If, however, information is available, I will happily write to him in due course.

The second point of considerable substance was made by my noble friends Lord Harrison and Lord Macdonald of Tradeston. Looking at my brief, it does not seem to do justice to their two powerful contributions and I should like, therefore, to respond in writing. A neutral question is not always the one that will find the answer. However, there is the question on the census form where there is a box to tick if you have no religious belief. Also in the consultation many people wanted the question to be asked in the form in which it is currently drafted. Finally, we are having a consultation to see whether a better question can be formed. However, both noble Lords made some powerful arguments that I should like to take away, consult with my colleagues on and provide at least a more thorough and comprehensive answer than the one I have just given.

I turn to some of the other points that were made. I did not recognise some of the points made by the noble Lord, Lord Hamilton, attacking the statistics body. First, Sir Michael Scholar, a very distinguished civil servant who I knew in a former existence, is a man of integrity and toughness who was clearly the right man to be chosen for the job. We have heard quite a lot about the December incident regarding the knife crime statistics. Let us be clear: it was a mistake. The Government acknowledged that from No. 10. The Home Secretary issued a fact sheet that was the cause of the problem and made an apology to Parliament. We immediately sought guidance from the statistics agency on how the matter should be addressed. The Cabinet Secretary was involved. Advice was given so that everyone—whether special advisers, Ministers or non-statistician civil servants—should understand the importance of that situation not occurring again. We have also made provision that there will be an investigation of any pre-release errors. There will be errors; there have been errors. The point of this is to have a position where something is in place that can take such errors on board and eradicate the possibility of them happening a second or third time. That is what we are seeking to do.

My Lords, does the Minister not accept that the knife crime figures came from an official in No. 10 and it was the Prime Minister who insisted that they were released? Should he then not have been the person who apologised?

My Lords, first, it was a Home Office statement. The question of No. 10 and special advisers, or whoever was involved at No. 10 saying that that was what No. 10 wanted to see, is not necessarily the same as the Prime Minister himself being directly and personally involved. I think that that issue has been put to bed in another place and indeed in your Lordships’ House. I do not think that there is anything more that we can add, other than to say, as I was saying, that we have put in—

My Lords, is the noble Lord saying that the Prime Minister is not responsible for what comes out of 10 Downing Street?

My Lords, I love it when noble Lords ask disingenuous questions to which they know the answer. It is the first lesson of politics never to ask a question when you do not know the answer. The noble Lord knows that that is not what I am saying at all. This issue is about building confidence in statistics, yet some noble Lords want to spend time discussing something that has been discussed several times before both here and in another place, for which apologies have been made and remedial action taken. It is not helpful in building confidence to dwell on that incident when we know that remedial action has been put in place. It is now a question of where we go from here in terms of the work of the statistics authority.

The issue referred to was information on foreign born workers in the United Kingdom. I do not know whether any noble Lord described it correctly. Many of the people included in the statistics had been or could have been British citizens in the United Kingdom for many years. Why those statistics were released is a question for the National Statistician because she is independent and works for an independent body. A criticism that I have is that, having put in place a system that appears robust, which has caused a public apology to be made and remedial action to be taken, should we not now be saying that that proves the value of the authority, its board of management, its head of assessment and its chairman? We should be congratulating them rather than trying to see whether there is some other motive.

We agree on several things. The ability of the statistics authority to comment independently on official statistics is an extremely important part of the Government’s reform programme. The Royal Statistical Society said in March, after the authority had been up and running for nearly a year:

“The authority is the independent and impartial body that Parliament intended”.

That is a very important endorsement and I am grateful for the work done by those in the authority to ensure that that is the true situation.

It is important to learn lessons from errors. The guidance note on the new code of practice—the one page affair—which was circulated, makes it clear how people who have doubts will be able to respond. A working party will be reporting shortly—some time this month—on the guidance that goes with the code.

I turn to the issue that has also been the subject of a number of contributions—pre-release access. When the statistical order came into effect in December last year it reduced the maximum permitted amount of pre-release access from five days to just 24 hours. The Government believe that it is important for civil servants to have the opportunity to advise Ministers on the statistics. It is true that Sir Michael would like to reduce that, and we will review it when it has been operating for one year. A number of other countries, including Canada, which is an exemplar when it comes to statistics, Australia and the United States, albeit briefly, have pre-release statistics to allow the Government of the country to put before the public the reasoning and arguments of their case when statistics are released.

A number of other questions were raised that I will not have the chance to answer in my 12 minutes. I will happily look at Hansard tomorrow morning and respond to all those requiring answers, although some are self-explanatory. I do not recognise some of the descriptions of the role of special advisers. It is not a question of the number of special advisers; it is a question of their role. The position has been made clear by the Prime Minister who revoked the ability of any special adviser to have authority over civil servants. I do not think that any special advisers are in any doubt about their role and their limitations. They do not make decisions; all decisions are made by Ministers. I will look through Hansard as I know that there are some questions that I have been unable to answer in the time allowed, and I will respond to noble Lords accordingly.

House adjourned at 8.39 pm.