My Lords, this is the most transparent Government ever, publishing more data more frequently than ever before. A key plank of our commitment to transparency is releasing public records after 20 years rather than 30 years, as was previously the case. For a transitional period up to 2023, there is a doubling of the information in scope. While that is a significant challenge, we are constantly improving to meet and build on the high standards we have set ourselves.
Why has the total number of documents released by the Government fallen so sharply? It is down from over 500 at the start of last year to under, I think, 60 at the start of this? Why should historians of events such as the Profumo affair of over 50 years ago and the Burgess and Maclean affair of over 60 years ago still be denied access to documents? Is there a particular problem with the release of papers by the Foreign and Commonwealth Office?
My Lords, starting with my noble friend’s last question, while the Foreign and Commonwealth Office is currently behind with its annual transfers, it remains the top transferring department in terms of volume, with nearly 15,000 files transferred in 2015. My noble friend also mentioned the Burgess and Maclean case. I gather that the relevant documents were released in October 2015. As regards the Profumo affair, the Cabinet Office is working with the National Archives to prepare the Denning papers for release, and an announcement will be made in due course.
Is not one of the basic problems about the Government’s good intentions—which I acknowledge—the operation of the interpretation of the so-called Freedom of Information Act? Is that not an obstacle to historians wanting to inquire about, for example, the invasion of Iraq? That was admittedly 13 years ago, but freedom of information has been applied to episodes many years before that. Does the Minister have any general observations about the approach that the Government could adopt in this area?
As far as the Freedom of Information Act is concerned, the noble Lord will be aware that there has been a recent commission report on this issue. Generally speaking, Her Majesty’s Government have accepted all proposals. I will let the noble Lord have details of the Government’s response. He also makes a good point as far as the history in relation to recent activities is concerned. As he is no doubt aware, the applications to retain certain records are scrutinised by the Advisory Council on National Records and Archives, which is chaired by the Master of the Rolls.
My Lords, I am second to none in my admiration for departmental record reviewers; they have a very intense workload. Not only do they have to prepare historical material for release to the National Archives but, as my noble friend and mentor Lord Morgan has just pointed out, they firefight on freedom of information requests, too. Would the Minister accept that the reduction in progressive stages of the 30-year rule of old to 20 years is not a “wouldn’t it be nice to if we had the time” requirement but a legal requirement and that, therefore, the human and financial resources must be found to make sure that it is done properly and on time?
My Lords, the noble Lord, Lord Hennessy, refers to the transitional arrangements, the timetable for which was established by the Constitutional Reform and Governance Act 2010. It set out the agreed timetable for transitioning from the 30-year to 20-year rule, which should be completed by 2023.
My Lords, I ask in particular about the Hanslope Park FCO archives. My wife, with a number of other scholars, visited them some time ago and we are grateful to the FCO for that. There was a huge set of files there—somewhere between 600,000 and 1 million—many of which have not been catalogued, let alone cleared. We were grateful to the Foreign Office for providing a number of retired diplomats to assist in this. Can we have a commitment from the Government that they will continue to provide additional funds so that those files can be sorted and cleared as quickly as possible, given how enormous the archive is?
The noble Lord makes a good point. The amount of files in various departments to go through—some relating way back to prisoner of war details and details of those serving in the armed services—is enormous. The Foreign and Commonwealth Office is working as hard as it can to clear this backlog.
My Lords, I declare an interest as a member and indeed secretary for some years of the All-Party Group on Archives and History. The special problem here—already alluded to by a number of speakers—is that the Foreign Office files, because of their great breadth and diversity, are creating a major management problem for archivists. I accept that, but is it not disappointing that, for example, the Irish Times, looking particularly at Anglo-Irish relations, has gone along for the last two tranches of 20-year releases and there has been nothing for the paper on either occasion? It is particularly disappointing because, ironically, the 20-year recommendation comes from a report by Sir Joseph Pilling, former Permanent Under-Secretary at the Northern Ireland Office. Can something be done, in other words, just to loosen things up a bit? I understand the difficulties with scale and the problems that archivists have, but this seems to be an unfortunate outcome.
My Lords, the noble Lord, Lord Bew, knows of my interest in matters relating to Ireland. There is a great paucity of records available in Dublin, to be perfectly honest. I will pass the noble Lord’s query back to the department but, as he knows, the archive that we have at Kew goes back to 937 AD. There is an enormous number of records there.
My Lords, the material we have been talking about up to now is almost entirely paper-based, but since 2008 most communications in Whitehall have been on email, policy papers have been generated electronically and announcements have been made through social media—believe it or not. The Minister will be aware of the excellent report produced by Sir Alex Allan in 2015 on digital records in Whitehall. Can he explain how far we have got in implementing that, and in particular the very important proposal Sir Alex makes for the emails of designated Ministers and senior officials to be automatically preserved? Is this happening?
I do not know the answer to the last part of the noble Lord’s question but I am aware of the review carried out by Sir Alex Allan in 2015. The National Archives have built the infrastructure to take in and present digital records, and have completed several successful pilots. I recommend looking at their website and Discovery, which is the National Archives’ online catalogue.