The Cabinet Office, 1916-2018: The Birth of Modern Government

By Anthony Seldon, Jonathan Meakin

The cabinet office, set up in December 1916 with the name the ‘Cabinet Secretariat’, has been at the very heart of the British government and state for the past 100 years. Modern British government can be dated back to its founding.

Almost all government decisions of enduring importance in the last 100 years have been overseen by the Cabinet Office.

An aim of this book is to elevate public perception of the role of the Cabinet Office.

Prime Ministers have been at their most effective when they have worked with the Cabinet Office and with the Cabinet Secretary in particular.

WPrime Ministers have been at their most effective when they have worked with the Cabinet Office and with the Cabinet Secretary in particular.

The Cabinet Office was heavily involved in Britain’s relations with the European Union from the outset..it will now have an important role to play in Britain’s exit from that organisation.

During the last 100 years, there have been eleven Cabinet Secretaries.

They have become more valuable to the Prime Minister than any other senior official..because their field and expertise cross all departments.

When both the PM and Cabinet Secretary work together in harmony, optimal results flow.

Maintaining objectivity and appropriate distance from the Prime Minister has been often a challenge, especially when they seek to make the Cabinet Secretary ‘part of the family’.

Cabinet Secretaries have come from a narrow social band: they were all male, all white, all from England, and all middle-class. All but two attended British public schools, and all but two went to university at Oxford or Cambridge as undergraduates.

*** The Cabinet emerged in the fourteenth century out of the Privy Council, which for several centuries was the most important body in the country under the monarch.

The year 1689, the first full year of William III’s reign, is pivotal to our understanding of the emergence of modern government.

Parliament was consolidated as a permanent feature of the constitution.

To avoid the monarchy subverting Parliament, the Triennial Act of 1694 decreed elections had to be held for the House of Commons at least every three years.

Debates ensued on the proper balance of power between Parliament and the King’s government, in the form of the Cabinet Council.

Even as early as the 1630s, during the reign of Charles I (1625–49), the term ‘Cabinet Council’ had been in common usage.

Robert Walpole, is rightly seen as Britain’s first Prime Minister, though the term was not common for more than a century.

Serving from 1721 to 1742, he remains Britain’s longest-serving Prime Minister..He moved into 10 Downing Street only in 1735.

The key figure in the Cabinet Council early on was not the First Lord, but the monarch. It was the King, not the First Lord, who appointed the ministers.

The ascendancy of the Hanoverian dynasty initially under George I (1714–27) began the process of distancing the monarch from the political running of the country.

Not without reason did the German Chancellor, Angela Merkel, jest with Prime Minister David Cameron (2010–16) that it was the Germans who were responsible for the emergence of his job.

By the late eighteenth century, Cabinet was becoming more established.

The American War of Independence (1775–83) saw Cabinet’s position at the head of the King’s government become even more consolidated.

The war also accelerated the doctrine of ‘collective responsibility’, with ministers being collectively bound by decisions taken in Cabinet.

The doctrine of collective responsibility was strengthened further when Pitt the Younger (1783–1801 and 1804–06) was in office.

The doctrine was to prove vital in ensuring effective government during the long wars with Revolutionary and then Napoleonic France from 1793 until 1815.

After a long period of stasis, these years saw the modernising of the country, with income tax, the Ordnance Survey and the introduction of the National Census from 1801.

The 1830s were pivotal: the decade was marked by ‘the change from the concept of government as the King’s government to that of … party government’.

By the second half of the nineteenth century, Walter Bagehot would argue that ‘real power is not in the Sovereign, it is in the Prime Minister and the Cabinet’.

Government in Victorian Britain muddled through. The doctrine of Cabinet ‘collective responsibility’ did, however, become consolidated during the Victorian era.

Cabinets continued to meet throughout the Victorian era into the twentieth century, with meetings largely unrecorded.

The most consistent record during these years remained the Prime Minister’s letters to the monarch, informing them of what had been discussed.

Cabinet was a legally unrecognised body that sprung from spontaneous gatherings of political friends and colleagues: the values of the public-school gentleman held sway.

A significant issue was the strain it placed on the Prime Minister to oversee the execution of Cabinet decisions, on top of all his other work.

For coordinating the national response in the event of war was indeed to be one of the core functions of the Cabinet Office throughout its first 100 years.

Maurice Hankey became the first and longest serving Cabinet Secretary.

It would be the First World War that swept away this system of government. The war was to prove the catalyst that transformed Britain’s inexpert central government into a highly professionalised one.

In 1914, Britain had not been involved in a major war, with the exceptions of the Crimea and Boer Wars, since 1815, and had no inkling of the demands that the first total war in history would make upon it.

As government spending as a percentage of GDP more than tripled from just 16 per cent in 1910 to 55 per cent by 1917 and the Civil Service swelled to its largest size in history, became clear that the apparatus of government in Whitehall was inadequate for the task.

The root difficulty was how to combine rapid and effective executive action in the various theatres with the maintenance of Cabinet responsibility and control.

The inadequacy of Asquith’s style of leadership became all the more apparent when, in May 1915, a coalition government formed with the Conservatives.

Now there were many more big beasts wanting to air their views, with a corresponding challenge to effective decision-taking.

Decision-making on ‘the simpler colonial warfare of the past’ was clearly inadequate for ‘the professional needs of modern warfare’.

Lack of top-level coordination in Whitehall provided a deeper problem.

The Battle of the Somme, which lasted from 1 July to 18 November 1916, was to be the eventual undoing of Asquith and the whole ad hoc system of government at the centre.

Hankey worked all hours to entrench the Cabinet Secretariat into the Whitehall architecture.

He delineated four main roles for it: to record the proceedings of the War Cabinet; to transmit the decisions of the War Cabinet on the same day to those departments required to bring them into effect; to prepare agenda papers in advance; and finaaly,to attend to the correspondence connected with the work of the War Cabinet.

As historian Andrew Roberts has said, ‘if you wanted anything done in Whitehall you’d have to get Hankey on side.’

Hankey was apt to believe that his grasp of both bureaucratic and strategic affairs was superior to that of the Prime Minister.

By the end of the war, he had acquired seventy-seven staff, with all the senior positions held by men.

‘We have created a system of government, which, though made specially for the war, I think cannot be without its effect on the permanent constitutional history of the country.’

When the war ended, there would be existential challenges to the Cabinet Office and to Hankey’s standing.

The Treasury and the Foreign Office were the two great departments that viewed Hankey and his Secretariat with suspicion, as did the Admiralty and War Office.

Hankey had no intention of letting the Cabinet Secretariat be disbanded once it was over.

He calculated that his own position, and the continuation of the Cabinet Secretariat in peacetime, would be dependent on Lloyd George’s remaining in power.

By 1919–20 the Cabinet system had settled into a postwar rhythm.

Cabinet was still the major decision-making body in Whitehall.

A significant difference from pre-war government, however, was the growth of Cabinet Committees.

Committee system would grow further in the twenty years that followed.

On 19 October 1922, Lloyd George resigned. The coalition was over. Suddenly, Hankey’s position looked very precarious.

Baldwin was to dominate the interwar years, yet he had other close allies, and Hankey never bonded with him as he had with Lloyd George and Bonar Law.

The final moment in the story of the postwar survival of the Cabinet Secretariat came with Britain’s first Labour government in January 1924.

Hankey had no personal cause for alarm. Incoming Prime Minister Ramsay MacDonald made it clear from the outset that he did not wish for change in the operation of Cabinet.

Hankey and the Cabinet Office, as it was increasingly being called, emerged with its position strengthened from the short-lived first Labour government.

The 1920s saw the Dominion governments in Australia and Canada send officials to London to learn about the Cabinet Office, and assist them in creating their own.

Following the election of the second Labour government on 30 May 1929, Hankey lost no time in getting back on terms with MacDonald.

The collapse of the Labour government in August 1931 provided a fresh crisis for Hankey.

One of his first tasks was to draft a manifesto for the three party leaders who had agreed to form the National Government.

The most dramatic event of Hankey’s final years in power was the Abdication Crisis at the end of 1936.

Hankey’s last Cabinet was on 26 July 1938. At the end of the meeting, Chamberlain paid him a handsome tribute. ‘He could fairly be called the creator of the modern Cabinet.’

Hankey reminisced about working for seven Prime Ministers, ‘fourteen or fifteen’ governments, and taking the minutes of ‘more than 1,100 Meetings of the Cabinet.’

He died in January 1963 in the middle of the long, cold harsh winter. Hankey was the most controversial of all the eleven Cabinet Secretaries, and the closest to breaching the invisible line separating the unelected official and the minister.

***

On 1 August 1938, Edward Bridges officially took up appointment as Cabinet Secretary.

Under Bridges, the Cabinet Office secured its central place and its importance was enshrined permanently in British government, ensuring it would remain at the heart of Whitehall ever after.

Bridges ran a different operation from that of Hankey, whose military background and experience were so different to his own.

Bridges, indeed, was the first Cabinet Secretary schooled in the Treasury and modern Civil Service norms of conduct, and from the outset thought very differently about the job to Hankey.

In Bridges’s first few weeks in September 1938, Chamberlain wrote to Hitler asking for a personal meeting in an attempt to avert a war.

No Cabinet Secretary ever had a more dramatic first two months.

On 5 September, two days after war broke out, Bridges sent around a confidential note about changes to the Cabinet Office in the light of war.

Bridges’s job became harder as the war ground on. Churchill became tired and his working patterns more erratic, inspiring discontent and anger among Chiefs of Staff and ministers.

Few Cabinets could have ever rivalled in intensity the desperate series of meetings in late May 1940, against a background of Belgium and the Netherlands conquered, France crumbling, and Hitler’s forces at the doorstep.

The committee system was critical to Allied victory.

The Cabinet Office would play a vital role during the Second World War in the organisation of the international conferences to coordinate the Allied war effort, an echo of the niche Hankey had carved out for it during and after the First World War.

By 1945, some twenty-four Cabinet Office staff were permanently stationed in Washington.

‘Without the unruffled competence of the Cabinet Office, which became, under Sir Edward Bridges, the powerful organisation it has ever since remained, the machinery of government would have broken under the strain.’

By the time the war ended in 1945, the Cabinet Office consisted of 123 senior staff, 453 ‘subordinates’, making 576 in total.

Bridges was the only Cabinet Secretary to go on to another position in the Civil Service. He continued to serve with distinction as Permanent Secretary at the Treasury.

*** Norman Brook [third cabinet secretary 1947 - 1962]. For the middle years of this century, he piloted the ship of the British state, steering one Labour and three Conservative Prime Ministers through the growth of the welfare state, the decline of empire and Britain as a world power, and the rise of government intervention in the economy.

Brook became the ‘Additional’ Cabinet Secretary working alongside Bridges, before becoming the sole Cabinet Secretary when Bridges went back to the Treasury in January 1947.

His influence, indeed, was all-pervasive throughout his seventeen years as Cabinet Secretary.

Brook had worked very closely with Attlee during the war and both men were pleased to find themselves working with each other again.

Never in the twentieth century had a new Prime Minister been better prepared for the job, having been an all-encompassing Deputy Prime Minister to Churchill since February 1942.

With Attlee’s willing support, Brook after 1945 consolidated the Cabinet Committee structure that Bridges had built during the war.

The plan had not bargained for the return of Winston Churchill as Prime Minister in October 1951.

He flatly demanded that Brook remain in office. Brook was not unhappy; he liked and admired Churchill.

The post of Cabinet Secretary had entrenched itself after 1945 as the key position in the Civil Service.

So speedily did Brook re-establish his relationship with Churchill after his return that he was invited to the party that sailed to Washington to see President Truman just after Christmas Day 1951.

Within no time, Churchill was asking for Brook’s opinion on all kinds of issues.

Brook always prevailed over Churchill in retaining his full complement of Cabinet Office staff.

Brook expended considerable energy drafting the agenda for Cabinet, for which he gave Churchill increasingly directed handling advice.

Brook played a sensitive and controversial role in ensuring efficient government continued after the Prime Minister had a stroke on 23 June 1953.

Every ounce of Brook’s discretion, tact and judgement was required to keep the government on the rails.

e advised Churchill, against the views of most of the Cabinet, that he should stay on if he wished.

Why did Brook believe in keeping on a seriously ill 79-year-old rather than letting natural political processes take their course. Delay would only postpone inevitable succession.

Churchill stood firm, determined to remain in office until he had reached an understanding with the Soviet Union, following the death of Stalin in March 1953.

Churchill eventually went to the Palace to see the new Queen Elizabeth II on 5 April 1955. A day later, in line with the Royal Prerogative, Eden went to the Palace and emerged as Prime Minister; a job he had anticipated for fifteen years and more.

Eden was about to drag Brook into the greatest moral crisis of his career. This was the Suez Crisis, precipitated by the Egyptian President Colonel Nasser’s decision for his forces to seize control and nationalise the Anglo-French canal on 26 July 1956.

Brook had long been alert to the possibility of trouble in the Middle East.

He was secretary to the Egypt Committee, chair of the Egypt (official) Committee and of the Defence (Transition) Committee overseeing Suez. He knew every single Suez secret, even the worst.

Britain requested Egypt leave the Suez Canal and, when this request was refused, Britain and France launched their invasion.

With a vote in the UN 64:5 against the military invasion, Cabinet divided, sterling falling dramatically, Eden was forced to end military operations after capturing only twenty-three miles of canal.

Brook was doubly at fault over Suez: for not preventing Eden from lying to the commons and for destroying the documents.

Bridges was scathing, saying that if he had been Cabinet Secretary, ‘Suez would never have happened.’ This is a significant indictment. Never before or since has a Cabinet Secretary been so critical of another.

Eden would eventually confront the truth in a private interview in 1967, which he said could not be used until after his death.

Macmillan, the last Prime Minister of four in succession to have served in the First World War, was admirably qualified to become Prime Minister.

He was one of those Conservative Prime Ministers – Heath and Major were others – who were happier working with civil servants than fellow politicians.

Mcmillan’s own description of the Cabinet Office is illuminating:

‘Parallel to Downing Street staff, but not exclusive to the service of the Prime Minister, is the Cabinet Office. It’s head acts as Secretary to the Cabinet, and he and his colleagues serve all Ministers alike.

The Secretary to the Cabinet acts in effect both as co-ordinator and friend in a very special degree. It was my good fortune to have from the beginning the outstanding services of Sir Norman Brook’.

Macmillan provided a stability and consistency that No. 10 had not seen since Attlee, daunting though the task confronting him was post-Suez.

Macmillan, like Churchill before him, wanted Brook to travel with him on overseas trips. The Cabinet Secretary proved his worth from the very outset.

A secret briefing for new President Kennedy from the US State Department in 1961 described Brook as having ‘the best mind in the Civil Service’.

Few Whitehall mandarins rivalled Brook’s understanding of postwar Britain.

Macmillan’s reliance on Brook can be seen in the crisis in July 1958 in Iraq, when a military coup saw the overthrow of the pro-Western Hashemite monarchy established by King Faisal in 1921.

The July 1958 revolution saw King Faisal II, the Regent and Crown Prince, and the Prime Minister, all assassinated, and an Arab nationalist Republic of Iraq established, which ended its alliances with the West, and later to Saddam Hussein emerging as Iraq’s leader in the 1970s.

Macmillan wanted the process of decolonisation to be orderly, and to lead to voluntary membership of the Commonwealth.

Brook accompanied Macmillan on his celebrated tour of Africa in January and February 1960.

‘Few of the [local officials] realised that he was the central cog in the British government machine’.

Brook helped to shape Macmillan’s centrepiece ‘Winds of Change’ speech, delivered to the South African Parliament in Cape Town on 3 February.

Brook ensured the Cabinet Office became the overseer of the Commonwealth and its regular conferences.

Between 1946 and 1962, he acted as secretary to eleven full-scale conferences for its Prime Ministers held in London.

Brook was intimately involved in the emergence of Britain as a nuclear power after the Second World War.

Macmillan was in power for two of the most serious incidents of the Cold War, including the Berlin crisis in 1961, when the Berlin Wall was erected, and the Cuban Missile Crisis in 1962.

The Cabinet Secretary’s job overseeing intelligence was extended under Brook, though oversight did not come fully under the incumbent until Trend succeeded him in 1962.

Brook was instrumental in the appointment of MI5 head Dick White to run the Secret Intelligence Service (MI6) in 1956 with a brief to shake it up.

Macmillan’s final period in power, and thus Brook’s, was overshadowed by a series of spy scandals.

Brook became very close to Macmillan on appointments. In his major reshuffle in July 1962, the so-called ‘Night of the Long Knives’, when he dismissed seven Cabinet ministers, Brook was by his side throughout.

Brook retired in December 1962 when the Cabinet Office was moving out of the Great George Street offices into 70 Whitehall.

On top of trying to run the Civil Service, he had also to do the regular Cabinet Secretary job as the Prime Minister’s chief adviser, head the Secretariat, oversee propriety and ethics, intelligence, the Commonwealth, and be the Prime Minister’s chief foreign policy adviser.

He died at his home in Chelsea in June 1967, the only Cabinet Secretary to die as young as sixty-five.

*** The sixteen years from 1963 to 1979 were years of satire and irreverence, with the establishment – a term coined only in 1955 by Henry Fairlie – no longer able to command an automatic respect, and with the ‘man in Whitehall’ becoming a figure of ridicule.

Brook saw Burke Trend as his natural successor as Cabinet Secretary in January 1963. Macmillan was only too happy to agree.

Henry Kissinger, later to be US Secretary of State, often spoke to Trend, describing him as ‘imperturbable, well-informed, discreet, tactful, [and] quite charming’.

Before Trend’s first month in office was over, French President Charles de Gaulle vetoed Britain’s application to join the EEC.

Brook might have engineered for Macmillan to stay on in office, as he did Churchill, but not Trend.

With Alec Douglas-Home, Trend was at his happiest. He found here a civilised, orderly and scrupulous Prime Minister.

Home deeply respected Trend and his oversight of Cabinet Committees, crucial to the smooth conduct of government.

The contrast between the aristocratic Etonian Douglas-Home, and the state-schooled Harold Wilson, could not have been starker.

Wilson burst on the Whitehall scene with a dynamic ‘First Hundred Days’ plan in emulation of President Roosevelt.He wanted Trend to be an integral part of his ambitious programme.

Trend’s relationship with his new Prime Minister started out very well: Wilson told Barbara Castle that Trend ‘was the best civil servant I have ever known’.

After Britain became a nuclear power in the 1950s, new responsibilities fell to the Cabinet Secretary.

Overseeing the process of appointing nuclear deputies for the Prime Minister was another of the Cabinet Secretary’s responsibilities. The Cabinet Secretary thus had a crucial role in a nuclear war.

When the British colony of Rhodesia declared unilateral independence in November 1965 it expressly rejected London’s demand that it consent to black majority rule.

Wilson’s response was to institute sanctions, coordinated by the Cabinet Office.

By early 1966, Wilson was tiring of his tiny majority and triggered a general election. Labour defeated the Tories in style on 31 March.

Trend wanted to ensure that Wilson’s second government would be a much tidier ship. He had become concerned by an excessive number of meetings, and by their lack of rigour.

It is a mark of Trend’s ability that he kept the system of Cabinet government fully operational and intact throughout, overseeing considerable expansion in the size of government.

The government introduced a wide range of liberal reforms, which Trend facilitated even if he did not always find them congenial.

Trend was rejuvenated by the arrival of a new Prime Minister in the shape of the Conservative, Edward Heath.

Heath wanted fewer Cabinet Committees and to make Cabinet itself much smaller.

Trend achieved success in persuading Heath to agree to important reforms to strengthen the Cabinet Office’s capacity for managing emergencies, reforms which he had failed to sell to Wilson in 1969.

In 2001, under Blair, the Cabinet Office took over responsibility for all crisis planning for both civil and military eventualities, in the Civil Contingency Secretariat.

In March, Heath suspended Stormont, and direct rule of Northern Ireland from London resumed.

Heath was in almost constant battle against the trade unions, inflamed by his Industrial Relations Act of 1971.

Not since the 1926 General Strike had union relations been so inflamed.

A desperate Heath asked Trend to increase surveillance of miners’ and other militant forces.

Trend’s heart remained with his staples, above all Anglo-American policy.

Trend found himself one of the foremost Atlanticists in Whitehall, which did not go unnoticed in Washington DC.

Kissinger and Trend spoke regularly on all manner of subjects.

Heath believed firmly that Britain’s destiny lay in Europe and, as his premiership evolved, he sought to distance the UK from the Americans.

Europe was the more serious issue dividing Prime Minister from his Cabinet Secretary than differences over the US.

Heath was disappointed that Trend’s Atlanticism and love of the Commonwealth meant he could not share his enthusiasm for Europe.

The European Community Act was passed in October 1972, and Britain joined in January 1973.

Heath, a lover of Europe to his fingertips, was very impressed by the French Civil Service model, where officials were far more active politically.

But was Trend necessarily at fault for not playing the game that Heath wanted? Rather he deserves credit for abiding by his Civil Service code and personal credo, and refusing to be overly drawn into the cauldron.

Trend began looking for an exit.

Burke Trend had dual motivations for leaving: ‘he thought that ten years was enough’, and he ran ‘out of sympathy with Heath’s European direction of international policy’.

His premature departure was a poignant end to the career of a great public servant.

***

With Trend clearly on his way out, John Hunt was brought into the Cabinet Office as his deputy, poised to take over.

The only Cabinet Secretary to serve in the Second World War, he always had the air of a military man about him, and was always eager to support the services.

In 1973, Heath invited Hunt to conduct an audit of the Cabinet Office, while Trend was still in situ and to make recommendations for improving it after he departed.

Hunt scrupulously avoided criticising Trend personally, maintaining that all changes must await his retirement, but he nevertheless pinpointed several inadequacies in the Cabinet Office’s organisation.

The first Catholic to be appointed Cabinet Secretary, he was said to be the most senior Catholic openly operating in Whitehall since Henry VIII disposed of the services of Chancellor Thomas Wolsey in 1529.

Hunt’s four months with Heath proved a baptism of fire.

Heath called an election for 28 February, posing the question ‘Who runs Britain? The government or the unions?’

Wilson returned to Downing Street again on 4 March.

For Hunt, it was out of the frying pan and into the fire.

Hunt was the supreme technician – less cerebral then Trend or Brook – but a master at getting the job done, and in comprehending the electric currents of power.

Hunt, like Trend, and indeed all their predecessors, was deeply interested in diplomatic and intelligence matters.

Hunt, nevertheless, found the space to achieve his most enduring legacy as Cabinet Secretary: the welding of the Cabinet Office into a strong and better staffed force in Whitehall.

Hunt was ‘the single most powerful official in Whitehall or indeed the country’, who succeeded in drawing power back into the office and recovering its prestige.

Both Wilson and Callaghan used Hunt’s guile to advance their own agenda, and to deter those opposed to them, notably Tony Benn.

Callaghan became Prime Minister on 5 April 1976, ushering in a period where Hunt was at the height of his powers.

He was undisputed master of the Cabinet Office, widely regarded as ‘a senior guy, who had been around a long time, with views of his own … he’d set his own standard’.

Callaghan made a deal with the SNP and Plaid Cymru, agreeing that the government would devolve powers to them.

Hunt was the first to highlight the significance of the famous ‘West Lothian’ question.

Hunt outlived the Labour government that defined his time as Cabinet Secretary. In May 1979, the election was won by the Conservatives under Margaret Thatcher, heralding the beginning of her turbulent, pivotal and decisive premiership.

Hunt was rapidly swept up into the Thatcher maelstrom. Thatcher had little time for either of the civil servants she inherited from Labour.

‘She did not trust Hunt,’he was too obviously a fixture of the previous era. ‘She found him too managing and too inclined to tell her what to do’.

Hunt was ready to retire at the age of sixty.

Following his death from cancer aged eighty-eight,Michael Quinlan, the leading Whitehall defence mandarin, summed him up. ‘Both what he did and what he was, he enriched the public life in this country.’

***

After two decades of relative difficulties and decline, Britain re-emerged on the world stage under a strong, if controversial, leader in Mrs Thatcher.

Robert Armstrong could not have been better qualified for the post of Cabinet Secretary, while simultaneously appearing utterly incompatible with the Prime Minister he was to serve for his entire tenure.

He was the only Cabinet Secretary in the twentieth century to serve just one figure in No. 10.

Armstrong served under Thatcher from 1979 to 1987, and became a master in understanding her.

However, he never became truly ‘one of us’, in Thatcher’s terms.

Thatcher’s initial sense was that Whitehall mandarins opposed her radical ideas, an instinct that grew over time.

Thatcher was keen for the Cabinet Office’s compass to be expanded ‘to mesh the work of the great Whitehall departments into a coherent whole.’

Armstrong could be a feisty defender of Thatcher.

The Falklands War, nevertheless, showed him to be a man of prescience on foreign policy, as well as a calm dispatcher of business in the very eye of the storm.

Thatcher’s second term (1983–87), bookended by two strong election victories, saw her most successful years as Prime Minister.

The Anglo-Irish Agreement of 1985 was very much the fruit of Armstrong’s personal endeavours.

His personal input as Cabinet Secretary was never seen more artfully than over Ireland.

Thatcher was content to give Armstrong wide rein. She did not trust the Foreign Office to deal with the Irish.

She became still closer to Butler during the miners’ strike. Butler was one of those, along with Armstrong, who saw the actions of Arthur Scargill’s NUM as a threat to the British state itself.

Defeating the NUM significantly weakened the trade union movement and constituted a major victory for Thatcher.

The Cabinet Office changed its size and ambitions under Armstrong’s tenure.

Following the abolition of the CPRS in 1983, a new ‘Science and Technology Secretariat’ was set up, resulting in further expansion of Cabinet Office staff. It became the fifth Secretariat operating under Armstrong.

The Economic Secretariat, responsible for the economy, energy and industry, was the one that interested Thatcher most. It had a succession of notably able deputies in charge, including Richard Wilson, the later Cabinet Secretary.

It was one of Armstrong’s great achievements that after the Falklands and Westland in particular, he reinstated Cabinet government and committees: but it was a constant battle for him to preserve the status quo.

For his successor Armstrong recommended a shortlist of two: Clive Whitmore and Robin Butler.

Butler saw the appointment as the very apex of a civil servant’s career.

***

Butler had been mentally prepared to be Cabinet Secretary for a long time, his mentor being Robert Armstrong.

Unsurprisingly, he found this task easier when Major, rather than Thatcher or later Blair, was Prime Minister.

He saw her reliance on Bernard Ingham and Charles Powell as ‘very dangerous’.

Matters came to a head three months later, when President Reagan was to visit London. Thatcher and Powell did not plan for Butler to be part of the home team.

A final straw came when Nicholas Gordon-Lennox was persuaded to retire early from the Embassy in Madrid to make way for Powell. Thatcher had approved it, but then she and Powell changed their minds. Butler later estimated that it was the clearest single occasion when she rejected his advice.

The more her energy waned, the more dependent she became upon Powell and Ingham for advice.

Her speech in Bruges proved highly controversial and became a pivotal historical moment, when the Conservative Party ceased to be ‘the party of Europe’ in British politics.

Butler believed that it was here that Thatcher laid the seeds of her future downfall.

By 1986, Thatcher was certainly becoming tense and frustrated with her Civil Service, and they with her.

Her final year was indeed her most fraught, and her behaviour its most volatile.

She and President Bush played a key role in galvanising the multinational coalition against Iraq.

Thatcher deliberately avoided constituting the War Cabinet as a Cabinet Committee, thereby bypassing the Cabinet Office.

‘Bureaucratically it was getting into a tangle. It was constitutionally pretty dangerous.’

By the time of her fall in November 1990, Butler was unable to prevent the inevitable.

Butler was pleased that Thatcher’s replacement in November 1990 was John Major. The succession meant a return to the conventions of Cabinet government

They were an odd couple. Major was state-school educated and never went to university, Butler was a Harrow and Oxford educated grand mandarin.

Major himself had no hesitation in placing Butler in charge of the War Cabinet minutes during the build-up to the First Gulf War (1990–91). Butler took his intelligence duties very seriously.

The rhythm of Cabinet Committee meetings for ministers and officials began to be re-established under Major.

Sleaze was the most difficult issue that Butler confronted as Cabinet Secretary, as the job involved advising the Prime Minister about ethical breaches.

Major’s greatest single achievement was staying in power for seven years.

Black Wednesday in September 1992 severely damaged Major’s reputation and the economic credibility of the government.

Butler found himself increasingly in despair, not at the conduct of Major, as he had been with Thatcher, but with Cabinet ministers in general.

One particular occasion where Major followed Butler’s advice came after a close defeat for the government on the passage of the controversial Maastricht Bill in the summer of 1993.

Butler’s clear advice was to hold a vote of confidence the following day.

Eurosceptic Conservative MPs, faced with the prospect of bringing down their own government, came back into line, which meant that it passed with forty votes, and the Maastricht Treaty was ratified.

Butler continued to work closely with Major until the end.

History can judge the seven-year Butler–Major relationship as one of the most effective of the 100 years.

***

Labour under Tony Blair came to power with an ambitious plan to modernise an outdated country.

Butler was swept up in the excitement of a new, youthful government coming into power, with the prospect of a fresh approach to policy.

One of Butler’s first tasks with Blair, as it had been with Major, was to explain the responsibility of a Prime Minister in the event of a nuclear war.

Only when Powell published his book The New Machiavelli in 2011 did Butler begin to fully appreciate the depth of hostility towards him from the Blair team.

Butler was not sorry to leave at the normal retirement age of sixty.

Blair’s team thought they knew it all.

Blair’s team had been eager to handpick a successor to Butler upon his retirement at the end of 1997.

Richard Wilson, Permanent Secretary at the Home Office, was selected as the outstanding candidate.

Wilson’s stellar reputation developed as Head of the Economic Secretariat in Thatcher’s last three years.

Wilson was hugely excited by the prospect of working with the new Blair government.

Here was a new generation coming into office in Britain, and he would have the chance to help them achieve their grand vision.

Wilson had a very clear conception of the job of Cabinet Secretary, and hoped it would chime with Blair’s.

Blair (and Brown after him) were idiosyncratic and dismissive of Cabinet government, and it took all of the skills of Wilson to establish a working relationship.

Procedure for Ministers was published: ‘Goodbye Cabinet government. Welcome to the Blair Presidency,’ wrote journalist Peter Riddell.

Difficulties arose because of New Labour’s vision of how government should be run, which sat uneasily alongside conventional Cabinet government.

Blair was fully committed to a command and control model of government.

This tension between two mutually incompatible approaches to government was apparent to Wilson from very early on.

Wilson believed in evolution: New Labour in revolution – the Blair revolution – a revolution that largely failed to materialise.

He noted the sharp contrast in the style of Cabinet meetings under Blair to those under Thatcher.

The profound fissure between Brown and Blair, and their highly partisan camps, made life much more difficult for Wilson than it already was.

Wilson was preparing for his final full year in office when, on 11 September, the American passenger planes struck the Twin Towers.

Wilson might have been only partially successful in blunting Blair’s centralising tendencies, but he did manage to bring about the most radical restructuring of Whitehall for several years.

In his last months, when his successor was being appointed, Wilson put it bluntly to Blair: ‘Your problem is that neither you nor anybody in No. 10 has managed anything on a large scale,’ he said. ‘I have managed the Labour Party,’ retorted Blair. ‘You have never managed them, you have led them, there is a big difference.’

It was Wilson’s vision, not Blair’s, however, which was the more accurate description of how government was to develop in the years that followed.

The deepest disagreement between Wilson and Blair’s team, however, came about over his successor.

At the end of July 2002, Blair hosted a farewell dinner for Wilson at No. 10 for his family. Whatever differences had come between them, the farewell between Wilson and Blair had all the appearance of being warm and sincere.

***

Andrew Turnbull emerged as the strongest and most acceptable candidate, becoming Cabinet Secretary on 1 September 2002.

Turnbull was the first Cabinet Secretary not to have attended a public school.

Turnbull was a very different figure to Wilson; less urbane and donnish. Turnbull described his own style as ‘quite functional, getting things done, making things happen’.

He came to reluctantly accept that ‘Blair was not looking towards his Cabinet Secretary to be a major policy adviser’.

The Cabinet Office was less immediately involved in the Iraq War than in any other British war over the previous century.

Critically, the Cabinet Office was not asked to provide a secretariat function recording minutes of Blair’s informal Iraq meetings.

One can only speculate how events might have turned out differently had there been a Cabinet Secretary occupying the traditional role.

Turnbull, like Wilson and Butler before him, found himself increasingly standing up for Cabinet conventions even if he was powerless to insist that Blair observed the correct decisions of Cabinet.

Turnbull formed the view that they knew exactly what they were doing, and had worked out their modus operandi before they came to power in 1997.

It was as if Blair wanted to wind the clock back to Lloyd George’s suburb, or even pre-1916, pre-Hankey.

The Cabinet Office was rarely involved in the most important decision post-Iraq: entering the single currency.

Brown stopped Britain joining the euro, though it proved to be the right decision for Britain, and hampered Blair’s entire public sector reform programme.

Blair did not have it in him to say: ‘I am going to get myself a new Chancellor.’

Turnbull, at fifty-seven, was the oldest Cabinet Secretary on appointment, and was the shortest serving of the eleven.

***

Gus O’Donnell was the second Roman Catholic Cabinet Secretary, and the first and only Cabinet Secretary to attend neither public school nor Oxbridge as an undergraduate.

O’Donnell was of very different ilk to all other Cabinet Secretaries, his easy charm and ‘bloke-ish’ manner chimed with Blair.

Less experienced as a policy adviser than some Cabinet Secretaries, he nevertheless proved a supreme operator as Cabinet Secretary.

Less experienced as a policy adviser than some Cabinet Secretaries, he nevertheless proved a supreme operator as Cabinet Secretary.

Turnbull and Wilson worked for just one Prime Minister, whereas O’Donnell went on to serve three: the tail end of Blair, three highly charged years of Brown, and David Cameron.

O’Donnell joined a Tony Blair bristling with confidence, having won a third election.

Blair knew he had wasted too much time early in his premiership, and he needed to work purposefully.

Despite his frantic pace in his final two years, Blair’s domestic agenda was far from complete when he left office.

The question, though, is could he have achieved more if he had worked with the norms of the Cabinet Office and the traditions of collective responsibility, rather than against it?

A new era seemed to have arrived when, on 27 June, after ten years of waiting, Brown walked through the door of No. 10 as Prime Minister.

Cabinet government was back in business, he said, and he wanted to listen to ministers’ views.

Within months, however, all Brown’s high-minded aspirations began to evaporate. He decided he could not trust Cabinet not to leak, and he became disenchanted with Cabinet Committees.

His aides were soon complaining, as had Blair’s: ‘Gordon has no concept of management. He is incapable of it, he is a hopeless team manager.’

‘It was total nightmare as no one ever knew what was going on,’ said another.

O’Donnell found himself catapulted into the heart of the most anarchic premiership of the last 100 years.

Brown’s mind turned increasingly towards Heywood, who returned to the Cabinet Office in June 2007 from the investment bank, Morgan Stanley.

A new post of ‘Permanent Secretary’ at No. 10 was negotiated between Brown and O’Donnell, and dangled before him.

Ironically, it was the banking crisis which saw a return, for a while, of a semblance of Cabinet Committee governance.

Brown’s biggest triumph was the G20 Conference, which he chaired in East London in April 2009. It helped rekindle optimism in the international community that the world would not descend into a 1930s-style depression.

O’Donnell became increasingly dismayed in the months that followed about Brown’s inability to run a stable Cabinet team, a core responsibility of the Cabinet Secretary.

Brown’s aggressive and erratic behaviour to his staff became a matter of public discussion, into which O’Donnell was dragged.

So close to the general election, the episode led to a deterioration of O’Donnell’s relationship with Brown, from which it never recovered.

When Blair became Prime Minister in 1997, Cabinet government and collective responsibility took its biggest hit since Lloyd George’s coalition government.

He was not comfortable using Cabinet or committees as decision-making forums, and this knowledge sent shockwaves through the system.

Blair was frustrated by the lack of power at the centre, and ended up being disappointed (as he was apt to be with people) by the lack of reforming zeal of many of those officials he encountered, with the notable exception of his No. 10 official Jeremy Heywood.

Blair and Brown’s government achieved more than their detractors maintain.

The real question is to ask how much more could have been achieved if they had come to power in 1997 not with hostility, but respect for the Cabinet Office and the conventions it upheld for eighty years?

The Cabinet Office was bloodied and more than a little bowed by the time the New Labour storm subsided. The inconclusive general election in May 2010 threw a new challenge at it.

***

Rarely had any of the eleven Cabinet Secretaries faced a more delicate task over the 100 years than that presented to O’Donnell on the morning following the inconclusive general election on 7 May 2010.

The average time for European countries to form a coalition was over forty days. It became clear the Lib Dems were not going to form a coalition with Labour.

O’Donnell and his fellow officials were constantly aware of their primary job, to ensure Britain had a government capable of functioning.

The shared determination of the two leaders to arrive at solutions produced speedy decisions on a whole range of fraught matters.

Clegg worked well with his Civil Service support and found it fair-handed.

O’Donnell helped devise a ‘Coalition Committee’ to meet weekly and more, as needed, to resolve disputes.

O’Donnell was all too easily painted by Conservative economic hardliners as anti-austerity, and as sympathetic to Labour’s policies.

He regularly asked Cameron whether he was fully aware of the implications of the proposed NHS and welfare policies.

It was a mark of O’Donnell’s ability that he retained the trust of Conservatives throughout his period in office.

In October 2010, O’Donnell launched the Cabinet Manual, laying out the laws, rules and conventions for government in the UK.

He also oversaw the establishment popularly known as the ‘Nudge Unit’.

Another innovation proposed by the Cameron government to which O’Donnell lent his personal support was the measurement of ‘well-being’.

Another task O’Donnell reaffirmed was the responsibility for the Cabinet Office for intelligence and security.

Cameron and Francis Maude, the minister responsible for the Civil Service, wanted Heywood to become Cabinet Secretary.

***

Jeremy Heywood joined the Civil Service at the age of twenty-one, starting at the Health and Safety Executive before moving quickly to the Treasury, where his first boss was none other than Gus O’Donnell.

So quickly did he impress the new Labour masters after the May 1997 election that, just two years later, he was appointed PPS to Blair.

Few civil servants have been as much relied upon by Labour and Conservative administrations as Heywood.

Two of his attributes, shared by many of his predecessors, are formidable hard work, and his ability to provide creative solutions to the myriad of problems crossing his desk.

The Cabinet Secretary job was the one Heywood was made for.

Central to the role of Cabinet Secretary, as Heywood has performed it, is to be the Prime Minister’s chief Civil Service policy adviser and fixer.

Cameron ‘never stopped regarding me as his senior Civil Service adviser’ as he said.

Heywood’s first five years found him having to pick a way through a number of delicate issues involving ministers, few more fraught than Andrew Mitchell and the ‘Plebgate affair’.

In the run-up to the EU referendum in June 2016 and, even more so, after the result, EU issues inevitably took up more and more of his time.

Only someone capable of immensely hard and relentless work could oversee such an Everest of a job.

Initially, he was very focused on delivering coalition policy and helping the coalition resolve its difficulties.

‘Heywood helped to create the conditions that allowed the government to last five years when the general belief was it wouldn't last the course.’

In the run-up to the 2015 general election, the Cabinet Office engaged in preparatory work in the event of a hung parliament, as in 2010.

Heywood worked for several months with a Conservative-only government, helping it get off to a running start implementing its manifesto before being swept up in the EU referendum in June 2016, and its aftermath.

Theresa May, the nineteenth Prime Minister to be served by the Cabinet Office since December 1916, began her premiership in a blaze of activity, with the Cabinet Office responding to the new challenges of exiting the European Union.

Hankey would have approved.