Defend the Realm: The Authorized History of MI5 Hardback - 2009

Excerpt

Introduction

The Origins of the Secret Service Bureau

The Security Service (MI5) and the Secret Intelligence Service (SIS orMI6) began operations in October 1909 as a single organization, the Secret Service Bureau, based in premises rented by a private detective, retired Chief Inspector Edward ‘Tricky’ Drew, at 64 Victoria Street, London SW1, opposite the Army and Navy Stores. The Bureau was staffed initially by only two officers, the fifty-year-old Commander Mansfield Cumming RN and an army captain fourteen years his junior, Vernon Kell, who met for the first time on 4 October when, according to Cumming’s diary, they ‘had a yarn over the future and agreed to work together for the success of the cause’. Cumming and Kell later parted company to become the first heads of, respectively, SIS and MI5. For several months, however, they were based in the same room, struggling, with minimal resources, ‘to deal both
with espionage in this country and with our foreign agents abroad’.

The Secret Service Bureau owed its foundation to the recommendations of a sub-committee of the Committee of Imperial Defence, the chief defence planning council of the realm, which had been instructed in March 1909 by the Liberal government of Herbert Asquith to consider ‘the nature and extent of foreign espionage that is at present taking place within this country and the danger to which it may expose us’. It reported on 24 July: ‘The evidence which was produced left no doubt in the minds of the
subcommittee that an extensive system of German espionage exists in this country and that we have no organisation for keeping in touch with that espionage and for accurately determining its extent or objectives.’ Most continental high commands would have been surprised to discover that British intelligence was in such an enfeebled state. There was a widespread myth that, ever since the days when a secret service run by Queen Elizabeth I’s Secretary of State, Sir Francis Walsingham, had successfully uncovered a number of Catholic plots, British intelligence, like the British Empire, had grown steadily in size and influence, spreading its tentacles across the globe.

The myth was encouraged by Edwardian spy novelists. The most prolific and successful of them, William Le Queux, allegedly Queen Alexandra’s favourite novelist, assured his readers: ‘The British Secret Service, although never so prominently before the public as those unscrupulous agents provocateurs of France and Russia, is nevertheless equally active. It works in silence and secrecy, yet many are its successful counterplots against the machinations of England’s enemies.’6 Le Queux (pronounced ‘Kew’) was a Walter Mitty figure who fantasized that he had played a personal part in some of these successes. In Secrets of the Foreign Office published in 1903, Le Queux, thinly disguised as Duckworth Drew, ‘secret agent in the employ of the Foreign Office, and, next to his Majesty’s Secretary of State for Foreign Affairs, one of the most powerful and important pillars of England’s supremacy’, quickly gets the better of the long-serving French Foreign Minister, The´ophile Delcasse´ (equally thinly disguised as Monsieur Delanne). Delcasse´, alias Delanne, ‘admitted that he longed to smoke one of my excellent light-coloured Corona Superbos’. But there was more to Drew’s cigars than met the Minister’s inattentive eye: ‘To this day Monsieur le Ministre is in ignorance that that particular Corona had been carefully prepared by me with a solution of cocculus indicus . . .’ Outwitted by the
cunningly prepared Corona, the disoriented Delanne revealed the secrets Drew (sometimes considered an Edwardian prototype of James Bond) had come to collect.8 Such fantasies found a ready market. Like Thomas Hardy and H. G. Wells, both vastly superior writers, Le Queux was paid the top rate of 12 guineas per thousand words and published far more than either.

At the opposite extreme of literary merit from Le Queux, Rudyard Kipling gave an equally optimistic assessment of British successes in the intelligence duel with Russia on India’s North-West Frontier. In Kim (probably the finest of all spy novels, though it transcends the world of espionage), unseen but ubiquitous agents of the British Raj play ‘the Great Game that never ceases day and night throughout India’. And they do so with a subtlety quite beyond the capacity of Tsarist Russia, ‘the dread Power of
the North’, and its French ally, whose emissaries are ‘smitten helpless’. So far as theWar Office were concerned, the myth of a far-flung intelligence network, whether promulgated by Kipling or by lesser literary talents, had the incidental advantage of avoiding public revelation of British intelligence weakness. ‘The only consolation’, they concluded in 1907, ‘is that every foreign government implicitly believes that we already have a thoroughly organised and efficient European Secret Service.’

All that Britain actually had were small and underfunded military and naval intelligence departments, both with little capacity to collect secret intelligence, and the Metropolitan Police Special Branch (MPSB), founded in 1883 to counter the threat to the capital from Fenian (Irish Republican) terrorism, which had moved on to small-scale investigation of other terrorist and subversive threats but had minimal expertise in counterespionage. The three agencies had little influence in Whitehall. Spenser Wilkinson, first Chichele Professor ofWar at Oxford University, compared the War Office’s use of their Intelligence Department (ID) during the Boer War (1899–1902) to a man who ‘kept a small brain for occasional use in his waistcoat pocket and ran his head by clockwork’. Although the 1903 Royal Commission on the War in South Africa concluded that the ID had been ‘undermanned for the work of preparation for a great war’,14 once the war was over the pressure for intelligence reform and more resources declined.

Within the Directorate of Military Operations at the War Office, however, two diminutive departments, MO2 and MO3, were established in 1903 with responsibility for, respectively, foreign intelligence and counterespionage. MO3 was the direct predecessor of MI5. Superintendent William Melville, who had been head of the Met’s Special Branch for the previous decade, was recruited to carry out secret investigations for both MO2 and MO3, later becoming chief detective of the Security Service during its first eight years. Since he qualified for a police pension of £240 and received an additional £400 from the War Office, the terms were
financially attractive. Melville’s appointment was not publicly announced. Officially, he simply retired from the Special Branch. The Times reported that Scotland Yard had lost the services of ‘the most celebrated detective of the day’.15 The award of the MVO (Member of the Royal Victorian Order) to Melville on his official retirement in 1903 also recognized his role in overseeing, with very limited resources, the security of Queen Victoria, King Edward VII and other members of the Royal Family both at home and during their continental travels at a time when European heads of state were more regularly threatened with assassination by revolutionary and anarchist groups than at any time before or since. Those assassinated on the continent included a Russian tsar, a French president, an empress of Austria-Hungary, a king of Italy, prime ministers of Spain and Russia, but no British royal or minister. Among foreign royals whose security Melville helped to protect during visits to Britain was Kaiser Wilhelm II, who presented him at various times with a gold watch and chain, a ring and a cigarette case.

The fact that early Security Service records date Melville’s employment from 1903, six years before MI5 was founded, is evidence that his work  for it after 1909 was seen at the time as a continuation and extension of his earlier War Office investigations. During his investigations for both the War Office and the Secret Service Bureau, Melville operated from an office at 25 Victoria Street, Westminster, using the alias ‘W. Morgan, General Agent’. Melville was well acquainted with Gustav Steinhauer, who became head in 1901 of the British section of the German Admiralty’s newly founded intelligence service, the Nachrichten-Abteilung, usually known as ‘N’. Former Kriminalkommissar of the Berlin police, Steinhauer, who grandly termed himself the ‘Kaiser’s spy’, had trained as a private detective at the Pinkerton Agency in Chicago and spoke English fluently with an American accent. He accompanied the Kaiser to England in 1901 as his personal bodyguard when Wilhelm II came to pay his last respects to his dying grandmother, Queen Victoria, and later to attend her funeral. A detective inspector in the MPSB described Steinhauer as ‘a handsome soldierly figure who had seen more courts than camps’. Steinhauer remembered Melville as ‘a silent, reserved man, never given to talking wildly’, who entertained him to dinner with cigars and ‘one or two bottles of wine’
at Simpson’s Grand Cigar Divan in the Strand. The presence of so much European royalty at Queen Victoria’s funeral inevitably led to fears of assassination attempts. Steinhauer later gave a melodramatic account of how he had accompanied Melville in a hunt for three homicidal Russian nihilists, who made their escape after allegedly killing a female informant of the Special Branch. Melville told the Kaiser he had been impressed by Steinhauer’s intelligence expertise. ‘Yes, Steinhauer is a splendid fellow!’ replied the Kaiser.

In the spring of 1904 Melville sent his assistant, Herbert Dale Long, on the first of several missions to Germany on behalf ofMO2 under commercial cover, probably to inquire into German naval construction. A fragmentary file on Melville’s early work for MO3 (renamed MO5 in 1907) suggests that his early priorities in Britain (particularly during the Russo-Japanese War of 1904–5) were to monitor the operations not of German intelligence but of the Okhrana, the Tsarist intelligence and security service. One of the documents in Melville’s file (received from a Colonel Dawson) dramatically describes the Okhrana chief, Pyotr Rachkovsky, as ‘Head of all the [Russian] secret service police in the whole world, & the most important man in Russia. Commander of the Legion of Honour in France, and has agents throughout the whole world.’ When stationed in the West, Rachkovsky lived in much greater opulence than his Soviet successors. Melville reported on 25 November 1904:


I know him personally, having frequently met him in London and he often called upon me in Scotland Yard when I introduced him to some of my superiors . . . When in London, Ratchkowsky always had some of his officers with him and invariably had a suite of rooms at the Savoy Hotel. I was told that he lived in a similar style in Paris, and know that he did so at Copenhagen.

Melville was probably aware that Rachkovsky and other Russian foreign intelligence officers were responsible for a series of explosions and agentprovocateur operations on the continent designed to discredit Russian revolutionary e´migre´s. He is unlikely to have known, however, that Rachkovsky was probably also responsible for the fabrication of the infamous anti-Semitic forgery The Protocols of the Elders of Zion, which purports to describe a Jewish plot for world domination. Between the wars, the Protocols, much praised by Hitler in Mein Kampf, emerged as one of the central texts in Nazi anti-Semitism, as well as later appearing on numerous early twenty-first-century Islamist websites.

From 1905 to 1907, Melville concentrated increasingly on German rather than Russian espionage. Reports of suspicious behaviour by German residents and visitors convinced him that German spies were reconnoitring invasion routes in England for the German army. In 1906 he believed that he had identified a group of spies in Epping:


I mentioned to the Superintendent of Police at Epping that the Germans might be spies; he laughed at the idea as being ridiculous, adding, ‘Spies! What could they spy here?’ Argument was useless. The fact remains that undoubtedly they were spies,
and their business, I should say, was to become thoroughly conversant with the routes from the sea coast to London, and thus to be able to guide a German army landed in this country.


There can be little doubt that the Epping Superintendent’s scepticism was fully justified. In other parts of the country, when making his inquiries about German spies, Melville also found the local police ‘absolutely useless’. He was not, however, to be deterred by the scepticism of the police from approaching the Home Office:


Owing to the almost continuous enquiries on the Eastern coast re suspected Germans, alleged staff rides by Germans, etc, from 1905 to 1907 I submitted reports outlining a scheme of surveillance on all suspected foreigners around the country. In them I suggested the utilisation of the Police, the Postal authorities and the Coast Guard Service.


Unsurprisingly, the Home Office failed to respond to Melville’s proposals. By the time Major (later Brigadier General Sir) James Edmonds became head of MO5 late in 1907, ‘its activities had been allowed to die down’. Save for Melville, Edmonds’s staff consisted only of another major whose main preoccupation was cultivating a parliamentary constituency which three years later elected him as its ConservativeMP. Apart from Melville’s reports, MO5 files when Edmonds took over ‘contained only papers relating to the South African [Boer] War and some scraps about France and Russia – nothing whatever about Germany’. Germany, however, was Edmonds’s main preoccupation. He seems to have been influenced by both Melville’s alarmist reports and the international tension generated by British–German naval rivalry. The Entente Cordiale of 1904, followed by the Triple Entente of 1907, had resolved Britain’s differences with France and Russia, both of which were to become wartime allies. The main threat to British security now came from the expanding German High Seas Fleet. The security of Victorian Britain had depended on Britannia’s ability to rule the waves with a navy which was by far the largest in the world. But, with the launching of the new British battleship Dreadnought in 1906, Anglo-German naval rivalry took a new and dangerous turn. By its size and firepower the Dreadnought threatened to make all other battleships obsolete. With ten 12-inch guns, each with a range of over 8 miles, it was
more than a match for any two of its predecessors. Overnight, the existing Grand Fleet of the Royal Navy, like every other navy in the world, seemed out of date. It was feared that the German High Seas Fleet, which also began building the dreadnought class of battleship, might soon catch up with the Grand Fleet and threaten the naval supremacy on which British security depended.

Media reviews

“Absolutely fascinating…a sweeping and highly readable account of a century of British intelligence. –Washington Times

"This unique publication is definitive and fascinating. Definitive because, after decades of ill-informed or partial accounts this book fully defines and describes its subject; no future writer can ignore it. Fascinating because the fluent clarity of Andrew’s narrative, his eye for colourful individual detail and the sheer interest of his subjects…this book is essential reading for anyone with even the slightest interest in intelligence in the modern period." – Alan Judd, Spectator

"A scholarly and hugely entertaining account…often enthralling…Professor Andrew is an entertaining and authoritative guide through the labyrinth of secret files, with an infectious fascination for the game of counter-espionage…an important part of Andrew’s achievement is to narrate with clarity an incredibly complex story in which bizarre and improbable reality often outruns the most rococo fabrications of the spy novelist…the reader is left in no doubt that the defence of the realm is being vigorously conducted by the secret state with all the extraordinary powers at its command." – Robert McCrum, Observer

"MI5 is the first major security or intelligence service in the world to give a historian free range of its records—nearly 400,000 paper files, some with many volumes, say Christopher Andrew with a touch of exhaustion…it has been well worth the effort. The Defence of the Realm throws new light on an important area of the running of the country, analysing the changing threats to national security over the 100 years and discussing the appropriateness or otherwise of the service’s response. But just as interestingly, the book gets inside the culture of this secret service, showing how attitudes have changed with those changing threats; how woman have worked their way from the fringes to the heart of the organisation and how a sense of humour has always been important. It will be enthusiastically scrutinised by historians, intelligence buffs and conspiracy theorists…[there are] anecdotes and operational details as gripping as any thriller." – Stella Rimington, Financial Times

"Andrew’s magisterial study is an authorised, but not official, history and is clearly written, brilliantly organised and extremely readable, not least because of something he shares with many of MI5’s staff over the past hundred years—a sense of humour…" – Oleg Gordievsky, The Times

"To mark the centenary, the service took the unprecedented step of inviting Christopher Andrew, Cambridge historian and doyen of intelligence chroniclers, to write an authorised history. The outcome is weighty, measured and compelling…with this book, the author has done a formidably good job for both the service and the public interest…I find it hard to disbelieve much he asserts or denies…his narrative offers a feast for students and politics." – Max Hastings, Sunday Times

"A ripping read and just the kind of work one would hope for from a well-qualified academic who has been given the run of MI5’s treasure-trove of files. It is scrupulously documented, covering both the glory days of war, when MI5’s deception operations outsmarted Hitler, and the later nightmare penetrations by the double agent Kim Philby and friends, in which the KGB thoroughly outsmarted the British…Dramatically, it is confirmed that there was indeed a secret Wilson MI5 file, under the pseudonym ‘Worthington’…There is more." – David Leigh, Guardian

"The book covers everything from the agency’s origins 100 years ago as a shoestring outfit hunting German spies to the duping of the Nazis during the second world war, the scandals and successes of the intrigues against the Soviet Union and, latterly, the counter-terrorist campaigns first against the IRA and then against Jihadist suicide bombers…It is a striking experiment in openness." – Economist

"Professor Andrew’s account is magisterial, authoritative, balanced, readable and, particularly in the first half, full of wry humour and with an eye for the absurd. Andrew shows himself to be in command of his vast amount of material and be able to fashion it into a coherent narrative which is as comfortable with MI5’s organisational structure and development and relations with Whitehall as with individual cases and personalities…MI5 has been well-served by this history and so have future historians, Service staff and the public in general." – Andrew Lownie, Sunday Telegraph
"The thousand pages of this book are brimming with some wonderful details…The Defence of the Realm is a valuable and important contribution to our understanding of the 20th century." – Susan Williams, Independent

"Andrew’s scholarship is meticulous and extensive. He deals at length with MI5’s role in the decolonization process in the Fifties and Sixties, something little known and still less appreciated. He also gives a superb account of the Service’s role in the struggle against IRA terrorism, casting new light on the Gibraltar shootings. There is a tantalizing section on MI5’s role in the battle against organized crime, especially drug trafficking; but that now seems to be taking second place to the Service’s counter-terrorism activities, in the wake of September 11 2001. MI5 could not have wanted a better historian than him. He has captured every important detail of the Service, but also its ethos and its place in our country as an institution. This book is unlikely to be surpassed for another 100 years, and until then will be the necessary starting point for anyone who wants to know what, exactly, MI5 is." – Simon Heffer, Telegraph