Ten lessons from the history of espionage

From a royal slipper thrown at Sir Francis Walsingham to Saddam Hussein's non-existent Weapons of Mass Destruction, via Napoleon's love of OSINT and Stalin's sweary outbursts

Belated happy new year readers!

I was going to write about why I think the conventional wisdom that 2021 will be a year of recovery is wrong, a triumph of hope over experience. But I figured that would be a pretty depressing way to kick things off.

So, instead, I decided to share ten lessons that emerged from, perhaps, the best book I read in 2020. Christopher Andrew’s The Secret World is a masterful history of the world of espionage. While he covers the usual suspects such as MI5, CIA, KGB and Mossad, he delves much deeper into the annals of intelligence, starting with the ancient Greeks and Romans.

I was lucky enough to take Prof Andrew’s course on the secret world while I was a student at the University of Cambridge. His book is packed full of pertinent historical tales that illuminate the world of espionage and will doubtless make you think twice before jumping to conclusions next time you’re writing up that intelligence estimate, research report or news analysis.

1. Spies should speak truth to power

As you can see from the above portrait, Sir Francis Walsingham, Elizabeth I’s omniscient, workaholic spy chief, was not the sort of person you’d want to mess with in a dark London alley. Even in the days of (near) absolute monarchy, when falling out of favour could mean literally losing your head, Walsingham believed it was his duty to tell it like it is. So much so that, after one disagreement in 1586, Queen Elizabeth took off a slipper and tossed it at him in rage. I’m not condoning violence in the workplace, but if your boss is so annoyed by your fearless advice that they reach for their footwear, then you’re probably doing something right.

2. Leaders should listen to their spies

Top intel is only useful if leaders are willing to listen to it. And, all too often across historical periods and political systems, rulers do not like to hear insights that challenge their existing biases, wants and plans. Napoleon Bonaparte, the French emperor (not the allegedly corrupt Indonesian police general) was unashamedly disdainful of his spies and scouts, particularly when they went against his preconceived notions. “I do not allow myself to be governed by advice,” he once wrote, a sentiment sadly shared by many politicians in 21st Century democracies.

If Napoleon disregarded dissenting views with a certain French élan, Ioseb Besarionis dze Jughashvili wasn’t one to beat around the bush. Jospeh Stalin, as he’s better known, stubbornly ignored repeated warnings from his spies, including the irrepressible Richard Sorge, that Hitler was going to invade the Soviet Union in 1941. Stalin’s response to one NKVD report along these lines was this pithy minute:

You can tell your ‘source’ in German airforce headquarters to go fuck himself. He’s not a ‘source’, he’s a disinformer. J Stalin.

Unfortunately for the patriotic spies risking it all to support the revolution, they had much more to worry about than Stalin’s Malcolm Tucker-esque rants. The sycophantic NKVD chief Lavrenti Beria turned on those who were sending the unwelcome reports of German intentions. Not one to throw slippers, he vowed that they would be “ground into concentration camp dust”. Beria himself somehow survived Stalin’s murderous reign but, within months of the great leader’s death, he was predictably arrested, tried and executed.

3. Just because it’s SECRET, doesn’t mean it’s true

As the buzz around the release of the United States Strategic Framework for the Indo-Pacific shows, anodyne government documents seem a lot more interesting when they have the word SECRET stamped at the top. The mysterious redactions only add to the sense of intrigue and heighten analysts’ FOMO.

Outside the intelligence world, we tend to think that the spies have access to a higher level of top secret information. But the increasing focus on open-source intelligence is testament to the fact that that’s not always the case.

Napoleon didn’t like listening to his spies, as I said above, but that was partly because he felt that open sources were more accurate. Andrew writes:

Napoleon seems to have read British newspaper reports with greater attention than most secret intelligence, believing that they frequently gave more reliable accounts of operations than his own generals.

It’s not just Napoleon. When I was an FT correspondent in China, Indonesia and Vietnam, more than a few foreign diplomats (and people doing other sorts of work who were formally attached to embassies) told me that they could often learn as much from reading the local and international press as their own classified cables.

4. Revolutions often produce revolutionaries, not the other way round

Having studied the revolutionary independence movements of Southeast Asia, and observed a brewing uprising in Hong Kong, I’m fascinated by what triggers people to take to the streets en masse in the face of overwhelming force. There’s always a complex interplay of deep structural forces, individual actions and sheer luck (which historians prefer to call contingency).

Andrew highlights how none of the nascent intelligence services in nineteenth century Europe predicted the 1848 revolutions, despite intensive efforts by secret police to monitor groups thought to be revolutionary. The reason, Andrews argues, is that as with the original French revolution of 1789, “revolution produced revolutionaries, rather than the other way around”.

Cass Sunstein, who co-wrote behavourial economics bestseller Nudge, produced a good short paper a couple of years ago on why revolutions are extremely difficult to predict. In essence, he argued, revolutions often succeed because of a “Cascade Effect” when people feel emboldened to act because others are doing the same thing and/or the authorities respond in a way that encourages more, not less, protesters.

If you’re the first protester facing the riot police, there’s a pretty high chance that you’re going to get pepper-sprayed/beaten up/arrested or all three. Those odds reduce dramatically as more people join the movement. So if you’re Person 100, you better have a facemask and lawyers’ number handy and if you’re Person 1,000 you might be feeling a bit nervous. But Person 1,000,000 will be strolling along happily updating their social media as they go.

Sometimes, the authorities can nip such movements in the bud by going hard and early. Other times, as we saw in Hong Kong, it’s the heavy-handed response that drives the movement forward.

5. Don’t blame technology for your own shortcomings…

Mark Zuckerberg and Jack Dorsey aren’t exactly plucky underdogs. But you don’t need to be a Facebook shareholder to feel that they’re being scapegoated for almost every problem under the sun right now. That should not be a surprise. Humans have always had a tendency to blame new technology for their own shortcomings.

The rapid spread of the telegraph in the mid-nineteenth century transformed communications, and espionage. At the start of the Crimean War in 1853, it took days for information from the front to reach London. Just two years later, thanks to new cables laid by the British, it took mere hours. William Howard Russell, the pioneering war correspondent for The Times, seized on the possibilities of the telegraph to send timely and unvarnished stories about the struggles of the campaign back to London.

The incompetent generals, predictably, tried to shoot the messenger, accusing Russell of treason and arguing that rapid modern communications meant that British armies could never again face down an enemy in the field.

Andrew quotes Lord Raglan, the British commander-in-chief, writing to his boss, the Secretary of War:

I ask you to consider whether the paid agent of the Emperor of Russia could better serve his Master than does the correspondent of the paper that has the largest circulation in Europe… I am very doubtful, now that Communications are so rapid, whether a British Army can long be maintained in the presence of a powerful Enemy, that Enemy having at his command thro’ the English press, and from London to his Head Quarters by telegraph, every detail that can be required of the numbers, condition, and equipment of his opponent’s force.

6. … but new tech can mean new risks

While the British generals were wrong to blame The Times for their own errors, the advent of the telegraph did present significant new intelligence risks, just as the rapid adoption of Zoom, LinkedIn and other new channels for communications has presented many opportunities for spies.

Andrew writes that “by greatly simplifying the interception of diplomatic correspondence, the use of the telegraph marked a turning point in intelligence history”. No longer did spies have to try and intercept or bribe couriers to get access to diplomatic letters. They could simply tap the traffic that was channelled through the telegraphic offices in their capital cities. From this point onwards, the struggle shifted from one of collection to one of decryption.

Interestingly, given its leading role in all things intelligence from the second half of the twentieth century onwards, the US was a vulnerable laggard when it came to information security. After the transatlantic cable between the US and Europe was opened in 1866, the US emissary in Paris wrote to the Secretary of State, William Seward, urging him to introduce new cyphers because “nothing goes over a French telegraph wire, that is not transmitted to the Ministry of the Interior”.

But Seward ignored the advice, insisting that the State Department code was the “the most inscrutable ever invented”. He compounded the problem when he sent his first ciphered transatlantic cable by releasing a copy of the full text to the press, allowing French code-breakers to compare the two and crack the code.

7. Your enemy’s enemy is not always your friend

Despite the old saying, trying to assist your enemy’s enemy can have dangerous unforeseen repercussions, particularly when said enemy’s enemy is a whip-smart subversive. The most successful German espionage operation of the First World War, according to Andrew, was the mission to transport Vladimir Lenin back to Russia from Switzerland, on a sealed train through Germany.

The Kaiser and the German high command thought that his “revolutionary defeatism” would accelerate Russian withdrawal from the war (about which they were right) and Germany’s ultimate victory over its adversaries (about which they were wrong). But, in helping to instigate the Russian revolution, the German leadership stored up a world of trouble for their nation, and the world, in the decades to come.

8. Sometimes spies cannot find a threat because it doesn’t exist

Alleged threats, whether from domestic subversives, global terrorist groups or foreign powers, are frequently exaggerated, especially before and during wartime and other crises. That’s a bad enough problem on its own. But it’s made worse by the fact that the inability to find evidence of such threats can make them seem even more sinister.

During the First World War, MI5 successfully tracked down all the potentially significant German agents who had been infiltrated into Britain. But it failed to find the conduits of German finance for British pacifists and other subversives - a major source of angst for the conspiracy-obsessed politicians of the day. They called in the police Special Branch to investigate further but they had no luck either, principally because there was no German funding for the British critics of the war.

Basil Thomson, the head of Special Branch, despaired in his diary:

The War Cabinet… are not disposed to take soothing syrup in these matters. Being persuaded that German money is supporting [pacifist and revolutionary] societies they want to be assured that the police are doing something.

9. Intel on what your opponent doesn’t have is as vital as intel on what they do

A natural follow-on from lesson 8, this pearl of wisdom comes courtesy of President Dwight D. Eisenhower, who said that information on “what the Soviets did not have” was as important as information on what they did. Andrews says that “ignorance of a feared opponent invariably leads to an overestimation of the opponent’s strength”. Thus, a lack of good intelligence in the early 1950s led American policy makers to believe (erroneously) that the Soviets were out-producing them in nuclear bombers and missiles.

The procurement of accurate imagery intelligence, thanks to the development of the U-2 high-altitude spy plane, helped to correct these misperceptions and give Washington greater confidence in understanding and countering the Soviet threat. The problem, of course, is that it’s hard for spies to know if they can’t find something because they haven’t been looking hard enough - or simply because it’s not there. Pressure to find the threat that politicians fear makes this task even harder, especially in authoritarian systems but in democracies too.

10. Major intel failures are often caused by deficiencies in analysis not collection

One of my formative experiences as a young student journalist was covering the Hutton inquiry into the death of former UN weapons inspector David Kelly, who had been named as the source of a BBC report that the British government had “sexed up” evidence that Saddam Hussein possessed weapons of mass destruction.

The disastrous invasion of Iraq and the flawed intelligence upon which it was based left deep scars in many Western societies, helping to create the atmosphere of widespread distrust towards the ruling establishment in which we live today. There were many errors made on the road to Baghdad but Andrew argues that the biggest mistake was a failure to challenge the assumption that Saddam did have WMD.

Andrew cites storied CIA analyst Dick Heuer’s seminal CIA paper on the Psychology of Intelligence Analysis (available as a PDF here), in which he writes:

Major intelligence failures are usually caused by failures of analysis, not failures of collection. Relevant information is discounted, misinterpreted, ignored, rejected, or overlooked because it fails to fit a prevailing mental model or mind-set.65 The “signals” are lost in the “noise.”.

Western officials presumed that if Saddam did not have WMD he would have admitted as much or allowed UN inspectors back into Iraq so as to avoid a US-led invasion. But, as Saddam later revealed under interrogation, he didn’t do as they presumed because he was more scared about revealing Iraq’s vulnerabilities to Iran than he was about Western threats.

Heuer calls this “mirror-imaging”, mistakenly assuming that your adversary thinks like you and you can simply ask what would would I do if I was in his shoes.

“The US perspective on what is in another country’s national interest is usually irrelevant in intelligence analysis,” he says. That’s advice that ought to be heeded as the Biden administration looks to reset its relationships in Asia and push back more effectively against China.

Take the case of Myanmar. Many Western analysts and diplomats are perplexed by the recent coup, wondering why would the military take direct control when they already had so much power in the pre-existing system and could let Aung San Suu Kyi struggle with the nitty gritty of day-to-day government? But the Tatmadaw, and chief general Min Aung Hlaing, had been busy telegraphing its intentions to carry out a coup, as well as its rationale.

Instead of asking what what I do if I was Min Aung Hlaing, it’s better to ask what have been the drivers of the Myanmar military’s behaviour over the last 75 years since independence?

I hope that was useful. If you want to know more, read Prof Andrew’s great book. Normal service will be resumed next time.

Stay safe,

Ben