The Secret War

nonmorse review of

The Secret War: Spies, Codes and Guerrillas 1939-1945

by Max Hastings

Max Hastings takes on a massive subject in his latest book, the role played by intelligence and secret operations in the second world war. He covers the activities and operations of all the major belligerents, on both sides of the conflict, gathering in one sizeable volume an overview of a war within a war.

There have been scores of books written about the British codebreakers of Bletchley Park, but the spies, codebreakers and guerrillas of the wider global secret war have been covered in less copious detail. Hastings succeeds in bringing some form of order to this unwieldy topic and he details the exploits of the variously brave, craven, frightened, shameless, devious, dedicated, comic and, above all, clever people who helped, or failed to help, provide the warring nations on all sides with ‘intelligence’.

Unknown

The term ‘on all sides’ is an important one as, while the major thrust of the war was the binary opposition between the Allies and the Axis, there were in fact all sorts of sub-plots and sub-conflicts, especially in Eastern Europe and the Balkans. Hastings deftly exposes how many spies and guerrillas, paid or harrassed into danger by one side, actually worked for another side (or both, or more, or none except their own bank balance).

One can’t help raising a quiet cheer on reading about Dr. Wilhlem Bitter, a psychiatrist at a Berlin hospital who agreed, late in the war, to find a channel for negotiations between the Western Allies and Germany to stave off total Soviet victory. As Hastings recounts, ‘Having got a safe distance from Germany, Bitter sent just one hysterical message home, saying that the only answer was to overthrow Hitler, then vanished forever.’

Inevitably, for all the side conflicts, the core of the book necessarily focuses on this titanic struggle between Nazi Germany and its opponents.

Hastings takes an unemotional look at the secret war. This is no thrill-seeking account of daring do. ‘The only question that matters,’ says Hastings right at the outset, ‘is how far secret knowledge changed outcomes.’

“The only question that matters is how far secret knowledge changed outcomes”

He illustrates this theme by pointing out that the ‘scale of Soviet espionage dwarfed that of every other belligerent and yielded a rich technological harvest from Britain and the United States, but Stalin’s paranoia crippled exploitation of his crop of other people’s political and military secrets.’ Stalin’s failure to accept ample intelligence of the German attack in June 1941 is one of the greatest intelligence failures in history, though, unlike Hitler, Stalin did learn to be slightly less impervious to things he didn’t want to hear, though his subordinates, fearful for their lives, were careful about what they reported to him.

Similarly, Hastings judges that Germany and Japan lost the secret war not so much because of failures by their intelligence agencies, but because the military and political leaderships discounted the need to know about those they had chosen to attack. ‘The German and Japanese leaderships made their decisions shrouded in bewilderment and ignorance about their enemies, partly because of an institutionalised resistance to the objective examination of evidence, emphasised by Hitler’s refusal to explore the economic potential of the Soviet Union and the United States before he declared war on them.’

“The German and Japanese leaderships made their decisions shrouded in bewilderment and ignorance about their enemies”

Hastings also lays bare the failures of Allied intelligence, including long periods when British or American wireless communications were intercepted and decrypted due to security weaknesses. He suggests, however, that these Allied lapses may, in the long run, have been unwittingly advantageous. The Nazi military machine may have been encouraged to believe that their Enigma and Geheimschreiber cipher machines were invulnerable to Allied codebreakers because there was no hint at all of any Allied breaks of German ciphers in any of the Allied messages, especially British naval communications, that the German military could read. This led them to conclude that an enemy who could crack German messages would have the sense to plug the holes in their own systems.

In reality, the work of British and American codebreakers was becoming critical to Western Allied military operations against Germany although Ultra did not start to fulfill this role until late 1942, by which time it had become obvious that the Germany would, sooner or later, be defeated. Access to the information made available by Ultra did not guarantee military success, as shown by the muddled operations that followed the great success of D-Day and the ‘break out’ from Normandy in 1944 and the failure to predict the Battle of the Bulge.

Hastings concludes that ‘The indispensable element in making all intelligence useful, in peace or war, is that it should pass into the hands of a wise and effective leader; if such a person is absent, whether general, admiral or statesman, then even the most privileged information is worthless’.

The global intelligence war during the second world war included many pointless operations that saw agents sent to certain death. Hastings estimates that perhaps as little as ‘one-thousandth of one percent of material garnered from secret sources by all the belligerents in World War II contributed to changing battlefield outcomes’.

“perhaps as little as ‘one-thousandth of one percent of material garnered from secret sources by all the belligerents in World War II contributed to changing battlefield outcomes”

For all the efforts of spies and agents, Hastings is clear that it was eavesdropping on enemy wireless communications that was the key aspect of the intelligence war. The achievements of British and American codebreakers, ‘were very great’ he writes. ‘They elevated intelligence, hitherto a little respected branch of staff work, to an unprecedented importance in operational planning.’

This aspect of the second world war is highly relevant in the modern world. As Hastings observes, ‘the importance to national security of intelligence eavesdropping, codebreaking and counter-insurgency has never been greater. Cyber warfare is a logical evolution of the process that began in Room 40 during World War I and expanded vastly’ among British, German and American codebreakers during World War II.

“the most deadly threat of all – a few hundred tweedy bespectacled young English academics labouring in drab suburban Bedfordshire”

On Bletchley Park he comes to the view that ‘Many things about the 1939-1945 era remain disputable, but few informed people would question the proposition that Bletchley was one of the most remarkable institutions the world has ever known, and one of the greatest achievements in British history, towering over any narrative of the nation’s part in the conflict … While the Third Reich executed wholesale spies, traitors and saboteurs who threatened its security, its functionaries remained insistently oblivious of the most deadly threat of all – a few hundred tweedy bespectacled young English academics labouring in drab suburban Bedfordshire.’

Advertisements

Gordon Welchman and Tom Flowers at Bletchley Park

nonmorse review of BBC2 programme, ‘Bletchley Park: Code-breaking’s Forgotten Genius‘ on Gordon Welchman and Joel Greenberg’s biography, ‘Gordon Welchman: Architect of Ultra Intelligence

Screen Shot 2015-09-09 at 09.02.03

On 7th September 2015 BBC Television broadcast a documentary about Gordon Welchman, one of the key figures at the British codebreaking operation Bletchley Park during the second world war. The programme set out to boost Welchman’s reputation which it implied had suffered from lack of attention in contrast with the much better known figure of Alan Turing.

The programme claimed that Welchman was  a ‘forgotten genius’ who was in reality on a par with Turing, in his contribution to both Bletchley Park’s wartime work and to the post-war information and communications revolution. The commentary declared that Welchman’s improvements to Turing’s codebreaking machine, known as the Bombe, deployed to decrypt Enigma messages, was an example of Welchman’s ‘fantastic mathematical intelligence, easily a match for that of Alan Turing.’

 Welchman had a ‘fantastic mathematical intelligence, easily a match for that of Alan Turing.’

A recent biography by Joel Greenberg, Gordon Welchman: Bletchley Park’s Architect of Ultra Intelligence, makes similar if less dramatic claims for Welchman. “Along with Alan Turing, he had in effect developed a radically new production-oriented approach to machine cryptanalysis. [Bletchley Park] then turned to Welchman to put his plan in place and he too on leadership of the group which would ultimately decrypt over one million German Air Force and Army signals. In 1943 he was given responsibility for all ‘machine’ developments at [Bletchley Park] and while he was not directly involved in its creation, the world’s first electronic computer, Colossus, was designed and built on his ‘watch’… Welchman’s contribution is seen by many historians and former [Bletchley Park] colleagues as being fundamental to its ultimate success.’

P1030276 copy

Welchman was undoubtedly a brilliant mathematician, consummate information processing engineer, talented administrator and celebrated figure in the development of systematic analysis of electronic surveillance intelligence. His work at Bletchley Park on ‘traffic analysis’, inventing a way of speeding up Turing’s bombe codebreaking machines, and from 1943 as head of mechanisation, as well as his post-war work, especially in the US on military communications systems, certainly places him among the founders of the modern electronic surveillance, communications and information processing world.

Welchman’s worked with Turing during the period when Turing conceived a a machine, the Bombe, that could help work out the initial wheel settings on the Enigma. Welchman also developed a technique, known as the ‘diagonal board’, which speeded up the work of the bombes and made them practical devices for timely decryption of intercepted German Enigma messages.

Traffic analysis, first developed during the first world war by the French and British codebreaking units, uses any information about intercepted messages excluding the message content. Today that sort of information is known as ‘metadata’ and is subject of controversy over the scope of security and police access to such data in democratic societies. Welchman certainly organised and extended traffici analysis into a significant tool alongside systematic decryption.

However, all this does not necessarily mean that Welchman was a genius on a par with Turing. He was undoubtedly one of many brilliant people, both at Bletchley and in the US who invented and developed electronic computers and telecommunications. To say more than that is difficult to justify.

R0039147a copy

Bletchley Park today

Welchman was a complex human being, who damaged his own life and career by breaking the lifelong loyalty to the code of silence imposed on Bletchley Park’s staff in his desire for recognition of his contribution. His 1982 book, The Hut Six Story: Breaking the Enigma Codes, led to his downfall within the eavesdropping and military intelligence community in Britain and the US. His security clearance was withdrawn and he could no longer work on classified projects in the US or Britain.

Both the BBC2 documentary and Greenberg’s book liken Welchman’s treatment with that meted out to Alan Turing who was prosecuted and chemically castrated by the British state in the early 1950s, probably causing Turing to commit suicide. However, Welchman’s downfall (comfortable retirement on the eastern seaboard of the US) was the outcome of his choice to speak out about things his former colleagues expected would be kept quiet. Turing did not, as far as we know, ever give away details of his wartime work. His downfall (prosecution and suicide) was the outcome of the state outlawing his essential nature. The two are not really comparable. Welchman’s motivation for publishing his book was primarily about gaining recognition for his wartime contribution and about ensuring an income in old age, though he added an ending to his book concentrating on modern lessons of traffic analysis and related military communications issues, warning the western world that it was ignoring lessons learnt during the second world war.

Tony Comer, GCHQ historian and former analyst says that, ‘What ‘Welchman and his advisors] didn’t stop and think was the way in which the cryptanalyst approached the breaking of Enigma was as sensitive in the 1980s, and is as sensitive today, as it was at the time’.

Unfortunately both the BBC2 documentary and Greenberg in his biography have decided to paint Welchman, not just as a genius equal to Turing, but also as a superhero. As well as being a fine administrator, perspicacious communications analyst and innovative cryptographer, they convey the story of a flawless personality hounded to his death by a cold, callous bureaucracy.

At one point Greenberg writes that, ‘Welchman would not let a particular episode with a colleague distort his overall view of him.’ Elsewhere he observes that, ‘Throughout his professional career, before, during and after the war, Welchman demonstrated the admirable quality of being able to separate personal issues from professional ones.’

Greenberg writes that, ‘Welchman would not let a particular episode with a colleague distort his overall view of him.’

To test the veracity of these statements we need to take a look at one particular episode that we do know about from Welchman’s own reports written at the time and now open to public view at the National Archives at Kew.

In 1943 Welchman was appointed Assistant Director (Mechanisation) at Bletchley Park. This put him in charge of all the projects using machines at the Park. As noted above, Greenberg writes in his biography of Welchman, that ‘while he was not directly involved in its creation, the world’s first electronic computer, Colossus, was designed and built on his ‘watch’.’ Colossus was invented, like the bombe, to work out the initial settings of a cipher machine, but rather than Enigma, the machine was known as the Geheimschreiber or ‘secret writer’ (or to be more precise a version known as the Lorenz SZ42). This was a much more complex machine than Enigma and demanded more complex machinery than the Bombe to work out the initial wheel settings. The first machine intended to do this was an electromechanical monster known as the Heath Robinson. It proved that the codebreakers, working under Max Newman, could use machinery to decrypt Geheimschreiber messages, but the codebreaking machine was plagued by mechanical faults and ran too slowly. Colossus, invented by Tom Flowers of the Post Office, was a massive electronic replacement.

Flowers began working on the machine at the same time that Welchman was head of machine developments at Bletchley Park. Yet Welchman was strangely diffident about Colossus and its inventors. Writing in his book, The Hut Six Story, Welchman said, ‘There were many developments of which I knew nothing until I saw [Professor Brian] Randell’s [1975] paper on the Colossus … Randell tells us that, some time after the start of the [Heath] Robinson project, T H Flowers – who was a high-ranking telephone engineer in the British Post Office research organization – was brought into the picture, probably at the suggestion of Turing who had worked with him on an earlier project and had acted as an advisor to the Testery [another part of the team working on the Geheimschreiber enciphered messages]. Flowers threw all his electronic expertise into solving Newman’s problem with the machine that became known as Colossus.’

But I was too occupied with other work to find out much about what had happened before I became A.D. (Mech). There were many developments of which I knew nothing until I saw Randell’s paper on the Colossus, mentioned above. In particular I was not aware of the major contribution that had been made by Max Newman … Toward the end of the war I worked a little with Max, but I was not aware of his early work at Bletchley. According to Randell, Newman volunteered his services, joined a research section under Major G W (Gerry) Morgan in September 1942, and was assigned to the team known as the “Testery”. This was a sub-section under Major (later Colonel) Tester that was struggling by manual means with the [Geheimschreiber] problem.’

‘I was never in close touch with Tester’s work, and have forgotten the details of the problem, but Randell claims that it was Newman who had the idea for tackling the problem by mechanical means. He went to Travis, as I had done in 1939, to obtain permission to set up a new section that came to be known as the “Newmanry”. He collected a team of mathematicians, and got Wynn-Williams to build the first machine, an electro-mechanical device known as [Heath] Robinson.’

bletchley park 01 copy

Bletchley Park mansion

This half-hearted account of Colossus, and Welchman’s connection with it, may well in part be due to the problems Welchman encountered from the authorities when he tried to write about Bletchley Park. The details of the Colossus story were then still considered too sensitive to be made public. Welchman cleverly tells us that he didn’t know about Max Newman’s early contribution or the early work of the Testery, but fails to give any other information – except what could be gleaned from Professor Randell’s paper. This neatly met the restrictions of the authorities. But as Assistant Director (Mechanisation) Welchman was more intimately involved with Flowers and Colossus than he lets on.

In fact there was much bad blood between Welchman and Tom Flowers (and his boss Gordon Radley at the Post Office). The Post Office men were at the pinnacle of the technical hierarchy of the Post Office telecommunications system and arrogantly criticised the work of those who had worked with Welchman on the bombes and other projects. It is little wonder then, that these men did not get on at all with Welchman.  He even tried to have Flowers reprimanded for his ‘reckless use’ of scarce electronic valves. If this reprimand had been issued it would have meant an end to Flowers’s work at Bletchley Park – so that Colossus would never have been invented, built. Eleven Colossus machines were in operation by the end of the war.

 Welchman even tried to have Flowers reprimanded for his ‘reckless use’ of scarce electronic valves.

The best documented row between Flowers and Welchman involves an intricate story involving different solutions studied at Bletchley Park for dealing with the introduction, at a critical stage in the Battle of the Atlantic, of a fourth wheel in German navy Enigma machines. Bletchley Park called on the help of the company, BTM, which built the Bombes, of the radar development centre TRE, and Flowers’s department at the Post Office. This mix of groups did not work well and some intense personal dislikes got dragged into different technological approaches. More detail of the technical aspects behind the row can be found in Colossus: Bletchley Park’s Greatest Secret (pages 250-253) and in part only in Greenberg’s biography of Welchman (pages 77-79).

R0039148a copy

Welchman wrote in a memo to his boss at Bletchley Park that he had visited a Mr. McLarren of the Admiralty supply division to discover the position with availability of electronic valves.

‘I also wanted to find out the truth about the valve supply situation, because Mr. Flowers does not seem to mind how many valves he uses, while BTM are being urged by the Admiralty to use as few valves as possible. Mr. McLaren said at once, apart from all other considerations, we should certainly use [electro-mechanical] relays instead of valves if we possibly can because of the serious shortage of valves. He said that Dr. Radley must know this perfectly well. He took a very serious view of the reckless use of valves by Mr. Flowers, and said that the Post Office could be very seriously reprimanded for this offence, if it could be established … He [Radley] might have informed us [Bletchley Park] of the acute shortage of valve instead of encouraging Flowers to squander them … The influence of Dr. Radley and Mr. Flowers must be completely removed.’ [File reference: National Archives, HW62/5, 8th June 1943]

Welchman wrote to the director of Bletchley Park: ‘The influence of Dr. Radley and Mr. Flowers must be completely removed’

Welchman’s assessment of Flowers’s technical and organizational brilliance was, in hindsight, extraordinarily wrong. Welchman wrote: ‘…Mr. Flowers’ idea of co-operation is to run things himself … It may be that Mr. Flowers honestly thinks that he is better able than Mr. Keen [of BTM], Dr. Wynn Williams [of TRE] and myself to direct the policy of Bombe production, but, if so, I am quite sure he is wrong. He is probably very good at his ordinary work, and also at designing apparatus for a definite problem that he can understand, but I have found him very slow at grasping the complications of our work and is minds seems altogether inflexible.’ Incredibly, Welchman went on to say that Flowers was ‘not very good with electronics.’ Clearly, there is an element of condescension here towards Flowers, the engineer of working class origins, a man who wasn’t even a university graduate, let alone a Cambridge academic.

Welchman said of Flowers: ‘He is probably very good at his ordinary work, and also at designing apparatus for a definite problem that he can understand, but I have found him very slow at grasping the complications of our work and is minds seems altogether inflexible.’

Joel Greenberg cites part of this document in his biography of Welchman, but surprisingly he makes no reference to Welchman’s grossly erroneous judgment on Flowers, whose work on Colossus, both in conceiving of how electronics could be brought to bear on the complex Geheimschreiber problem and in getting the machines constructed in an incredibly short time and getting them to work, was one of the great technological advances of the information age.

If Welchman had had his way in removing Flowers’s influence, the result would have been that Colossus would never have happened and the intelligence it revealed, especially in helping the success of the D-Day landings in north-west France would have been lacking.

The D-Day and post-invasion intelligence produced with the aid of Colossus ‘has the highest strategic value of all sources of Signal Intelligence’ [Intelligence report: HW14/110, 25th August 1944]

In promoting Welchman’s claim to be ‘architect of Ultra intelligence’ or classifying him as a genius that shortened the war by two years, we do history a disservice by redacting his great blunder.

Max Hastings, in his overview of the role of intelligence in the second world war, The Secret War: Spies, Codes and Guerrillas 1939-1945, concluded that ‘while he [Flowers] forged a close working relationship with Turing, who often visited Dollis Hill, the formidable and influential Gordon Welchman took against the engineer, who was certainly no ‘gentleman’ in the parlance of those days. Welchman treated him with disdain, as a mere artisan with ideas above his station.’

‘Welchman treated [Flowers] with disdain, as a mere artisan with ideas above his station’, Max Hastings

It is likely that it was Alan Turing, who had long discussions with Flowers and could see his ability to understand Bletchley Park’s problems and to apply technological innovation to solving them, who spoke up for Flowers and prevented Bletchley Park management from removing his influence as demanded by Welchman.

Colossus was in fact only one of several electronic machines developed for working on breaking the Geheimschreiber ciphers. Other machines had names such as Dragon and Proteus. They played specific roles within the Newmanry and were not as advanced or complex as Colossus, but Flowers’s invention showed that electronics could be harnessed to the codebreaking effort of Bletchley Park.

Greenberg writes in his biography of Welchman that, ‘Throughout his professional career, before, during and after the war, Welchman demonstrated the admirable quality of being able to separate personal issues from professional ones.’ Yet this row with Flowers shows that this was not the case. Clearly Welchman allowed his disagreement with Flowers to cloud his judgment. Flowers’s achievement is on a historic scale that dwarfs that of Welchman.

 Greenberg wrongly claims that ‘Throughout his professional career, before, during and after the war, Welchman demonstrated the admirable quality of being able to separate personal issues from professional ones.’

Let Welchman’s achievements stand on their own, part of the array of brilliance displayed by many at Bletchley Park. But let’s not promote him to mythic, flawless status. Welchman was a brilliant mathematician, information scientist and administrator, but on one of the biggest questions facing Bletchley Park he was a fool who allowed his personal dislike of Flowers and Radley to get in the way. Luckily, the true genius at Bletchley Park, Alan Turing, did appreciate Flowers’s understanding of the cryptographers’ needs and of the potential for electronics to address those needs and was able to undermine Welchman’s foolishness.