Nigel West - In his evidence to Lord Justice Scott, who investigated the Matrix Churchill arms-to-Iraq fiasco in 1992-96, the then prime minister, John Major, revealed that 40,000 separate items of intelligence are delivered annually to the Foreign Office by the Secret Intelligence Service (MI6) and GCHQ, with the latter agency supplying about two-thirds of finished intelligence to Whitehall’s consumers.

Major’s disclosure surprised many, just as the ‘D-Notice affair’ of 1967 apparently shocked the veteran Fleet Street reporter Chapman Pincher by confirming that an unidentified government agency routinely retained copies of all private and commercial cables sent in and out of the country. A huge political row ensued, but actually, the practice had been authorised by the 1919 Official Secrets Act - although nobody had complained, or maybe noticed, at the time. Furthermore, GCHQ was by then Britain’s largest intelligence agency, with staff greater than MI5 and MI6 combined, yet its existence went unreported.

Now Angela Merkel, the German Chancellor, is complaining to the US about the alleged tapping of her mobile phone. Merkel feels that she must protest for the sake of domestic public opinion - just as Francois Hollande, the French president, did about the National Security Agency’s acquisition of French telephone records - but she cannot pretend to be surprised that the capability to monitor her phone calls exists, because she will have authorised similar operations herself.

The unpalatable truth is that eavesdropping is as old as any occupation. The term sub rosa comes from the flower placed over the door of the Roman senate to indicate to outsiders that the members were in secret session. Espionage became a growth industry in the Elizabethan era when Sir Francis Walsingham was credited with obtaining the evidence, by clandestine means, that condemned Mary Queen of Scots to her execution.

During the First World War, the interception and decryption of the notorious Zimmermann telegram, which exposed the Kaiser’s scheme to persuade Mexico to join the Central Powers and attack the US, led Woodrow Wilson to declare war on Germany. While T.E Lawrence became a hero for his adoption of unconventional warfare in Arabia, only a small number of cognoscenti understood that his true value had been in cutting Turkish telegraph lines, which ran alongside the railways in the Hijaz, thereby forcing the Ottoman Empire to rely on radio communications to maintain contact with far-flung garrisons.

Happily, the same strategy was adopted in 1940, when Special Operations Executive saboteurs attacked telephone exchanges in occupied Europe to ensure that the Axis would come to distrust landlines and to depend on encrypted wireless links. It was not until 1974 that the MI6 officer responsible for maintaining security at Bletchley Park, Fred Winterbotham, revealed that Anglo-American cryptanalysts had solved many of the enemy’s machine-generated ciphers, including those of the supposedly unbreakable three-rotor, and later the four-rotor Enigma machine. It was later still that the public learnt that the more complex 10-rotor Geheimeschreiber (secret writer) had also succumbed to the world’s first programmable analogue computer, the Colossus, which provided a glimpse into the German High Command’s top secret automated high-speed teletype network.

Whitehall imposed strict secrecy on everything cryptographic. Even after historians gained access to what had been accomplished at Bletchley, Eastcote and in America at Arlington Hall and the US Naval Annex on Nebraska Avenue in Washington DC, they had little idea that even the most complex of systems, such as the Floradora one time pad ciphers used by the Reich foreign ministry, had also been defeated. Experience had shown that politicians generally could not be trusted to protect the most prized sources.

In May 1927, Stanley Baldwin, while justifying the suspension of diplomatic ties to Moscow, informed the House of Commons about the content of intercepted Soviet diplomatic telegrams, thereby alerting the Kremlin to what was perceived as a hideous breach of security, and ensuring their systems were changed and made inviolable for some decades. Even Churchill had to be persuaded to remove compromising references to ‘Most Secret Sources’ in his magisterial history of the war. And after the German surrender, when GCHQ turned its attention to the Jewish Agency’s private communications, Clement Attlee ignored the pleas from MI5, GCHQ and MI6, and published extracts of intercepts codenamed Oats and Istria in 1946 that proved the organisation’s complicity with the terrorists of the Irgun and Stern gang.

Once again, by referring directly to the source, generically known as Ispal, it dried up, leaving the frustrated GCHQ cryptographers to turn to other targets.

In the immediate post-war era, when a bankrupt Britain could hardly hope to maintain the profitable partnership with Arlington Hall, the fading empire gave an opportunity for a beneficial trade: real estate - British possessions across the world where the phenomenon known as atmospheric bounce could be exploited by ground intercept stations - in return for the finished product. In a unique intelligence pact, American personnel manned British intercept stations from Ascension to Cyprus and Diego Garcia.

While Britain could not begin to match the technical and intellectual resources applied to international cryptographic challenges, it most certainly could offer collection facilities in Yorkshire, St Helena, Gibraltar, Malta, Aden, Masirah, and even Sarafand in Palestine and Habbaniya in Iraq. Since then, successive governments, increasingly reliant on secret sources to provide background to vital Cabinet decisions, especially in the foreign policy arena, have taken some draconian steps to ensure GCHQ is uninterrupted. Even the publication of a fairly innocuous article by Time Out in May 1976, entitled ‘The Eavesdroppers’ and based on information provided by a former National Service wireless operator, led to a major criminal trial because it contained details of the kind of work undertaken at Cheltenham.

Finding itself ‘blinded’ by trade union action during the unanticipated Soviet invasion of Afghanistan in December 1979, Margaret Thatcher’s administration banned unions at Cheltenham, thereby reducing the risk of militancy and sabotage within the agency. It also remedied management’s fear of external union interference or knowledge of its personnel, activities and operational sites. When Prof Lawrence Freedman came to write an official history of the 1982 Falklands conflict, he was finally allowed in 2007, after a lengthy and bitter debate within Whitehall, to include a single Argentine intercept, the first such admission since the 1946 White Paper justifying the police raid on the ostensibly untarnished Jewish Agency in Palestine.

This policy of discretion has been adopted universally, and even when a particular operation has achieved results of historic importance, such as Venona which continued from 1943 until 1979, and was only declassified in 1996, there were strong voices heard in the corridors of the covert world arguing that this particular triumph should never be disclosed. However, with GCHQ’s reluctant acquiescence, the publication of the Venona texts exposed the cryptonyms and activities, and often the true identities of literally hundreds of Soviet spies in the US, Great Britain, France, Australia, Mexico and Sweden. Among the traitors named were those who had penetrated deep into the Manhattan project, the White House, virtually every branch of the Roosevelt administration including the US Office of Strategic Services; the British Foreign Office and MI6; Australia’s Department of External Affairs; ministers close to General de Gaulle; and even Senator, who turned out to be the son of Sweden’s prime minister.

GCHQ argues that any informed discussion of its methods and capabilities handicaps its operational effectiveness, and details of its relationships with its foreign counterparts, especially those in Australia, New Zealand, Canada and the US, must be regarded as sacrosanct to ensure continued cooperation. These “Five Eyes” nations form the backbone of a global reach that can identify the precise location of a terrorist’s cellphone in the ‘denied territories’ of Yemen and Somalia, where no amount of local liaison or attempts to infiltrate physical surveillance teams or snatch squads will ever match the instantly actionable intelligence acquired from electronic collection systems. But the Five Eyes are not the only players in field.

The French GCR, the Swedish FRA, the Dutch AVI, the Norwegian SIGINT agency at Saeter and, yes, the German BND, Angela Merkel’s own foreign intelligence agency, all toil in the same vineyards, hoping to assemble an electronic haystack of metadata - that is, the time, duration and location information required for telephone billing purposes, in which the tell-tale needles may be traced. –Telegraph