[Image courtesy of NSA’s official Twitter account.]

At the end of part 1 of our look at the history of the NSA and American codebreaking, we left off with the pivotal Black Friday event.

On November 1, 1948, all intel coming from monitored Soviet signals went quiet. All traffic on military, naval, and police radio links was replaced with dummy messages. It was such an unprecedented and alarming event that London and Washington briefly considered that it might’ve been the first indication of preparations for war.

According to Code Warriors author Stephen Budiansky:

The full extent of the disaster only became apparent the following spring when real traffic started reappearing on the radio nets, now employing greatly improved — and completely unbreakable — technical and security procedures. The keying errors or other mistakes that had allowed most of the Soviets’ machine-enciphered military traffic to be routinely read by US and British codebreakers for the last several years had been corrected, and the much more disciplined systems that now replaced them slammed the cryptanalytic door shut.

Even the one-time pads that had offered some hope to attentive American codebreakers were updated, eliminating the ability to sort messages by which organization they originated from.

Codemakers had suddenly outpaced codebreakers.

[The Kryptos sculpture outside CIA Headquarters. The NSA cracked
several of its codes before the CIA did. Image courtesy of Slate.com.]

The Office of Naval Intelligence wanted to take over from Signals Intelligence (SIGINT), demanding to see “everything” so they could do the job. They claimed SIGINT should limit their work to message translation, leaving interpretation to “the real experts.” This sort of territorial gamesmanship would continue to hamper government organizations for decades to come.

And that demand to see everything? That probably sounds familiar, in light of the revelations about government data collection and the PRISM program that were revealed in Edward Snowden’s leaks.

Black Friday was the start of all that, a shift from codecracking to the massive data collection and sifting operation that characterized the NSA for decades to come.

More amazingly, there was SO MUCH information collected during World War II that SIGINT was still poring over it all in 1949, decrypting what they could to reveal Soviet agents in the U.S. and England.

The fact that a high-ranking member of British Intelligence at the time, Kim Philby, was actually a Soviet double agent complicated things. After a decade under suspicion, Philby would flee to the Soviet Union in 1963, stunning many friends and colleagues who had believed in his innocence.

[The spy and defector, honored with a Soviet stamp.
Image courtesy of Britannica.com.]

Although the Russians had flummoxed SIGINT, other countries weren’t so lucky. The East German police continued to use ENIGMA codes as late as 1956. Many of the early successes in the Korean War were tied to important decryption and analysis work by SIGINT. Those successes slowed in July of 1951, when North Korea began mimicking Russia’s radio procedures, making it much harder to gain access to North Korean intel.

Finally, the chaotic scramble for control over signal-based data gathering and codebreaking between the government and the military resulted in the birth of the National Security Agency on November 4, 1952, by order of President Truman.

One of the first things the NSA did? Reclassify all sorts of material involving historical codebreaking, including books and papers dating back to the Civil War and even the American Revolution.

[The actual report that recommended the creation of the NSA.
Image courtesy of NSA’s official Twitter account.]

The creation of the NSA had finally, for a time at least, settled the issue of who was running the codebreaking and signals intelligence operation for the United States. And they were doing fine work refining the art of encryption, thanks to the work of minds like mathematician and cryptographer Claude Shannon.

One of Shannon’s insights was the inherent redundancy that is built into written language. Think of the rules of spelling, of syntax, of logical sentence progression. Those rules define the ways that letters are combined to form words (and those words form sentences, and those sentences form paragraphs, and so on).

The result? Well, if you know the end goal of the encoded string of characters is a functioning sentence in a given language, that helps narrow down the amount of possible information contained in that string. For instance, a pair of characters can’t be ANYTHING, because letter combinations like TD, ED, LY, OU, and ING are common, while combos like XR, QA, and BG are rare or impossible.

By programming codecracking computers to recognize some of these rules, analysts were developing the next generation of codebreakers.

Unfortunately, the Russian line was holding. The NSA’s failure to read much, if any, Soviet encrypted traffic since Black Friday was obviously becoming more than just a temporary setback.

Something fundamental had changed in the nature of the Russian cryptographic systems, and in the eyes of some scientific experts called in to assess the situation, the NSA had failed to keep up with the times.

I hope you’re enjoying this look at the early days of America’s 20th-century codebreaking efforts. Part 3 will continue next week, with the sea change from active codebreaking to data mining, plus Vietnam, the space race, and more!

Thanks for visiting PuzzleNation Blog today! Be sure to sign up for our newsletter to stay up-to-date on everything PuzzleNation!

You can also share your pictures with us on Instagram, friend us on Facebook, check us out on Twitter, Pinterest, and Tumblr, and explore the always-expanding library of PuzzleNation apps and games on our website!