graduate

Codes


Photograph of Ultra decrypt

secretcodebreaker.com. Fair use may apply.

Radio revolutionized naval warfare by allowing a measure of command and control at great distances. However, this came at a price in security. Because radio is a broadcast, receivable by anyone, it can broadcast information to enemy as well as friend.

There are two significant aspects to this. First, it is possible for an enemy to analyze the pattern of broadcasts and detect enemy preparations. This is known as traffic analysis and it was practiced successfully by both sides in the Pacific War.

The second aspect is cryptography and cryptanalysis. Radio messages are encoded to prevent an enemy from discerning their contents (cryptography), but coded messages can be analyzed, the code broken, and the contents revealed (cryptanalysis). Both sides in the Pacific War were able to break at least some of the other’s codes, but the Allies were much more successful at cryptanalysis than the Japanese. Whereas Japanese code breakthroughs were usually into less important codes, such as weather ship codes, the Allies were able to break into every Japanese naval code except the flag officer's code (Ko). In particular, American cryptanalysts broke successive versions of the main Japanese naval operational code (D or Ro), which they dubbed JN-25. The Allies also broke successive versions of the Japanese diplomatic machine code, dubbed PURPLE. Intelligence derived from enemy military communications was referred to as ULTRA, while that from enemy diplomatic communications was referred to as MAGIC.

Allied code breaking was one of the decisive capabilities of the war. It contributed directly to victory at Midway and was helpful in almost every other major operation of the war (the Guadalcanal campaign being a prominent exception.) Allied success at reading the Japanese merchant shipping code often put American submarines in the right places at the right times, and Allied success at reading the Japanese submarine code greatly reduced the effectiveness of Japanese submarines.

The Allies took great pains to conceal the source of their superb intelligence. The cryptanalysts at Corregidor were given priority for evacuation by submarine to avoid having them fall into the hands of the Japanese. No such escape was possible for Captain John P. Cromwell, who chose to remain aboard a sinking submarine, Sculpin, rather than be captured by the Japanese and risk being tortured into revealing ULTRA intelligence. Cromwell was posthumously awarded the Medal of Honor. However, the Japanese missed important clues that their codes had been broken. The most glaring was a report in the fanatically anti-Roosevelt Chicago Tribune that the Navy had known the Japanese battle plan before Midway. This glaring security breach was somehow missed by the Japanese. The reporter responsible was not prosecuted because this would only have drawn attention to the incident. Accounts of how the reporter came across his information differ: One account has a sympathetic naval officer showing the reporter an intelligence report, while another has the reporter finding the report left open on the captain's desk.

MacArthur distributed ULTRA intelligence much more widely than any other Allied commander. This included distribution through radio channels, a practice that was supposed to be strictly forbidden. But the worst incident was a press release from MacArthur's headquarters on 27 April 1942 which mentioned a buildup of Japanese merchant shipping in the Marshalls. This intelligence had come from cryptanalysis, prompting a furious message from Marshall to MacArthur, but the general proved impossible to restrain. When a copy of the strict ULTRA regulations was sent to his headquarters, he had his staff water them down, then sent a signed copy back to Washington.

The Allied success in breaking Japanese codes was possible in part because the far-flung Japanese garrisons in the Pacific had no way to communicate except by radio. This produced a huge volume of messages that could be intercepted and analyzed using statistical methods. A bonus (from the Allied point of view) of the strategic bombing campaign was that it destroyed communications links in the Japanese home islands, forcing the Japanese to continue relying on radio messages that could be intercepted. This provided crucial intelligence during the final weeks of the war.

Principles of Cryptography and Cryptanalysis

A detailed discussion of the theory and practice of cryptography and cryptanalysis is beyond the scope of this Encyclopedia. However, a brief overview will help the reader understand the work of the code makers and breakers.

Many languages, including Western languages, are written using a phonetic alphabet in which a small set of letters is used to represent the sounds that make up a statement in the language. Other languages, including Japanese, use a large number of ideographs to represent entire syllables.In either case, the information in a message is contained in the patterns formed by the symbols used to write the message. Cryptography seeks to conceal these patterns in such a way that only a friendly recipient who knows how the patterns were concealed can remove the concealment and understand the message. However, information patterns can be surprisingly difficult to hide, requiring elaborate encryption schemes; and the more effective the encryption, the more expensive and difficult it is to use. 

Generally speaking, a code is any scheme for representing information. Security is only occasionally one of the purposes of a code, and such famous codes as Morse code or Hollerith code were invented for their suitability for representing information in a new medium rather than with any intent of concealment. However, to the cryptanalyst, a code is a set of numbers or short sequences of characters used to represent letters, words, or entire phrases that frequently appeared in messages. These codes usually reduce the length of messages, a useful feature, but they also help conceal the message contents.  A cipher is a scheme for  transforming one set of symbols to another set of symbols specifically for reasons of security. The transformation may involved both replacing symbols with different symbols and scrambling the order of the symbols. All of the encryption systems used in the Pacific War were ciphers, but in many cases the message was encoded before being enciphered. Thus cryptanalysis had to both strip the cipher and penetrate the underlying code.

The very simplest encryption scheme is monoalphabetic substitution. One simply replaces each of the conventional letters of the alphabet with some other symbol, often a different letter from the same alphabet. The encrypter looks up the symbol for each letter from a table and transmits the resulting string of symbols, and the decrypter looks up the letter for each symbol from a reverse table to reveal the message. This form of encryption provides virtually no real security, because the patterns in the language of the message are unchanged except for being represented with unconventional symbols.  For example, in written English, one expects the letters E, T, and A to appear most frequently. One also expects the digraphs (letter pairs) EE and TT to be common, but not AA. Thus if a symbol appears frequently in a message, but is almost never doubled, there is a good chance it is the symbol for A. By looking for common letters, digraphs, and trigraphs (sequences of three letters) the cryptanalyst can easily break the cipher. This kind of statistical analysis does require enough encrypted text (enough depth) to build up the necessary statistics on likely patterns. However, for a simple monoalphabetic substitution, a single message a hundred characters long may be adequate to provide the necessary depth, as is demonstrated by the many readers worldwide who solve the daily cryptogram puzzle in their newspapers.  

The next most sophisticated cipher is polyalphabetic substitution, in which a set of different monoalphabetic substitutions is applied cyclically to the message. For example, one can make up a set of ten monoalphabetic substitutions, then apply the first to the first letter of the message, the second to the second letter, and so on, starting over with the first cipher on the eleventh character. If the number of monoalphabetic substitutions is large and they are well chosen, straightforward statistical analysis will be baffled. However, if the cryptanalyst deduces that a polyalphabetic cipher is being used, and if he has sufficient enciphered material to analyze, he can try calculating letter frequencies for every second letter, every third letter, and so on until he finds a value at which the letter frequencies show the right distribution for the underlying language. This reveals the number of monoalphabetic ciphers in the polyalphabetic cipher, and it is then straightforward to crack each monoalphabetic cipher.

Of course, the encryption system may call for different messages to start at different places in the cycle of monoalphabetic substitutions. The cryptographer must then not only determine the length of the cycle, but the starting place in the cycle used for each message. Solomon Kullback discovered in 1935 that this can be done calculating the index of coincidences between two messages, which is done by lining up two messages, with a varying offset between the two, and calculating what percentage of letters match. If the index of coincidence is around 3.8 percent or so, the matches are probably random chance, and the two messages were likely encrypted at different points in the polyalphabetic cycle. The cryptanalyst then shifts the second message one place to the right and computes the index of coincidence again. When he has shifted a number of places equal to the difference in the starting point in the polyalphabetic cycle between the two messages, the index of coincidence will  jump to around 6.7 percent. The cryptanalyst can repeat the process for a large number of messages, thus putting them all in depth so that the individual monoalphabetic ciphers can be analyzed and broken.

The polyalphabetic substitution cipher illustrates another concept. The encrypter must somehow tell the decrypter where in the cycle the encryption of the message begins. In practice, this information must be embedded in the message itself. Such embedded information is known as an indicator and it is a common feature of almost all cipher systems of any complexity. Just as a lock can become a point of weakness if it is picked, so the indicator can be an important weakness in the encryption system if it is cracked.

The greater the number of monoalphabetic ciphers making up the polyalphabetic cipher, the more messages must be intercepted to get the necessary depth, and the harder the cipher is to break. The ultimate polyalphabetic cipher uses a different monoalphabetic cipher for every letter of every message ever sent. For example, with a one-time pad the encrypter replaces each letter in a message with a number, so A becomes 1, B becomes 2, and so on. He then takes a pad of random numbers and adds them to each number from the original message, transmitting the results. The decrypter has a copy of the encrypter's pad of random numbers, and he decrypts the message by subtracting the random numbers off the message. The decrypter then converts 1 back to A, 2 back to B, and so on to reveal the original message. If there are only two copies of each random pad (one each for the encrypter and decrypter), if the numbers on the pad are truly random, and if none of the sheets in the pad are ever reused, then the system is theoretically unbreakable. However, for a large organization like an army that sends a huge volume of traffic, the cost of a one-time pad encryption system is prohibitive, and no power involved in the Pacific War used a true one-time pad system for any but the most sensitive messages.

Another form of decryption is transposition, in which the letters of a message are shuffled into a different order by the encrypter and unshuffled by the decrypter. Transposition was used in a number of hand ciphers, but it was unsuitable as a machine cipher, since machines capable of rapid operations on stored data (a necessary requirement for transposition ciphers) could not be made compact and robust enough for military use in the 1940s.

The chief operational ciphers of the major powers were essentially polyalphabetic ciphers using very long cycles. Many of these, such as the German Enigma, the American ECM Mark II or SIGABA, and the Japanese PURPLE, were machine ciphers with cycle lengths of thousands to tens of millions.  Additional settings on the machines changed the basic cycle, so that there were more than a million million possible ways to begin encrypting a message.  Such a system sounds utterly inpenetrable, and the fact that the Allies nevertheless penetrated Enigma and PURPLE shows how difficult it is to completely erase telltale patterns with an encryption scheme.

A code, as opposed to a cipher, provides less security than one might expect. Even when code groups replace entire phrases, messages in the code still show characteristic patterns that can be analyzed to break the code. Stereotypical words or phrases that a cryptanalyst has reason to believe will appear in many messages, such as "I have the honour to report to your excellency that ..." in a Japanese report to a superior, are known as cribs and can be crucial wedges for breaking into a code or cipher. Position reports also provided cribs in the form of latitudes and longitudes. Many messages were sequentially numbered, which was a dead giveaway for the associated code groups. The Japanese tried unsuccessfully to foil such cribs by bisection, which consisted of splitting messages in two at an arbitrary point and placing the second half before the first half.

Cryptography and Cryptanalysis by the Major Powers

United States. The United States had successfully broken the Japanese diplomatic codes in the 1920s, and thereby learned what instructions the Japanese government had given its negotiators at the Washington Conference.  However, budgets for cryptanalysis were repeatedly slashed thereafter.  Joseph Rochefort, who enlisted in the Navy in 1918 with vague hopes of becoming a naval aviator, was assigned to code research duty because the officer then in charge recalled that Rochefort liked to solve crossword puzzles. When Rochefort took charge of the Navy's code research desk in February 1926, he had just two assistants, which was still triple the size of the group run by his predecessor. The Director of Naval Intelligence had a "slush fund" dating from the First World War which, for lack of any other ideas, was used to fund the burgling of the Japanese consulate in New York City. This highly illegal activity provided valuable information on Japanese code systems.

In 1929 then-Secretary of State Henry Stimson shut down the State Department's Cryptographic Office (the "Black Chamber") on the grounds that gentlemen do not read each other's mail. This reflected the spirit of international optimism prevalent in the West at the time. However, the unstable former director of the "Black Chamber", Herbert O. Yardley, who was disgruntled and short of cash, secretly sold information on the "Black Chamber" to the Japanese, then publicly sold the story of the "Black Chamber" to the Saturday Evening Post in 1931. This created a sensation in Japan and led to tighter cryptographic practices.

However, Japan was not alone in its cryptographic weakness between the wars. Even as the United States was reading Japan's diplomatic traffic, the British and Japanese were reading the American diplomatic traffic. One British cryptanalysis told an American visitor in 1943 that he regularly read U.S. diplomatic ciphers in the 1930s and that it had been "lots of fun" (Budiansky 2000). By 1932 the Japanese had reconstructed five thousand code groups from the American Gray Code, and Japanese cryptanalysts penetrated the subsequent Brown Code by breaking into the Kobe consulate and photographing the code book. In August 1941 a pro-American Japanese official told U.S. ambassador Joseph Grew which code should be used to transmit to Washington the information he was giving the ambassador, explaining that all the other diplomatic codes had been broken.

The U.S. Navy did not entirely neglect radio intelligence between the wars. Code breaking requires proficiency in the enemy's language, and after 1927 the Navy sent two or three officer per year to Japan to study the language. The Navy set up its first listening post at Shanghai in 1924 and set up additional posts over the next three years. Navy intelligence officers also had the foresight to design special typewriters for taking down the 48 kana characters of the Japanese Morse code, and in July 1928 a secret training program was established that eventually taught 176 enlisted men Japanese Morse. Both were paid for out of the ONI slush fund.

By 1939 the U.S. Navy was able to read both the Japanese diplomatic RED machine cipher and the Japanese Navy's Blue Book operational code, which revealed, among other things, that the Nagato class battleships were two and a half knots faster than previously believed. This may have influenced the design of the North Carolinas. However, on 20 February 1939, the RED machine cipher was replaced by a new machine cipher the Americans dubbed PURPLE, which was vastly more complicated. A little over three months later, on 1 June 1939, the Blue Book code was replaced by Ro, which became known to the Americans as "AN" or JN-25, and which was unlike any previous Japanese Navy operational code. It appeared to be totally impenetrable. American intelligence thus lost its ability to read Japanese secret communications during the crucial period between June 1939 and September 1940, when PURPLE was finally cracked. JN-25 would prove an even harder nut, and it was not until March 1942 that enough of the code was cracked to begin providing useful intelligence.

More resources became available for cryptanalysis after 8 September 1939, when Roosevelt declared a "limited national emergency", a largely rhetorical gesture having no particular standing in American law. However, Civil Service rules prevented the Americans from recruiting academic talent until after war had actually broken out; new hires had to be made from candidates who had passed a Civil Service exam. Fortunately for the Americans, William F. Friedman, the brilliant head of the U.S. Army's cryptanalysis effort, had a knack for recruiting cryptanalytic talent. Many of the new recruits were Jewish Americans or women who were highly educated but faced discrimination that prevented them from finding jobs in academia or business.

Sauce for the goose is sauce for the gander, and success in cracking other people's secret codes led the United States to look to its own cryptographic security. Friedman helped the U.S. armed services develop the ECM (Electronic Cipher Machine) Mark II, also known as SIGABA, which superficially resembled the German Enigma or Japanese Purple enciphering machines. However, it used three banks of five rotors each and had on the order of 100,000,000,000,000 possible settings. Operators were strictly admonished to avoid the kinds of procedural errors that allowed the Allies to penetrate Axis machine codes, and it is claimed that the ECM Mark II was never penetrated. Part of this was due to a seemingly trivial design decision: On Enigma, the rightmost rotor was the "fast" rotor that moved one place with each successive character, while the ECM Mark II placed the "fast" rotor in the middle of each bank of rotors. The German placement allowed cryptanalysts to deduce the wiring of the first rotor without needing to know the wiring of the successive rotors, a significant weakness that did not exist in the ECM Mark II.

On the other hand, the M-209 tactical machine cipher was relatively insecure, and its rotors were so difficult to reset that operators tended to use the same settings day after day. The Germans regularly broke it, and the Japanese may have broken it from time to time as well. However, it was only used for low-level tactical communications in the field.

Impressive as SIGABA was, American cryptographic security sometimes went even further. Nimitz at Pearl Harbor communicated with King in Washington by cable using a special telegraph machine that automatically encoded the messages using a punched paper tape of random numbers. Only Washington had a duplicate and it was never reused, thus constituting a true one-time pad system.

The American code breaking effort was characterized by its informality and by its heavy reliance on the primitive computing machinery available in the 1940s. The Army effort relied heavily on civilian specialists, and officers who tried to impose military-style discipline, in a misguided effort to increase productivity, were quietly shuffled aside. The Navy was much less inclined to use civilians, but OP-20-G (the Navy's signals intelligence unit) got around this by assigning Navy Reserve officer rank to its cryptanalysts strictly on the basis of age, then largely ignored rank. The Navy leased large numbers of IBM data processing machines (IBM's business model at the time was to never sell a piece of equipment), to the bemusement of the less resource-rich British. This led the Americans to rely more heavily on brute force attacks than the cryptanalyists of other powers, but this approach was not without success. Vast amounts of data (by 1940s standards) could be stored as punched paper tape or on film and processed by machine to calculate statistics or look for matching code groups in intercepted messages.

Priority in resources was given to the European war and to the Japanese Navy, and Japanese Army codes were not seriously tackled until 1943, when the Americans discovered that Japanese Army communications often dealt with the ambiguity of written Japanese by spelling out words using the Chinese Telegraph Code. This provided the necessary wedge, and by June 1943 the Water Transport system (used by the Army's fleet of ships) was providing useful intelligence. The capture of the entire code library of 20 Division at Sio (147.371E 5.954S) in New Guinea in January 1944 broke the Japanese Army codes wide open, yielding a flood of intelligence. This included a 13-part report by a "Staff Officer Izumi" detailing high-level conversations between the Japanese Army and Navy on strategy for the New Guinea and Solomons area.

By the end of the war, Radio Analysis Group Forward Area (RAGFOR) at Guam had over 600 cryptanalysts working for it.

Britain. The British had been very successful in breaking German codes during the First World War, but this capability languished between the wars, and it was not until the debacle in Norway in 1940 that signals intelligence began to be taken seriously again by British military leaders. This change in attitude was largely driven by Churchill, who insisted on seeing the most important raw decrypts and who liked to use them as ammunition during arguments with his military leaders.

Most of the British effort was focused on the German Enigma machine cipher, a story that has been widely told elsewhere and does not directly bear on the Pacific War. However, the British in the Far East put considerable effort into breaking the Japanese codes, and made important contributions to the Allied effort to recover code and addend groups for the JN-25 code. The British Army in Burma also had significant success breaking the Japanese Army codes, which contributed significantly to the British victory in the Burma campaign of 1944-1945. 

The British code breaking effort recruited "men of the professor type" (Budiansky 2000) but only three of the first 21 academics recruited were mathematicians. Partly this reflected the network of the contacts of the leaders of the effort, but it also reflected the contemporary British attitude that a proper education was focused on the study of Latin and Greek. Science and mathematics were tainted by their usefulness and the implied association with "trade." Such mathematics as was studied at the most prestigious British universities was of a highly abstract character. Thus the three mathematicians hired early on were pure mathematicians of the first order, and included the brilliant Alan Turing.

Japan. The Japanese Army code breaking effort was almost entirely focused on the Russians and Chinese, regarding Britain and the United States as the concern of the Navy. A number of lower-level Russian codes were broken by Kwantung Army intelligence officers, which yielded considerably information on the German-Russian war. The ability of the Army to read Chinese diplomatic communications tipped Army leaders off to the fact that Japanese Prime Minister Konoye was sending diplomats to negotiate with the Chinese in July 1937, and the Army found an excuse to arrest the couriers and thus hinder the negotiations. The Chinese Mi-ma code was broken by May 1940, allowing the Japanese to track the movements of practically every Chinese division involved in the battle of I'chang. 33 Army in northern Burma had a cryptanalysis team and veterans claimed that this allowed the Japanese to anticipate almost every move by the Allies during hte Myitkyina campaign. Japanese intelligence veterans have claimed that the Japanese attempted to shoot down U.S. Vice President Wallace during a trip to China in 1943 based on cryptographic intelligence. The Chinese Communists were more security-conscious than the Kuomintang and tended to use Russian code systems, and their codes were broken only intermittently by the Japanese during the war.

Army intelligence belated shifted its main effort to the Americans in July 1943, rather late in the game, but Japanese Army codebreakers did have some successes against the western Allies, penetrating both the American prewar diplomatic Brown Code (by breaking into the Kobe consulate and photographing the code book) and, by 1943, the diplomatic strip cipher. As with many cryptographic breakthroughs, the cracking of the strip cipher was aided by American operator carelessness. The Japanese also broke some lower-level prewar British diplomatic codes, which hinted that the defenses of Malaya were inadequate.  Japanese Army intelligence veterans later claimed that they made significant inroads into the M-209 machine cipher and thus were able to anticipate some B-29 raids. These included the Hiroshima raid, though its full significance was not realized in time to take any defensive action.

Japanese Navy cryptography had only limited successes. The Navy succeeded in cracking the American diplomatic Gray Code in the 1920s by assuming that NADED, which appeared frequently, meant "stop", a classic use of a crib. The Navy also broke Brown, the successor to Gray, independently of the Army. In July 1937 the Owada Receiving Station, which monitored Chinese and American signal traffic, successfully intercepted and decoded Chinese defense plans for 27 Army. Owada evaluated the results of Japanese air strikes during the Shanghai Incident later than year by intercepting and decoding Chinese damage reports. However, there were no Japanese breakthroughs into Allied operational codes comparable to those made by Allied cryptographers into JN-25.  The most damaging breakthrough was into the aircraft movement codes, which revealed the numbers and types of aircraft departing from and arriving at airbases in the Southwest Pacific.

China. The western Allies were well aware that the Japanese had cracked the Kuomintang codes. However, the western Allies valued their own ability to read Kuomintang diplomatic dispatches, and therefore avoided giving sensitive information to the Kuomintang rather than inform the Kuomintang that their codes had been compromised. Concerns for Chinese security were also cited as a reason why Chiang Kai-shek was refused a seat on the Combined Chiefs of Staff.

Penetrating the JN-25 code

The most significant code penetration of the Pacific War was of the main Japanese operational code, JN-25. This code was introduced on 1 June 1939 and proved far more resistant to cryptanalysis that previous Japanese Navy codes. The American naval intelligence unit at Pearl Harbor (HYPO), under the leadership of Rochefort, had correctly deduced the nature of JN-25 by the time war broke out, but it took time to reconstruct the Japanese code books. Useful intelligence did not start to become available until March 1942.

A Japanese communications technician would prepare a message for transmission by first looking up the five-digit code number for each phrase or syllable in the message from a basic code book. This contained over 30,000 code groups for the most common words or phrases used in naval messages, with the most frequently used being represented by several groups to frustrate frequency analysis -- an example of an injective code. The technician would then select a random location in a second addend code book containing over 30,000 random five-digit numbers. The sequence of random numbers from the second book would be added to the sequence of code numbers representing the message (ignoring all carries in the addition) and transmitted along with the start location in the addend book. The communications technician responsible for decoding the message at the other end would reverse the process, subtracting the random addends from the coded message (ignoring all borrows) to reveal the original code numbers, which were then looked up in the basic code book to reveal the original message.

Breaking the system required reconstructing both the basic code book and the addend code book. This was done largely by inspired guesswork, aided by the primitive computing machines available in 1941. (The U.S. Navy became an important customer of IBM as war approached.) Certain stereotypical phrases, such as "Long live the Emperor!", appeared in many messages, and by finding these cribs the code breakers could recover addends. As more and more addends were recovered, the code breakers could interpolate the gaps in the messages, filling out both the basic and the addend code books. But this was a tedious process that relied on having a very large volume of intercepted messages to work with. Little progress had been made prior to the attack on Pearl Harbor, but the enormous volume of messages intercepted after war broke out, combined with the highly predictable contents of certain messages, allowed the code breakers to reconstruct about 15% of the code books by the time of the Midway operation.

The code breaking process was made easier by Japanese complacence. The Japanese seemed to feel that their language was so difficult for Westerners to master, and their code system so cleverly constructed, that it was invulnerable to penetration by the Allies. The Japanese issued new addend books about every six months, but the resulting ULTRA intelligence blackouts were brief, due of the Japanese habit of continuing to send messages in the old code while the new code was distributed across the vast reaches of the Pacific. A message sent in both the old and new codes immediately gave the code breakers part of the new addend book. Another weakness of the system was that all the five-digit numbers were chosen such that the sum of the digits was divisible by three; this made it easier for the intended recipient to detect garbles, but it was also a weakness that was exploited by the code breakers.

Japanese refusal to believe JN-25 could be compromised prevented the Navy from drawing the correct conclusions after Yamamoto was shot down. The Navy investigation placed the blame on the Army for retransmitting Yamamoto's itinerary in a low-level code. Hence, interservice rivalry played a role in the lack of Japanese cryptographic security.

PURPLE

In addition to breaking JN-25, the Allies penetrated the machine cipher used by the Japanese diplomatic service, which was dubbed PURPLE by the Americans. This code was first introduced on 20 February 1939 and proved far more complex than any machine cipher previously used by the Japanese. The American cryptologists worked on the cipher for a year without success, but on 8 September 1939 Roosevelt had declared a "limited national emergency" (a rhetorical gesture having no particular standing in U.S. law) and more resources were made available to the cryptanalysts. The Japanese had continued using RED for a time after PURPLE was introduced, and it was not difficult to figure out which messages in RED were duplicated in PURPLE. This provided a wealth of long cribs.

The cryptanalysis also discovered that six letters appeared with different frequency from the others, showing that they were being encrypted separately, but they were not being shuffled in a way that could be carried out with rotors. Eventually it was realized that the Japanese were using telephone exchange stepping switches rather than rotors.  Unlike the rotors in the German Enigma, these could not be swapped out, greatly reducing the number of potential settings. Soon the "sixes" could be reliably cracked, but the "twenties" (the other twenty letters of the alphabet) remained unsolved. The "twenties" were finally cracked when the cryptanalysts realized that they were being encrypted using a cascade of scrambler units. Within a few weeks the cryptanalysts had constructed their own PURPLE machine, and in September 1940 they began reading PURPLE traffic almost as quickly as the Japanese diplomats themselves.

The Americans thus knew by the morning of 7 December 1942 that the Japanese were about to inform the U.S. government that there was no point to further negotiations. While this was certainly ominous, and the instructions to the Japanese ambassadors to deliver the note at a precise time that morning was obviously significant, the note was not an actual declaration of war nor did it give any hints what the Japanese opening move would be. Most American leaders assumed the Philippines would be the target, and an alert message to Hawaii did not arrive until hours after the attack on Pearl Harbor had taken place.

The American effort was hampered by bureaucratic turf wars once the value of the intelligence PURPLE produced was fully recognized. The two services saw the value of combining their efforts but could not agree on who would be in charge. The problem was finally solved by an informal arrangement in which the Navy would decipher PURPLE on odd-numbered days and the Army on even-numbered days.

PURPLE continued to be read throughout the war.

Other Japanese Codes

The British succeeded in breaking the Navy's dockyard code in 1939, which revealed in 1941 that the Japanese were recalling their warships to port for reorganization two months earlier than usual. This was correctly interpreted as a move to prepare Combined Fleet for war.

The Japanese merchant ship code, dubbed JN-40 by the Allies, was cracked by the British in September 1942 after a Japanese operator repeated a message word for word except for adding his ship's position. This allowed the cryptanalysts to determine that the code was a simple transposition cipher based on daily tables. These were found to change according to a predictable formula, and soon all themerchant message traffic was being read routinely.

The Japanese military attaché code was also cracked by the British, in early 1942. Among other things, this revealed Japanese plans to construct the Burma-Siam railroad. Some of the most useful intelligence came in the Tokyo Circular, a digest of Japanese military activity on all fronts that was circulated to Japanese military attachés around the world.

The WE code, used by the Japanese Navy when sending military traffic, was a very insecure code used for low-level messages, but these provided much order of battle information and cribs for more secure codes.

The General Army Administration Code, known as 7890 from the code group used to identify messages encrypted with this system, was cracked by the Americans in April 1943 due to Japanese carelessness. Several operators sent similar messages without bothering to change position in the addend book, giving enough depth to crack the code.

The JADE machine cipher, used by Japanese naval attachés, was cracked in August 1943 by the Americans. The more difficult CORAL machine was cracked by a joint British-American effort in March 1944.

The British cracked the 6633 cipher used for air units by late 1944, giving the Allies a very accurate order of battle for aircraft. The British also cracked the BULBIL air-to-ground code at about the same time.

Historians and Code Breaking

Both ULTRA (breaking of Japanese operational codes) and MAGIC (breaking of Japanese diplomatic codes) remained largely classified postwar. The fact that Japanese diplomatic codes were being read at the time of Pearl Harbor became public knowledge in the postwar Congressional investigations of that disaster. A small number of MAGIC messages from later in the war were released in the 1950s, but the first relatively complete set of MAGIC digests were not released until 1978. Even these had many sections whited out. A complete set of uncensored MAGIC digests was finally published in 1995, revealing (among other things) that MAGIC  was reading the diplomatic traffic of a large number of neutral countries, and even a few Allied powers (such as Nationalist China and the Free French.) ULTRA did not begin to be revealed until the 1970s.

The fact that ULTRA and MAGIC remained secret for so long has had an enormous influence on postwar historians. The greatest single flaw in Morison's otherwise excellent History of United States Naval Operations of World War II is that it was written when ULTRA was still highly classified, and Morison recites the official cover story for every important ULTRA-based intelligence coup in the war. A useful companion while reading Morison today is Prados' Combined Fleet Decoded, which gives the ULTRA background for many important command decisions during the war.

The gradual disclosure of MAGIC has had its effect on historiography of the Japanese surrender, as documented by Frank (1999). Early and fragmentary MAGIC disclosures, indicating that the Japanese were seeking peace terms through the Russians, have been seized upon by critics of Truman's decision to use nuclear weapons as evidence that the Japanese were on the verge of surrendering and the bombings were unjustified. Frank has argued that the complete MAGIC record released in 1995 reveals a much more complex picture that is much less supportive of this interpretation. For example, the ULTRA decrypts reveal a uniform picture of determination on the part of the Japanese military to continue the war.

References

Budiansky (2000)

Evans and Peattie (1997)

Frank (1990, 1999)

Kotani (2009)

Larrabee (1987)

Lundstrom (2006)

Peattie et al. (2011)

Nakagawa (1993)

Prados (1995)

San Francisco Maritime National Park Association (accessed 2010-4-19)

Smith (2000)

Weinberg (1994)



Valid HTML 4.01 Transitional