Intelligence

Navy History and Heritage Command #NH 84826

Via Wikimedia Commons

Military intelligence is that branch of the military art concerned with deducing the enemy's dispositions, capabilities, and intentions. It includes both data collection and analysis. The Japanese started the Pacific War with a significant advantage in intelligence, but rapidly fell behind the Allies. This was primarily due to superior Allied cryptanalysis, but also due to institutional attitudes that hindered effective use of intelligence.

1. The Intelligence Process
   1. Acquisition.
   2. Delivery.
   3. Acceptance.
   4. Interpretation.
   5. Implementation.
2. Intelligence Sources
   1. Human intelligence.
   2. Interrogation.
   3. Captured documents. 
   4. Captured equipment.
   5. Reconnaissance.
   6. Signals intelligence.
   7. Open source intelligence.
3. Analysis
4. Counterintelligence
   1. Counterespionage.
   2. Screening.
   3. 
      
   4. Operational security.
   5. Deception.
5. Intelligence and Special Operations
6. Allied Intelligence
7. Japanese Intelligence

The Intelligence Process

Keegan (2003) identifies five phases to the intelligence process:

Acquisition.

 Intelligence must first be obtained. This can be as simple as visiting a library to collect information from open publications, or as elaborate as developing a network of radio monitoring stations and a cryptanalysis organization to intercept and decode enemy transmissions. The least elaborate method should be employed to obtain a particular piece of intelligence: It is wasteful to employ secret agents to obtain information available from newspapers easily obtained by diplomats in neutral nations.

Delivery.

 Intelligence must get from its collection point to the intelligence center. For some forms of intelligence, such as human intelligence, this may be the most difficult phase of the process. The transmission of intelligence is susceptible to detection and interception, with consequent loss of the intelligence source, yet delivery of intelligence usually must be done under great time pressure for it to be of any use.

Acceptance.

 The intelligence must be vetted for credibility. Spies can be doubled (turned against their original employers) and signals can be faked.

Interpretation.

  Credible, timely intelligence still requires interpretation. The vast majority of intelligence takes the form of small items of information that mean little until they are collated to form a larger picture. The picture is rarely complete, and ultimately an intelligence chief must make his best guess what the picture really means.

Implementation.

 Once a picture has been assembled, it must be put to use by policy makers and military commanders. These leaders must take into account their own capabilities and intentions, which are often not shared with the intelligence apparatus, and choose a suitable course of action. Intelligence by itself never wins a battle.

While this breakdown is a reasonable general framework for understanding the intelligence process, the phases are not necessarily clear cut. For example, acceptance and interpretation tend to run together under the more general category of analysis. One may also speak of an intelligence cycle, as part of the implementation of intelligence may be directives to obtain further intelligence of specific enemy activities.

Intelligence Sources

Data came from many sources.

Human intelligence.

Traditional human intelligence, such as espionage, played an important role for the Japanese as war approached. Japanese consulates in target areas assiduously collected information and passed it back to Japan for war planning purposes. For example, Yoshikawa Takeo, a Japanese Navy officer dropped from flight training because of ill health, was assigned to the Honolulu consulate as vice-consul under a cover identity. He collected extensive intelligence on military activities on Oahu prior to the attack on Pearl Harbor. When the Taiyo Maru arrived at Honolulu on 1 November 1941, it carried three Japanese intelligence agents who managed to smuggle reports from Yoshikawa on board and who made their own extensive observations of air and sea activity around Oahu. Asaka Maru carried out a similar mission to the Panama Canal and Germany in early 1941. Japanese "tourists" were also active, and Japanese fishing vessels conducted extensive hydrographic surveys of potential landing areas. Japanese intelligence officers posed as foresters along the Thai-Burma border to scout out invasion routes.

A waiter at an officers' club at Singapore was discovered in 1941 to be a Japanese intelligence officer, Colonel Kadomatsu Tsugunori. He was arrested after suspicious British officers deliberately leaked false information and caught him taking notes. Another intelligence officer, a Colonel Yakematu, collected intelligence throughout Malaya by hiring himself out as a day laborer. The official photographer Changi Naval Base was later identified as a Colonel Nakajima. Deacon (1983) alleges that there were a large number of Japanese barbers in the Panama Canal Zone who were clandestinely collecting information and reporting to Morasawa Chiyo, a Japanese woman who had opened a small clothes shop in Panama in the 1920s. British prisoners of war at Hong Kong recognized one of their captors as a former barber in the British barracks, and natives on Guadalcanal recognized a Japanese officer, "Ishimoto", as a former itinerant carpenter at Tulagi. These spies turned occupation officers were often most brutal towards those they had formerly spied upon. For instance, "Ishimoto" is alleged to have arrested and murdered two Roman Catholic priests and two nuns from Ruavato mission.

Not all Japanese spies were Japanese subjects. H.O. Yardley, the psychologically unstable former director of the U.S. State Department's "Black Chamber", sold information on U.S. cryptanalytic activities to the Japanese shortly before publishing the same information openly in the Saturday Evening Post in 1931. Harry Thomas Thompson, a former U.S. Navy yeoman, was recruited by the Japanese in 1935. He collected information by boarding U.S. warships in uniform and striking up conversations with sailors as if he was still in the Navy and visiting old friends. Thompson was turned in by a roommate with whom he had confided part of his story. John Semer Farnsworth, a former U.S. Navy commander who had been cashiered for refusing to repay a debt to an enlisted sailor, was also recruited in 1935, and he managed to photocopy a number of very sensitive documents for the Japanese before being turned in by an officer's wife, whom he had pressured to provide him with documents. British cryptanalysts in the Far East uncovered espionage by Frederick Rutland, a disgruntled former British military pilot who also carried out espionage on the U.S. West Coast. Rutland was quietly interned when he returned to Britain in 1941.

Japanese intelligence officers also maintained contacts with members of secret societies in Japan, such as the Black Dragon Society (from the Japanese name for the Amur River, the boundary between Manchuria and Russia.) Chinese secret societies, such as the Triads, were in contact with both Japanese and Chinese intelligence, and are alleged to have played a significant role in providing the Japanese with intelligence at Hong Kong.

Many of these sources dried up once war broke out. The Japanese attempted to organize a spy ring on Oahu to replace intelligence gathering by the Honolulu consulate, but the leader of the ring, Otto Kuehn, a German national, was unfit by training and temperament to be an effective spy, and the FBI arrested him the day after the attack on Pearl Harbor. Officials who commandeered the Vichy French liner Maréchal Joffre in Manila shortly after war broke out were embarrassed to discover that one of the crew had been working for U.S. Navy Intelligence, reporting on Japanese ship movements in French Indochina. However, Axis agents remained active in Noumea, the Allies recruited a number of Thai agents in southeast Asia and sympathetic French in French Indochina, and pretty much everyone spied on everyone else in China. The Sorge ring in Tokyo provided some excellent intelligence to Russia, but this information was not passed on to the powers at war with Japan. 

One of the few Japanese spy rings still operating in North America after war broke out was directed from Buenos Aires. An older American couple, Lee and Velavee Dickinson, had been recruited by Japanese agents in California in the mid-1930s, and after moving to New York City in October 1941, Velavee opened a doll shop catering to the rich and famous. After postal censors flagged a letter purportedly from a Portland, Oregon, woman as suspicious, the FBI traced the letter back to Velavee. Navy cryptanalysts concluded that she was using a "doll code" to transmit information about warships and naval installations. For example, the phrase "three Old English dolls" referred to three warships under repair in a naval shipyard. By this time Lee Dickinson had died of a heart attack, and Velavee was able to strike a plea bargain and avoid the most serious charges against her by claiming Lee was the real villain. She was sentenced to ten years' imprisonment,

One of the greatest difficulties with wartime espionage was getting the information out fast enough to be of any use. Radio transmission was usually ruled out by the difficulty of avoiding detection. Diplomatic channels that might provide cover for intelligence delivery were disrupted, and information often had to go out through mail through neutral countries, as in the Dickinson case. Such intelligence was therefore typically strategic intelligence of more lasting value.

A remarkable exception to this rule was the British spy ring among prisoners of war at Singapore. The prisoners contrived to manufacture a transmitter powerful enough to be picked up in India, and concealed it in the flooded ammunition rooms under one of the demolished naval rifles at the old naval base. The Japanese detected the transmissions but could not locate the transmitter until its operators were betrayed by an Indonesian soldier who had changed sides.

No one was better than the Russians in human intelligence. Not only did Russian agents penetrate the German and Japanese governments, but there were an estimated 329 Soviet agents within the U.S. government, including Roosevelt's senior economic adviser, Lauchlin Currie; assistant Treasury secretary Harry Dexter White; and Bill Donovan's personal assistant at OSS, Duncan Chapin Lee. The British government was at least as heavily penetrated, including by the "Cambridge Five." Intelligence supplied by these agents doubtless played a role in the end game in the Pacific.

The Russians may not have been the only ones to spy on their own allies. Sasgen (2010) claims that the papers of Charles Lockwood show that the U.S. Navy attempted to infiltrate a spy into Vladivostok to collect information from Russian shipmasters on Japanese antisubmarine defenses in the Sea of Japan, particularly around La Perouse Strait. 

Another source of human intelligence used by both sides was coast watchers. These were agents left in the jungle in areas overrun by the enemy. The Australian "Ferdinand" organization, which operated in the Solomons and New Guinea, was particularly famous, but the Japanese also employed coast watchers once the tide had turned. Closely related to coast watchers, and sometimes synonymous with them, were various guerrilla organizations, which were particularly active in the Philippines. Philippine guerrillas succeeded in capturing Fukudome Shigeru, chief of staff of Combined Fleet, after his plane crashed in a typhoon in mid-1944. He was released out of fear of reprisals, but the documents he was carrying found their way to Allied headquarters.

Interrogation.

Fear of interrogation under torture led to strict rules for employment of personnel who were privy to the most sensitive Allied intelligence. High-ranking officers who were cleared for cryptanalytic data ("ULTRA") were kept from situations in which they might be captured. Junior intelligence officers were almost never permitted to transfer to front-line duty. Joseph Rochefort (pictured above), who led the cryptanalysts during the crucial Coral Sea-Midway period, was given command of a floating dry dock when he requested a sea assignment; this looked like a calculated insult, but it reflected the fear that he might be captured and tortured by the Japanese if assigned to a warship. Captain John Cromwell unwisely led a submarine wolf pack off the Gilbert Islands in spite of his knowledge of ULTRA, and when his pennant submarine was crippled, he chose to go down with the boat rather than risk capture and interrogation. He was subsequently awarded the Medal of Honor for his sacrifice.

Allied interrogators did not as a rule employ any form of torture. They did not need to. Because the Japanese military code of honor absolutely forbade surrender, Japanese soldiers received no instruction on how to behave in captivity, and those captured felt such shame that they had little psychological resistance to interrogation. Many sang like canaries. However, the fact that officers almost never surrendered meant that almost all prisoners were enlisted men with little or no high-level information to impart. For example, the British in Burma never took a live prisoner of rank higher than captain. Straus (2003) estimates that 87% of Japanese prisoners of war had no useful intelligence to impart.

One of the most successful U.S. Navy interrogators was Lieutenant Otis Cary, who had been born on Hokkaido to missionary parents and attended Japanese schools through fourth grade. He was fluent in casual Japanese and felt considerable empathy for those he interrogated: "Japanese were used to being coerced and knew how to take evasive measures; if treated humanely, they lost the will to resist" (quoted by Straus 2003). Car's most remarkable accomplishment was persuading a group of Japanese POWs to assist in drafting surrender leaflets, out of his own sincere desire to see a new democratic Japan arise from the ashes of war.

So successful were Allied interrogators in building trust with prisoners that an apocryphal story began circulating around JICPOA, Nimitz' intelligence organization (Straus 2003):

... Admiral Nimitz [asked] for some information from his chief intelligence officer, who then telephoned the language officer in charge of the Japanese prisoners. The voice answering said that the officer was unavailable as he was out buying sports equipment for the POWs. Nimitz' chief intelligence officer then asked to speak to the deputy officer in charge, only to be informed that this individual was also out of the office, taking some of the prisoners to the movies in Honolulu. Becoming quite annoyed, the chief of intelligence asked to talk to the only other Japanese language officer there but was informed that this officer too was regrettably unavailable as he was purchasing phonograph records that he thought the POWs might enjoy. Now totally exasperated, the senior officer wanted to know to whom he was speaking. "Major Tanaka, Imperial Japanese Army, Sir ..."

Japanese officers were particularly prized as prisoners, but many tried to pass themselves off as enlisted men. They could sometimes be detected by watching their interactions with other prisoners, where the huge social gap between Japanese officers and their men was hard to conceal (Straus 2003):

Toyota brought two stools into the enclosure, sat down on one, and waited to see what might develop. As expected, one of the prisoners promptly offered the seat to the other, thus establishing the rank difference between the two. The lower-ranked soldier was taken away, while Toyota proceeded to interrogate the higher ranked one on the stool. He gave him a hard time for lying about his rank. Toyota pointed out that by removing the other POW, he had avoided making the officer lose face in front of his subordinate. The officer promptly confessed his status and admitted the other prisoner was his orderly.

Surprisingly, it was discovered that interrogation was more effective after a prisoner had been allowed to mingle with other Japanese prisoners in the POW map. Possibly the realization that he was not the only one in his situation seemed to give a prisoner a sense of diffused responsibility.

A special category of prisoners was defectors, men who voluntarily sought the opportunity to surrender because of disaffection with their own side. I have found one documented case of Japanese soldiers deserting to the Americans. A handful of enlisted men from a communications unit on Okinawa, who were privy to more information on the course of the war than the average Japanese soldier, concluded that "they might be better off if Japan were defeated; Japan might then become one of the states of the United States or a republic like France" (Straus 2003). The men slipped away from their unit, took a small boat to the nearby island of Kume Jima, were picked up by a Japanese naval unit from which they deserted a second time, and surrendered to the first Americans they could find. There were a small number of defections by Western soldiers early in the war, but defection became all but unknown once Western troops had the measure of their enemy. Colonial troops were far more likely to defect, and defecting Indians made up the bulk of Indian National Army. Japanese intelligence officers in Manchuria found that they could gauge the state of morale in the Russian Army by the number of defectors; this peaked in 1942 but dropped considerably by 1945.

The need for skillful interrogation was not limited to prisoners of war. Friendly local civilians were sometimes important sources of information, though such civilians risked being treated as spies if they crossed enemy lines. Debriefing, the interrogation of one's own pilots and soldiers returning from sorties and patrols, was an important aspect of intelligence, and one sometimes requiring considerable tact and skill. Pilots on all sides tended to greatly overestimate their own effectiveness in combat, claiming far too many kills of enemy aircraft and far too much damage to enemy shipping. Because the Allies had access to other superb sources of intelligence, they were able to confirm the survival of many ships thought by attacking pilots to have sunk. Allied intelligence officers came to be skeptical of claims of aircraft kills as well, and began to require kills to be confirmed by gun cameras or a second pilot. The Japanese were far less careful about damage assessment, and grossly inflated claims of ships sunk and aircraft destroyed characterized Japanese battle damage assessment throughout the war. This often led the Japanese to adopt unrealistic courses of action, such as the order to Shima to mop up 3 Fleet after it was supposedly shattered off Formosa in the fall of 1944.

The difficulty of battle damage assessment was not entirely the fault of the pilots. An armor-piercing bomb penetrating an armored ship produced almost no visible signature, and the effects even of a general-purpose bomb could be hard to distinguish by aircrew of a dive bomber pulling out of a steep dive.

Captured documents. 

Also important for the Allies was the capture of documents. Allied divers became proficient at recovering important documents from wrecked ships, including very sensitive documents recovered from the sunken cruiser Nachi in Manila harbor. Documents were also recovered from overrun Japanese positions. In one incident, a Japanese commander in New Guinea buried a cache of code books rather than burning them, and they were discovered by Allied forces, giving a tremendous boost to cryptanalysts. Diaries recovered from Japanese bodies had considerable intelligence value, though sometimes it was difficult to get Allied combat troops to give up their "souvenirs."  The Allies actively discouraged their own men from keeping diaries, though individuals like Eugene Sledge (With The Old Breed) and James Fahey (Pacific War Diary) found ways to surreptitiously record their experiences, to the delight of later historians.

Capture of documents was not a purely wartime activity. U.S. Navy Intelligence burgled the Japanese Consulate in New York to obtain vital cryptographic information. This proved so valuable that the Navy Department persuaded the State Department to leave Japanese consulates open when the German consulates were closed on 10 July 1941.

The Japanese also made use of captured documents. On 11 November 1940 the German merchant raider Atlantis captured the British cargo liner Automedon off Sumatra. The British ship was carrying a number of very high-level dispatches to Commander in Chief, Far East, which the crew were unable to destroy before they could be seized by the German boarding party. The Germans turned the documents over to the Japanese at Tokyo. The documents laid bare Britain's weakness in the Far East, particularly her inability to hold Hong Kong in the event of war with Japan.

Captured equipment.

The Americans were keenly interested in Japanese aircraft technology, and the capture of a nearly intact Japanese Zero fighter may have had some influence on the final design of the F6F Hellcat, which was in the prototype stage at the time. The Technical Air Intelligence Center, based at Anacostia (outside Washington, D.C.), received all captured Japanese air equipment from the Pacific and reconstructed a number of Japanese aircraft designs. Flight instruments, radios, and oxygen equipment were usually replaced with American designs in those reconstructed aircraft intended for flight, but the original equipment was carefully examined and catalogued. TAIC eventually reconstructed were several Zeros, two Oscars, a Tony, a Dinah, and a Kate.

Reconnaissance.

Reconnaissance was another important form of intelligence. Tactical reconnaissance consisted of patrols in land combat and of air searches in naval combat. The Japanese debacle at Midway was partially a consequence of ineffective air search. Strategic reconnaissance could be conducted by submarines or by long-range aircraft equipped with special cameras. One of the most important functions of aerial reconnaissance in the Pacific was to obtain accurate topographic maps of future battlefields for tactical use.

Signals intelligence.

Signals intelligence was decisive to the course of the conflict. Traffic analysis is the art of deducing enemy dispositions from the pattern of radio signals and was practiced with success by both sides. Cryptanalysis, the actual decoding of enemy signals, was practiced most successfully by the Allies, though the Japanese did succeed in breaking some low-level codes, such as those used to transmit weather reports.

Strategic radio intelligence was complemented by battle intelligence. Following the failure of the Wake relief effort in December 1941, Fletcher asked that his task force be assigned "specially trained radiomen to copy Japanese sending, and specially trained officers to interpret was is being intercepted" (Lundstrom 2006). He received a Marine captain fluent in Japanese in January, and shortly thereafter Nimitz assigned a radio intelligence team to each carrier task force. These proved increasingly effective as the war went on. At the Battle of the Philippine Sea, the intelligence team was able to obtain such useful information from monitoring the Japanese strike leader that they humorously requested that the combat air patrol not shoot him down at the end of the engagement, as he had served the Americans so well.

Tactical signals intelligence included radio direction finding to pinpoint enemy transmitters. The British had become so adept at locating Japanese headquarters in Burma by 1944 that the Japanese were forced to locate transmitters some kilometers from their headquarters to avoid disruptive air raids.

Open source intelligence.

  Open source intelligence was intelligence derived from newspapers, news broadcasts, and other sources that were openly circulated and freely obtainable. This is sometimes also described as document mining. Wartime censorship was directed at preventing the open circulation of material of obvious military value, but disconnected and seemingly innocuous information could sometimes be drawn together by a skilled analyst to reveal valuable information. 

Analysis

Collection and delivery of data was only half the intelligence story. Raw data required analysis, which was the process of turning data into information and information into knowledge. The Allies devoted considerable resources to analysis and had a clear edge over the Japanese in this area. One consequence was the Japanese failure to deduce that their codes were being read, in spite of several glaring slips by the Allies. But even Allied intelligence could occasionally go wrong. A mistaken deduction about Japanese intentions in March 1943 led Ralph Christie to order the west coast of Australia cleared of shipping in anticipation of a carrier raid that never materialized.  Ironically, documents from Fukudome's captured briefcase (mentioned above) led Spruance to believe that the Japanese might attempt an "end run" at the Battle of the Philippine Sea, which makes his cautious strategy in that battle easier to understand.

Analysis can be broken down further into capabilities and intentions. Capabilities are relatively easy to deduce, since (in principle) their deduction is a straightforward mathematical exercise. Intentions are much more difficult, and it has long been a military maxim to base one's strategy on what the enemy is capable of doing, rather than what you think he intends to do. However, the Allied success in cryptanalysis often revealed enemy intentions as well as capabilities.

Because the data were almost always incomplete, analysis resembled assembling a jigsaw puzzle with most of the pieces missing. The process of reconstructing the missing pieces was susceptible to the biases and preconceptions of the analyst. Good analysis requires getting into the enemy's mind. Allied contempt for the Japanese hindered this process before war broke out, most spectacularly at Pearl Harbor. The shock of that debacle led to improved analysis. The Japanese, in turn, misread the Americans because of their own contempt for such a "soft" race. The most important example was the failure to appreciate how the Pearl Harbor attack would infuriate the Americans and make a negotiated peace impossible. This is particularly surprising, since the chief advocate of the Pearl Harbor attack, Yamamoto, had spent considerable time in America. On a tactical level, the assumption that Americans were soft led Japanese intelligence to grossly underestimate how many Marines had landed on Guadalcanal. The Japanese assumed that the American desire for "the amenities" meant that twenty transports were required to carry an American regiment, when in fact the 23 transports off Guadalcanal had brought in a full division.

Analysis could also be biased in favor of the answers desired by those asking the questions. One of the more serious Japanese intelligence failures concerned the Kokoda Trail. The Japanese captured a book in Manila containing a vague, second-hand account of a road from Buna to Port Moresby. Aerial reconnaissance showed that there was a road from Buna to Kokoda that continued on to the edge of the thick jungle. Based on the vague references in the Manila book, Japanese intelligence concluded that the road continued on under the jungle canopy, a conclusion that was "confirmed" by a second aerial observer who thought he saw a serviceable road from Kokoda to Isurava. However, the observer was disoriented, was attempting to evade Allied fighter cover, and misjudged the width and condition of the trail. Further failures in command led to the disastrous Kokoda campaign.

Counterintelligence

Just as military forces seek intelligence concerning the enemy, they also seek to deny the enemy intelligence about themselves. There are basically three ways to do this. The first is to damage the enemy's intelligence collection apparatus. The second is to reduce one's own information signature. The third is to swamp one's own information signature with false information.

Counterespionage.

  Counterespionage is basically detective work to identify spies. It relies heavily on leads from ordinary soldiers and citizens who think they have seen something odd. Since the overwhelming majority of leads go nowhere, this is dull, grinding work, contrary to the glamorous James Bond image. It could also be an ugly business, as spies enjoy little protection under international law and espionage during the war years was universally regarded as a capital offense. The Allies were remarkably successful at counterespionage in the European theater, where it appears that every agent the Germans attempted to infiltrate into Britain was immediately arrested and forced either to "turn" or be executed by hanging. Allied counterespionage was less prominent in the Pacific, but the Japanese breaking of the Sorge ring was a major scandal, while the Kempeitai broke a spy ring in a Hong Kong prisoner of war camp and other spy rings in the Netherlands East Indies and the Philippines.

In Manchuria, the Japanese Army exposed Soviet agents infiltrated across the border by putting subtle scratches on the lettering of passports and by instructing passport issuing agents to stamp passports at slightly different angles depending on the date of departure. This uncovered a number of Soviet agents traveling with forged passports. A Russian agent disguised as a Japanese Army officer, Yuan Wang, was uncovered when he confused the equipment of officers and enlisted men and was found to be carrying a scabbard for his sword that was not genuine issue. Radio transmitters in urban areas were uncovered by selectively cutting power to separate city blocks.

Counterespionage overlaps counterinsurgency when guerrillas serve as a source of intelligence. The Japanese degree of success at counterespionage in the Philippines closely correlated with their degree of success at counterinsurgency.

Screening.

  Screening is the process of foiling enemy reconnaissance. The most obvious form of screening is to intercept and destroy the reconnaissance unit. This is sometimes harder than it would appear. Reconnaissance aircraft were designed for high speed and approached at high altitude, making them difficult to intercept. Submarines on reconnaissance missions were almost impossible to thwart, since they could choose the time and place to expose themselves. Infantry patrols were easier to defeat: Effective patrolling requires skill and nerve, and patrols could be thwarted by constructing obstacles, such as barbed wire or minefields, or by deploying listening posts or noisemakers. A favorite technique of the Marines was to hang empty tin cans with pebbles on barbed wire in front of their lines. These created noise if the barbed wire was disturbed by infiltrators.

Operational security.

  Operational security. is the art of reducing one's own intelligence signal. Means for accomplishing this included maintaining radio silence whenever possible; using means other than radio to communicate sensitive messages; using various forms of camouflage to evade or confuse enemy reconnaissance; and limiting distribution of information strictly to those persons with a genuine need for the information. Allied personnel were discouraged from keeping diaries, and deploying soldiers and sailors were not told their destination until they were already at sea.

There were some significant lapses in operational security, particularly early in the war. Supplies for operation BOBCAT were left on docks on the West Coast with labels clearly indicating their destinations. U.S. reconnaissance teams on New Georgia left behind scraps of food and a torn up letter that tipped the Japanese off that an invasion was imminent.

Deception.

  Deception is the art of contriving false or misleading information that is sufficiently convincing to fool the enemy. Since enemy intelligence is likely already flooded with a mass of observations, from which they must somehow sift the few observations of real value, one need not plant much false information to make the enemy's task harder. However, there is always the risk that the deception will be detected and give the enemy genuine clues to one's own intentions. This risk can be minimized by planting some true but out of date or otherwise useless information in order to give the sources of false information greater credibility.

Sources of false information during the Pacific War included simulated radio traffic to mislead enemy traffic analysis; deployment of dummy aircraft or tanks to attract the attention of enemy reconnaissance; and use of double agents to return false or misleading human intelligence. In November 1941, the Japanese Navy redeployed a replacement air group to the bases vacated by the Pearl Harbor strike force, granted shore leave to as many sailors as possible, and had the replacement air group send out dummy messages to conceal the sailing of the strike force. Allied troops headed for the Aleutians were lecture on tropical diseases and senior officers allowed themselves to be seen studying charts of the Atlantic or the Argentine coast in hopes of planting false rumors about their destination.

One form of deception is the Haversack Deception, in which faked documents are planted where they will be discovered by the enemy. The Allies had great success in Europe with Operation Mincemeat, in which a dead body was floated ashore in Spain dressed as a Royal Marines officer and bearing a suitcase full of documents suggesting an Allied assault on Greece. This successfully drew Axis attention away from Sicily, where the real assault was planned, with the added bonus that genuine documents captured later in the war were discounted by the Germans as plants. No such success was achieved in the Pacific, in spite of efforts such as leaving a wrecked jeep containing fake documents in the path of the Japanese advance in Burma. The Japanese simply ignored the wreckage.

The best deception schemes played to the enemy's prejudices, and if one's own intelligence service was reading enemy codes, it was possible to closely monitor the effect that deception schemes were having on the enemy's thinking and adjust accordingly.

Intelligence and Special Operations

Special forces often play an important role in intelligence gathering. In fact, a good case can be made that intelligence gathering is the only role for special forces that has consistently proven to be cost effective. Intelligence organizations, in turn, have historically almost always expanded their mission from simple intelligence and counterintelligence to more active covert operations. 

The Allies had a mixed record in this regard during the Pacific War. For the most part, the Alamo Scouts were recognized as an intelligence asset that should not be squandered on combat missions. However, the Philippine guerrillas were an invaluable intelligence source, and Alamo Scouts working with guerrilla groups could not entirely avoid involvement in guerrilla combat operations, though this involvement was usually limited to providing supplies in return for intelligence. Alamo Scouts were also assigned to locate high-ranking Japanese officers in the Philippines, with the implication that assassination was the ultimate goal. However, none were ever located.

Underwater demolition teams simultaneously collected intelligence and destroyed beach obstacles as a routine part of their missions, but in their case the dual mission made perfect sense. However, the UDTs avoided firefights with enemy troops, for which they lacked training and equipment.

The OSS was nominally an intelligence organization, but its director, William Donovan, insisted almost from the start that the OSS should engage in covert operations as well. Its closest British counterpart, the SOE, began as a covert operations organization and played only a modest role in intelligence, mostly because MI6, the regular British military intelligence bureau, was distrustful of the professionalism of SOE and jealous of its own turf.

The Japanese ran both intelligence and covert operations out of its Eighth Section, which ran both the Nakano School for intelligence officers and the Noborito Research Institute for development of special equipment for covert operations. The Japanese seem to have had a particularly poor record for muddling intelligence and covert operations, and it is difficult to examine the history of the Nakano School graduates without concluding that the romance of covert operations too often took priority over sound intelligence work.

Allied Intelligence

Allied intelligence benefited tremendously from the efforts of cryptanalysts. However, this was only one element of a comprehensive intelligence program that gave equal weight to information gathering and analysis. The Joint Intelligence Center, Pacific Ocean Areas (JICPOA) eventually had a strength of 1,767 specialists drawn from all branches of the armed forces. MacArthur received valuable intelligence from the Central Bureau, a joint American-Australian organization, and 7 Fleet's Fleet Radio Unit Melbourne (FRUMEL). When changes to Japanese codes temporarily thwarted the cryptanalysts, the intelligence community fell back on traffic analysis. This information was supplemented with direct reconnaissance from submarines and long-range reconnaissance aircraft.

Intelligence from the HYPO station at Oahu, which made heavy use of cryptanalysis, was decisive in the Coral Sea and Midway campaigns. However, HYPO did not formally report to Nimitz and the information from HYPO might not have come to Nimitz' attention in time but for the close friendship between Rochefort, the commander of Hypo, and Layton, Nimitz' intelligence officer.The officer in charge of communications intelligence in Washington, John Redman, was skeptical of Rochefort's assessments and believed in any case that they should go to Washington for further analysis before being forwarded back to Nimitz. It was likely Redman's influence that led to the subsequent shameful treatment of Rochefort, who was denied a Distinguished Service Medal for what may have been the single most important intelligence coup in American history until long after his death.

The Allies took considerable pains to conceal their signals intelligence from Japanese counterintelligence. However, there were occasional blunders in the war which the Japanese failed to exploit. Sensitive intelligence was much too widely disseminated until after the Battle of Midway, as when Spruance announced to his task force that a Japanese admiral was thought to be present on a "battleship" (actually heavy cruiser Mikuma) that was about to be attacked. The announcement was intended to raise morale, but was a textbook example of too wide dissemination of (in this case faulty) intelligence.

Even the Allies did not have unlimited resources for intelligence gathering and analysis. Writing in 1956, long before ULTRA was declassified, Slim complained that (Allen 1984):

I had throughout been conscious that ... our intelligence ... was far from being as complete or as accurate as in other theatres. We never made up for the lack of methodically collected intelligence or the intelligence organization which should have been available to us when the war began. We knew something of the Japanese intentions, but little of the dispositions of their reserves, and practically nothing about one of the most important factors that a general has to consider— the character of the opposing commanders.

ULTRA was not going to tell the analysts much about the character of enemy commanders, but Slim's other complaints appear to be veiled complaints about the low priority given to the China-Burma-India theater by the ULTRA analysts. Allen (1984) suggests that the low priority given to CBI by ULTRA is responsible for the Japanese ability to achieve surprise at the start of U-Go and nearly trap 17 Indian Division.

Chinese intelligence was inept, particularly early in the war. The Chinese intelligence chief, Tai Li, was a sinister figure described by some Westerners as "China's Himmler", who was aligned with the Green Gang and had no scruples about means. Dai was more interested in keeping tabs on the Chinese Communists than the Japanese, and his heavy handed treatment of his own agents could not have helped matters.

Japanese Intelligence

Japanese Army intelligence officers were trained at the Rear Staff Training School, later known as the Nakano School, which was established in December 1937 and graduated its first 18 students in August 1939. The school emphasized espionage, propaganda, security, and black operations. It published the Introduction to Covert War (Himitsu-sen Gairon) and taught a curriculum that ranged from modern signals intelligence and cryptology to Kokutai-gaku ("Study for National Structure and Mind") and the 14th-century Jinno Shoto-ki ("Record of the Legitimate Succession of the Divine Emperors") "which became a mental and spiritual 'Bible' for the students" (Kotani 2005). This spiritual training was thought necessary to strengthen officers against bribery or "honey traps." The school emphasized survival skills and one of its graduates, Hiroo Onoda, who was sent to the Philippines, did not surrender until 1974. The Guadalcanal political officer, "Ishimoto", disappeared from history with the Japanese evacuation of the island and has never been identified, but may have been a Nakano graduate. A total of 1900 students eventually graduated from the Nakano School.

Japanese prewar intelligence had a number of successes. Koa Kikan ("Asia Development Agency") was active in Hong Kong prior to and during the battle of Hong Kong. Sakata Shigemori, who organized Sakata Kikan in Shanghai, made contact with the Triad secret society in Hong Kong and was actually arrested by the British on 12 May 1941, but was aided in escaping by the Triads. The Japanese military attaché in Bangkok, Tamura Hiroshi, controlled the Japanese intelligence network throughout southeast Asia and collected extensive topographical and order of battle information, particularly of the area from Kota Baharu to Singora that would later be the main landing area for the Japanese invasion of Malaya. The Japanese seem to have had excellent intelligence on the American contingency plan for war in the Pacific, Plan Orange, whose source remains uncertain today.

However, once the war was well underway, Japanese intelligence fell badly behind the Allies. The situation was summarized by a staff officer in the intelligence section of South Asian Army (Hastings 2007):

Only in 1944 did the war situation really begin to alarm us. The Japanese army did not take intelligence nearly seriously enough. At South Asia Army HQ, we had no proper system, no analytical section, no resources -- that's how bad it was. Perhaps our attitude reflected Japan's historic isolation from the rest of the world. We had no tradition of being interested in other societies and what they were doing. It came as a shock to realize how powerful the Allies were becoming, and how much they they knew about our actions and intentions.

Historian  Hando Kazutoshi added (op. cit.):

Intelligence became a backwater for officers who were perceived as unfit for more responsible postings. Strategic decision-making was concentrated in the hands of perhaps twenty people, military and naval. Even if our intelligence services had gained access to important information, it would have remained unexploited if it ran against the convictions of the decision makers. They would not have wanted to know.

The Japanese Army intelligence effort was chronically understaffed, a problem further aggravated by the tendency to overspecialize, so that, for example, Canton-tsu (specialists on Canton affairs) and Manmo-tsu (experts on Manchuria and Mongolia) rarely communicated with each other.  There were numerous special agencies with overlapping responsibilities, and these agencies tended to emphasize black operations over intelligence gathering. For example, the Matsu Kikan ("Pine Agency") engaged in extensive counterfeiting of Chinese currency, which was shipped to Shanghai at a peak of 200 million yuan per month. The Ume Kikan ("Plum Agency") was created to work with the collaborationist government of Wang Ching-wei. The Minami Kikan ("Little Tree Agency") sponsored the Burma Independence Army.

The Japanese Army emphasized tactical intelligence and directed almost all its intelligence efforts against the Chinese and Russians, regarding Britain and the United States as the concern of the Navy. Code breaking initially received little emphasis, though the Japanese received some assistance from the Poles in decoding Russian communications. The Japanese fared poorly in their attempts at espionage across the Russian-Manchurian border; almost all the Japanese agents send across the border were promptly arrested, and some were turned. When the Russians rolled across the border in August 1945, almost the entire staff of Harbin Special Duty Agency, which carried out clandestine operations, were promptly arrested based on information from double agents.

Army intelligence against the Kuomintang scored numerous successes, but the Chinese Communists were more security-conscious, were trained by the Russians, and proved a harder nut to crack. The Japanese Army was caught by surprise by the "Hundred Regiments Offensive" of July 1940 and took heavy losses, but the Communists continued to be regarded as a much less significant threat than the Kuomintang by the Army General Staff.

Army intelligence did not shift its main effort to the Americans until July 1943, rather late in the game, but Japanese Army code breakers did have some successes against the western Allies, penetrated both the American diplomatic Brown Code (by breaking into the Kobe consulate and photographing the code book) and, by 1943, the diplomatic strip cipher. As with many cryptographic breakthroughs, the cracking of the strip cipher was aided by American operator carelessness. The Japanese also broke some lower-level British diplomatic codes, which hinted that the defenses of Malaya were inadequate.  Japanese Army intelligence veterans later claimed that they made significant inroads into the tactical M-209 machine cipher and thus were able to anticipate some B-29 raids. These included the Hiroshima raid, though its full significance was not realized in time to take any defensive action.

Matsu Kikan was redeployed to the Netherlands East Indies by the end of 1943, and on 14  January 1944 elements of the unit departed from Kupang aboard a 25-ton fishing vessel, Hiroshi Maru. The crew consisted of Lieutenant Mizuno Susihika, nine Japanese soldiers and sailors, and fifteen natives of Timor to man the boat and put off any inquisitive Allied patrols. Their mission was to reconnoiter Admiralty Bay (125.892E 14.286S) to determine if the Americans were constructing a naval base in the area. The party was forced back to port by a tropical storm but departed again on 16 January. They were provided with a Ki-51 "Sonia" escort, a rather strange accompaniment to what was supposed to be a covert mission, and the aircraft forced down and depth charged an Allied submarine at one point. However, the mission was otherwise completely uneventful, and the Japanese scouts made their first landing on 17 January. They continued to scout the area until 20 January, finding absolutely nothing of military interest, and then returned to Timor. Rumors of additional scouting missions to northern Australia have never been confirmed.

Some local units created their own intelligence organizations. 28 Army in Burma, for example, organized Hayate Tai ("Gale Unit") in October 1944 to conduct guerrilla operations from the Pegu Yomas hills. The unit ended up conducting intelligence gathering in preparation for the breakout across the Sittang instead. Its members were chosen for their physical resemblance to Burmese natives and were steeped in Burmese customs and mannerisms so that they could blend with the local population. They even plucked the hairs out of their legs to better resemble the very smooth-skinned Burmese. Nonetheless, casualties were high, and just 34 men out of the original 160 were left when the unit was disbanded at Moulmein at the time of the Japanese surrender.

The Japanese Navy had no intelligence school before war broke out nor any courses on intelligence in its other schools. Funding and manpower were grossly inadequate, with just ten men assigned to the General Staff section on American intelligence prior to the attack on Pearl Harbor.  Staffing remained inadequate even under wartime conditions. Furthermore, turnover was excessive, with five different officers serving as chiefs of the navy's intelligence department from 1940 to 1945. Japanese intelligence officers attributed this lack of attention to the ease with which the open U.S. society was penetrated in peacetime, which led to complacence. The Japanese conducted peacetime espionage from their consulates and embassies, but also put in place "sleeper" organizations with no connection to consulates to be activated in time of war. Espionage from the Japanese consulate on Oahu was highly effective, but the local sleeper organization proved almost completely ineffective once war broke out, and it was rapidly rolled up by the FBI.

Japanese intelligence failures had the perverse consequence of lowering the prestige of the intelligence apparatus, causing it to receive less funding and manpower in a downwards spiral.

Japanese Navy cryptography had some limited successes. The Navy succeeded in cracking the American diplomatic Gray Code in the 1920s by assuming that NADED, which appeared frequently, meant "stop", a classic use of a crib. The Navy also broke Brown, the successor to Gray, independently of the Army. The Germans gave the Japanese part of the code book for BAMS (Broadcast for Allied Merchant Shipping), which had been captured by the merchant raider Atlantis, and the Japanese had some success thereafter predicting American operations from the movement of merchant shipping.

Japanese Navy sources of intelligence late in the war are summarized by Kotani (2009):

A number of things stand out in these statistics. First, the majority of intelligence was gathered from open sources such as radio broadcasts and newspapers. Signals intelligence made up most of the rest of the reports. Open sources can be valuable; Japanese intelligence built up a surprisingly accurate picture of the B-29 based almost entirely on open sources well before the first Superfortress appeared over any Japanese target. However, Allied operational security meant that open sources were a poor source of intelligence on Allied strategy.

Second, cooperation with the Army was practically nonexistent, with only eleven Army intelligence reports being shared with the Navy. This lack of cooperation between the services was evident elsewhere, even extending to the Army concealing from the Navy the fact that the Army had broken the American strip cipher system used for lower-level communications. Lack of cooperation also hindered Japanese cipher security, which was not taken seriously until at least late 1943 with the organization of the joint Cipher Security Committee.

Lack of cooperation was also evident between between the Army and the civilian government. The Cabinet Intelligence Department was effectively "castrated" by the Army (Kotani 2009) and became a mere propaganda department. Bureaucratic turf wars were a serious impediment even with the services, as operational departments often ignored intelligence departments and even carried out independent intelligence activities without consultation. Because the operations departments rarely communicated their requirements to the intelligence departments, the intelligence departments often gathered the wrong information. The collegiality that characterized British intelligence was entirely lacking.

Exchange of intelligence with other Axis powers was very limited. Major General Oshima Hiroshi, military attaché to Germany, signed an agreement in 1937 with Admiral Wilhelm Canaris, head of the German Abwehr, to exchange intelligence on the Soviets, but little information was actually exchanged. An agreement in 1941 to exchange signals intelligence between the navies also proved of limited value.

Japanese intelligence also appears to have suffered from arrogance. One consequence was the refusal to believe that JN-25 had been broken. Another was the failure to act in a timely manner on partial intelligence. Imai Nobuhiko, a lieutenant colonel serving with the Navy General Staff, observed that (Kotani 2009)

They [the US Forces] immediately reported raw news and data that they saw through observation or heard on radio transmission, unlike our technique: to collect information by various means and to examine the enemy's condition by assembling the information through mathematical means. Although their way of forming intelligence is too simple and direct to be compared with our extremely complicated system, I dare classify the former as "intelligence by power" and the later as "intelligence by mind."

One cannot help concluding that the Japanese expended too much effort cooking their data and too little effort getting information to the operational commands in a timely manner.

References

Allen (1984)

Collie and Marutani (2009)

Deacon (1983)

Federal Bureau of Investigation Website (accessed 2014-8-6)

Felton (2009)

Fleming (2001)

Frank (1990)

Harmsen (2013)

Hastings (2007)

Keegan (2003)

Kotani (2009)

Lindsay (2005)

Lundstrom (2006)

Marston (2005)

Mercado (2002)

Mikesh (2004)

Morison (1951)

Nakagawa (1993)

Prados (1995)
Prange (1981)

Sasgen (2010)

Smith (2000)

Straus (2003)

Symonds (2011)

Tamayama and Nunneley (2000)

Thompson (2005)

Tillman (2010)

Womack (2006)

Zimm (2011)

The Pacific War Online Encyclopedia © 2007-2015 by Kent G. Budge.  Index

What's new

Comment on this article