When I was a manager of analysis at CIA, I would hear our customers, often senior policymakers, demand more INSIGHT in our analysis. And I would go back and tell the analysts they needed to produce more insight. Until one day an analyst asked me “Well, what is INSIGHT?” And I thought to myself, that’s a good question; a “good question” is ipso facto a question for which you do not have a ready answer.
I put on my tattered thinking cap and worked to come up with some type of answer—a “formula for INSIGHT” that was reproducible and generated a set of actions that analysts could actually perform. I asked many colleagues to describe how they thought. This is another good question. Almost nobody could describe their own thinking process.
“I read and then I write.”
What happens in between?
“I shake my head until some ideas fall out,” one analyst offered.
Eventually I came up with a formula—the steps of analysis—that I thought enough of to share with others. Like everything I do, it’s imperfect but hopefully it offers a starting point.
- COMPREHENSION. When we’re thinking about a problem, first we try to comprehend it. We assemble relevant information and consume it however we prefer.
- CATEGORIZATION. Once we’ve achieved some comfort in our level of understanding, the next step is categorization. We sort what we know into various categories and patterns. (Actually, this starts to happen organically during the Comprehension stage. This is unavoidable and can be the place where cognitive biases take root. Some information you consume early on colors how you think about every subsequent report, and you fall victim to the anchoring bias. I’ve always wanted to run an experiment where the same 100 pieces of information were presented to analytic teams, but in different orders. Would their analysis differ? My bet is yes!)
The categories can be as simple as Old Information, New Information, but they eventually evolve into a complex taxonomy that forms the backbone of your Analytic Lines. These Analytic Lines are powerful beings and resist change. This is usually very bad. - PRIMARY INSIGHT. INSIGHT occurs when you see things you’ve never seen before or in ways that are new to you. When an individual takes an item of information and argues that it belongs in a different category, you have produced a moment of INSIGHT. Recategorization of information is a way of generating INSIGHT. Is President Xi’s third term in China an indicator of his strength or of China’s weakness? The conventional wisdom probably is to categorize the event as the former but making a credible argument that it is the latter generates INSIGHT. The INSIGHT argument doesn’t have to be convincing; just provoking others to take a second look is useful.
- PROFOUND INSIGHT. A harder but more powerful way to generate INSIGHT is to renovate and/or rebuild your categorization schema. For example, analysts realize that a significant amount of information remains uncategorized—it doesn’t easily fit the current taxonomy. Do you ignore it, or do you begin to experiment with new categories that might better explain the information? And at some point you can experiment with rethinking your categorization scheme—your Analytic Line—from scratch. To return to the China example, how best should we think of it—as an emerging Superpower, as a declining power, or as a country destined for the middle-income trap? Each of these options can generate significantly different categorization schemes. (When your Analytic Line is long in the tooth, lots of information will no longer easily fit your existing categories. This is a “sell signal” for how you currently think about your problem, but not enough analysts recognize it as such.)
Analytic teams need to be hawk-like in policing their categorization schemes because they often sneakily embed themselves in the back-office processes of the organization. Take, for example, the reading profiles of an analytic team—the algorithms that determine which information flows into their inboxes. Ask your analysts how often these reading profiles are updated. You will not be happy with their answers.
What inspired me to natter on about analysis and insight on this beautiful fall day? Reading Adrian Wolfberg’s excellent monograph In Pursuit of Insight: The Everyday Work of Intelligence Analysts Who Solve Real World Novel Problems. It’s not a quick read but luckily there’s a shorter version available here. Based on extensive interviews with analysts of varying experience, Wolfberg seeks to unpack how insight actually happens from a cognitive, neurological perspective. It tackles the all-important step that my all-too-neat process completely ignores: how does the idea for new categories enter your brain? What leads to its emergence?
Wolfberg writes that the insight process begins with a trigger phase, “an initiating event that, seemingly by chance, brings the analyst’s attention to a particular problem to address; alternatively, after an analyst has been working on a given problem, a random event contributes to focusing their attention more intently on the problem. Entering into and navigating through the trigger phase takes cognitive and emotional courage on the analyst’s part.”
After the trigger phase, Wolfberg identifies emergence as the next step. Two activities promote the emergence of insight: internalized tensions and priming. Quoting from the shorter paper:
Internalized Tensions: As analysts start working on a novel problem, they become aware of inconsistencies that can be cognition-based (i.e., inconsistencies between pairs of thoughts or ideas) or emotion-based (i.e., inconsistencies between an analyst’s action and interpretation of others’ reactions). Tensions induced by these inconsistencies can originate within the individual (i.e., self-initiated) or in the individual’s social environment (i.e., related to organizational structure and the behavior of others). An analyst who concludes that exploring a diversity of ways to represent a problem would lead to the most accurate assessment, while others judge that a standardized process would be best, is an example of cognition-based tension at the individual level. An analyst who presents a unique methodology in a detailed, transparent way to address skeptical concerns, while worried this could lead to being discredited in a production system that values standard product lines, is an example of emotion-based tension at the social level.
Priming: Analysts draw upon memories of past experiences unrelated to their present novel problem in order to make progress toward overcoming tensions and reaching insight. Priming sources also occur across the intersection of the emotion-cognition and individual-social dimensions. In an example of cognition-based priming at the individual level, an analyst who studied in graduate school how physical factors in the environment could trigger social or political outcomes applies that cause-and-effect knowledge to a national security novel problem. In an example of emotion-based priming at the social level, an analyst who had lived in a foreign country appreciates that even in countries where the same language is spoken, cultures can be very different.
What’s clear is that insight emerges from a rich casserole of experiences, emotions, and feelings in an analyst’s mind. Our intuition, what Daniel Kahneman calls our System 1, is the primary custodian of these insight-generating prompts. Wolfberg notes that “although these past experiences were unrelated to the problem at hand, an aspect of these past experiences brought forth a combination of emotional and cognitive meaning that informed how the analysts thought…” Every analyst interviewed by Wolfberg reflected on past experiences unrelated to the problem.
Clarity and INSIGHT are the most sought-after products from intelligence analysts. Clarity is getting the facts straight, sorting out complicated situations. Intelligence organizations usually do clarity well, but not always (think Iraq WMD). INSIGHT requires going beyond the facts to consider hidden causes and motivations, anticipate unexpected changes in trajectory, and appreciate the nonlinearity of events. The work processes of most intelligence teams are suited more to producing clarity than generating INSIGHT. Analysts often describe having to buck the established way of doing things to explore their emerging INSIGHT.
As Wolfberg notes, leaders of intelligence organizations need to appreciate the conditions necessary for the generation of INSIGHT and work to allow the time and space necessary for its emergence. Many of the work processes of the Intelligence Community emphasize order and consistency over thoughtfulness and contemplation. Working 8+ hours a day in a cubicle is also not ideal. As the science writer Annie Murphy Paul notes in her excellent book The Extended Mind, human brains evolved to think best when we’re physically active. My favorite “structured analytic technique” has always been to take a walk, preferably with a favorite thinking partner.
Wolfberg’s study has many other insights about INSIGHT. It’s a rewarding read for anyone wanting to make intelligence analysis better.
RecoveringFed 