Borrowing from Military Doctrine

In his report Intelligence Preparation for Operational Resilience (IPOR), Douglas Gray describes common military approaches towards frequent operational actions such as Military Decision-Making Process (MDMP) and Intelligence Preparation for the Battlefield (IPB) as highly useful for the private sector – yet, in need of adjustment since these approaches are heavy in military jargon (2015).  Thus, the Intelligence Preparation for Operational Resilience was born, which has a foundational basis in ways intelligence is prepared in conventional military approaches and should be “used during operational resilience planning to provide context to risk management decisions” within private sector organizations (Gray, 2015). 

IPB is employed by the military to study the enemy, its resources, tactics, strengths, and weaknesses.  As lines continue to blur between nation states and its proxies conducting cyberattacks against the private sector, critical infrastructure, and the public sector, it makes sense to leverage time-tested processes developed by the military.  The IPB approach is well-suited to cyber operations in the private sector because, in many cases, both public and private sectors face the same adversary (Gray, 2015).  This process takes threat intelligence and attempts to put context to it to make it actionable (Gray, 2015).  Developing actionable intelligence is achieved by leveraging the practice of decomposition, which is essentially the process of taking a complicated topic and breaking it into smaller, and therefore measurable, elements (Hubbard, 2014).  Decomposition is a valuable tool in intelligence analysis, statistics, and building risk models. 

By applying decomposition practices, Gray breaks down “sources of situational awareness” into three “Voices”:

1. The Voice of the Environment
2. The Voice of the Organization
3. The Voice of the Threat Actor

These three Voices can be further decomposed, as well.  Another factor to account for in this process is bias.  In traditional intelligence analysis, especially in the Post 9/11 environment, intelligence analysts leverage structured analytic techniques where possible because it reduces the risk of bias and promotes consistency in results over time (Heuer & Pherson, 2015).  While not expressly discussed in Gray’s report, an important technique in traditional intelligence analysis called calibration might also serve well in the IPOR process. 

Calibration feedback, which Rieber describes as analyst feedback concerning how well their probability estimates correspond to number of accurate predictions (Rieber, 2004).  The value of calibration can be different for every field, for example, a physician making a probability estimate for a patient with cancer can have tremendous impact.  Cyber threat analysts making probability estimates for threat actors or TTPs targeting specific business assets or industries may result in a company investing large portions of its budget to prevent and defend against such attacks.  It is clear intelligence analysts, whether cyber threat or traditional, have a similar responsibility to seek calibration feedback to improve probability estimates.

In the study of cause-and-effect, we learn there can be several subsequent outcomes which stem from a single causal outcome, also known as second and third-order effects (Miller, 2006).  The practice of calibration matters to an intelligence analyst because they may be responsible for first, second, and third order effects of a single overconfident or underconfident prediction or probability estimate.  Calibration feedback assists the analyst to identify whether they are over or under confident, and how to improve their probability estimates accuracy (Rieber, 2004).  Specifically, in the virtual and physical environment where rapid change is a given, calibration can assist in reducing risk ahead of incidents. A good way to calibrate oneself in situations of high uncertainty or rapid change is to map out assumptions and several counter assumptions, which forces considering an issue from multiple angles.  Further, it forces the analyst to consider other likely factors. Used as a supplement to the IPOR process Gray outlines, calibration would be a powerful tool for organizational planning and risk management for cyber threats.

References

Gray, D., (2015, Dec.). Intelligence Preparation for Operational Resilience (IPOR). Software Engineering Institute, Carnegie Mellon University. Special Report CMU/SEI-2015-SR-033.

Hubbard, D. W. (2014). How to Measure Anything: Finding the Value of “Intangibles” in Business (3rd Ed). Hoboken, NJ: John Wiley & Sons

Heuer, R. & Pherson, R. (2015). Structured Analytic Techniques for Intelligence Analysis. Thousand Oaks, CA: CQ Press.

Miller, M. G. (2006). Thinking About Second & Third Order Effects: A Sample (And Simple) Methodology. IO Sphere, (Summer), 36-39. Retrieved August 21, 2017, from http://www.au.af.mil/info-ops/iosphere/iosphere_summer06_miller.pdf

Rieber, S. (2004). Intelligence Analysis and Judgmental Calibration. International Journal of Intelligence and Counterintelligence, 17(1), 97-112.

  ACH, analysis of competing hypothesis, cognitive bias, PIR analysis