Competition is usually a good thing. It allows us to push our limits and attempt to better our own performance. In analysis, having a variety of competing explanations or estimates (perhaps a result of Multiple Hypothesis Generation) prevents us from several pitfalls, including being overly influenced by first impressions, selecting the first answer that appears “good enough,” focusing on a narrow range of alternatives, and Confirmation Bias. However, when faced with a set of mutually exclusive alternative explanations or outcomes, how do we weed out the choices? Analysis of Competing Hypotheses (ACH), the fifth and final technique in our summer series of popular Structured Analytic Techniques, is a useful tool when you have disparate data to evaluate.

Analysis of Competing Hypotheses (ACH) is an analytic process that identifies a complete set of alternative hypotheses, systematically evaluates data that are consistent or inconsistent with each hypothesis, and rejects hypotheses rather than trying to confirm what appears to be the most likely hypothesis.

Use ACH:

• When you need a systematic approach to prevent being surprised by an unforeseen outcome.
• On controversial issues when it is desirable to identify precise areas of disagreement and to leave an audit trail to show what relevant information was considered.
• When you have a robust flow of data to absorb and evaluate.
• When you have a small team whose members can question one another’s evaluation of the relevant information.

The Method:

1. Identify and list the hypotheses to be considered. The list should allow for all reasonable possibilities, including a deception hypothesis – if that is appropriate. Develop a brief scenario or “story” to explain how each hypothesis might be true.
2. Make a list of significant information – including evidence,  assumptions, and the absence of things one would expect to see if a hypothesis were true.
3. Create a matrix* with each hypothesis displayed across the top and each item of relevant information listed down the left side. Analyze each input by asking, “Is this consistent (C) with the
   hypothesis, inconsistent (I), or is it not relevant or applicable (NA)?” Use “CC” for particularly compelling items of information and “II” if the piece of information strongly undercuts the hypothesis. Complete the matrix by either filling in each cell row-by-row or using a survey method that randomly selects a cell in the matrix for the analyst to rate. After the items of information have been sorted for diagnosticity**, note how many of the “II” ratings are based on assumptions. Consider how much confidence you should have in those assumptions and then adjust the confidence in the ACH Inconsistency scores accordingly.
4. If working with several analysts, review where they differ in their assessments and decide if adjustments are needed in the ratings. Differences in ratings can often be traced back to different assumptions about the hypotheses.
5. Refine the matrix by reconsidering the hypotheses. Does it make sense for two hypotheses to be combined into one or should a new, previously unconsidered hypothesis be added? Relevant information can be added at any time.
6. Draw tentative conclusions about the relative likelihood of each hypothesis, basing your conclusions on an analysis of the diagnosticity rating of each item of relevant information. The ACH software* helps the analyst by ranking each hypothesis from least Inconsistent to most Inconsistent. The hypothesis with the lowest Inconsistency score is tentatively the most likely hypothesis.
7. Analyze the sensitivity of your tentative conclusion to a change in the interpretation of a few critical items of relevant information. Consider the consequences for your analysis if one or more of these critical items of relevant information were wrong or deceptive. If a different interpretation would be sufficient to change your conclusion, go back and doublecheck the accuracy of your interpretation.
8. Report the conclusions. Consider the relative likelihood of all the hypotheses. State which items of relevant information were the most diagnostic and how compelling a case they make in identifying the most likely hypothesis.
9. Identify indicators or milestones for future observation. Generate two lists: the first focusing on future events or what might be developed through additional research that would help prove the validity of your judgment; the second, a list of indicators that would suggest your judgment is less likely to be correct or the situation has changed. Validate the indicators and monitor both lists on a regular basis, remaining alert to whether new information strengthens or weakens your case.

*Click here for more information about ACH as well as a link to free matrix software.

** To perform sort for diagnosticity and adjustment of confidence level, use the ACH software (link shown above).

SATs can (literally) save your life!

Structured Analytic Techniques (SATs) improve efficiency in analysis, but can be just as important outside the workplace. As explained in Randy Pherson’s upcoming publication, Questions You Should Ask Your Doctor If You Don’t Want To Die!, SATs (including four of those discussed in recent issues of the Analytic Insider) can benefit your personal life – and perhaps even save your life.

If you want to get your doctor’s attention and regain control over how you will be treated, ask these five questions:

1. What key assumptions are you making and could any of them be wrong? (Key Assumptions Check)
2. What alternative explanations might there be for my problem? (Multiple Hypothesis Generation)
3. What confirming indicators should I look for that would suggest a particular explanation is correct? (Indicators and Indicators Validator)
4. What disconfirming indicators should I look for that would suggest a particular explanation cannot be correct and can be dismissed? (Analysis of Competing Hypotheses and Disconfirming Evidence)
5. If six months from now, you had to explain in hindsight why I died, and the current diagnosis was found to be incorrect or the current treatment of no value, how would you explain my death? (Premortem Analysis and Structured Self-Critique)

Clear, concise instructions for each question/technique are included in the book as well as obstacles to anticipate and tips to help stay alive. Questions You Should Ask Your Doctor If You Don’t Want To Die! will be published in 2016.

Learn to Crack The Code – Register before it’s too late!

Seats are still available for Globalytica’s popular certificate course later this month:

What: Crack the Code-Diagnostic Structured Analytic Techniques Certificate Course (DSAT)

When: 22-24 September; 0900-1300 Daily

Where: Globalytica Training Facility in Reston, VA

**Click here for details and to register**

Our DSAT Course is designed for analysts interested in learning  techniques to help uncover information gaps and inform future research design. DSAT provides students with a set of analytic tools and techniques to help formulate and refine ideas about what has happened or is currently occurring. Students will:

• Learn to identify the dynamics at play in an issue or problem.
• Practice reframing issues to understand better how forces or elements might combine to generate different outcomes in the future.

For more information or to find out about group pricing, please contact us at: think@globalytica.com.

TH!NK Suite®

Enhance the Impact of Your Analysis.

Our collaborative webbased tools help analysts employ Structured Analytic Techniques effectively. They can be used routinely making the analysis more rigorous while saving time.

Learn more